2023-10-08T13:29:03.644 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-08T13:29:03.644 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/default mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} --archive /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991 --name pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi --verbose -- /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991/orig.config.yaml 2023-10-08T13:29:03.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T13:29:03.799 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991 branch: wip-batrick-testing-20231007.154158 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/default mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7417991' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-batrick-testing-20231007.154158 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231007.154158 sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8022 sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de sleep_before_teardown: 0 subset: 128/256 suite: fs suite_branch: wip-batrick-testing-20231007.154158 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-07_20:01:29 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-08T13:29:03.800 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa; will attempt to use it 2023-10-08T13:29:03.801 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks 2023-10-08T13:29:03.801 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-08T13:29:03.802 INFO:teuthology.task.internal:Checking packages... 2023-10-08T13:29:03.824 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de' 2023-10-08T13:29:03.825 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-08T13:29:03.825 INFO:teuthology.packaging:ref: None 2023-10-08T13:29:03.825 INFO:teuthology.packaging:tag: None 2023-10-08T13:29:03.825 INFO:teuthology.packaging:branch: wip-batrick-testing-20231007.154158 2023-10-08T13:29:03.825 INFO:teuthology.packaging:sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:29:03.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20231007.154158 2023-10-08T13:29:03.963 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6639.g7b3143ba 2023-10-08T13:29:03.963 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-08T13:29:03.969 INFO:teuthology.task.internal:no buildpackages task found 2023-10-08T13:29:03.970 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-08T13:29:03.985 INFO:teuthology.task.internal:Saving configuration 2023-10-08T13:29:04.005 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-08T13:29:04.010 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T13:29:04.034 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 13:23:40.455017', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T13:29:04.054 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 13:23:40.452819', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T13:29:04.072 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 13:23:40.454000', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T13:29:04.072 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-08T13:29:04.079 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-08T13:29:04.079 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-08T13:29:04.079 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-08T13:29:04.079 INFO:teuthology.run_tasks:Running task console_log... 2023-10-08T13:29:04.179 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd98f694b80>, signals=[15]) 2023-10-08T13:29:04.179 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-08T13:29:04.186 INFO:teuthology.task.internal:Opening connections... 2023-10-08T13:29:04.186 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T13:29:04.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T13:29:04.256 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T13:29:04.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T13:29:04.324 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2023-10-08T13:29:04.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T13:29:04.391 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-08T13:29:04.400 DEBUG:teuthology.orchestra.run.smithi057:> uname -m 2023-10-08T13:29:04.420 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2023-10-08T13:29:04.421 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2023-10-08T13:29:04.480 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T13:29:04.481 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="8.6 (Ootpa)" 2023-10-08T13:29:04.481 INFO:teuthology.orchestra.run.smithi057.stdout:ID="rhel" 2023-10-08T13:29:04.481 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="fedora" 2023-10-08T13:29:04.481 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="8.6" 2023-10-08T13:29:04.481 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el8" 2023-10-08T13:29:04.481 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-08T13:29:04.481 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31" 2023-10-08T13:29:04.481 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-08T13:29:04.482 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T13:29:04.482 INFO:teuthology.orchestra.run.smithi057.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T13:29:04.482 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T13:29:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:29:04.482 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T13:29:04.482 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-08T13:29:04.482 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T13:29:04.482 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-08T13:29:04.483 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2023-10-08T13:29:04.501 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2023-10-08T13:29:04.522 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2023-10-08T13:29:04.522 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2023-10-08T13:29:04.580 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T13:29:04.581 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8.6 (Ootpa)" 2023-10-08T13:29:04.581 INFO:teuthology.orchestra.run.smithi099.stdout:ID="rhel" 2023-10-08T13:29:04.581 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="fedora" 2023-10-08T13:29:04.581 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8.6" 2023-10-08T13:29:04.581 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2023-10-08T13:29:04.581 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-08T13:29:04.581 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2023-10-08T13:29:04.581 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-08T13:29:04.581 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T13:29:04.581 INFO:teuthology.orchestra.run.smithi099.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T13:29:04.582 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T13:29:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:29:04.582 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T13:29:04.582 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-08T13:29:04.582 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T13:29:04.582 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-08T13:29:04.582 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2023-10-08T13:29:04.601 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2023-10-08T13:29:04.620 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2023-10-08T13:29:04.621 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2023-10-08T13:29:04.680 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T13:29:04.680 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="8.6 (Ootpa)" 2023-10-08T13:29:04.680 INFO:teuthology.orchestra.run.smithi115.stdout:ID="rhel" 2023-10-08T13:29:04.680 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="fedora" 2023-10-08T13:29:04.680 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="8.6" 2023-10-08T13:29:04.680 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el8" 2023-10-08T13:29:04.680 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-08T13:29:04.680 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2023-10-08T13:29:04.680 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-08T13:29:04.681 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T13:29:04.681 INFO:teuthology.orchestra.run.smithi115.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T13:29:04.681 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T13:29:04.681 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:29:04.681 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T13:29:04.681 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-08T13:29:04.681 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T13:29:04.681 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-08T13:29:04.681 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2023-10-08T13:29:04.701 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-08T13:29:04.714 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-08T13:29:04.720 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-08T13:29:04.720 DEBUG:teuthology.orchestra.run.smithi057:> test '!' -e /home/ubuntu/cephtest 2023-10-08T13:29:04.723 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2023-10-08T13:29:04.725 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2023-10-08T13:29:04.743 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-08T13:29:04.749 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-08T13:29:04.750 DEBUG:teuthology.orchestra.run.smithi057:> test -z $(ls -A /var/lib/ceph) 2023-10-08T13:29:04.783 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2023-10-08T13:29:04.786 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2023-10-08T13:29:04.815 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-08T13:29:04.893 INFO:teuthology.run_tasks:Running task kernel... 2023-10-08T13:29:04.906 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-08T13:29:04.907 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-10-08T13:29:04.907 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-10-08T13:29:04.907 DEBUG:teuthology.orchestra.run.smithi057:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T13:29:04.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T13:29:04.926 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2023-10-08T13:29:04.987 INFO:teuthology.orchestra.run.smithi057.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-08T13:29:04.987 INFO:teuthology.task.kernel:Running kernel on smithi057: 4.18.0-372.9.1.el8.x86_64 2023-10-08T13:29:04.987 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum install -y kernel 2023-10-08T13:29:06.180 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-08T13:29:06.181 INFO:teuthology.orchestra.run.smithi057.stdout:Unable to read consumer identity 2023-10-08T13:29:07.626 INFO:teuthology.orchestra.run.smithi057.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-10-08T13:29:08.469 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2023-10-08T13:29:12.252 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 438 kB/s | 24 kB 00:00 2023-10-08T13:29:13.005 INFO:teuthology.orchestra.run.smithi057.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-10-08T13:29:13.063 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-08T13:29:13.064 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2023-10-08T13:29:13.064 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-08T13:29:13.184 DEBUG:teuthology.orchestra.run.smithi057:> echo no | sudo yum reinstall kernel || true 2023-10-08T13:29:13.535 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-08T13:29:13.536 INFO:teuthology.orchestra.run.smithi057.stdout:Unable to read consumer identity 2023-10-08T13:29:13.827 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:00:01 ago on Sun 08 Oct 2023 01:29:12 PM UTC. 2023-10-08T13:29:13.876 INFO:teuthology.orchestra.run.smithi057.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-08T13:29:13.905 INFO:teuthology.orchestra.run.smithi057.stderr:Error: No packages marked for reinstall. 2023-10-08T13:29:13.952 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum reinstall -y kernel || true 2023-10-08T13:29:14.300 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-08T13:29:14.301 INFO:teuthology.orchestra.run.smithi057.stdout:Unable to read consumer identity 2023-10-08T13:29:14.585 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:00:02 ago on Sun 08 Oct 2023 01:29:12 PM UTC. 2023-10-08T13:29:14.631 INFO:teuthology.orchestra.run.smithi057.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-08T13:29:14.651 INFO:teuthology.orchestra.run.smithi057.stderr:Error: No packages marked for reinstall. 2023-10-08T13:29:14.703 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q kernel | sort -rV | head -n 1 2023-10-08T13:29:14.786 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-10-08T13:29:14.787 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-10-08T13:29:14.787 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-10-08T13:29:14.787 DEBUG:teuthology.parallel:result is None 2023-10-08T13:29:14.788 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-08T13:29:14.796 INFO:teuthology.task.internal:Creating test directory... 2023-10-08T13:29:14.796 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T13:29:14.798 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T13:29:14.800 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T13:29:14.821 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-08T13:29:14.828 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-08T13:29:14.864 INFO:teuthology.task.internal:Creating archive directory... 2023-10-08T13:29:14.864 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T13:29:14.867 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T13:29:14.869 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T13:29:14.910 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-08T13:29:14.917 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-08T13:29:14.917 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T13:29:14.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T13:29:14.962 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T13:29:14.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T13:29:14.981 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T13:29:14.984 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T13:29:15.004 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T13:29:15.017 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T13:29:15.028 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T13:29:15.055 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T13:29:15.062 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T13:29:15.093 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T13:29:15.098 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T13:29:15.099 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-08T13:29:15.107 INFO:teuthology.task.internal:Configuring sudo... 2023-10-08T13:29:15.107 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T13:29:15.110 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T13:29:15.112 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T13:29:15.169 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-08T13:29:15.234 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-08T13:29:15.234 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T13:29:15.239 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T13:29:15.243 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T13:29:15.261 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T13:29:15.338 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T13:29:15.391 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T13:29:15.475 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T13:29:15.517 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:29:15.517 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T13:29:15.585 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T13:29:15.630 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T13:29:15.685 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T13:29:15.771 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T13:29:15.818 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:29:15.818 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T13:29:15.888 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T13:29:15.933 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T13:29:15.986 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T13:29:16.071 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T13:29:16.115 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:29:16.115 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T13:29:16.184 DEBUG:teuthology.orchestra.run.smithi057:> sudo service rsyslog restart 2023-10-08T13:29:16.187 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2023-10-08T13:29:16.189 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2023-10-08T13:29:16.244 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T13:29:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T13:29:16.288 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T13:29:16.699 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-08T13:29:16.707 INFO:teuthology.task.internal:Starting timer... 2023-10-08T13:29:16.707 INFO:teuthology.run_tasks:Running task pcp... 2023-10-08T13:29:16.717 INFO:teuthology.run_tasks:Running task selinux... 2023-10-08T13:29:16.725 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-10-08T13:29:16.726 DEBUG:teuthology.orchestra.run.smithi057:> sudo service auditd rotate 2023-10-08T13:29:16.790 INFO:teuthology.orchestra.run.smithi057.stdout:Rotating logs: 2023-10-08T13:29:16.792 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2023-10-08T13:29:16.858 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2023-10-08T13:29:16.860 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2023-10-08T13:29:16.915 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2023-10-08T13:29:16.916 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-08T13:29:16.917 DEBUG:teuthology.orchestra.run.smithi057:> /usr/sbin/getenforce 2023-10-08T13:29:16.940 INFO:teuthology.orchestra.run.smithi057.stdout:Permissive 2023-10-08T13:29:16.941 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2023-10-08T13:29:16.971 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2023-10-08T13:29:16.971 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2023-10-08T13:29:16.991 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2023-10-08T13:29:16.991 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi057.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive'} 2023-10-08T13:29:16.991 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T13:29:17.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T13:29:17.021 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T13:29:17.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T13:29:17.048 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T13:29:17.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T13:29:17.075 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-08T13:29:17.075 DEBUG:teuthology.orchestra.run.smithi057:> sudo /usr/sbin/setenforce permissive 2023-10-08T13:29:17.103 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2023-10-08T13:29:17.129 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2023-10-08T13:29:17.158 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-08T13:29:17.168 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-08T13:29:17.173 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-08T13:29:17.284 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-08T13:29:17.299 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-08T13:29:17.300 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi057.front.sepia.ceph.com,smithi099.front.sepia.ceph.com,smithi115.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-08T13:38:30.085 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi057.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com')] 2023-10-08T13:38:30.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-08T13:38:30.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T13:38:30.157 DEBUG:teuthology.orchestra.run.smithi057:> true 2023-10-08T13:38:30.229 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-08T13:38:30.229 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-08T13:38:30.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T13:38:30.297 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-10-08T13:38:30.363 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-08T13:38:30.363 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-10-08T13:38:30.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T13:38:30.433 DEBUG:teuthology.orchestra.run.smithi115:> true 2023-10-08T13:38:30.505 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-10-08T13:38:30.506 INFO:teuthology.run_tasks:Running task clock... 2023-10-08T13:38:30.518 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-08T13:38:30.519 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T13:38:30.519 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T13:38:30.521 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T13:38:30.522 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T13:38:30.523 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T13:38:30.524 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T13:38:30.552 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T13:38:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T13:38:30.566 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T13:38:30.570 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T13:38:30.580 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T13:38:30.596 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T13:38:30.632 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2023-10-08T13:38:30.644 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2023-10-08T13:38:30.651 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: ntpd: command not found 2023-10-08T13:38:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T13:38:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2023-10-08T13:38:30.664 INFO:teuthology.orchestra.run.smithi057.stdout:506 Cannot talk to daemon 2023-10-08T13:38:30.672 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T13:38:30.673 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2023-10-08T13:38:30.679 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T13:38:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T13:38:30.695 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T13:38:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T13:38:30.746 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2023-10-08T13:38:30.749 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T13:38:30.750 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-10-08T13:38:30.750 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.750 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.750 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.750 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.769 INFO:teuthology.orchestra.run.smithi115.stderr:bash: ntpq: command not found 2023-10-08T13:38:30.773 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T13:38:30.773 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2023-10-08T13:38:30.773 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.773 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.773 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.774 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.774 INFO:teuthology.orchestra.run.smithi057.stderr:bash: ntpq: command not found 2023-10-08T13:38:30.777 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T13:38:30.777 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2023-10-08T13:38:30.777 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.777 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.778 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.778 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T13:38:30.779 INFO:teuthology.run_tasks:Running task pexec... 2023-10-08T13:38:30.788 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-08T13:38:30.788 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T13:38:30.788 DEBUG:teuthology.orchestra.run.smithi057:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-08T13:38:30.789 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T13:38:30.789 DEBUG:teuthology.orchestra.run.smithi099:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-08T13:38:30.789 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi115.front.sepia.ceph.com 2023-10-08T13:38:30.789 DEBUG:teuthology.orchestra.run.smithi115:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-08T13:38:31.165 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:31.188 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:31.210 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:32.356 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:40 ago on Sun 08 Oct 2023 01:36:51 PM UTC. 2023-10-08T13:38:32.431 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:32 ago on Sun 08 Oct 2023 01:36:59 PM UTC. 2023-10-08T13:38:32.432 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:43 ago on Sun 08 Oct 2023 01:36:48 PM UTC. 2023-10-08T13:38:34.117 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2023-10-08T13:38:34.118 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2023-10-08T13:38:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2023-10-08T13:38:34.119 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2023-10-08T13:38:34.119 INFO:teuthology.orchestra.run.smithi115.stdout:Resetting modules: 2023-10-08T13:38:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: container-tools 2023-10-08T13:38:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:38:34.119 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2023-10-08T13:38:34.119 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2023-10-08T13:38:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:38:34.162 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-10-08T13:38:34.163 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-10-08T13:38:34.163 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2023-10-08T13:38:34.163 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-10-08T13:38:34.163 INFO:teuthology.orchestra.run.smithi099.stdout:Resetting modules: 2023-10-08T13:38:34.164 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools 2023-10-08T13:38:34.164 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:38:34.164 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-10-08T13:38:34.164 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-10-08T13:38:34.164 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:38:34.393 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-08T13:38:34.394 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-08T13:38:34.394 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2023-10-08T13:38:34.394 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-08T13:38:34.394 INFO:teuthology.orchestra.run.smithi057.stdout:Resetting modules: 2023-10-08T13:38:34.395 INFO:teuthology.orchestra.run.smithi057.stdout: container-tools 2023-10-08T13:38:34.395 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:38:34.395 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-10-08T13:38:34.395 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2023-10-08T13:38:34.395 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:38:35.127 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:46 ago on Sun 08 Oct 2023 01:36:48 PM UTC. 2023-10-08T13:38:35.138 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:43 ago on Sun 08 Oct 2023 01:36:51 PM UTC. 2023-10-08T13:38:35.412 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:35 ago on Sun 08 Oct 2023 01:36:59 PM UTC. 2023-10-08T13:38:38.372 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2023-10-08T13:38:38.837 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:38.940 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-10-08T13:38:38.958 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-08T13:38:39.412 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:39.427 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:39.730 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:48 ago on Sun 08 Oct 2023 01:36:51 PM UTC. 2023-10-08T13:38:40.369 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:51 ago on Sun 08 Oct 2023 01:36:48 PM UTC. 2023-10-08T13:38:40.410 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:40 ago on Sun 08 Oct 2023 01:36:59 PM UTC. 2023-10-08T13:38:41.449 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2023-10-08T13:38:41.542 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================================ 2023-10-08T13:38:41.543 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2023-10-08T13:38:41.543 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================================ 2023-10-08T13:38:41.543 INFO:teuthology.orchestra.run.smithi115.stdout:Installing group/module packages: 2023-10-08T13:38:41.543 INFO:teuthology.orchestra.run.smithi115.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-08T13:38:41.543 INFO:teuthology.orchestra.run.smithi115.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-08T13:38:41.543 INFO:teuthology.orchestra.run.smithi115.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-08T13:38:41.543 INFO:teuthology.orchestra.run.smithi115.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-08T13:38:41.543 INFO:teuthology.orchestra.run.smithi115.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-08T13:38:41.543 INFO:teuthology.orchestra.run.smithi115.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-08T13:38:41.543 INFO:teuthology.orchestra.run.smithi115.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:Installing module profiles: 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout: container-tools/common 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling module streams: 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout: container-tools rhel8 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2023-10-08T13:38:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================================ 2023-10-08T13:38:41.545 INFO:teuthology.orchestra.run.smithi115.stdout:Install 10 Packages 2023-10-08T13:38:41.545 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:38:41.545 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 18 M 2023-10-08T13:38:41.545 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 65 M 2023-10-08T13:38:41.545 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2023-10-08T13:38:41.633 INFO:teuthology.orchestra.run.smithi115.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.0 MB/s | 94 kB 00:00 2023-10-08T13:38:41.658 INFO:teuthology.orchestra.run.smithi115.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 219 kB/s | 25 kB 00:00 2023-10-08T13:38:41.683 INFO:teuthology.orchestra.run.smithi115.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 295 kB/s | 41 kB 00:00 2023-10-08T13:38:41.709 INFO:teuthology.orchestra.run.smithi115.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-10-08T13:38:41.742 INFO:teuthology.orchestra.run.smithi115.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-08T13:38:41.768 INFO:teuthology.orchestra.run.smithi115.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-08T13:38:41.868 INFO:teuthology.orchestra.run.smithi115.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.7 MB/s | 213 kB 00:00 2023-10-08T13:38:42.018 INFO:teuthology.orchestra.run.smithi115.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 24 MB/s | 8.0 MB 00:00 2023-10-08T13:38:42.110 INFO:teuthology.orchestra.run.smithi115.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 21 MB/s | 7.0 MB 00:00 2023-10-08T13:38:42.168 INFO:teuthology.orchestra.run.smithi115.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 7.3 MB/s | 2.2 MB 00:00 2023-10-08T13:38:42.169 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2023-10-08T13:38:42.169 INFO:teuthology.orchestra.run.smithi115.stdout:Total 29 MB/s | 18 MB 00:00 2023-10-08T13:38:42.213 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-10-08T13:38:42.309 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================================ 2023-10-08T13:38:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-10-08T13:38:42.309 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================================ 2023-10-08T13:38:42.309 INFO:teuthology.orchestra.run.smithi099.stdout:Installing group/module packages: 2023-10-08T13:38:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-08T13:38:42.310 INFO:teuthology.orchestra.run.smithi099.stdout:Installing module profiles: 2023-10-08T13:38:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools/common 2023-10-08T13:38:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:38:42.311 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling module streams: 2023-10-08T13:38:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools rhel8 2023-10-08T13:38:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:38:42.311 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-10-08T13:38:42.311 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================================ 2023-10-08T13:38:42.311 INFO:teuthology.orchestra.run.smithi099.stdout:Install 10 Packages 2023-10-08T13:38:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:38:42.312 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 18 M 2023-10-08T13:38:42.312 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 65 M 2023-10-08T13:38:42.312 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-10-08T13:38:42.369 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2023-10-08T13:38:42.375 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-08T13:38:42.388 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2023-10-08T13:38:42.388 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2023-10-08T13:38:42.464 INFO:teuthology.orchestra.run.smithi099.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 162 kB/s | 25 kB 00:00 2023-10-08T13:38:42.475 INFO:teuthology.orchestra.run.smithi057.stdout:============================================================================================================ 2023-10-08T13:38:42.475 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2023-10-08T13:38:42.475 INFO:teuthology.orchestra.run.smithi057.stdout:============================================================================================================ 2023-10-08T13:38:42.475 INFO:teuthology.orchestra.run.smithi057.stdout:Installing group/module packages: 2023-10-08T13:38:42.475 INFO:teuthology.orchestra.run.smithi057.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-08T13:38:42.475 INFO:teuthology.orchestra.run.smithi057.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-08T13:38:42.475 INFO:teuthology.orchestra.run.smithi057.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout:Installing module profiles: 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout: container-tools/common 2023-10-08T13:38:42.476 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:38:42.477 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling module streams: 2023-10-08T13:38:42.477 INFO:teuthology.orchestra.run.smithi057.stdout: container-tools rhel8 2023-10-08T13:38:42.477 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:38:42.477 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-10-08T13:38:42.477 INFO:teuthology.orchestra.run.smithi057.stdout:============================================================================================================ 2023-10-08T13:38:42.477 INFO:teuthology.orchestra.run.smithi057.stdout:Install 10 Packages 2023-10-08T13:38:42.477 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:38:42.478 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 18 M 2023-10-08T13:38:42.478 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 65 M 2023-10-08T13:38:42.478 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2023-10-08T13:38:42.481 INFO:teuthology.orchestra.run.smithi099.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 240 kB/s | 41 kB 00:00 2023-10-08T13:38:42.540 INFO:teuthology.orchestra.run.smithi099.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 411 kB/s | 94 kB 00:00 2023-10-08T13:38:42.569 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2023-10-08T13:38:42.574 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2023-10-08T13:38:42.634 INFO:teuthology.orchestra.run.smithi057.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 261 kB/s | 41 kB 00:00 2023-10-08T13:38:42.649 INFO:teuthology.orchestra.run.smithi099.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.1 MB/s | 524 kB 00:00 2023-10-08T13:38:42.650 INFO:teuthology.orchestra.run.smithi057.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 144 kB/s | 25 kB 00:00 2023-10-08T13:38:42.666 INFO:teuthology.orchestra.run.smithi099.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 245 kB/s | 49 kB 00:00 2023-10-08T13:38:42.675 INFO:teuthology.orchestra.run.smithi057.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 474 kB/s | 94 kB 00:00 2023-10-08T13:38:42.826 INFO:teuthology.orchestra.run.smithi057.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 257 kB/s | 49 kB 00:00 2023-10-08T13:38:42.841 INFO:teuthology.orchestra.run.smithi099.stdout:(6/10): buildah-1.26.2-1.module+el8.6.0+16771+2 26 MB/s | 8.0 MB 00:00 2023-10-08T13:38:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.5 MB/s | 524 kB 00:00 2023-10-08T13:38:42.866 INFO:teuthology.orchestra.run.smithi099.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 688 kB/s | 149 kB 00:00 2023-10-08T13:38:42.900 INFO:teuthology.orchestra.run.smithi099.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 910 kB/s | 213 kB 00:00 2023-10-08T13:38:42.952 INFO:teuthology.orchestra.run.smithi057.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.2 MB/s | 149 kB 00:00 2023-10-08T13:38:42.967 INFO:teuthology.orchestra.run.smithi099.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-10-08T13:38:43.045 INFO:teuthology.orchestra.run.smithi057.stdout:(7/10): buildah-1.26.2-1.module+el8.6.0+16771+2 22 MB/s | 8.0 MB 00:00 2023-10-08T13:38:43.056 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2023-10-08T13:38:43.069 INFO:teuthology.orchestra.run.smithi057.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.0 MB/s | 213 kB 00:00 2023-10-08T13:38:43.084 INFO:teuthology.orchestra.run.smithi099.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 29 MB/s | 7.0 MB 00:00 2023-10-08T13:38:43.084 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-10-08T13:38:43.084 INFO:teuthology.orchestra.run.smithi099.stdout:Total 24 MB/s | 18 MB 00:00 2023-10-08T13:38:43.211 INFO:teuthology.orchestra.run.smithi057.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 27 MB/s | 7.0 MB 00:00 2023-10-08T13:38:43.228 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-08T13:38:43.277 INFO:teuthology.orchestra.run.smithi057.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 9.4 MB/s | 2.2 MB 00:00 2023-10-08T13:38:43.278 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2023-10-08T13:38:43.278 INFO:teuthology.orchestra.run.smithi057.stdout:Total 23 MB/s | 18 MB 00:00 2023-10-08T13:38:43.285 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-10-08T13:38:43.304 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-10-08T13:38:43.304 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-10-08T13:38:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T13:38:43.483 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2023-10-08T13:38:43.484 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-10-08T13:38:43.489 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-10-08T13:38:43.503 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2023-10-08T13:38:43.503 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2023-10-08T13:38:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-08T13:38:43.697 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2023-10-08T13:38:43.701 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2023-10-08T13:38:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-08T13:38:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-10-08T13:38:44.190 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-08T13:38:44.225 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2023-10-08T13:38:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-08T13:38:44.431 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-08T13:38:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T13:38:44.600 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T13:38:44.705 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-08T13:38:44.805 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-08T13:38:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-08T13:38:44.979 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-08T13:38:45.011 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-08T13:38:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-08T13:38:45.398 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-08T13:38:45.827 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-08T13:38:46.062 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-08T13:38:46.186 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-08T13:38:46.193 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-08T13:38:46.219 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-08T13:38:46.438 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T13:38:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T13:38:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-08T13:38:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T13:38:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-08T13:38:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-08T13:38:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-08T13:38:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-08T13:38:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-08T13:38:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-08T13:38:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-08T13:38:47.095 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-08T13:38:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-08T13:38:47.347 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-08T13:38:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-08T13:38:47.515 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-08T13:38:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-08T13:38:47.699 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T13:38:47.845 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T13:38:47.905 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-08T13:38:47.905 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:56 ago on Sun 08 Oct 2023 01:36:51 PM UTC. 2023-10-08T13:38:47.980 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T13:38:47.980 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-08T13:38:47.980 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T13:38:47.980 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-08T13:38:47.980 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-08T13:38:47.980 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-08T13:38:47.981 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-08T13:38:47.981 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-08T13:38:47.981 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-08T13:38:47.981 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-08T13:38:48.101 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T13:38:48.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-08T13:38:48.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T13:38:48.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-08T13:38:48.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-08T13:38:48.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-08T13:38:48.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-08T13:38:48.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-08T13:38:48.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-08T13:38:48.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-08T13:38:49.145 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-08T13:38:49.145 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:49 ago on Sun 08 Oct 2023 01:36:59 PM UTC. 2023-10-08T13:38:49.214 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-08T13:38:49.214 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:00 ago on Sun 08 Oct 2023 01:36:48 PM UTC. 2023-10-08T13:38:50.997 INFO:teuthology.orchestra.run.smithi115.stdout:Installed products updated. 2023-10-08T13:38:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:38:51.184 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2023-10-08T13:38:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T13:38:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-08T13:38:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-08T13:38:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-08T13:38:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T13:38:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-08T13:38:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:38:51.185 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2023-10-08T13:38:51.330 DEBUG:teuthology.parallel:result is None 2023-10-08T13:38:52.516 INFO:teuthology.orchestra.run.smithi057.stdout:Installed products updated. 2023-10-08T13:38:52.538 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2023-10-08T13:38:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:38:52.733 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-10-08T13:38:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T13:38:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-08T13:38:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-08T13:38:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-08T13:38:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T13:38:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-08T13:38:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:38:52.734 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-10-08T13:38:52.836 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:38:52.836 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2023-10-08T13:38:52.836 INFO:teuthology.orchestra.run.smithi057.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:38:52.837 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-08T13:38:52.896 DEBUG:teuthology.parallel:result is None 2023-10-08T13:38:53.081 DEBUG:teuthology.parallel:result is None 2023-10-08T13:38:53.081 INFO:teuthology.run_tasks:Running task install... 2023-10-08T13:38:53.090 DEBUG:teuthology.task.install:project ceph 2023-10-08T13:38:53.091 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de'}} 2023-10-08T13:38:53.091 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de'} 2023-10-08T13:38:53.091 INFO:teuthology.task.install:Using flavor: default 2023-10-08T13:38:53.097 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-08T13:38:53.097 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-08T13:38:53.097 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'tag': None, 'wait_for_package': False} 2023-10-08T13:38:53.098 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:38:53.100 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'tag': None, 'wait_for_package': False} 2023-10-08T13:38:53.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:38:53.101 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'tag': None, 'wait_for_package': False} 2023-10-08T13:38:53.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:38:53.244 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/ 2023-10-08T13:38:53.245 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6639.g7b3143ba 2023-10-08T13:38:53.318 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/ 2023-10-08T13:38:53.318 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6639.g7b3143ba 2023-10-08T13:38:53.326 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/ 2023-10-08T13:38:53.326 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6639.g7b3143ba 2023-10-08T13:38:53.424 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T13:38:53.424 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:38:53.424 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T13:38:53.453 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T13:38:53.453 DEBUG:teuthology.orchestra.run.smithi057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T13:38:53.483 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T13:38:53.484 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:38:53.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T13:38:53.498 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T13:38:53.498 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:38:53.498 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T13:38:53.513 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T13:38:53.514 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T13:38:53.522 DEBUG:teuthology.orchestra.run.smithi057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T13:38:53.531 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T13:38:53.531 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T13:38:53.582 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T13:38:53.603 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T13:38:53.603 DEBUG:teuthology.orchestra.run.smithi057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T13:38:53.661 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T13:38:53.670 INFO:teuthology.orchestra.run.smithi057.stdout:check_obsoletes = 1 2023-10-08T13:38:53.672 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum clean all 2023-10-08T13:38:53.684 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T13:38:53.727 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2023-10-08T13:38:53.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2023-10-08T13:38:53.752 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2023-10-08T13:38:53.753 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2023-10-08T13:38:54.064 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:54.104 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:54.127 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:54.440 INFO:teuthology.orchestra.run.smithi099.stdout:44 files removed 2023-10-08T13:38:54.444 INFO:teuthology.orchestra.run.smithi057.stdout:44 files removed 2023-10-08T13:38:54.487 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T13:38:54.494 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T13:38:54.533 INFO:teuthology.orchestra.run.smithi115.stdout:44 files removed 2023-10-08T13:38:54.579 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T13:38:54.816 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:54.845 INFO:teuthology.orchestra.run.smithi057.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:54.908 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2023-10-08T13:38:55.628 INFO:teuthology.orchestra.run.smithi057.stdout:ceph packages for x86_64 314 kB/s | 81 kB 00:00 2023-10-08T13:38:55.629 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 302 kB/s | 81 kB 00:00 2023-10-08T13:38:55.694 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 262 kB/s | 81 kB 00:00 2023-10-08T13:38:55.929 INFO:teuthology.orchestra.run.smithi057.stdout:ceph noarch packages 35 kB/s | 9.4 kB 00:00 2023-10-08T13:38:55.971 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-10-08T13:38:56.037 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-10-08T13:38:56.270 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 7.0 kB/s | 1.5 kB 00:00 2023-10-08T13:38:56.280 INFO:teuthology.orchestra.run.smithi057.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-10-08T13:38:56.280 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 5.2 kB/s | 1.5 kB 00:00 2023-10-08T13:38:56.413 INFO:teuthology.orchestra.run.smithi115.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-10-08T13:38:56.423 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-10-08T13:38:56.430 INFO:teuthology.orchestra.run.smithi057.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-10-08T13:38:58.130 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 16 MB 00:01 2023-10-08T13:38:58.418 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 8.0 MB/s | 16 MB 00:01 2023-10-08T13:39:01.142 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 16 MB 00:04 2023-10-08T13:39:01.630 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 399 kB/s | 24 kB 00:00 2023-10-08T13:39:01.899 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 420 kB/s | 24 kB 00:00 2023-10-08T13:39:02.526 INFO:teuthology.orchestra.run.smithi115.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-10-08T13:39:02.884 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-10-08T13:39:04.856 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 415 kB/s | 24 kB 00:00 2023-10-08T13:39:06.224 INFO:teuthology.orchestra.run.smithi057.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 MB/s | 53 MB 00:01 2023-10-08T13:39:10.576 INFO:teuthology.orchestra.run.smithi115.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-10-08T13:39:11.359 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 MB/s | 47 MB 00:01 2023-10-08T13:39:15.183 INFO:teuthology.orchestra.run.smithi057.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 MB/s | 47 MB 00:01 2023-10-08T13:39:21.592 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T13:39:21.593 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T13:39:21.608 INFO:teuthology.orchestra.run.smithi115.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-08T13:39:21.609 INFO:teuthology.orchestra.run.smithi115.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-08T13:39:21.610 INFO:teuthology.orchestra.run.smithi115.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-08T13:39:21.611 INFO:teuthology.orchestra.run.smithi115.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T13:39:21.613 INFO:teuthology.orchestra.run.smithi115.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T13:39:21.616 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-08T13:39:21.617 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-08T13:39:21.750 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2023-10-08T13:39:21.762 INFO:teuthology.orchestra.run.smithi115.stdout:=========================================================================================================================== 2023-10-08T13:39:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2023-10-08T13:39:21.763 INFO:teuthology.orchestra.run.smithi115.stdout:=========================================================================================================================== 2023-10-08T13:39:21.763 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2023-10-08T13:39:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-08T13:39:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T13:39:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.4 k 2023-10-08T13:39:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 5.4 M 2023-10-08T13:39:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 918 k 2023-10-08T13:39:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 167 k 2023-10-08T13:39:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 1.6 M 2023-10-08T13:39:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 132 k 2023-10-08T13:39:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 1.8 M 2023-10-08T13:39:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 7.4 M 2023-10-08T13:39:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 47 k 2023-10-08T13:39:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 14 M 2023-10-08T13:39:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 49 M 2023-10-08T13:39:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 261 k 2023-10-08T13:39:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 190 k 2023-10-08T13:39:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 261 k 2023-10-08T13:39:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 18 k 2023-10-08T13:39:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-08T13:39:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T13:39:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T13:39:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 31 k 2023-10-08T13:39:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 840 k 2023-10-08T13:39:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 139 k 2023-10-08T13:39:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T13:39:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-08T13:39:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T13:39:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-08T13:39:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-08T13:39:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-08T13:39:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 197 k 2023-10-08T13:39:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 371 k 2023-10-08T13:39:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 365 k 2023-10-08T13:39:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 107 k 2023-10-08T13:39:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 94 k 2023-10-08T13:39:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.6 M 2023-10-08T13:39:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 179 k 2023-10-08T13:39:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-08T13:39:21.767 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2023-10-08T13:39:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-08T13:39:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T13:39:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T13:39:21.768 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2023-10-08T13:39:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T13:39:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 23 M 2023-10-08T13:39:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 23 k 2023-10-08T13:39:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 2.3 M 2023-10-08T13:39:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 242 k 2023-10-08T13:39:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 4.6 M 2023-10-08T13:39:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 18 M 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 15 k 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 24 k 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 184 k 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-08T13:39:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 518 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.9 M 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T13:39:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T13:39:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T13:39:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-08T13:39:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T13:39:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T13:39:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T13:39:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T13:39:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T13:39:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 45 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 125 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T13:39:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T13:39:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T13:39:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T13:39:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T13:39:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T13:39:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T13:39:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T13:39:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T13:39:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T13:39:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T13:39:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T13:39:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T13:39:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T13:39:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T13:39:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T13:39:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T13:39:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T13:39:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T13:39:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-08T13:39:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T13:39:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T13:39:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T13:39:21.776 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling module streams: 2023-10-08T13:39:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql 10 2023-10-08T13:39:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:39:21.777 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2023-10-08T13:39:21.777 INFO:teuthology.orchestra.run.smithi115.stdout:=========================================================================================================================== 2023-10-08T13:39:21.777 INFO:teuthology.orchestra.run.smithi115.stdout:Install 128 Packages 2023-10-08T13:39:21.777 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 3 Packages 2023-10-08T13:39:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:39:21.777 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 209 M 2023-10-08T13:39:21.778 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2023-10-08T13:39:22.040 INFO:teuthology.orchestra.run.smithi115.stdout:(1/131): ceph-18.0.0-6639.g7b3143ba.el8.x86_64. 55 kB/s | 6.4 kB 00:00 2023-10-08T13:39:22.224 INFO:teuthology.orchestra.run.smithi115.stdout:(2/131): ceph-fuse-18.0.0-6639.g7b3143ba.el8.x8 4.9 MB/s | 918 kB 00:00 2023-10-08T13:39:22.257 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T13:39:22.258 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T13:39:22.272 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-08T13:39:22.273 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-08T13:39:22.274 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-08T13:39:22.274 INFO:teuthology.orchestra.run.smithi115.stdout:(3/131): ceph-immutable-object-cache-18.0.0-663 3.3 MB/s | 167 kB 00:00 2023-10-08T13:39:22.275 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T13:39:22.277 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T13:39:22.280 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-08T13:39:22.281 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-08T13:39:22.415 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-10-08T13:39:22.428 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2023-10-08T13:39:22.428 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-10-08T13:39:22.428 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2023-10-08T13:39:22.428 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-10-08T13:39:22.428 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-08T13:39:22.428 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T13:39:22.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.4 k 2023-10-08T13:39:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 5.4 M 2023-10-08T13:39:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 918 k 2023-10-08T13:39:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 167 k 2023-10-08T13:39:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 1.6 M 2023-10-08T13:39:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 132 k 2023-10-08T13:39:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 1.8 M 2023-10-08T13:39:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 7.4 M 2023-10-08T13:39:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 47 k 2023-10-08T13:39:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 14 M 2023-10-08T13:39:22.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 49 M 2023-10-08T13:39:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 261 k 2023-10-08T13:39:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 190 k 2023-10-08T13:39:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 261 k 2023-10-08T13:39:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 18 k 2023-10-08T13:39:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-08T13:39:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T13:39:22.430 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T13:39:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 31 k 2023-10-08T13:39:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 840 k 2023-10-08T13:39:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 139 k 2023-10-08T13:39:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T13:39:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-08T13:39:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T13:39:22.431 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-08T13:39:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-08T13:39:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-08T13:39:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 197 k 2023-10-08T13:39:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 371 k 2023-10-08T13:39:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 365 k 2023-10-08T13:39:22.432 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 107 k 2023-10-08T13:39:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 94 k 2023-10-08T13:39:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.6 M 2023-10-08T13:39:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 179 k 2023-10-08T13:39:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-08T13:39:22.433 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2023-10-08T13:39:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-08T13:39:22.433 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T13:39:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T13:39:22.434 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-10-08T13:39:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T13:39:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 23 M 2023-10-08T13:39:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 23 k 2023-10-08T13:39:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 2.3 M 2023-10-08T13:39:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 242 k 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 4.6 M 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 18 M 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 15 k 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 24 k 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 184 k 2023-10-08T13:39:22.435 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 518 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.9 M 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T13:39:22.436 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T13:39:22.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T13:39:22.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T13:39:22.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 45 k 2023-10-08T13:39:22.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 125 k 2023-10-08T13:39:22.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T13:39:22.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-08T13:39:22.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T13:39:22.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T13:39:22.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T13:39:22.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T13:39:22.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T13:39:22.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T13:39:22.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T13:39:22.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T13:39:22.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T13:39:22.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T13:39:22.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T13:39:22.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T13:39:22.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T13:39:22.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T13:39:22.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T13:39:22.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T13:39:22.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T13:39:22.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T13:39:22.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T13:39:22.441 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T13:39:22.441 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T13:39:22.441 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T13:39:22.441 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-08T13:39:22.441 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T13:39:22.441 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T13:39:22.441 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-08T13:39:22.442 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T13:39:22.442 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T13:39:22.442 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T13:39:22.442 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T13:39:22.442 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T13:39:22.442 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-08T13:39:22.442 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T13:39:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T13:39:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T13:39:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T13:39:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T13:39:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T13:39:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T13:39:22.443 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T13:39:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T13:39:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T13:39:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-08T13:39:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T13:39:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-08T13:39:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T13:39:22.444 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T13:39:22.444 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling module streams: 2023-10-08T13:39:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql 10 2023-10-08T13:39:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:39:22.445 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-10-08T13:39:22.445 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2023-10-08T13:39:22.445 INFO:teuthology.orchestra.run.smithi099.stdout:Install 128 Packages 2023-10-08T13:39:22.445 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 3 Packages 2023-10-08T13:39:22.445 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:39:22.446 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 209 M 2023-10-08T13:39:22.446 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-10-08T13:39:22.525 INFO:teuthology.orchestra.run.smithi115.stdout:(4/131): ceph-mds-18.0.0-6639.g7b3143ba.el8.x86 9.3 MB/s | 2.3 MB 00:00 2023-10-08T13:39:22.742 INFO:teuthology.orchestra.run.smithi115.stdout:(5/131): ceph-mgr-18.0.0-6639.g7b3143ba.el8.x86 7.5 MB/s | 1.6 MB 00:00 2023-10-08T13:39:22.793 INFO:teuthology.orchestra.run.smithi099.stdout:(1/131): ceph-18.0.0-6639.g7b3143ba.el8.x86_64. 33 kB/s | 6.4 kB 00:00 2023-10-08T13:39:22.926 INFO:teuthology.orchestra.run.smithi115.stdout:(6/131): ceph-base-18.0.0-6639.g7b3143ba.el8.x8 5.4 MB/s | 5.4 MB 00:01 2023-10-08T13:39:23.259 INFO:teuthology.orchestra.run.smithi115.stdout:(7/131): ceph-mon-18.0.0-6639.g7b3143ba.el8.x86 8.9 MB/s | 4.6 MB 00:00 2023-10-08T13:39:23.726 INFO:teuthology.orchestra.run.smithi099.stdout:(2/131): ceph-fuse-18.0.0-6639.g7b3143ba.el8.x8 984 kB/s | 918 kB 00:00 2023-10-08T13:39:23.912 INFO:teuthology.orchestra.run.smithi099.stdout:(3/131): ceph-immutable-object-cache-18.0.0-663 904 kB/s | 167 kB 00:00 2023-10-08T13:39:25.027 INFO:teuthology.orchestra.run.smithi099.stdout:(4/131): ceph-base-18.0.0-6639.g7b3143ba.el8.x8 2.2 MB/s | 5.4 MB 00:02 2023-10-08T13:39:25.085 INFO:teuthology.orchestra.run.smithi099.stdout:(5/131): ceph-mds-18.0.0-6639.g7b3143ba.el8.x86 2.0 MB/s | 2.3 MB 00:01 2023-10-08T13:39:25.125 INFO:teuthology.orchestra.run.smithi115.stdout:(8/131): ceph-common-18.0.0-6639.g7b3143ba.el8. 7.1 MB/s | 23 MB 00:03 2023-10-08T13:39:25.311 INFO:teuthology.orchestra.run.smithi099.stdout:(6/131): ceph-mgr-18.0.0-6639.g7b3143ba.el8.x86 5.7 MB/s | 1.6 MB 00:00 2023-10-08T13:39:25.326 INFO:teuthology.orchestra.run.smithi115.stdout:(9/131): ceph-osd-18.0.0-6639.g7b3143ba.el8.x86 7.5 MB/s | 18 MB 00:02 2023-10-08T13:39:25.351 INFO:teuthology.orchestra.run.smithi115.stdout:(10/131): ceph-selinux-18.0.0-6639.g7b3143ba.el 108 kB/s | 24 kB 00:00 2023-10-08T13:39:25.435 INFO:teuthology.orchestra.run.smithi115.stdout:(11/131): cephfs-mirror-18.0.0-6639.g7b3143ba.e 3.1 MB/s | 261 kB 00:00 2023-10-08T13:39:25.607 INFO:teuthology.orchestra.run.smithi115.stdout:(12/131): ceph-radosgw-18.0.0-6639.g7b3143ba.el 5.9 MB/s | 14 MB 00:02 2023-10-08T13:39:25.635 INFO:teuthology.orchestra.run.smithi115.stdout:(13/131): libcephfs-devel-18.0.0-6639.g7b3143ba 155 kB/s | 31 kB 00:00 2023-10-08T13:39:25.727 INFO:teuthology.orchestra.run.smithi115.stdout:(14/131): libcephsqlite-18.0.0-6639.g7b3143ba.e 2.0 MB/s | 184 kB 00:00 2023-10-08T13:39:25.752 INFO:teuthology.orchestra.run.smithi115.stdout:(15/131): libcephfs2-18.0.0-6639.g7b3143ba.el8. 5.7 MB/s | 840 kB 00:00 2023-10-08T13:39:25.803 INFO:teuthology.orchestra.run.smithi115.stdout:(16/131): librados-devel-18.0.0-6639.g7b3143ba. 1.8 MB/s | 139 kB 00:00 2023-10-08T13:39:25.861 INFO:teuthology.orchestra.run.smithi115.stdout:(17/131): libradosstriper1-18.0.0-6639.g7b3143b 4.7 MB/s | 518 kB 00:00 2023-10-08T13:39:25.920 INFO:teuthology.orchestra.run.smithi115.stdout:(18/131): python3-ceph-argparse-18.0.0-6639.g7b 770 kB/s | 45 kB 00:00 2023-10-08T13:39:25.987 INFO:teuthology.orchestra.run.smithi115.stdout:(19/131): python3-ceph-common-18.0.0-6639.g7b31 1.8 MB/s | 125 kB 00:00 2023-10-08T13:39:26.054 INFO:teuthology.orchestra.run.smithi115.stdout:(20/131): python3-cephfs-18.0.0-6639.g7b3143ba. 2.9 MB/s | 197 kB 00:00 2023-10-08T13:39:26.196 INFO:teuthology.orchestra.run.smithi115.stdout:(21/131): python3-rados-18.0.0-6639.g7b3143ba.e 2.6 MB/s | 371 kB 00:00 2023-10-08T13:39:26.269 INFO:teuthology.orchestra.run.smithi099.stdout:(7/131): ceph-mon-18.0.0-6639.g7b3143ba.el8.x86 3.9 MB/s | 4.6 MB 00:01 2023-10-08T13:39:26.296 INFO:teuthology.orchestra.run.smithi115.stdout:(22/131): python3-rbd-18.0.0-6639.g7b3143ba.el8 3.6 MB/s | 365 kB 00:00 2023-10-08T13:39:26.355 INFO:teuthology.orchestra.run.smithi115.stdout:(23/131): python3-rgw-18.0.0-6639.g7b3143ba.el8 1.8 MB/s | 107 kB 00:00 2023-10-08T13:39:26.430 INFO:teuthology.orchestra.run.smithi115.stdout:(24/131): rbd-fuse-18.0.0-6639.g7b3143ba.el8.x8 1.2 MB/s | 94 kB 00:00 2023-10-08T13:39:26.808 INFO:teuthology.orchestra.run.smithi057.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T13:39:26.809 INFO:teuthology.orchestra.run.smithi057.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T13:39:26.824 INFO:teuthology.orchestra.run.smithi057.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-08T13:39:26.825 INFO:teuthology.orchestra.run.smithi057.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-08T13:39:26.826 INFO:teuthology.orchestra.run.smithi057.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-08T13:39:26.827 INFO:teuthology.orchestra.run.smithi057.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T13:39:26.830 INFO:teuthology.orchestra.run.smithi057.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T13:39:26.833 INFO:teuthology.orchestra.run.smithi057.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-08T13:39:26.834 INFO:teuthology.orchestra.run.smithi057.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-08T13:39:26.980 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2023-10-08T13:39:26.993 INFO:teuthology.orchestra.run.smithi057.stdout:=========================================================================================================================== 2023-10-08T13:39:26.994 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2023-10-08T13:39:26.994 INFO:teuthology.orchestra.run.smithi057.stdout:=========================================================================================================================== 2023-10-08T13:39:26.994 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2023-10-08T13:39:26.994 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-08T13:39:26.994 INFO:teuthology.orchestra.run.smithi057.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T13:39:26.994 INFO:teuthology.orchestra.run.smithi057.stdout: ceph x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.4 k 2023-10-08T13:39:26.994 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 5.4 M 2023-10-08T13:39:26.994 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 918 k 2023-10-08T13:39:26.994 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 167 k 2023-10-08T13:39:26.994 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 1.6 M 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 132 k 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 1.8 M 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 7.4 M 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 47 k 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 14 M 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 49 M 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 261 k 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 190 k 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 261 k 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-top noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 18 k 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-08T13:39:26.995 INFO:teuthology.orchestra.run.smithi057.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 31 k 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 840 k 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 139 k 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 197 k 2023-10-08T13:39:26.996 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 371 k 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 365 k 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 107 k 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 94 k 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.6 M 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 179 k 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrading: 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout: librados2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T13:39:26.997 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 23 M 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 23 k 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 2.3 M 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 242 k 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 4.6 M 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 18 M 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 15 k 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 24 k 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T13:39:26.998 INFO:teuthology.orchestra.run.smithi057.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 184 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 518 k 2023-10-08T13:39:26.999 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.9 M 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T13:39:27.000 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 45 k 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 125 k 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T13:39:27.001 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T13:39:27.002 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-08T13:39:27.003 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T13:39:27.004 INFO:teuthology.orchestra.run.smithi057.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T13:39:27.005 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling module streams: 2023-10-08T13:39:27.006 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql 10 2023-10-08T13:39:27.006 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:39:27.006 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2023-10-08T13:39:27.006 INFO:teuthology.orchestra.run.smithi057.stdout:=========================================================================================================================== 2023-10-08T13:39:27.006 INFO:teuthology.orchestra.run.smithi057.stdout:Install 128 Packages 2023-10-08T13:39:27.006 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrade 3 Packages 2023-10-08T13:39:27.006 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:39:27.007 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 209 M 2023-10-08T13:39:27.007 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2023-10-08T13:39:27.139 INFO:teuthology.orchestra.run.smithi115.stdout:(25/131): rbd-mirror-18.0.0-6639.g7b3143ba.el8. 5.1 MB/s | 3.6 MB 00:00 2023-10-08T13:39:27.222 INFO:teuthology.orchestra.run.smithi115.stdout:(26/131): librgw2-18.0.0-6639.g7b3143ba.el8.x86 4.9 MB/s | 6.9 MB 00:01 2023-10-08T13:39:27.247 INFO:teuthology.orchestra.run.smithi115.stdout:(27/131): rbd-nbd-18.0.0-6639.g7b3143ba.el8.x86 1.6 MB/s | 179 kB 00:00 2023-10-08T13:39:27.289 INFO:teuthology.orchestra.run.smithi115.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6639.g 346 kB/s | 23 kB 00:00 2023-10-08T13:39:27.331 INFO:teuthology.orchestra.run.smithi115.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6639.g7b3143b 1.5 MB/s | 132 kB 00:00 2023-10-08T13:39:27.400 INFO:teuthology.orchestra.run.smithi057.stdout:(1/131): ceph-18.0.0-6639.g7b3143ba.el8.x86_64. 27 kB/s | 6.4 kB 00:00 2023-10-08T13:39:27.707 INFO:teuthology.orchestra.run.smithi115.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6639.g7b314 4.2 MB/s | 1.8 MB 00:00 2023-10-08T13:39:27.824 INFO:teuthology.orchestra.run.smithi115.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6639.g7b 2.0 MB/s | 242 kB 00:00 2023-10-08T13:39:27.916 INFO:teuthology.orchestra.run.smithi115.stdout:(32/131): ceph-mgr-rook-18.0.0-6639.g7b3143ba.e 519 kB/s | 47 kB 00:00 2023-10-08T13:39:28.041 INFO:teuthology.orchestra.run.smithi115.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6639.g7 116 kB/s | 15 kB 00:00 2023-10-08T13:39:28.150 INFO:teuthology.orchestra.run.smithi115.stdout:(34/131): ceph-volume-18.0.0-6639.g7b3143ba.el8 2.4 MB/s | 261 kB 00:00 2023-10-08T13:39:28.275 INFO:teuthology.orchestra.run.smithi115.stdout:(35/131): cephadm-18.0.0-6639.g7b3143ba.el8.noa 1.5 MB/s | 190 kB 00:00 2023-10-08T13:39:28.383 INFO:teuthology.orchestra.run.smithi115.stdout:(36/131): cephfs-top-18.0.0-6639.g7b3143ba.el8. 167 kB/s | 18 kB 00:00 2023-10-08T13:39:28.542 INFO:teuthology.orchestra.run.smithi115.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.9 MB/s | 306 kB 00:00 2023-10-08T13:39:28.576 INFO:teuthology.orchestra.run.smithi115.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2023-10-08T13:39:28.626 INFO:teuthology.orchestra.run.smithi115.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2023-10-08T13:39:28.684 INFO:teuthology.orchestra.run.smithi057.stdout:(2/131): ceph-fuse-18.0.0-6639.g7b3143ba.el8.x8 715 kB/s | 918 kB 00:01 2023-10-08T13:39:28.743 INFO:teuthology.orchestra.run.smithi115.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 5.3 MB/s | 7.4 MB 00:01 2023-10-08T13:39:28.769 INFO:teuthology.orchestra.run.smithi115.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2023-10-08T13:39:28.827 INFO:teuthology.orchestra.run.smithi115.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2023-10-08T13:39:28.842 INFO:teuthology.orchestra.run.smithi057.stdout:(3/131): ceph-immutable-object-cache-18.0.0-663 1.0 MB/s | 167 kB 00:00 2023-10-08T13:39:28.844 INFO:teuthology.orchestra.run.smithi115.stdout:(43/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 434 kB/s | 44 kB 00:00 2023-10-08T13:39:28.861 INFO:teuthology.orchestra.run.smithi115.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 987 kB/s | 33 kB 00:00 2023-10-08T13:39:28.886 INFO:teuthology.orchestra.run.smithi115.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 389 kB/s | 16 kB 00:00 2023-10-08T13:39:28.920 INFO:teuthology.orchestra.run.smithi115.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-10-08T13:39:28.970 INFO:teuthology.orchestra.run.smithi115.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-10-08T13:39:29.012 INFO:teuthology.orchestra.run.smithi115.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 267 kB/s | 11 kB 00:00 2023-10-08T13:39:29.062 INFO:teuthology.orchestra.run.smithi115.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-10-08T13:39:29.088 INFO:teuthology.orchestra.run.smithi115.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2023-10-08T13:39:29.130 INFO:teuthology.orchestra.run.smithi115.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-10-08T13:39:29.165 INFO:teuthology.orchestra.run.smithi115.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 12 MB/s | 1.2 MB 00:00 2023-10-08T13:39:29.189 INFO:teuthology.orchestra.run.smithi115.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-10-08T13:39:29.214 INFO:teuthology.orchestra.run.smithi115.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-10-08T13:39:29.247 INFO:teuthology.orchestra.run.smithi115.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-10-08T13:39:29.269 INFO:teuthology.orchestra.run.smithi099.stdout:(8/131): ceph-osd-18.0.0-6639.g7b3143ba.el8.x86 4.5 MB/s | 18 MB 00:03 2023-10-08T13:39:29.273 INFO:teuthology.orchestra.run.smithi115.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2023-10-08T13:39:29.298 INFO:teuthology.orchestra.run.smithi115.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 667 kB/s | 34 kB 00:00 2023-10-08T13:39:29.324 INFO:teuthology.orchestra.run.smithi115.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-10-08T13:39:29.349 INFO:teuthology.orchestra.run.smithi115.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2023-10-08T13:39:29.361 INFO:teuthology.orchestra.run.smithi099.stdout:(9/131): ceph-selinux-18.0.0-6639.g7b3143ba.el8 266 kB/s | 24 kB 00:00 2023-10-08T13:39:29.374 INFO:teuthology.orchestra.run.smithi115.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-10-08T13:39:29.400 INFO:teuthology.orchestra.run.smithi115.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 476 kB/s | 24 kB 00:00 2023-10-08T13:39:29.425 INFO:teuthology.orchestra.run.smithi115.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 584 kB/s | 29 kB 00:00 2023-10-08T13:39:29.450 INFO:teuthology.orchestra.run.smithi115.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 550 kB/s | 27 kB 00:00 2023-10-08T13:39:29.492 INFO:teuthology.orchestra.run.smithi115.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-10-08T13:39:29.517 INFO:teuthology.orchestra.run.smithi115.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2023-10-08T13:39:29.543 INFO:teuthology.orchestra.run.smithi115.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-10-08T13:39:29.559 INFO:teuthology.orchestra.run.smithi115.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2023-10-08T13:39:29.576 INFO:teuthology.orchestra.run.smithi115.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 684 kB/s | 23 kB 00:00 2023-10-08T13:39:29.592 INFO:teuthology.orchestra.run.smithi057.stdout:(4/131): ceph-base-18.0.0-6639.g7b3143ba.el8.x8 2.2 MB/s | 5.4 MB 00:02 2023-10-08T13:39:29.632 INFO:teuthology.orchestra.run.smithi115.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-10-08T13:39:29.727 INFO:teuthology.orchestra.run.smithi115.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 10 MB/s | 1.7 MB 00:00 2023-10-08T13:39:29.794 INFO:teuthology.orchestra.run.smithi115.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-10-08T13:39:29.828 INFO:teuthology.orchestra.run.smithi115.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-10-08T13:39:29.870 INFO:teuthology.orchestra.run.smithi115.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-10-08T13:39:29.920 INFO:teuthology.orchestra.run.smithi115.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-10-08T13:39:29.951 INFO:teuthology.orchestra.run.smithi057.stdout:(5/131): ceph-mds-18.0.0-6639.g7b3143ba.el8.x86 2.1 MB/s | 2.3 MB 00:01 2023-10-08T13:39:29.962 INFO:teuthology.orchestra.run.smithi115.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 933 kB/s | 39 kB 00:00 2023-10-08T13:39:29.985 INFO:teuthology.orchestra.run.smithi057.stdout:(6/131): ceph-mgr-18.0.0-6639.g7b3143ba.el8.x86 4.1 MB/s | 1.6 MB 00:00 2023-10-08T13:39:30.045 INFO:teuthology.orchestra.run.smithi099.stdout:(10/131): ceph-common-18.0.0-6639.g7b3143ba.el8 3.0 MB/s | 23 MB 00:07 2023-10-08T13:39:30.121 INFO:teuthology.orchestra.run.smithi115.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 28 MB/s | 14 MB 00:00 2023-10-08T13:39:30.154 INFO:teuthology.orchestra.run.smithi115.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 3.5 MB/s | 688 kB 00:00 2023-10-08T13:39:30.210 INFO:teuthology.orchestra.run.smithi115.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 6.0 MB/s | 538 kB 00:00 2023-10-08T13:39:30.238 INFO:teuthology.orchestra.run.smithi115.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.8 MB/s | 157 kB 00:00 2023-10-08T13:39:30.240 INFO:teuthology.orchestra.run.smithi099.stdout:(11/131): cephfs-mirror-18.0.0-6639.g7b3143ba.e 1.3 MB/s | 261 kB 00:00 2023-10-08T13:39:30.263 INFO:teuthology.orchestra.run.smithi115.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.4 MB/s | 187 kB 00:00 2023-10-08T13:39:30.297 INFO:teuthology.orchestra.run.smithi115.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-10-08T13:39:30.337 INFO:teuthology.orchestra.run.smithi099.stdout:(12/131): libcephfs-devel-18.0.0-6639.g7b3143ba 322 kB/s | 31 kB 00:00 2023-10-08T13:39:30.339 INFO:teuthology.orchestra.run.smithi115.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 639 kB/s | 27 kB 00:00 2023-10-08T13:39:30.390 INFO:teuthology.orchestra.run.smithi115.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-10-08T13:39:30.456 INFO:teuthology.orchestra.run.smithi115.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-10-08T13:39:30.482 INFO:teuthology.orchestra.run.smithi115.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 477 kB/s | 44 kB 00:00 2023-10-08T13:39:30.532 INFO:teuthology.orchestra.run.smithi115.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-10-08T13:39:30.607 INFO:teuthology.orchestra.run.smithi115.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 25 MB/s | 3.7 MB 00:00 2023-10-08T13:39:30.621 INFO:teuthology.orchestra.run.smithi099.stdout:(13/131): libcephfs2-18.0.0-6639.g7b3143ba.el8. 2.9 MB/s | 840 kB 00:00 2023-10-08T13:39:30.633 INFO:teuthology.orchestra.run.smithi115.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-10-08T13:39:30.658 INFO:teuthology.orchestra.run.smithi115.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-08T13:39:30.683 INFO:teuthology.orchestra.run.smithi115.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-10-08T13:39:30.709 INFO:teuthology.orchestra.run.smithi115.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 396 kB/s | 20 kB 00:00 2023-10-08T13:39:30.712 INFO:teuthology.orchestra.run.smithi099.stdout:(14/131): libcephsqlite-18.0.0-6639.g7b3143ba.e 2.0 MB/s | 184 kB 00:00 2023-10-08T13:39:30.734 INFO:teuthology.orchestra.run.smithi115.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-10-08T13:39:30.801 INFO:teuthology.orchestra.run.smithi115.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2023-10-08T13:39:30.804 INFO:teuthology.orchestra.run.smithi099.stdout:(15/131): librados-devel-18.0.0-6639.g7b3143ba. 1.5 MB/s | 139 kB 00:00 2023-10-08T13:39:30.843 INFO:teuthology.orchestra.run.smithi115.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-10-08T13:39:30.935 INFO:teuthology.orchestra.run.smithi115.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 24 MB/s | 4.9 MB 00:00 2023-10-08T13:39:30.977 INFO:teuthology.orchestra.run.smithi115.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-10-08T13:39:31.052 INFO:teuthology.orchestra.run.smithi115.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-10-08T13:39:31.086 INFO:teuthology.orchestra.run.smithi115.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-10-08T13:39:31.130 INFO:teuthology.orchestra.run.smithi099.stdout:(16/131): ceph-radosgw-18.0.0-6639.g7b3143ba.el 2.9 MB/s | 14 MB 00:04 2023-10-08T13:39:31.152 INFO:teuthology.orchestra.run.smithi115.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 12 MB/s | 808 kB 00:00 2023-10-08T13:39:31.155 INFO:teuthology.orchestra.run.smithi099.stdout:(17/131): libradosstriper1-18.0.0-6639.g7b3143b 1.4 MB/s | 518 kB 00:00 2023-10-08T13:39:31.244 INFO:teuthology.orchestra.run.smithi115.stdout:(100/131): postgresql-server-10.21-2.module+el8 26 MB/s | 5.1 MB 00:00 2023-10-08T13:39:31.272 INFO:teuthology.orchestra.run.smithi099.stdout:(18/131): python3-ceph-argparse-18.0.0-6639.g7b 383 kB/s | 45 kB 00:00 2023-10-08T13:39:31.295 INFO:teuthology.orchestra.run.smithi115.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-10-08T13:39:31.320 INFO:teuthology.orchestra.run.smithi115.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 622 kB/s | 47 kB 00:00 2023-10-08T13:39:31.364 INFO:teuthology.orchestra.run.smithi099.stdout:(19/131): python3-ceph-common-18.0.0-6639.g7b31 1.3 MB/s | 125 kB 00:00 2023-10-08T13:39:31.370 INFO:teuthology.orchestra.run.smithi115.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.6 MB/s | 200 kB 00:00 2023-10-08T13:39:31.395 INFO:teuthology.orchestra.run.smithi115.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 565 kB/s | 43 kB 00:00 2023-10-08T13:39:31.448 INFO:teuthology.orchestra.run.smithi099.stdout:(20/131): python3-cephfs-18.0.0-6639.g7b3143ba. 2.3 MB/s | 197 kB 00:00 2023-10-08T13:39:31.598 INFO:teuthology.orchestra.run.smithi099.stdout:(21/131): python3-rados-18.0.0-6639.g7b3143ba.e 2.4 MB/s | 371 kB 00:00 2023-10-08T13:39:31.669 INFO:teuthology.orchestra.run.smithi057.stdout:(7/131): ceph-mon-18.0.0-6639.g7b3143ba.el8.x86 2.7 MB/s | 4.6 MB 00:01 2023-10-08T13:39:31.774 INFO:teuthology.orchestra.run.smithi099.stdout:(22/131): python3-rbd-18.0.0-6639.g7b3143ba.el8 2.0 MB/s | 365 kB 00:00 2023-10-08T13:39:31.829 INFO:teuthology.orchestra.run.smithi115.stdout:(105/131): ceph-test-18.0.0-6639.g7b3143ba.el8. 7.5 MB/s | 49 MB 00:06 2023-10-08T13:39:31.846 INFO:teuthology.orchestra.run.smithi115.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 107 kB/s | 48 kB 00:00 2023-10-08T13:39:31.857 INFO:teuthology.orchestra.run.smithi099.stdout:(23/131): python3-rgw-18.0.0-6639.g7b3143ba.el8 1.3 MB/s | 107 kB 00:00 2023-10-08T13:39:31.880 INFO:teuthology.orchestra.run.smithi115.stdout:(107/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 433 kB/s | 220 kB 00:00 2023-10-08T13:39:31.897 INFO:teuthology.orchestra.run.smithi115.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 952 kB/s | 43 kB 00:00 2023-10-08T13:39:31.924 INFO:teuthology.orchestra.run.smithi099.stdout:(24/131): rbd-fuse-18.0.0-6639.g7b3143ba.el8.x8 1.4 MB/s | 94 kB 00:00 2023-10-08T13:39:32.085 INFO:teuthology.orchestra.run.smithi057.stdout:(8/131): ceph-common-18.0.0-6639.g7b3143ba.el8. 4.6 MB/s | 23 MB 00:04 2023-10-08T13:39:32.144 INFO:teuthology.orchestra.run.smithi057.stdout:(9/131): ceph-selinux-18.0.0-6639.g7b3143ba.el8 419 kB/s | 24 kB 00:00 2023-10-08T13:39:32.148 INFO:teuthology.orchestra.run.smithi115.stdout:(109/131): libsepol-devel-2.9-3.el8.x86_64.rpm 274 kB/s | 87 kB 00:00 2023-10-08T13:39:32.172 INFO:teuthology.orchestra.run.smithi115.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 373 kB/s | 109 kB 00:00 2023-10-08T13:39:32.198 INFO:teuthology.orchestra.run.smithi115.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 761 kB/s | 229 kB 00:00 2023-10-08T13:39:32.223 INFO:teuthology.orchestra.run.smithi115.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 374 kB/s | 19 kB 00:00 2023-10-08T13:39:32.257 INFO:teuthology.orchestra.run.smithi115.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.6 MB/s | 155 kB 00:00 2023-10-08T13:39:32.291 INFO:teuthology.orchestra.run.smithi115.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.8 MB/s | 560 kB 00:00 2023-10-08T13:39:32.316 INFO:teuthology.orchestra.run.smithi115.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-10-08T13:39:32.349 INFO:teuthology.orchestra.run.smithi115.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.3 MB/s | 559 kB 00:00 2023-10-08T13:39:32.375 INFO:teuthology.orchestra.run.smithi115.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-10-08T13:39:32.400 INFO:teuthology.orchestra.run.smithi115.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-10-08T13:39:32.425 INFO:teuthology.orchestra.run.smithi115.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-10-08T13:39:32.450 INFO:teuthology.orchestra.run.smithi115.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-08T13:39:32.476 INFO:teuthology.orchestra.run.smithi115.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-10-08T13:39:32.518 INFO:teuthology.orchestra.run.smithi115.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.4 MB/s | 98 kB 00:00 2023-10-08T13:39:32.543 INFO:teuthology.orchestra.run.smithi115.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 864 kB/s | 58 kB 00:00 2023-10-08T13:39:32.568 INFO:teuthology.orchestra.run.smithi115.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-10-08T13:39:32.610 INFO:teuthology.orchestra.run.smithi115.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-10-08T13:39:32.657 INFO:teuthology.orchestra.run.smithi115.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-10-08T13:39:32.660 INFO:teuthology.orchestra.run.smithi115.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-10-08T13:39:32.719 INFO:teuthology.orchestra.run.smithi115.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.3 MB/s | 2.3 MB 00:00 2023-10-08T13:39:32.769 INFO:teuthology.orchestra.run.smithi115.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-08T13:39:32.799 INFO:teuthology.orchestra.run.smithi099.stdout:(25/131): rbd-mirror-18.0.0-6639.g7b3143ba.el8. 4.1 MB/s | 3.6 MB 00:00 2023-10-08T13:39:32.925 INFO:teuthology.orchestra.run.smithi099.stdout:(26/131): librgw2-18.0.0-6639.g7b3143ba.el8.x86 3.9 MB/s | 6.9 MB 00:01 2023-10-08T13:39:32.950 INFO:teuthology.orchestra.run.smithi099.stdout:(27/131): rbd-nbd-18.0.0-6639.g7b3143ba.el8.x86 1.2 MB/s | 179 kB 00:00 2023-10-08T13:39:32.977 INFO:teuthology.orchestra.run.smithi057.stdout:(10/131): ceph-osd-18.0.0-6639.g7b3143ba.el8.x8 6.0 MB/s | 18 MB 00:02 2023-10-08T13:39:32.984 INFO:teuthology.orchestra.run.smithi099.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6639.g 394 kB/s | 23 kB 00:00 2023-10-08T13:39:33.051 INFO:teuthology.orchestra.run.smithi099.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6639.g7b3143b 1.3 MB/s | 132 kB 00:00 2023-10-08T13:39:33.061 INFO:teuthology.orchestra.run.smithi057.stdout:(11/131): cephfs-mirror-18.0.0-6639.g7b3143ba.e 3.1 MB/s | 261 kB 00:00 2023-10-08T13:39:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:(12/131): libcephfs-devel-18.0.0-6639.g7b3143ba 266 kB/s | 31 kB 00:00 2023-10-08T13:39:33.486 INFO:teuthology.orchestra.run.smithi057.stdout:(13/131): libcephfs2-18.0.0-6639.g7b3143ba.el8. 2.7 MB/s | 840 kB 00:00 2023-10-08T13:39:33.587 INFO:teuthology.orchestra.run.smithi057.stdout:(14/131): libcephsqlite-18.0.0-6639.g7b3143ba.e 1.8 MB/s | 184 kB 00:00 2023-10-08T13:39:33.654 INFO:teuthology.orchestra.run.smithi057.stdout:(15/131): librados-devel-18.0.0-6639.g7b3143ba. 2.0 MB/s | 139 kB 00:00 2023-10-08T13:39:33.657 INFO:teuthology.orchestra.run.smithi115.stdout:(130/131): librados2-18.0.0-6639.g7b3143ba.el8. 3.7 MB/s | 3.7 MB 00:01 2023-10-08T13:39:33.659 INFO:teuthology.orchestra.run.smithi099.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6639.g7b314 2.6 MB/s | 1.8 MB 00:00 2023-10-08T13:39:33.711 INFO:teuthology.orchestra.run.smithi115.stdout:(131/131): librbd1-18.0.0-6639.g7b3143ba.el8.x8 3.5 MB/s | 3.7 MB 00:01 2023-10-08T13:39:33.714 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2023-10-08T13:39:33.714 INFO:teuthology.orchestra.run.smithi115.stdout:Total 17 MB/s | 209 MB 00:11 2023-10-08T13:39:33.801 INFO:teuthology.orchestra.run.smithi099.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6639.g7b 1.7 MB/s | 242 kB 00:00 2023-10-08T13:39:33.843 INFO:teuthology.orchestra.run.smithi099.stdout:(32/131): ceph-mgr-rook-18.0.0-6639.g7b3143ba.e 1.1 MB/s | 47 kB 00:00 2023-10-08T13:39:33.854 INFO:teuthology.orchestra.run.smithi057.stdout:(16/131): libradosstriper1-18.0.0-6639.g7b3143b 2.5 MB/s | 518 kB 00:00 2023-10-08T13:39:33.885 INFO:teuthology.orchestra.run.smithi099.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6639.g7 350 kB/s | 15 kB 00:00 2023-10-08T13:39:34.060 INFO:teuthology.orchestra.run.smithi099.stdout:(34/131): ceph-volume-18.0.0-6639.g7b3143ba.el8 1.5 MB/s | 261 kB 00:00 2023-10-08T13:39:34.386 INFO:teuthology.orchestra.run.smithi099.stdout:(35/131): cephadm-18.0.0-6639.g7b3143ba.el8.noa 585 kB/s | 190 kB 00:00 2023-10-08T13:39:34.461 INFO:teuthology.orchestra.run.smithi099.stdout:(36/131): cephfs-top-18.0.0-6639.g7b3143ba.el8. 241 kB/s | 18 kB 00:00 2023-10-08T13:39:34.526 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2023-10-08T13:39:34.647 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2023-10-08T13:39:34.647 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2023-10-08T13:39:34.703 INFO:teuthology.orchestra.run.smithi099.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-10-08T13:39:34.762 INFO:teuthology.orchestra.run.smithi099.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-10-08T13:39:34.878 INFO:teuthology.orchestra.run.smithi099.stdout:(39/131): ceph-mgr-diskprediction-local-18.0.0- 4.1 MB/s | 7.4 MB 00:01 2023-10-08T13:39:34.912 INFO:teuthology.orchestra.run.smithi099.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 500 kB/s | 75 kB 00:00 2023-10-08T13:39:34.921 INFO:teuthology.orchestra.run.smithi057.stdout:(17/131): librgw2-18.0.0-6639.g7b3143ba.el8.x86 6.5 MB/s | 6.9 MB 00:01 2023-10-08T13:39:34.971 INFO:teuthology.orchestra.run.smithi099.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 810 kB/s | 44 kB 00:00 2023-10-08T13:39:34.979 INFO:teuthology.orchestra.run.smithi057.stdout:(18/131): python3-ceph-argparse-18.0.0-6639.g7b 771 kB/s | 45 kB 00:00 2023-10-08T13:39:35.038 INFO:teuthology.orchestra.run.smithi057.stdout:(19/131): python3-ceph-common-18.0.0-6639.g7b31 2.1 MB/s | 125 kB 00:00 2023-10-08T13:39:35.097 INFO:teuthology.orchestra.run.smithi057.stdout:(20/131): python3-cephfs-18.0.0-6639.g7b3143ba. 3.3 MB/s | 197 kB 00:00 2023-10-08T13:39:35.188 INFO:teuthology.orchestra.run.smithi057.stdout:(21/131): python3-rados-18.0.0-6639.g7b3143ba.e 4.0 MB/s | 371 kB 00:00 2023-10-08T13:39:35.221 INFO:teuthology.orchestra.run.smithi099.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.4 MB/s | 476 kB 00:00 2023-10-08T13:39:35.263 INFO:teuthology.orchestra.run.smithi099.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 795 kB/s | 33 kB 00:00 2023-10-08T13:39:35.364 INFO:teuthology.orchestra.run.smithi099.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2023-10-08T13:39:35.397 INFO:teuthology.orchestra.run.smithi057.stdout:(22/131): ceph-radosgw-18.0.0-6639.g7b3143ba.el 3.7 MB/s | 14 MB 00:03 2023-10-08T13:39:35.422 INFO:teuthology.orchestra.run.smithi057.stdout:(23/131): python3-rbd-18.0.0-6639.g7b3143ba.el8 1.5 MB/s | 365 kB 00:00 2023-10-08T13:39:35.464 INFO:teuthology.orchestra.run.smithi099.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-10-08T13:39:35.464 INFO:teuthology.orchestra.run.smithi057.stdout:(24/131): python3-rgw-18.0.0-6639.g7b3143ba.el8 1.6 MB/s | 107 kB 00:00 2023-10-08T13:39:35.490 INFO:teuthology.orchestra.run.smithi057.stdout:(25/131): rbd-fuse-18.0.0-6639.g7b3143ba.el8.x8 1.4 MB/s | 94 kB 00:00 2023-10-08T13:39:35.506 INFO:teuthology.orchestra.run.smithi099.stdout:(46/131): python3-beautifulsoup4-4.6.3-2.el8.1. 347 kB/s | 185 kB 00:00 2023-10-08T13:39:35.540 INFO:teuthology.orchestra.run.smithi057.stdout:(26/131): rbd-nbd-18.0.0-6639.g7b3143ba.el8.x86 3.5 MB/s | 179 kB 00:00 2023-10-08T13:39:35.590 INFO:teuthology.orchestra.run.smithi057.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6639.g 461 kB/s | 23 kB 00:00 2023-10-08T13:39:35.649 INFO:teuthology.orchestra.run.smithi057.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6639.g7b3143b 2.2 MB/s | 132 kB 00:00 2023-10-08T13:39:35.665 INFO:teuthology.orchestra.run.smithi099.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 513 kB/s | 81 kB 00:00 2023-10-08T13:39:35.690 INFO:teuthology.orchestra.run.smithi099.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.7 MB/s | 384 kB 00:00 2023-10-08T13:39:35.715 INFO:teuthology.orchestra.run.smithi099.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-10-08T13:39:35.749 INFO:teuthology.orchestra.run.smithi099.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 300 kB/s | 18 kB 00:00 2023-10-08T13:39:35.785 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2023-10-08T13:39:35.789 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2023-10-08T13:39:35.841 INFO:teuthology.orchestra.run.smithi099.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 532 kB/s | 49 kB 00:00 2023-10-08T13:39:35.916 INFO:teuthology.orchestra.run.smithi057.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6639.g7b314 6.6 MB/s | 1.8 MB 00:00 2023-10-08T13:39:35.949 INFO:teuthology.orchestra.run.smithi099.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 543 kB/s | 59 kB 00:00 2023-10-08T13:39:36.042 INFO:teuthology.orchestra.run.smithi099.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 657 kB/s | 60 kB 00:00 2023-10-08T13:39:36.250 INFO:teuthology.orchestra.run.smithi057.stdout:(30/131): rbd-mirror-18.0.0-6639.g7b3143ba.el8. 4.6 MB/s | 3.6 MB 00:00 2023-10-08T13:39:36.325 INFO:teuthology.orchestra.run.smithi057.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6639.g7b 3.2 MB/s | 242 kB 00:00 2023-10-08T13:39:36.358 INFO:teuthology.orchestra.run.smithi099.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 893 kB/s | 283 kB 00:00 2023-10-08T13:39:36.400 INFO:teuthology.orchestra.run.smithi057.stdout:(32/131): ceph-mgr-rook-18.0.0-6639.g7b3143ba.e 633 kB/s | 47 kB 00:00 2023-10-08T13:39:36.417 INFO:teuthology.orchestra.run.smithi099.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-10-08T13:39:36.467 INFO:teuthology.orchestra.run.smithi057.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6639.g7 219 kB/s | 15 kB 00:00 2023-10-08T13:39:36.468 INFO:teuthology.orchestra.run.smithi099.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 675 kB/s | 34 kB 00:00 2023-10-08T13:39:36.559 INFO:teuthology.orchestra.run.smithi057.stdout:(34/131): ceph-volume-18.0.0-6639.g7b3143ba.el8 2.8 MB/s | 261 kB 00:00 2023-10-08T13:39:36.626 INFO:teuthology.orchestra.run.smithi057.stdout:(35/131): cephadm-18.0.0-6639.g7b3143ba.el8.noa 2.8 MB/s | 190 kB 00:00 2023-10-08T13:39:36.668 INFO:teuthology.orchestra.run.smithi099.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 983 kB/s | 196 kB 00:00 2023-10-08T13:39:36.693 INFO:teuthology.orchestra.run.smithi057.stdout:(36/131): cephfs-top-18.0.0-6639.g7b3143ba.el8. 271 kB/s | 18 kB 00:00 2023-10-08T13:39:36.743 INFO:teuthology.orchestra.run.smithi099.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 857 kB/s | 64 kB 00:00 2023-10-08T13:39:36.777 INFO:teuthology.orchestra.run.smithi099.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 560 kB/s | 19 kB 00:00 2023-10-08T13:39:36.827 INFO:teuthology.orchestra.run.smithi099.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 480 kB/s | 24 kB 00:00 2023-10-08T13:39:36.861 INFO:teuthology.orchestra.run.smithi099.stdout:(61/131): python3-kubernetes-11.0.0-6.el8.noarc 1.0 MB/s | 1.2 MB 00:01 2023-10-08T13:39:36.910 INFO:teuthology.orchestra.run.smithi057.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 7.5 MB/s | 7.4 MB 00:00 2023-10-08T13:39:36.919 INFO:teuthology.orchestra.run.smithi099.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-10-08T13:39:36.945 INFO:teuthology.orchestra.run.smithi099.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 329 kB/s | 27 kB 00:00 2023-10-08T13:39:37.170 INFO:teuthology.orchestra.run.smithi099.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.0 MB/s | 255 kB 00:00 2023-10-08T13:39:37.212 INFO:teuthology.orchestra.run.smithi099.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 941 kB/s | 251 kB 00:00 2023-10-08T13:39:37.246 INFO:teuthology.orchestra.run.smithi099.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 813 kB/s | 61 kB 00:00 2023-10-08T13:39:37.288 INFO:teuthology.orchestra.run.smithi099.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 552 kB/s | 23 kB 00:00 2023-10-08T13:39:37.330 INFO:teuthology.orchestra.run.smithi099.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 728 kB/s | 85 kB 00:00 2023-10-08T13:39:37.357 INFO:teuthology.orchestra.run.smithi099.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.5 MB/s | 70 kB 00:00 2023-10-08T13:39:37.424 INFO:teuthology.orchestra.run.smithi099.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 703 kB/s | 47 kB 00:00 2023-10-08T13:39:37.458 INFO:teuthology.orchestra.run.smithi099.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 5.9 MB/s | 200 kB 00:00 2023-10-08T13:39:37.500 INFO:teuthology.orchestra.run.smithi099.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-10-08T13:39:37.542 INFO:teuthology.orchestra.run.smithi099.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-10-08T13:39:37.584 INFO:teuthology.orchestra.run.smithi099.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-10-08T13:39:37.610 INFO:teuthology.orchestra.run.smithi057.stdout:(38/131): ceph-test-18.0.0-6639.g7b3143ba.el8.x 9.0 MB/s | 49 MB 00:05 2023-10-08T13:39:37.627 INFO:teuthology.orchestra.run.smithi057.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 82 kB/s | 59 kB 00:00 2023-10-08T13:39:37.653 INFO:teuthology.orchestra.run.smithi057.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 319 kB/s | 306 kB 00:00 2023-10-08T13:39:37.656 INFO:teuthology.orchestra.run.smithi099.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.2 MB/s | 87 kB 00:00 2023-10-08T13:39:37.701 INFO:teuthology.orchestra.run.smithi099.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 952 kB/s | 43 kB 00:00 2023-10-08T13:39:37.728 INFO:teuthology.orchestra.run.smithi057.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 581 kB/s | 44 kB 00:00 2023-10-08T13:39:37.743 INFO:teuthology.orchestra.run.smithi099.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-10-08T13:39:37.787 INFO:teuthology.orchestra.run.smithi099.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-10-08T13:39:37.827 INFO:teuthology.orchestra.run.smithi099.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 13 MB/s | 560 kB 00:00 2023-10-08T13:39:37.828 INFO:teuthology.orchestra.run.smithi057.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-10-08T13:39:37.852 INFO:teuthology.orchestra.run.smithi099.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 749 kB/s | 19 kB 00:00 2023-10-08T13:39:37.853 INFO:teuthology.orchestra.run.smithi057.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2023-10-08T13:39:37.895 INFO:teuthology.orchestra.run.smithi057.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 493 kB/s | 33 kB 00:00 2023-10-08T13:39:37.910 INFO:teuthology.orchestra.run.smithi099.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.7 MB/s | 155 kB 00:00 2023-10-08T13:39:37.920 INFO:teuthology.orchestra.run.smithi057.stdout:(45/131): libunwind-1.3.1-3.el8.x86_64.rpm 242 kB/s | 75 kB 00:00 2023-10-08T13:39:37.946 INFO:teuthology.orchestra.run.smithi057.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 177 kB/s | 16 kB 00:00 2023-10-08T13:39:37.961 INFO:teuthology.orchestra.run.smithi099.stdout:(82/131): python3-cryptography-3.2.1-5.el8.x86_ 11 MB/s | 559 kB 00:00 2023-10-08T13:39:37.986 INFO:teuthology.orchestra.run.smithi099.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 715 kB/s | 18 kB 00:00 2023-10-08T13:39:37.987 INFO:teuthology.orchestra.run.smithi057.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-10-08T13:39:38.021 INFO:teuthology.orchestra.run.smithi057.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-10-08T13:39:38.028 INFO:teuthology.orchestra.run.smithi099.stdout:(84/131): pcre2-devel-10.32-2.el8.x86_64.rpm 14 MB/s | 605 kB 00:00 2023-10-08T13:39:38.063 INFO:teuthology.orchestra.run.smithi057.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2023-10-08T13:39:38.070 INFO:teuthology.orchestra.run.smithi099.stdout:(85/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2023-10-08T13:39:38.097 INFO:teuthology.orchestra.run.smithi057.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2023-10-08T13:39:38.147 INFO:teuthology.orchestra.run.smithi057.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.7 MB/s | 384 kB 00:00 2023-10-08T13:39:38.172 INFO:teuthology.orchestra.run.smithi057.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 648 kB/s | 49 kB 00:00 2023-10-08T13:39:38.172 INFO:teuthology.orchestra.run.smithi099.stdout:(86/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 23 MB/s | 2.3 MB 00:00 2023-10-08T13:39:38.213 INFO:teuthology.orchestra.run.smithi099.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.8 MB/s | 72 kB 00:00 2023-10-08T13:39:38.255 INFO:teuthology.orchestra.run.smithi099.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 2.0 MB/s | 83 kB 00:00 2023-10-08T13:39:38.289 INFO:teuthology.orchestra.run.smithi099.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 1.8 MB/s | 60 kB 00:00 2023-10-08T13:39:38.331 INFO:teuthology.orchestra.run.smithi099.stdout:(90/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-10-08T13:39:38.364 INFO:teuthology.orchestra.run.smithi099.stdout:(91/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2023-10-08T13:39:38.365 INFO:teuthology.orchestra.run.smithi057.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 271 kB/s | 59 kB 00:00 2023-10-08T13:39:38.389 INFO:teuthology.orchestra.run.smithi057.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 278 kB/s | 60 kB 00:00 2023-10-08T13:39:38.398 INFO:teuthology.orchestra.run.smithi099.stdout:(92/131): libcom_err-devel-1.45.6-4.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-10-08T13:39:38.423 INFO:teuthology.orchestra.run.smithi057.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2023-10-08T13:39:38.465 INFO:teuthology.orchestra.run.smithi099.stdout:(93/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-10-08T13:39:38.465 INFO:teuthology.orchestra.run.smithi057.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.7 MB/s | 283 kB 00:00 2023-10-08T13:39:38.490 INFO:teuthology.orchestra.run.smithi057.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-10-08T13:39:38.516 INFO:teuthology.orchestra.run.smithi099.stdout:(94/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 3.3 MB/s | 171 kB 00:00 2023-10-08T13:39:38.516 INFO:teuthology.orchestra.run.smithi057.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 365 kB/s | 34 kB 00:00 2023-10-08T13:39:38.558 INFO:teuthology.orchestra.run.smithi099.stdout:(95/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-10-08T13:39:38.558 INFO:teuthology.orchestra.run.smithi057.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-10-08T13:39:38.583 INFO:teuthology.orchestra.run.smithi057.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 695 kB/s | 64 kB 00:00 2023-10-08T13:39:38.608 INFO:teuthology.orchestra.run.smithi057.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 203 kB/s | 19 kB 00:00 2023-10-08T13:39:38.633 INFO:teuthology.orchestra.run.smithi057.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2023-10-08T13:39:38.659 INFO:teuthology.orchestra.run.smithi057.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 388 kB/s | 29 kB 00:00 2023-10-08T13:39:38.684 INFO:teuthology.orchestra.run.smithi057.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2023-10-08T13:39:38.726 INFO:teuthology.orchestra.run.smithi057.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-10-08T13:39:38.751 INFO:teuthology.orchestra.run.smithi057.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-10-08T13:39:38.776 INFO:teuthology.orchestra.run.smithi057.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 665 kB/s | 61 kB 00:00 2023-10-08T13:39:38.802 INFO:teuthology.orchestra.run.smithi057.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-10-08T13:39:38.827 INFO:teuthology.orchestra.run.smithi057.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2023-10-08T13:39:38.841 INFO:teuthology.orchestra.run.smithi099.stdout:(96/131): python3-scipy-1.0.0-21.module+el8.5.0 48 MB/s | 14 MB 00:00 2023-10-08T13:39:38.883 INFO:teuthology.orchestra.run.smithi099.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-10-08T13:39:38.927 INFO:teuthology.orchestra.run.smithi057.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 556 kB/s | 70 kB 00:00 2023-10-08T13:39:38.958 INFO:teuthology.orchestra.run.smithi099.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 724 kB/s | 54 kB 00:00 2023-10-08T13:39:39.000 INFO:teuthology.orchestra.run.smithi099.stdout:(99/131): thrift-0.13.0-2.el8.x86_64.rpm 1.0 MB/s | 1.7 MB 00:01 2023-10-08T13:39:39.019 INFO:teuthology.orchestra.run.smithi057.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 7.2 MB/s | 1.7 MB 00:00 2023-10-08T13:39:39.025 INFO:teuthology.orchestra.run.smithi099.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-10-08T13:39:39.053 INFO:teuthology.orchestra.run.smithi057.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 207 kB/s | 47 kB 00:00 2023-10-08T13:39:39.076 INFO:teuthology.orchestra.run.smithi099.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 869 kB/s | 39 kB 00:00 2023-10-08T13:39:39.086 INFO:teuthology.orchestra.run.smithi057.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.3 MB/s | 200 kB 00:00 2023-10-08T13:39:39.101 INFO:teuthology.orchestra.run.smithi099.stdout:(102/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.1 MB/s | 323 kB 00:00 2023-10-08T13:39:39.112 INFO:teuthology.orchestra.run.smithi057.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 510 kB/s | 43 kB 00:00 2023-10-08T13:39:39.137 INFO:teuthology.orchestra.run.smithi057.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 956 kB/s | 48 kB 00:00 2023-10-08T13:39:39.143 INFO:teuthology.orchestra.run.smithi099.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-10-08T13:39:39.162 INFO:teuthology.orchestra.run.smithi057.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-08T13:39:39.188 INFO:teuthology.orchestra.run.smithi057.stdout:(77/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-10-08T13:39:39.210 INFO:teuthology.orchestra.run.smithi099.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 4.8 MB/s | 538 kB 00:00 2023-10-08T13:39:39.213 INFO:teuthology.orchestra.run.smithi057.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 564 kB/s | 43 kB 00:00 2023-10-08T13:39:39.230 INFO:teuthology.orchestra.run.smithi057.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-10-08T13:39:39.235 INFO:teuthology.orchestra.run.smithi099.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2023-10-08T13:39:39.261 INFO:teuthology.orchestra.run.smithi099.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-10-08T13:39:39.263 INFO:teuthology.orchestra.run.smithi057.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-10-08T13:39:39.280 INFO:teuthology.orchestra.run.smithi057.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 371 kB/s | 19 kB 00:00 2023-10-08T13:39:39.303 INFO:teuthology.orchestra.run.smithi099.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-10-08T13:39:39.314 INFO:teuthology.orchestra.run.smithi057.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-10-08T13:39:39.347 INFO:teuthology.orchestra.run.smithi057.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-10-08T13:39:39.364 INFO:teuthology.orchestra.run.smithi057.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 357 kB/s | 18 kB 00:00 2023-10-08T13:39:39.406 INFO:teuthology.orchestra.run.smithi057.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.3 MB/s | 559 kB 00:00 2023-10-08T13:39:39.432 INFO:teuthology.orchestra.run.smithi057.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-10-08T13:39:39.457 INFO:teuthology.orchestra.run.smithi057.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-10-08T13:39:39.482 INFO:teuthology.orchestra.run.smithi057.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-10-08T13:39:39.508 INFO:teuthology.orchestra.run.smithi057.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-08T13:39:39.533 INFO:teuthology.orchestra.run.smithi057.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-10-08T13:39:39.550 INFO:teuthology.orchestra.run.smithi057.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-10-08T13:39:39.575 INFO:teuthology.orchestra.run.smithi057.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2023-10-08T13:39:39.601 INFO:teuthology.orchestra.run.smithi057.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 767 kB/s | 39 kB 00:00 2023-10-08T13:39:39.651 INFO:teuthology.orchestra.run.smithi057.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 8.4 MB/s | 644 kB 00:00 2023-10-08T13:39:39.707 INFO:teuthology.orchestra.run.smithi057.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.6 MB/s | 171 kB 00:00 2023-10-08T13:39:39.734 INFO:teuthology.orchestra.run.smithi057.stdout:(96/131): NetworkManager-initscripts-updown-1.3 1.6 MB/s | 138 kB 00:00 2023-10-08T13:39:39.762 INFO:teuthology.orchestra.run.smithi099.stdout:(108/131): ceph-test-18.0.0-6639.g7b3143ba.el8. 4.7 MB/s | 49 MB 00:10 2023-10-08T13:39:39.787 INFO:teuthology.orchestra.run.smithi099.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 55 kB/s | 27 kB 00:00 2023-10-08T13:39:39.793 INFO:teuthology.orchestra.run.smithi057.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.0 MB/s | 2.3 MB 00:00 2023-10-08T13:39:39.810 INFO:teuthology.orchestra.run.smithi057.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-10-08T13:39:39.838 INFO:teuthology.orchestra.run.smithi057.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-10-08T13:39:39.869 INFO:teuthology.orchestra.run.smithi057.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-10-08T13:39:39.903 INFO:teuthology.orchestra.run.smithi057.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-10-08T13:39:39.928 INFO:teuthology.orchestra.run.smithi057.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 665 kB/s | 39 kB 00:00 2023-10-08T13:39:39.970 INFO:teuthology.orchestra.run.smithi057.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-10-08T13:39:40.003 INFO:teuthology.orchestra.run.smithi057.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-10-08T13:39:40.029 INFO:teuthology.orchestra.run.smithi057.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-08T13:39:40.037 INFO:teuthology.orchestra.run.smithi099.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 180 kB/s | 44 kB 00:00 2023-10-08T13:39:40.055 INFO:teuthology.orchestra.run.smithi057.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-10-08T13:39:40.071 INFO:teuthology.orchestra.run.smithi099.stdout:(111/131): flex-2.6.1-9.el8.x86_64.rpm 1.0 MB/s | 320 kB 00:00 2023-10-08T13:39:40.096 INFO:teuthology.orchestra.run.smithi057.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-10-08T13:39:40.121 INFO:teuthology.orchestra.run.smithi099.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-10-08T13:39:40.138 INFO:teuthology.orchestra.run.smithi057.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 641 kB/s | 27 kB 00:00 2023-10-08T13:39:40.163 INFO:teuthology.orchestra.run.smithi099.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-10-08T13:39:40.180 INFO:teuthology.orchestra.run.smithi057.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-08T13:39:40.222 INFO:teuthology.orchestra.run.smithi057.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-10-08T13:39:40.238 INFO:teuthology.orchestra.run.smithi099.stdout:(114/131): python3-babel-2.5.1-7.el8.noarch.rpm 4.9 MB/s | 4.8 MB 00:00 2023-10-08T13:39:40.294 INFO:teuthology.orchestra.run.smithi099.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 491 kB/s | 64 kB 00:00 2023-10-08T13:39:40.322 INFO:teuthology.orchestra.run.smithi057.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-10-08T13:39:40.322 INFO:teuthology.orchestra.run.smithi099.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.5 MB/s | 126 kB 00:00 2023-10-08T13:39:40.364 INFO:teuthology.orchestra.run.smithi099.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 285 kB/s | 20 kB 00:00 2023-10-08T13:39:40.380 INFO:teuthology.orchestra.run.smithi057.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-10-08T13:39:40.389 INFO:teuthology.orchestra.run.smithi099.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.5 MB/s | 103 kB 00:00 2023-10-08T13:39:40.457 INFO:teuthology.orchestra.run.smithi099.stdout:(119/131): python3-numpy-1.14.3-10.el8.x86_64.r 8.9 MB/s | 3.7 MB 00:00 2023-10-08T13:39:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-10-08T13:39:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.0 MB/s | 225 kB 00:00 2023-10-08T13:39:40.507 INFO:teuthology.orchestra.run.smithi099.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2023-10-08T13:39:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-08T13:39:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-10-08T13:39:40.557 INFO:teuthology.orchestra.run.smithi099.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.9 MB/s | 198 kB 00:00 2023-10-08T13:39:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2023-10-08T13:39:40.590 INFO:teuthology.orchestra.run.smithi057.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-08T13:39:40.599 INFO:teuthology.orchestra.run.smithi099.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-10-08T13:39:40.625 INFO:teuthology.orchestra.run.smithi057.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-10-08T13:39:40.649 INFO:teuthology.orchestra.run.smithi099.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 7.0 MB/s | 354 kB 00:00 2023-10-08T13:39:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2023-10-08T13:39:40.725 INFO:teuthology.orchestra.run.smithi099.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 15 MB/s | 4.9 MB 00:00 2023-10-08T13:39:40.775 INFO:teuthology.orchestra.run.smithi057.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-10-08T13:39:40.800 INFO:teuthology.orchestra.run.smithi057.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-10-08T13:39:40.825 INFO:teuthology.orchestra.run.smithi099.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-10-08T13:39:40.842 INFO:teuthology.orchestra.run.smithi057.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-10-08T13:39:40.867 INFO:teuthology.orchestra.run.smithi099.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-10-08T13:39:40.892 INFO:teuthology.orchestra.run.smithi057.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-10-08T13:39:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-08T13:39:40.917 INFO:teuthology.orchestra.run.smithi099.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-10-08T13:39:40.976 INFO:teuthology.orchestra.run.smithi057.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-10-08T13:39:41.001 INFO:teuthology.orchestra.run.smithi099.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-10-08T13:39:41.060 INFO:teuthology.orchestra.run.smithi057.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-10-08T13:39:41.073 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-08T13:39:41.076 INFO:teuthology.orchestra.run.smithi099.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 10 MB/s | 798 kB 00:00 2023-10-08T13:39:41.101 INFO:teuthology.orchestra.run.smithi057.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-10-08T13:39:41.178 INFO:teuthology.orchestra.run.smithi057.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-10-08T13:39:41.260 INFO:teuthology.orchestra.run.smithi057.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-10-08T13:39:41.310 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-08T13:39:41.310 INFO:teuthology.orchestra.run.smithi057.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-08T13:39:41.479 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-08T13:39:41.577 INFO:teuthology.orchestra.run.smithi099.stdout:(130/131): librados2-18.0.0-6639.g7b3143ba.el8. 5.3 MB/s | 3.7 MB 00:00 2023-10-08T13:39:41.594 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-08T13:39:41.635 INFO:teuthology.orchestra.run.smithi099.stdout:(131/131): librbd1-18.0.0-6639.g7b3143ba.el8.x8 5.2 MB/s | 3.7 MB 00:00 2023-10-08T13:39:41.638 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-10-08T13:39:41.638 INFO:teuthology.orchestra.run.smithi099.stdout:Total 11 MB/s | 209 MB 00:19 2023-10-08T13:39:41.652 INFO:teuthology.orchestra.run.smithi057.stdout:(130/131): librbd1-18.0.0-6639.g7b3143ba.el8.x8 7.8 MB/s | 3.7 MB 00:00 2023-10-08T13:39:41.845 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-08T13:39:41.955 INFO:teuthology.orchestra.run.smithi057.stdout:(131/131): librados2-18.0.0-6639.g7b3143ba.el8. 4.4 MB/s | 3.7 MB 00:00 2023-10-08T13:39:41.958 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2023-10-08T13:39:41.958 INFO:teuthology.orchestra.run.smithi057.stdout:Total 14 MB/s | 209 MB 00:14 2023-10-08T13:39:42.256 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-08T13:39:42.472 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-10-08T13:39:42.478 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T13:39:42.593 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-10-08T13:39:42.593 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-10-08T13:39:42.825 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2023-10-08T13:39:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T13:39:42.958 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2023-10-08T13:39:42.958 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2023-10-08T13:39:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T13:39:43.433 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T13:39:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T13:39:43.730 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-10-08T13:39:43.732 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T13:39:43.735 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-10-08T13:39:43.903 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T13:39:44.156 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-08T13:39:44.324 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2023-10-08T13:39:44.329 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2023-10-08T13:39:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-08T13:39:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-08T13:39:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-08T13:39:45.102 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T13:39:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T13:39:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T13:39:45.463 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T13:39:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-08T13:39:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-08T13:39:45.919 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-08T13:39:46.536 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-08T13:39:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-08T13:39:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-08T13:39:47.460 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-08T13:39:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-08T13:39:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-08T13:39:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-08T13:39:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-10-08T13:39:49.226 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2023-10-08T13:39:49.231 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-08T13:39:49.249 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-08T13:39:49.390 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-08T13:39:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-08T13:39:49.482 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-08T13:39:49.576 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-08T13:39:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-08T13:39:49.709 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-08T13:39:49.818 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-08T13:39:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-08T13:39:49.992 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-08T13:39:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 26/134 2023-10-08T13:39:50.107 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-08T13:39:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 27/134 2023-10-08T13:39:50.309 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-08T13:39:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-08T13:39:50.732 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-08T13:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-08T13:39:50.910 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 5/134 2023-10-08T13:39:51.104 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 28/134 2023-10-08T13:39:51.159 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 5/134 2023-10-08T13:39:51.338 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 5/134 2023-10-08T13:39:51.415 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-08T13:39:51.517 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T13:39:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 5/134 2023-10-08T13:39:51.652 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T13:39:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-08T13:39:51.825 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T13:39:51.825 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T13:39:51.925 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T13:39:51.939 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T13:39:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-08T13:39:52.114 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T13:39:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T13:39:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-08T13:39:52.277 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-08T13:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T13:39:52.502 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-08T13:39:52.505 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T13:39:52.692 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-08T13:39:52.696 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-08T13:39:52.970 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-08T13:39:53.125 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-08T13:39:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-08T13:39:53.367 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-08T13:39:53.525 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T13:39:53.635 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T13:39:53.687 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-08T13:39:53.775 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T13:39:53.928 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T13:39:53.933 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-08T13:39:53.949 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-08T13:39:54.138 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-08T13:39:54.158 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T13:39:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-08T13:39:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T13:39:54.349 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-08T13:39:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T13:39:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-08T13:39:54.515 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-08T13:39:54.603 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T13:39:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-08T13:39:54.641 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-08T13:39:54.800 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-08T13:39:54.814 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 36/134 2023-10-08T13:39:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-08T13:39:54.975 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-08T13:39:54.979 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 37/134 2023-10-08T13:39:55.089 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-08T13:39:55.157 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-08T13:39:55.314 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-08T13:39:55.332 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-08T13:39:55.466 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-08T13:39:55.503 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-08T13:39:55.642 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-08T13:39:55.647 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-08T13:39:55.679 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-08T13:39:55.799 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-08T13:39:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-08T13:39:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-08T13:39:56.149 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-08T13:39:56.414 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-08T13:39:56.479 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-08T13:39:56.591 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-08T13:39:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-08T13:39:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-08T13:39:56.991 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-08T13:39:57.010 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-08T13:39:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-08T13:39:57.324 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-08T13:39:57.505 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-08T13:39:57.564 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-08T13:39:57.567 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-08T13:39:57.768 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-08T13:39:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-08T13:39:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-08T13:39:58.005 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-08T13:39:58.058 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-08T13:39:58.164 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 26/134 2023-10-08T13:39:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-08T13:39:58.303 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 27/134 2023-10-08T13:39:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-08T13:39:58.513 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-08T13:39:58.560 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 28/134 2023-10-08T13:39:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-08T13:39:58.749 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-08T13:39:58.780 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-08T13:39:58.828 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-08T13:39:59.013 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-08T13:39:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-08T13:39:59.276 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-08T13:39:59.369 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-08T13:39:59.480 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 26/134 2023-10-08T13:39:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-08T13:39:59.660 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 27/134 2023-10-08T13:39:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-08T13:39:59.886 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 28/134 2023-10-08T13:40:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-08T13:40:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-08T13:40:00.195 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-08T13:40:00.391 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-08T13:40:00.451 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-08T13:40:00.642 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-08T13:40:00.703 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-08T13:40:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-08T13:40:01.091 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-08T13:40:01.163 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-08T13:40:01.378 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-08T13:40:01.541 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-08T13:40:01.745 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-08T13:40:01.748 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-08T13:40:01.864 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-08T13:40:01.909 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-08T13:40:02.063 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-08T13:40:02.181 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-08T13:40:02.313 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-08T13:40:02.414 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-08T13:40:02.535 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-08T13:40:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-08T13:40:02.759 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-08T13:40:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-08T13:40:02.923 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-08T13:40:02.948 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-08T13:40:02.961 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-08T13:40:03.139 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 36/134 2023-10-08T13:40:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-08T13:40:03.286 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-08T13:40:03.314 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 37/134 2023-10-08T13:40:03.386 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-08T13:40:03.498 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-08T13:40:03.552 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-08T13:40:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-08T13:40:03.652 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-08T13:40:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-08T13:40:03.831 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-08T13:40:03.839 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-08T13:40:03.907 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-08T13:40:03.919 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-08T13:40:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-08T13:40:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-08T13:40:04.135 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-08T13:40:04.316 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-08T13:40:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-08T13:40:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-08T13:40:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-08T13:40:04.528 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 36/134 2023-10-08T13:40:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-08T13:40:04.602 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-08T13:40:04.729 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 37/134 2023-10-08T13:40:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-08T13:40:04.856 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-08T13:40:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-08T13:40:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-08T13:40:05.031 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-08T13:40:05.082 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-08T13:40:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-08T13:40:05.249 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-08T13:40:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-08T13:40:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-08T13:40:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-08T13:40:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-08T13:40:05.637 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-08T13:40:05.640 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-08T13:40:05.817 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-08T13:40:05.893 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-08T13:40:06.025 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-08T13:40:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-08T13:40:06.199 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-08T13:40:06.366 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-08T13:40:06.404 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-08T13:40:06.487 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-08T13:40:06.652 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-08T13:40:06.667 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-08T13:40:06.693 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-08T13:40:06.839 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-08T13:40:06.911 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-08T13:40:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-08T13:40:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-08T13:40:07.176 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-08T13:40:07.265 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-08T13:40:07.291 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-08T13:40:07.396 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-08T13:40:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-08T13:40:07.532 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-08T13:40:07.704 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-08T13:40:07.718 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-08T13:40:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-08T13:40:07.931 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-08T13:40:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-08T13:40:08.128 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-08T13:40:08.355 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-08T13:40:08.395 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-08T13:40:08.528 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-08T13:40:08.562 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-08T13:40:08.689 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T13:40:08.728 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-08T13:40:08.763 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T13:40:08.798 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-08T13:40:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 93/134 2023-10-08T13:40:09.042 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-08T13:40:09.066 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-08T13:40:09.283 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-08T13:40:09.324 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-08T13:40:09.551 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-08T13:40:09.582 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-08T13:40:09.783 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-08T13:40:09.920 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-08T13:40:09.923 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-08T13:40:10.146 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-08T13:40:10.171 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-08T13:40:10.411 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-08T13:40:10.423 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-08T13:40:10.647 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-08T13:40:10.693 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-08T13:40:10.828 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-08T13:40:10.925 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-08T13:40:10.986 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-08T13:40:11.141 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-08T13:40:11.195 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-08T13:40:11.405 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-08T13:40:11.501 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-08T13:40:11.628 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-08T13:40:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-08T13:40:11.801 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-08T13:40:11.946 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-08T13:40:12.093 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-08T13:40:12.160 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-08T13:40:12.372 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-08T13:40:12.428 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-08T13:40:12.537 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-08T13:40:12.604 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-08T13:40:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T13:40:12.767 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-08T13:40:12.845 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-08T13:40:12.939 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T13:40:13.069 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-08T13:40:13.098 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-08T13:40:13.264 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-08T13:40:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-08T13:40:13.494 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-08T13:40:13.536 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T13:40:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-08T13:40:13.732 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-08T13:40:13.782 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T13:40:13.866 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T13:40:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-08T13:40:14.005 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T13:40:14.217 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-08T13:40:14.441 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-08T13:40:14.599 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-08T13:40:14.668 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-08T13:40:14.776 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-08T13:40:14.929 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-08T13:40:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-08T13:40:15.158 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-08T13:40:15.326 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-08T13:40:15.462 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-08T13:40:15.559 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-08T13:40:15.764 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-08T13:40:15.834 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-08T13:40:15.991 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-08T13:40:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-08T13:40:16.328 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-08T13:40:16.603 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-08T13:40:16.815 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-08T13:40:17.004 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T13:40:17.087 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T13:40:17.239 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 93/134 2023-10-08T13:40:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-08T13:40:18.115 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-08T13:40:18.411 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-08T13:40:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-08T13:40:19.125 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-08T13:40:19.361 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-08T13:40:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-08T13:40:20.444 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T13:40:20.523 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T13:40:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 93/134 2023-10-08T13:40:21.207 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T13:40:21.838 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T13:40:22.464 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T13:40:22.722 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T13:40:22.798 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T13:40:22.929 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T13:40:24.954 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T13:40:25.243 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T13:40:25.835 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T13:40:26.151 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T13:40:26.229 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T13:40:26.393 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T13:40:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T13:40:32.643 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2023-10-08T13:40:32.644 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2023-10-08T13:40:32.644 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /dev 2023-10-08T13:40:32.644 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /run 2023-10-08T13:40:32.644 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2023-10-08T13:40:32.644 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2023-10-08T13:40:32.644 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2023-10-08T13:40:32.644 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2023-10-08T13:40:32.644 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /dev 2023-10-08T13:40:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:40:33.001 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T13:40:34.586 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T13:40:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T13:40:34.989 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T13:40:35.150 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T13:40:35.625 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T13:40:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T13:40:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T13:40:37.484 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T13:40:37.704 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T13:40:37.822 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 102/134 2023-10-08T13:40:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 103/134 2023-10-08T13:40:38.208 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T13:40:38.428 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T13:40:38.668 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T13:40:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T13:40:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T13:40:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T13:40:40.255 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 107/134 2023-10-08T13:40:40.475 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T13:40:40.510 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T13:40:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T13:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T13:40:41.982 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 110/134 2023-10-08T13:40:42.154 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T13:40:42.187 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T13:40:42.348 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T13:40:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T13:40:42.540 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T13:40:42.658 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T13:40:42.658 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /sys 2023-10-08T13:40:42.658 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /proc 2023-10-08T13:40:42.658 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /dev 2023-10-08T13:40:42.659 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /run 2023-10-08T13:40:42.659 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /mnt 2023-10-08T13:40:42.659 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /var/tmp 2023-10-08T13:40:42.659 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /home 2023-10-08T13:40:42.659 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /tmp 2023-10-08T13:40:42.659 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /dev 2023-10-08T13:40:42.659 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:40:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T13:40:43.026 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T13:40:43.040 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T13:40:44.722 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T13:40:44.869 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T13:40:45.102 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T13:40:45.276 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T13:40:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T13:40:45.319 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2023-10-08T13:40:45.319 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2023-10-08T13:40:45.319 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2023-10-08T13:40:45.319 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /run 2023-10-08T13:40:45.319 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2023-10-08T13:40:45.319 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2023-10-08T13:40:45.319 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2023-10-08T13:40:45.320 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2023-10-08T13:40:45.320 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2023-10-08T13:40:45.320 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:40:45.714 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T13:40:45.776 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T13:40:45.942 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T13:40:45.960 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T13:40:47.301 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T13:40:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T13:40:47.681 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T13:40:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T13:40:47.785 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T13:40:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T13:40:47.955 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 115/134 2023-10-08T13:40:47.979 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T13:40:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-08T13:40:48.123 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 102/134 2023-10-08T13:40:48.302 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-08T13:40:48.318 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 103/134 2023-10-08T13:40:48.427 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T13:40:48.500 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T13:40:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T13:40:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T13:40:48.729 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T13:40:49.026 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T13:40:49.154 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-08T13:40:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-08T13:40:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 119/134 2023-10-08T13:40:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-08T13:40:50.051 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T13:40:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 121/134 2023-10-08T13:40:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T13:40:50.249 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T13:40:50.347 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T13:40:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T13:40:50.607 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T13:40:50.705 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 107/134 2023-10-08T13:40:50.760 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 102/134 2023-10-08T13:40:50.931 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T13:40:50.968 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T13:40:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 103/134 2023-10-08T13:40:51.124 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T13:40:51.137 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T13:40:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T13:40:51.460 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T13:40:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T13:40:52.429 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T13:40:52.565 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 110/134 2023-10-08T13:40:52.610 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T13:40:52.721 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T13:40:52.739 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T13:40:52.754 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T13:40:52.790 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T13:40:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T13:40:52.880 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 124/134 2023-10-08T13:40:52.916 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T13:40:52.918 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T13:40:52.960 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T13:40:53.133 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T13:40:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 125/134 2023-10-08T13:40:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 107/134 2023-10-08T13:40:53.505 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 126/134 2023-10-08T13:40:53.523 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T13:40:53.525 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T13:40:53.561 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T13:40:53.658 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T13:40:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T13:40:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T13:40:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-08T13:40:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T13:40:53.921 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T13:40:54.105 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-08T13:40:54.320 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-08T13:40:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-08T13:40:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-08T13:40:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-08T13:40:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T13:40:54.849 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T13:40:54.849 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T13:40:55.089 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 110/134 2023-10-08T13:40:55.101 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T13:40:55.286 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T13:40:55.321 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T13:40:55.356 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T13:40:55.545 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T13:40:55.587 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T13:40:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T13:40:55.810 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T13:40:55.811 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 1/134 2023-10-08T13:40:55.811 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2/134 2023-10-08T13:40:55.811 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 3/134 2023-10-08T13:40:55.811 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 4/134 2023-10-08T13:40:55.811 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 5/134 2023-10-08T13:40:55.811 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T13:40:55.811 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T13:40:55.811 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T13:40:55.811 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 10/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 11/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 12/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 13/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 14/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 15/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 16/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 17/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 18/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 19/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 20/134 2023-10-08T13:40:55.812 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 21/134 2023-10-08T13:40:55.813 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 22/134 2023-10-08T13:40:55.813 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 23/134 2023-10-08T13:40:55.813 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 24/134 2023-10-08T13:40:55.813 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 25/134 2023-10-08T13:40:55.813 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 26/134 2023-10-08T13:40:55.813 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 27/134 2023-10-08T13:40:55.813 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 28/134 2023-10-08T13:40:55.813 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 29/134 2023-10-08T13:40:55.813 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 30/134 2023-10-08T13:40:55.823 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 31/134 2023-10-08T13:40:55.823 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 32/134 2023-10-08T13:40:55.823 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 33/134 2023-10-08T13:40:55.823 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 34/134 2023-10-08T13:40:55.823 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 35/134 2023-10-08T13:40:55.823 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 36/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 37/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 38/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-08T13:40:55.824 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-08T13:40:55.825 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-08T13:40:55.826 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-08T13:40:55.826 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-08T13:40:55.826 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-08T13:40:55.826 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-08T13:40:55.826 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-08T13:40:55.826 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-08T13:40:55.826 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-08T13:40:55.826 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-08T13:40:55.826 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-08T13:40:55.827 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-08T13:40:55.828 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-08T13:40:55.829 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-08T13:40:55.830 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-08T13:40:55.831 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-08T13:40:55.832 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-08T13:40:55.832 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-08T13:40:55.832 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T13:40:55.832 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 129/134 2023-10-08T13:40:55.832 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-08T13:40:55.832 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 131/134 2023-10-08T13:40:55.832 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T13:40:55.832 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-08T13:40:56.157 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T13:40:56.361 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T13:40:58.325 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T13:40:58.325 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:48 ago on Sun 08 Oct 2023 01:39:10 PM UTC. 2023-10-08T13:40:58.629 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T13:40:59.565 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 115/134 2023-10-08T13:40:59.865 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-08T13:41:00.035 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 117/134 2023-10-08T13:41:00.313 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-08T13:41:00.448 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-08T13:41:00.654 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-08T13:41:00.878 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-08T13:41:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T13:41:01.077 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 121/134 2023-10-08T13:41:01.216 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T13:41:01.871 INFO:teuthology.orchestra.run.smithi115.stdout:Installed products updated. 2023-10-08T13:41:01.959 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 115/134 2023-10-08T13:41:02.083 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:41:02.083 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2023-10-08T13:41:02.083 INFO:teuthology.orchestra.run.smithi115.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:02.084 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:02.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: jq-1.6-3.el8.x86_64 2023-10-08T13:41:02.086 INFO:teuthology.orchestra.run.smithi115.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: libpq-13.5-1.el8.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T13:41:02.087 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T13:41:02.088 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-08T13:41:02.089 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T13:41:02.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T13:41:02.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T13:41:02.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T13:41:02.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-08T13:41:02.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T13:41:02.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T13:41:02.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T13:41:02.091 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-08T13:41:02.092 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T13:41:02.093 INFO:teuthology.orchestra.run.smithi115.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T13:41:02.094 INFO:teuthology.orchestra.run.smithi115.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-08T13:41:02.095 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-08T13:41:02.095 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T13:41:02.095 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-08T13:41:02.095 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:41:02.095 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2023-10-08T13:41:02.270 DEBUG:teuthology.parallel:result is None 2023-10-08T13:41:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-08T13:41:02.529 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 117/134 2023-10-08T13:41:02.828 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-08T13:41:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-08T13:41:03.208 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-08T13:41:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-08T13:41:03.583 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T13:41:03.705 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 121/134 2023-10-08T13:41:03.848 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T13:41:03.867 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T13:41:03.877 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T13:41:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 124/134 2023-10-08T13:41:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 125/134 2023-10-08T13:41:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 126/134 2023-10-08T13:41:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-08T13:41:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-08T13:41:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-08T13:41:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-08T13:41:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-08T13:41:05.589 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-08T13:41:05.597 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-08T13:41:05.716 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-08T13:41:05.716 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T13:41:05.884 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T13:41:05.884 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T13:41:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T13:41:06.124 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T13:41:06.314 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T13:41:06.384 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T13:41:06.403 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T13:41:06.618 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 124/134 2023-10-08T13:41:06.854 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T13:41:06.854 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 1/134 2023-10-08T13:41:06.854 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2/134 2023-10-08T13:41:06.854 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 3/134 2023-10-08T13:41:06.854 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 4/134 2023-10-08T13:41:06.854 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 5/134 2023-10-08T13:41:06.854 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T13:41:06.854 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 10/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 11/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 12/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 13/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 14/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 15/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 16/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 17/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 18/134 2023-10-08T13:41:06.855 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 19/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 20/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 21/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 22/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 23/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 24/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 25/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 26/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 27/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 28/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 29/134 2023-10-08T13:41:06.856 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 30/134 2023-10-08T13:41:06.857 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 31/134 2023-10-08T13:41:06.867 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 32/134 2023-10-08T13:41:06.868 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 33/134 2023-10-08T13:41:06.868 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 34/134 2023-10-08T13:41:06.868 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 35/134 2023-10-08T13:41:06.868 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 36/134 2023-10-08T13:41:06.868 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 37/134 2023-10-08T13:41:06.868 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 38/134 2023-10-08T13:41:06.868 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-08T13:41:06.869 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-08T13:41:06.869 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-08T13:41:06.869 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-08T13:41:06.869 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-08T13:41:06.869 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-08T13:41:06.869 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-08T13:41:06.869 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-08T13:41:06.869 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-08T13:41:06.870 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-08T13:41:06.870 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-08T13:41:06.870 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-08T13:41:06.870 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-08T13:41:06.870 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-08T13:41:06.873 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-08T13:41:06.873 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-08T13:41:06.873 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-08T13:41:06.873 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-08T13:41:06.873 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-08T13:41:06.873 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-08T13:41:06.873 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-08T13:41:06.873 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-08T13:41:06.873 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-08T13:41:06.874 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-08T13:41:06.874 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-08T13:41:06.874 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-08T13:41:06.874 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-08T13:41:06.874 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-08T13:41:06.874 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-08T13:41:06.874 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-08T13:41:06.874 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-08T13:41:06.874 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-08T13:41:06.874 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-08T13:41:06.875 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-08T13:41:06.876 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-08T13:41:06.877 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-08T13:41:06.878 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-08T13:41:06.878 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-08T13:41:06.878 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-08T13:41:06.878 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-08T13:41:06.878 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-08T13:41:06.878 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-08T13:41:06.878 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-08T13:41:06.878 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-08T13:41:06.879 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-08T13:41:06.880 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-08T13:41:06.880 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-08T13:41:06.880 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-08T13:41:06.880 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 129/134 2023-10-08T13:41:06.880 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-08T13:41:06.880 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 131/134 2023-10-08T13:41:06.880 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T13:41:06.880 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-08T13:41:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 125/134 2023-10-08T13:41:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 126/134 2023-10-08T13:41:07.752 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-08T13:41:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-08T13:41:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-08T13:41:08.308 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-08T13:41:08.321 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-08T13:41:08.509 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-08T13:41:08.517 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-08T13:41:08.702 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-08T13:41:08.702 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T13:41:08.945 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T13:41:08.945 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T13:41:09.224 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T13:41:09.423 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T13:41:09.423 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:55 ago on Sun 08 Oct 2023 01:39:14 PM UTC. 2023-10-08T13:41:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T13:41:09.968 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T13:41:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 1/134 2023-10-08T13:41:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2/134 2023-10-08T13:41:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 3/134 2023-10-08T13:41:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 4/134 2023-10-08T13:41:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 5/134 2023-10-08T13:41:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T13:41:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T13:41:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T13:41:09.969 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T13:41:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 10/134 2023-10-08T13:41:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 11/134 2023-10-08T13:41:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 12/134 2023-10-08T13:41:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 13/134 2023-10-08T13:41:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 14/134 2023-10-08T13:41:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 15/134 2023-10-08T13:41:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 16/134 2023-10-08T13:41:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 17/134 2023-10-08T13:41:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 18/134 2023-10-08T13:41:09.970 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 19/134 2023-10-08T13:41:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 20/134 2023-10-08T13:41:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 21/134 2023-10-08T13:41:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 22/134 2023-10-08T13:41:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 23/134 2023-10-08T13:41:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 24/134 2023-10-08T13:41:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 25/134 2023-10-08T13:41:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 26/134 2023-10-08T13:41:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 27/134 2023-10-08T13:41:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 28/134 2023-10-08T13:41:09.971 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 29/134 2023-10-08T13:41:09.972 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 30/134 2023-10-08T13:41:09.972 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 31/134 2023-10-08T13:41:09.980 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 32/134 2023-10-08T13:41:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 33/134 2023-10-08T13:41:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 34/134 2023-10-08T13:41:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 35/134 2023-10-08T13:41:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 36/134 2023-10-08T13:41:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 37/134 2023-10-08T13:41:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 38/134 2023-10-08T13:41:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-08T13:41:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-08T13:41:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-08T13:41:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-08T13:41:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-08T13:41:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-08T13:41:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-08T13:41:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-08T13:41:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-08T13:41:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-08T13:41:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-08T13:41:09.982 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-08T13:41:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-08T13:41:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-08T13:41:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-08T13:41:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-08T13:41:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-08T13:41:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-08T13:41:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-08T13:41:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-08T13:41:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-08T13:41:09.983 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-08T13:41:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-08T13:41:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-08T13:41:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-08T13:41:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-08T13:41:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-08T13:41:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-08T13:41:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-08T13:41:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-08T13:41:09.984 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-08T13:41:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-08T13:41:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-08T13:41:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-08T13:41:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-08T13:41:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-08T13:41:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-08T13:41:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-08T13:41:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-08T13:41:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-08T13:41:09.985 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-08T13:41:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-08T13:41:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-08T13:41:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-08T13:41:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-08T13:41:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-08T13:41:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-08T13:41:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-08T13:41:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-08T13:41:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-08T13:41:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-08T13:41:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-08T13:41:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-08T13:41:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-08T13:41:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-08T13:41:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-08T13:41:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-08T13:41:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-08T13:41:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-08T13:41:09.987 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-08T13:41:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-08T13:41:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-08T13:41:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-08T13:41:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-08T13:41:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-08T13:41:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-08T13:41:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-08T13:41:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-08T13:41:09.988 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-08T13:41:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-08T13:41:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-08T13:41:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-08T13:41:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-08T13:41:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-08T13:41:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-08T13:41:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-08T13:41:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-08T13:41:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-08T13:41:09.989 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-08T13:41:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-08T13:41:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-08T13:41:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-08T13:41:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-08T13:41:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-08T13:41:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-08T13:41:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-08T13:41:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-08T13:41:09.990 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-08T13:41:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-08T13:41:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-08T13:41:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 129/134 2023-10-08T13:41:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-08T13:41:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 131/134 2023-10-08T13:41:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T13:41:09.991 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-08T13:41:12.520 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T13:41:12.520 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:02:02 ago on Sun 08 Oct 2023 01:39:10 PM UTC. 2023-10-08T13:41:17.233 INFO:teuthology.orchestra.run.smithi057.stdout:Installed products updated. 2023-10-08T13:41:17.467 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:41:17.467 INFO:teuthology.orchestra.run.smithi057.stdout:Upgraded: 2023-10-08T13:41:17.467 INFO:teuthology.orchestra.run.smithi057.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-08T13:41:17.467 INFO:teuthology.orchestra.run.smithi057.stdout: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.468 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.469 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: jq-1.6-3.el8.x86_64 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-08T13:41:17.470 INFO:teuthology.orchestra.run.smithi057.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: libpq-13.5-1.el8.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.471 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-08T13:41:17.472 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T13:41:17.473 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-08T13:41:17.474 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T13:41:17.475 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T13:41:17.476 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T13:41:17.477 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T13:41:17.478 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T13:41:17.478 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.478 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.478 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:17.478 INFO:teuthology.orchestra.run.smithi057.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-08T13:41:17.478 INFO:teuthology.orchestra.run.smithi057.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T13:41:17.478 INFO:teuthology.orchestra.run.smithi057.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-08T13:41:17.478 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-08T13:41:17.478 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T13:41:17.479 INFO:teuthology.orchestra.run.smithi057.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-08T13:41:17.479 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:41:17.479 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2023-10-08T13:41:17.669 DEBUG:teuthology.parallel:result is None 2023-10-08T13:41:21.641 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2023-10-08T13:41:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:41:21.854 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2023-10-08T13:41:21.854 INFO:teuthology.orchestra.run.smithi099.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-08T13:41:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.855 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-10-08T13:41:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-08T13:41:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T13:41:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T13:41:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.855 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.856 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.857 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: jq-1.6-3.el8.x86_64 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.858 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: libpq-13.5-1.el8.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.859 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-08T13:41:21.860 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T13:41:21.861 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T13:41:21.862 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-08T13:41:21.863 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T13:41:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T13:41:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T13:41:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T13:41:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-08T13:41:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T13:41:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T13:41:21.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T13:41:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T13:41:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T13:41:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T13:41:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T13:41:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T13:41:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T13:41:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T13:41:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.866 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T13:41:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-08T13:41:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T13:41:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-08T13:41:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-08T13:41:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T13:41:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-08T13:41:21.868 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-08T13:41:21.868 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-10-08T13:41:22.045 DEBUG:teuthology.parallel:result is None 2023-10-08T13:41:22.046 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:41:22.266 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T13:41:22.296 INFO:teuthology.orchestra.run.smithi057.stdout:18.0.0-6639.g7b3143ba.el8 2023-10-08T13:41:22.296 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6639.g7b3143ba.el8 2023-10-08T13:41:22.297 INFO:teuthology.task.install:The correct ceph version 18.0.0-6639.g7b3143ba is installed. 2023-10-08T13:41:22.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:41:22.430 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T13:41:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:18.0.0-6639.g7b3143ba.el8 2023-10-08T13:41:22.455 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6639.g7b3143ba.el8 2023-10-08T13:41:22.455 INFO:teuthology.task.install:The correct ceph version 18.0.0-6639.g7b3143ba is installed. 2023-10-08T13:41:22.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:41:22.666 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T13:41:22.695 INFO:teuthology.orchestra.run.smithi115.stdout:18.0.0-6639.g7b3143ba.el8 2023-10-08T13:41:22.698 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6639.g7b3143ba.el8 2023-10-08T13:41:22.698 INFO:teuthology.task.install:The correct ceph version 18.0.0-6639.g7b3143ba is installed. 2023-10-08T13:41:22.698 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-08T13:41:22.698 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:41:22.698 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T13:41:22.749 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:41:22.749 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T13:41:22.792 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:41:22.793 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T13:41:22.840 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-08T13:41:22.841 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:41:22.841 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T13:41:22.875 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T13:41:22.947 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:41:22.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T13:41:22.977 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T13:41:23.046 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:41:23.046 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T13:41:23.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T13:41:23.149 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-08T13:41:23.149 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:41:23.149 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T13:41:23.182 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T13:41:23.255 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:41:23.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T13:41:23.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T13:41:23.352 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:41:23.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T13:41:23.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T13:41:23.454 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-08T13:41:23.454 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:41:23.454 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T13:41:23.487 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T13:41:23.559 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:41:23.560 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T13:41:23.589 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T13:41:23.659 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:41:23.659 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T13:41:23.690 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T13:41:23.761 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-08T13:41:23.842 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-08T13:41:23.842 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:41:23.843 INFO:tasks.cephadm:Cluster fsid is 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:41:23.843 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-08T13:41:23.843 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.57', 'mon.b': '172.21.15.99', 'mon.c': '172.21.15.115'} 2023-10-08T13:41:23.843 INFO:tasks.cephadm:First mon is mon.a on smithi057 2023-10-08T13:41:23.844 INFO:tasks.cephadm:First mgr is x 2023-10-08T13:41:23.844 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-08T13:41:23.844 DEBUG:teuthology.orchestra.run.smithi057:> sudo hostname $(hostname -s) 2023-10-08T13:41:23.877 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2023-10-08T13:41:23.910 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2023-10-08T13:41:23.943 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-08T13:41:23.943 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:41:24.081 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73096/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6639-g7b3143ba', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6639.g7b3143ba'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-07 17:16:01.199993', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20231007.154158', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-10-08T13:41:24.304 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20231007.154158, sha1 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:41:24.305 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm 2023-10-08T13:41:24.305 DEBUG:teuthology.orchestra.run.smithi057:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-08T13:41:25.537 INFO:teuthology.orchestra.run.smithi057.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 8 13:41 /home/ubuntu/cephtest/cephadm 2023-10-08T13:41:25.538 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-08T13:41:26.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 8 13:41 /home/ubuntu/cephtest/cephadm 2023-10-08T13:41:26.795 DEBUG:teuthology.orchestra.run.smithi115:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-08T13:41:27.333 INFO:teuthology.orchestra.run.smithi115.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 8 13:41 /home/ubuntu/cephtest/cephadm 2023-10-08T13:41:27.333 DEBUG:teuthology.orchestra.run.smithi057:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-08T13:41:27.356 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-08T13:41:27.378 DEBUG:teuthology.orchestra.run.smithi115:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-08T13:41:27.415 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de on all hosts... 2023-10-08T13:41:27.415 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de pull 2023-10-08T13:41:27.418 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de pull 2023-10-08T13:41:27.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de pull 2023-10-08T13:41:29.144 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T13:41:30.237 INFO:teuthology.orchestra.run.smithi057.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T13:41:31.226 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T13:41:55.933 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-10-08T13:41:55.934 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T13:41:55.934 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492", 2023-10-08T13:41:55.934 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2023-10-08T13:41:55.935 INFO:teuthology.orchestra.run.smithi115.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75b5faa12438709dee04dcec4961d708afb26e7c47ec80421dede7e5d40615e1" 2023-10-08T13:41:55.935 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-10-08T13:41:55.935 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-10-08T13:41:56.935 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:41:56.935 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T13:41:56.935 INFO:teuthology.orchestra.run.smithi057.stdout: "image_id": "09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492", 2023-10-08T13:41:56.936 INFO:teuthology.orchestra.run.smithi057.stdout: "repo_digests": [ 2023-10-08T13:41:56.936 INFO:teuthology.orchestra.run.smithi057.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75b5faa12438709dee04dcec4961d708afb26e7c47ec80421dede7e5d40615e1" 2023-10-08T13:41:56.936 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T13:41:56.936 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:41:58.211 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T13:41:58.212 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T13:41:58.212 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492", 2023-10-08T13:41:58.212 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2023-10-08T13:41:58.212 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75b5faa12438709dee04dcec4961d708afb26e7c47ec80421dede7e5d40615e1" 2023-10-08T13:41:58.212 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2023-10-08T13:41:58.212 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T13:41:58.234 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /etc/ceph 2023-10-08T13:41:58.271 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2023-10-08T13:41:58.300 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2023-10-08T13:41:58.336 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 777 /etc/ceph 2023-10-08T13:41:58.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2023-10-08T13:41:58.395 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2023-10-08T13:41:58.425 INFO:tasks.cephadm:Writing seed config... 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-08T13:41:58.427 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-08T13:41:58.428 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-08T13:41:58.429 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:41:58.429 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-08T13:41:58.449 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5f25708e-65e0-11ee-8db5-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2023-10-08T13:41:58.449 DEBUG:teuthology.orchestra.run.smithi057:mon.a> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.a.service 2023-10-08T13:41:58.492 DEBUG:teuthology.orchestra.run.smithi057:mgr.x> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.x.service 2023-10-08T13:41:58.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:-- Logs begin at Sun 2023-10-08 13:28:06 UTC. -- 2023-10-08T13:41:58.525 INFO:tasks.cephadm:Bootstrapping... 2023-10-08T13:41:58.526 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de -v bootstrap --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.57 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-08T13:41:58.555 INFO:journalctl@ceph.mgr.x.smithi057.stdout:-- Logs begin at Sun 2023-10-08 13:28:06 UTC. -- 2023-10-08T13:41:58.706 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2023-10-08T13:41:58.706 INFO:teuthology.orchestra.run.smithi057.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', '-v', 'bootstrap', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.57', '--skip-admin-label'] 2023-10-08T13:41:58.851 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.1.1 2023-10-08T13:41:58.851 INFO:teuthology.orchestra.run.smithi057.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-08T13:41:58.852 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying podman|docker is present... 2023-10-08T13:41:58.997 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.1.1 2023-10-08T13:41:58.997 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying lvm2 is present... 2023-10-08T13:41:58.997 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying time synchronization is in place... 2023-10-08T13:41:59.005 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-08T13:41:59.005 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-08T13:41:59.010 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-08T13:41:59.011 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-08T13:41:59.016 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2023-10-08T13:41:59.022 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2023-10-08T13:41:59.023 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2023-10-08T13:41:59.023 INFO:teuthology.orchestra.run.smithi057.stdout:Repeating the final host check... 2023-10-08T13:41:59.166 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 4.1.1 2023-10-08T13:41:59.166 INFO:teuthology.orchestra.run.smithi057.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-08T13:41:59.167 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl is present 2023-10-08T13:41:59.167 INFO:teuthology.orchestra.run.smithi057.stdout:lvcreate is present 2023-10-08T13:41:59.173 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-08T13:41:59.173 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-08T13:41:59.179 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-08T13:41:59.179 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-08T13:41:59.185 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2023-10-08T13:41:59.191 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2023-10-08T13:41:59.191 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2023-10-08T13:41:59.191 INFO:teuthology.orchestra.run.smithi057.stdout:Host looks OK 2023-10-08T13:41:59.191 INFO:teuthology.orchestra.run.smithi057.stdout:Cluster fsid: 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:41:59.191 INFO:teuthology.orchestra.run.smithi057.stdout:Acquiring lock 140100319439392 on /run/cephadm/5f25708e-65e0-11ee-8db5-212e2dc638e7.lock 2023-10-08T13:41:59.192 INFO:teuthology.orchestra.run.smithi057.stdout:Lock 140100319439392 acquired on /run/cephadm/5f25708e-65e0-11ee-8db5-212e2dc638e7.lock 2023-10-08T13:41:59.192 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 3300 ... 2023-10-08T13:41:59.192 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 6789 ... 2023-10-08T13:41:59.193 INFO:teuthology.orchestra.run.smithi057.stdout:Base mon IP(s) is [172.21.15.57:3300, 172.21.15.57:6789], mon addrv is [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2023-10-08T13:41:59.195 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-10-08T13:41:59.195 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.57 metric 100 2023-10-08T13:41:59.198 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-08T13:41:59.198 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-10-08T13:41:59.198 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1781sec hoplimit 64 pref medium 2023-10-08T13:41:59.201 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-08T13:41:59.201 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-08T13:41:59.201 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-08T13:41:59.201 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-10-08T13:41:59.201 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-08T13:41:59.201 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-08T13:41:59.202 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2023-10-08T13:41:59.202 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2023-10-08T13:41:59.202 INFO:teuthology.orchestra.run.smithi057.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-08T13:41:59.202 INFO:teuthology.orchestra.run.smithi057.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-08T13:41:59.203 INFO:teuthology.orchestra.run.smithi057.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T13:41:59.648 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492 2023-10-08T13:41:59.648 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T13:41:59.648 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Getting image source signatures 2023-10-08T13:41:59.648 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:1016a809630cb60ade9603521911a0725a35bac1f67b6518836ae2dd33bb1dad 2023-10-08T13:41:59.648 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:be9ed931e0f930233626f1c4d225977fbcc33e056e1a8fc67c7f63553d9455a2 2023-10-08T13:41:59.648 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying config sha256:09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492 2023-10-08T13:41:59.648 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-08T13:41:59.649 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Storing signatures 2023-10-08T13:42:00.494 INFO:teuthology.orchestra.run.smithi057.stdout:ceph: stdout ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev) 2023-10-08T13:42:00.495 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph version: ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev) 2023-10-08T13:42:00.495 INFO:teuthology.orchestra.run.smithi057.stdout:Extracting ceph user uid/gid from container image... 2023-10-08T13:42:01.748 INFO:teuthology.orchestra.run.smithi057.stdout:stat: stdout 167 167 2023-10-08T13:42:01.749 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial keys... 2023-10-08T13:42:02.502 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCqsSJlIeshCRAA2S2xzLsIrhvB+K3RYOgWrA== 2023-10-08T13:42:03.255 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCqsSJlGzuCNxAALqtxjXiubvKdWTNhfFrTGA== 2023-10-08T13:42:04.002 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCrsSJlYT2cJhAAAOArur3YTESMnsNeQDiyFg== 2023-10-08T13:42:04.002 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial monmap... 2023-10-08T13:42:04.757 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-08T13:42:04.758 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-08T13:42:04.758 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:42:04.758 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-08T13:42:04.758 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool for a [v2:172.21.15.57:3300,v1:172.21.15.57:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-08T13:42:04.758 INFO:teuthology.orchestra.run.smithi057.stdout:setting min_mon_release = pacific 2023-10-08T13:42:04.758 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: set fsid to 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:42:04.758 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-08T13:42:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:42:04.759 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mon... 2023-10-08T13:42:05.509 INFO:teuthology.orchestra.run.smithi057.stdout:create mon.a on 2023-10-08T13:42:05.663 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-08T13:42:05.761 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-08T13:42:05.882 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7.target. 2023-10-08T13:42:05.882 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7.target. 2023-10-08T13:42:06.072 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.a 2023-10-08T13:42:06.072 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.a.service: Unit ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.a.service not loaded. 2023-10-08T13:42:06.163 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7.target.wants/ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@.service. 2023-10-08T13:42:06.435 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:06 smithi057 systemd[1]: Starting Ceph mon.a for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:42:06.676 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T13:42:06.676 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2023-10-08T13:42:06.684 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T13:42:06.684 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-08T13:42:06.684 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2023-10-08T13:42:06.684 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2023-10-08T13:42:06.684 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon to start... 2023-10-08T13:42:06.685 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon... 2023-10-08T13:42:06.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:06 smithi057 podman[98941]: 2023-10-08T13:42:06.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:06 smithi057 bash[98941]: a8a9c0f2222349e5a3b1d64e19421cc538a25e363e2b46dd94aa79cb40edd225 2023-10-08T13:42:06.705 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:06 smithi057 systemd[1]: Started Ceph mon.a for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:42:06.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:06 smithi057 ceph-mon[98981]: mkfs 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:42:06.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:06 smithi057 ceph-mon[98981]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-08T13:42:07.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:07 smithi057 ceph-mon[98981]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-08T13:42:07.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:07 smithi057 ceph-mon[98981]: monmap e1: 1 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0]} removed_ranks: {} 2023-10-08T13:42:07.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:07 smithi057 ceph-mon[98981]: fsmap 2023-10-08T13:42:07.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:07 smithi057 ceph-mon[98981]: osdmap e1: 0 total, 0 up, 0 in 2023-10-08T13:42:07.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:07 smithi057 ceph-mon[98981]: mgrmap e1: no daemons active 2023-10-08T13:42:08.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:08 smithi057 ceph-mon[98981]: from='client.? 172.21.15.57:0/1385972856' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-08T13:42:09.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout cluster: 2023-10-08T13:42:09.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout id: 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:42:09.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-08T13:42:09.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:09.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout services: 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 1.45533s) 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout data: 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pgs: 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:mon is available 2023-10-08T13:42:09.240 INFO:teuthology.orchestra.run.smithi057.stdout:Assimilating anything we can from ceph.conf... 2023-10-08T13:42:10.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:09 smithi057 ceph-mon[98981]: from='client.? 172.21.15.57:0/1128927584' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-08T13:42:10.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:09 smithi057 ceph-mon[98981]: from='client.? 172.21.15.57:0/1128927584' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-08T13:42:10.288 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2023-10-08T13:42:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-08T13:42:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-08T13:42:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-08T13:42:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-08T13:42:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-08T13:42:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:Generating new minimal ceph.conf... 2023-10-08T13:42:11.219 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:10 smithi057 ceph-mon[98981]: from='client.? 172.21.15.57:0/3156935200' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:42:11.229 INFO:teuthology.orchestra.run.smithi057.stdout:Restarting the monitor... 2023-10-08T13:42:11.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:11 smithi057 systemd[1]: Stopping Ceph mon.a for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:42:11.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a[98957]: 2023-10-08T13:42:11.558+0000 7f4243eaa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T13:42:11.841 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a[98957]: 2023-10-08T13:42:11.558+0000 7f4243eaa700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-08T13:42:12.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:11 smithi057 bash[99315]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a 2023-10-08T13:42:12.400 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 systemd[1]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-08T13:42:12.400 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 systemd[1]: Stopped Ceph mon.a for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:42:12.400 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 systemd[1]: Starting Ceph mon.a for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:42:12.669 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 podman[99436]: 2023-10-08T13:42:12.669 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 bash[99436]: be6997884e4da97a4719c83a632ae990ca984e4c0897976b7dc40e58617e6bc0 2023-10-08T13:42:12.669 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: set uid:gid to 167:167 (ceph:ceph) 2023-10-08T13:42:12.669 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev), process ceph-mon, pid 7 2023-10-08T13:42:12.669 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: pidfile_write: ignore empty --pid-file 2023-10-08T13:42:12.669 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: load: jerasure load: lrc 2023-10-08T13:42:12.669 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: RocksDB version: 7.9.2 2023-10-08T13:42:12.669 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Git sha 0 2023-10-08T13:42:12.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Compile date 2023-10-07 16:15:44 2023-10-08T13:42:12.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: DB SUMMARY 2023-10-08T13:42:12.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: DB Session ID: 1FJC2ED2IND1MHNYXDXQ 2023-10-08T13:42:12.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: CURRENT file: CURRENT 2023-10-08T13:42:12.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: IDENTITY file: IDENTITY 2023-10-08T13:42:12.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-08T13:42:12.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-08T13:42:12.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 88957 ; 2023-10-08T13:42:12.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.error_if_exists: 0 2023-10-08T13:42:12.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.create_if_missing: 0 2023-10-08T13:42:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.paranoid_checks: 1 2023-10-08T13:42:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-08T13:42:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-08T13:42:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-08T13:42:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.env: 0x5628317e2da0 2023-10-08T13:42:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.fs: PosixFileSystem 2023-10-08T13:42:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.info_log: 0x562833813640 2023-10-08T13:42:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_file_opening_threads: 16 2023-10-08T13:42:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.statistics: (nil) 2023-10-08T13:42:12.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.use_fsync: 0 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_log_file_size: 0 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.keep_log_file_num: 1000 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.recycle_log_file_num: 0 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.allow_fallocate: 1 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.allow_mmap_reads: 0 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.allow_mmap_writes: 0 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.use_direct_reads: 0 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.create_missing_column_families: 0 2023-10-08T13:42:12.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.db_log_dir: 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.wal_dir: 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.advise_random_on_open: 1 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.db_write_buffer_size: 0 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.write_buffer_manager: 0x5628337cb360 2023-10-08T13:42:12.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-08T13:42:12.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-08T13:42:12.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-08T13:42:12.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.rate_limiter: (nil) 2023-10-08T13:42:12.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-08T13:42:12.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.wal_recovery_mode: 2 2023-10-08T13:42:12.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.enable_thread_tracking: 0 2023-10-08T13:42:12.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.enable_pipelined_write: 0 2023-10-08T13:42:12.674 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.unordered_write: 0 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.row_cache: None 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.wal_filter: None 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.allow_ingest_behind: 0 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.two_write_queues: 0 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.manual_wal_flush: 0 2023-10-08T13:42:12.675 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.wal_compression: 0 2023-10-08T13:42:12.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.atomic_flush: 0 2023-10-08T13:42:12.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-08T13:42:12.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-08T13:42:12.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-08T13:42:12.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.log_readahead_size: 0 2023-10-08T13:42:12.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-08T13:42:12.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.best_efforts_recovery: 0 2023-10-08T13:42:12.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-08T13:42:12.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-08T13:42:12.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.allow_data_in_errors: 0 2023-10-08T13:42:12.677 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.db_host_id: __hostname__ 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_background_jobs: 2 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_background_compactions: -1 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_subcompactions: 1 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_total_wal_size: 0 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-08T13:42:12.678 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_open_files: -1 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bytes_per_sync: 0 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_readahead_size: 0 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_background_flushes: -1 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Compression algorithms supported: 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: kZSTD supported: 0 2023-10-08T13:42:12.679 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: kXpressCompression supported: 0 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: kLZ4HCCompression supported: 1 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: kZlibCompression supported: 1 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: kSnappyCompression supported: 1 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: kLZ4Compression supported: 1 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: kBZip2Compression supported: 0 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-08T13:42:12.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.merge_operator: 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_filter: None 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_filter_factory: None 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.sst_partitioner_factory: None 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562833813740) 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks: 1 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_top_level_index_and_filter: 1 2023-10-08T13:42:12.681 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_type: 0 2023-10-08T13:42:12.682 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_index_type: 0 2023-10-08T13:42:12.682 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_shortening: 1 2023-10-08T13:42:12.682 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-08T13:42:12.682 INFO:journalctl@ceph.mon.a.smithi057.stdout: checksum: 4 2023-10-08T13:42:12.682 INFO:journalctl@ceph.mon.a.smithi057.stdout: no_block_cache: 0 2023-10-08T13:42:12.682 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache: 0x5628338691f0 2023-10-08T13:42:12.682 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_name: BinnedLRUCache 2023-10-08T13:42:12.682 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_options: 2023-10-08T13:42:12.682 INFO:journalctl@ceph.mon.a.smithi057.stdout: capacity : 536870912 2023-10-08T13:42:12.682 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_shard_bits : 4 2023-10-08T13:42:12.683 INFO:journalctl@ceph.mon.a.smithi057.stdout: strict_capacity_limit : 0 2023-10-08T13:42:12.683 INFO:journalctl@ceph.mon.a.smithi057.stdout: high_pri_pool_ratio: 0.000 2023-10-08T13:42:12.683 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_compressed: (nil) 2023-10-08T13:42:12.683 INFO:journalctl@ceph.mon.a.smithi057.stdout: persistent_cache: (nil) 2023-10-08T13:42:12.683 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size: 4096 2023-10-08T13:42:12.683 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size_deviation: 10 2023-10-08T13:42:12.683 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_restart_interval: 16 2023-10-08T13:42:12.683 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_block_restart_interval: 1 2023-10-08T13:42:12.683 INFO:journalctl@ceph.mon.a.smithi057.stdout: metadata_block_size: 4096 2023-10-08T13:42:12.683 INFO:journalctl@ceph.mon.a.smithi057.stdout: partition_filters: 0 2023-10-08T13:42:12.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: use_delta_encoding: 1 2023-10-08T13:42:12.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: filter_policy: bloomfilter 2023-10-08T13:42:12.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: whole_key_filtering: 1 2023-10-08T13:42:12.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: verify_compression: 0 2023-10-08T13:42:12.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: read_amp_bytes_per_bit: 0 2023-10-08T13:42:12.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: format_version: 5 2023-10-08T13:42:12.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_index_compression: 1 2023-10-08T13:42:12.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_align: 0 2023-10-08T13:42:12.684 INFO:journalctl@ceph.mon.a.smithi057.stdout: max_auto_readahead_size: 262144 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: prepopulate_block_cache: 0 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: initial_auto_readahead_size: 8192 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_file_reads_for_auto_readahead: 2 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.write_buffer_size: 33554432 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_write_buffer_number: 2 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compression: NoCompression 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bottommost_compression: Disabled 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.prefix_extractor: nullptr 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.num_levels: 7 2023-10-08T13:42:12.685 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-08T13:42:12.686 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compression_opts.level: 32767 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compression_opts.strategy: 0 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compression_opts.enabled: false 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-08T13:42:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.target_file_size_base: 67108864 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-08T13:42:12.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.arena_block_size: 1048576 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.disable_auto_compactions: 0 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-08T13:42:12.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.table_properties_collectors: 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.inplace_update_support: 0 2023-10-08T13:42:12.690 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.bloom_locality: 0 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.max_successive_merges: 0 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.paranoid_file_checks: 0 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.force_consistency_checks: 1 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.report_bg_io_stats: 0 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.ttl: 2592000 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-08T13:42:12.691 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.enable_blob_files: false 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.min_blob_size: 0 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.blob_file_size: 268435456 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.blob_file_starting_level: 0 2023-10-08T13:42:12.692 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 607a2531-2b1c-4820-8df6-dc03483fdda3 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696772532597425, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696772532598311, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 85150, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 276, "table_properties": {"data_size": 83242, "index_size": 254, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11130, "raw_average_key_size": 48, "raw_value_size": 76921, "raw_average_value_size": 332, "num_data_blocks": 11, "num_entries": 231, "num_filter_entries": 231, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696772532, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "607a2531-2b1c-4820-8df6-dc03483fdda3", "db_session_id": "1FJC2ED2IND1MHNYXDXQ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696772532598381, "job": 1, "event": "recovery_finished"} 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562833906000 2023-10-08T13:42:12.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: DB pointer 0x5628338f2000 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** DB Stats ** 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T13:42:12.694 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: L0 2/0 84.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: Sum 2/0 84.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T13:42:12.695 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative compaction: 0.00 GB write, 25.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval compaction: 0.00 GB write, 25.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T13:42:12.696 INFO:journalctl@ceph.mon.a.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache BinnedLRUCache@0x5628338691f0#7 capacity: 512.00 MB usage: 1.27 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: starting mon.a rank 0 at public addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] at bind addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: mon.a@-1(???) e1 preinit fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: mon.a@-1(???).mds e1 new map 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: mon.a@-1(???).mds e1 print_map 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: e1 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-08T13:42:12.697 INFO:journalctl@ceph.mon.a.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: legacy client fscid: -1 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout: No filesystems configured 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: monmap e1: 1 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0]} removed_ranks: {} 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: fsmap 2023-10-08T13:42:12.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: osdmap e1: 0 total, 0 up, 0 in 2023-10-08T13:42:12.699 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 ceph-mon[99476]: mgrmap e1: no daemons active 2023-10-08T13:42:12.699 INFO:teuthology.orchestra.run.smithi057.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-08T13:42:12.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:12 smithi057 systemd[1]: Started Ceph mon.a for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:42:13.637 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-08T13:42:13.683 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-08T13:42:13.683 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mgr... 2023-10-08T13:42:13.683 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-08T13:42:13.684 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-08T13:42:13.914 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.x 2023-10-08T13:42:13.914 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.x.service: Unit ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.x.service not loaded. 2023-10-08T13:42:14.008 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7.target.wants/ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@.service. 2023-10-08T13:42:14.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:14 smithi057 systemd[1]: Starting Ceph mgr.x for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:42:14.582 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1861238368' entity='client.admin' 2023-10-08T13:42:14.583 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:14 smithi057 podman[99711]: 2023-10-08T13:42:14.618 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T13:42:14.618 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2023-10-08T13:42:14.626 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T13:42:14.626 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-08T13:42:14.626 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2023-10-08T13:42:14.626 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2023-10-08T13:42:14.634 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T13:42:14.634 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2023-10-08T13:42:14.642 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T13:42:14.642 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-08T13:42:14.642 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2023-10-08T13:42:14.643 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-08T13:42:14.643 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr to start... 2023-10-08T13:42:14.643 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr... 2023-10-08T13:42:14.841 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:14 smithi057 bash[99711]: 3a499252d3bec00770d5490c26679c74b37f7c5d2d58a1a20afb5d067ff59523 2023-10-08T13:42:14.841 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:14 smithi057 systemd[1]: Started Ceph mgr.x for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:42:14.841 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:14.730+0000 7fbdc8283200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T13:42:14.842 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:14.828+0000 7fbdc8283200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T13:42:15.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:14.945+0000 7fbdc8283200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T13:42:15.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:15.059+0000 7fbdc8283200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T13:42:15.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:15.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-08T13:42:15.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:42:15.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T13:42:15.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T13:42:15.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T13:42:15.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T13:42:15.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:15.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T13:42:15.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T13:42:15.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2023-10-08T13:42:15.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:15.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T13:42:15.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T13:42:15.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:15.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-10-08T13:42:15.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T13:42:15.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:15.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T13:42:15.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T13:42:15.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:15.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T13:42:15.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:15.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T13:42:15.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T13:42:15.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T13:42:15.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T13:42:15.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T13:42:15.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T13:42:15.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:15.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T13:42:15.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T13:42:15.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T13:42:15.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T13:42:15.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T13:42:15.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T13:42:15.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T13:42:15.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T13:42:15.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T13:42:15.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:15.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T13:42:15.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:15.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T13:42:15.601 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T13:42:15.601 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:15.601 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T13:42:15.601 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-08T13:42:15.601 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T13:42:15.601 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T13:42:15.601 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T13:42:15.602 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T13:42:15.602 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T13:42:15.602 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:15.602 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T13:42:15.602 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:15.602 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T13:42:15.602 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:15.602 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T13:42:06.689805+0000", 2023-10-08T13:42:15.603 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T13:42:15.603 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:15.603 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T13:42:15.603 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-08T13:42:15.603 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (1/15)... 2023-10-08T13:42:15.670 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:15 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4269687987' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T13:42:15.670 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:15.270+0000 7fbdc8283200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T13:42:15.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:15.756+0000 7fbdc8283200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T13:42:15.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:15.915+0000 7fbdc8283200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T13:42:16.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:16.023+0000 7fbdc8283200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T13:42:16.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:16.138+0000 7fbdc8283200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T13:42:16.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:16.649+0000 7fbdc8283200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T13:42:17.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:17 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:17.444+0000 7fbdc8283200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T13:42:17.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:17 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:17.542+0000 7fbdc8283200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T13:42:18.545 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:18.545 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-08T13:42:18.545 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:42:18.545 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T13:42:18.545 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T13:42:18.546 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T13:42:18.546 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T13:42:18.546 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:18.546 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T13:42:18.546 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T13:42:18.546 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2023-10-08T13:42:18.546 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:18.546 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:18.547 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T13:42:18.548 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T13:42:18.548 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T13:42:18.548 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T13:42:18.548 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T13:42:18.548 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T13:42:18.548 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:18.548 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T13:42:18.548 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T13:42:18.548 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T13:42:18.548 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T13:42:18.549 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T13:42:18.549 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T13:42:18.549 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T13:42:18.549 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T13:42:18.549 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T13:42:18.549 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:18.549 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T13:42:18.549 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:18.549 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T13:42:18.549 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T13:42:18.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:18.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T13:42:18.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-08T13:42:18.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T13:42:18.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T13:42:18.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T13:42:18.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T13:42:18.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T13:42:18.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:18.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T13:42:18.551 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:18.551 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T13:42:18.551 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:18.551 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T13:42:06.689805+0000", 2023-10-08T13:42:18.551 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T13:42:18.551 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:18.551 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T13:42:18.551 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-08T13:42:18.551 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (2/15)... 2023-10-08T13:42:18.881 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:18 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1971556578' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T13:42:18.881 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:18.545+0000 7fbdc8283200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:42:18.882 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:18.658+0000 7fbdc8283200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T13:42:18.882 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:18.880+0000 7fbdc8283200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:42:19.486 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:19 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:19.232+0000 7fbdc8283200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T13:42:19.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:19 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:19.485+0000 7fbdc8283200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T13:42:19.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:19 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:19.668+0000 7fbdc8283200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T13:42:20.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:20 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:20.001+0000 7fbdc8283200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T13:42:21.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:20 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:20.845+0000 7fbdc8283200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T13:42:21.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:21 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:21.003+0000 7fbdc8283200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T13:42:21.549 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:21 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:21.250+0000 7fbdc8283200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T13:42:21.549 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:21 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:21.386+0000 7fbdc8283200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T13:42:21.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:21 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:21.668+0000 7fbdc8283200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T13:42:22.725 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:22 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1463646451' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T13:42:22.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:22 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:22.725+0000 7fbdc8283200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T13:42:23.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: Activating manager daemon x 2023-10-08T13:42:23.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: mgrmap e2: x(active, starting, since 0.00327106s) 2023-10-08T13:42:23.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: from='mgr.14100 172.21.15.57:0/2262455225' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T13:42:23.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: from='mgr.14100 172.21.15.57:0/2262455225' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T13:42:23.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: from='mgr.14100 172.21.15.57:0/2262455225' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T13:42:23.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: from='mgr.14100 172.21.15.57:0/2262455225' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:42:23.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: from='mgr.14100 172.21.15.57:0/2262455225' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T13:42:23.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: Manager daemon x is now available 2023-10-08T13:42:23.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: from='mgr.14100 172.21.15.57:0/2262455225' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T13:42:23.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: from='mgr.14100 172.21.15.57:0/2262455225' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T13:42:23.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: from='mgr.14100 172.21.15.57:0/2262455225' entity='mgr.x' 2023-10-08T13:42:23.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: from='mgr.14100 172.21.15.57:0/2262455225' entity='mgr.x' 2023-10-08T13:42:23.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:23 smithi057 ceph-mon[99476]: from='mgr.14100 172.21.15.57:0/2262455225' entity='mgr.x' 2023-10-08T13:42:24.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:24 smithi057 ceph-mon[99476]: mgrmap e3: x(active, since 1.00737s) 2023-10-08T13:42:24.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:24.998 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:24.999 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T13:42:25.000 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T13:42:25.001 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T13:42:25.007 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T13:42:25.007 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T13:42:25.008 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:25.008 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T13:42:25.008 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T13:42:25.008 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T13:42:25.008 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T13:42:25.008 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T13:42:25.009 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T13:42:25.009 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T13:42:25.009 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T13:42:25.009 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T13:42:25.009 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:25.010 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T13:42:25.010 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:25.010 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T13:42:25.010 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T13:42:25.011 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:25.011 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T13:42:25.011 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-08T13:42:25.011 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T13:42:25.011 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T13:42:25.012 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T13:42:25.012 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T13:42:25.012 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T13:42:25.012 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:25.013 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T13:42:25.013 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:25.013 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T13:42:25.013 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:25.013 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T13:42:06.689805+0000", 2023-10-08T13:42:25.013 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T13:42:25.013 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:25.013 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T13:42:25.013 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-08T13:42:25.014 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (3/15)... 2023-10-08T13:42:27.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:26 smithi057 ceph-mon[99476]: mgrmap e4: x(active, since 3s) 2023-10-08T13:42:29.405 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:29 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2135193771' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T13:42:29.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2023-10-08T13:42:29.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:29.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T13:42:29.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T13:42:29.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T13:42:29.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T13:42:29.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T13:42:29.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T13:42:29.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:29.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T13:42:29.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T13:42:29.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:29.463 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T13:42:29.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T13:42:29.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T13:42:06.689805+0000", 2023-10-08T13:42:29.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T13:42:29.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2023-10-08T13:42:29.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T13:42:29.464 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-08T13:42:29.464 INFO:teuthology.orchestra.run.smithi057.stdout:mgr is available 2023-10-08T13:42:30.652 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:30 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/167786589' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-08T13:42:30.701 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:30.701 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2023-10-08T13:42:30.701 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:42:30.701 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-08T13:42:30.701 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-08T13:42:30.702 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-08T13:42:30.703 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-08T13:42:30.703 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling cephadm module... 2023-10-08T13:42:32.050 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:31 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3334358853' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-08T13:42:32.050 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:31 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: ignoring --setuser ceph since I am not root 2023-10-08T13:42:32.050 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:31 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: ignoring --setgroup ceph since I am not root 2023-10-08T13:42:32.050 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:31 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:31.954+0000 7f93e3009200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T13:42:32.051 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:32 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:32.050+0000 7f93e3009200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T13:42:32.380 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:32 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:32.161+0000 7f93e3009200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T13:42:32.380 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:32 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:32.276+0000 7f93e3009200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T13:42:32.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:32 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:32.490+0000 7f93e3009200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T13:42:33.127 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:32 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:32.972+0000 7f93e3009200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T13:42:33.128 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:32 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3334358853' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-08T13:42:33.128 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:32 smithi057 ceph-mon[99476]: mgrmap e5: x(active, since 9s) 2023-10-08T13:42:33.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:33 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:33.126+0000 7f93e3009200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T13:42:33.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:33 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:33.241+0000 7f93e3009200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T13:42:33.488 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:33 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:33.359+0000 7f93e3009200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T13:42:33.587 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-08T13:42:33.588 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-08T13:42:33.588 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-08T13:42:33.588 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-08T13:42:33.588 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-08T13:42:33.588 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-08T13:42:33.588 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2023-10-08T13:42:33.588 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 5... 2023-10-08T13:42:33.880 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:33 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/10929112' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-08T13:42:34.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:33 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:33.879+0000 7f93e3009200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T13:42:34.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:34 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:34.671+0000 7f93e3009200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T13:42:34.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:34 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:34.769+0000 7f93e3009200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T13:42:36.074 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:35 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:35.741+0000 7f93e3009200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:42:36.074 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:35 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:35.851+0000 7f93e3009200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T13:42:36.074 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:36.073+0000 7f93e3009200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:42:36.676 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:36.422+0000 7f93e3009200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T13:42:36.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:36.675+0000 7f93e3009200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T13:42:36.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:36.857+0000 7f93e3009200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T13:42:37.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:37 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:37.187+0000 7f93e3009200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T13:42:38.415 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:38.011+0000 7f93e3009200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T13:42:38.415 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:38.168+0000 7f93e3009200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T13:42:38.415 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:38.414+0000 7f93e3009200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T13:42:38.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:38.551+0000 7f93e3009200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T13:42:39.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:38.824+0000 7f93e3009200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T13:42:40.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:39.869+0000 7f93e3009200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T13:42:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: Active manager daemon x restarted 2023-10-08T13:42:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: Activating manager daemon x 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: osdmap e2: 0 total, 0 up, 0 in 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: mgrmap e6: x(active, starting, since 0.00485319s) 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: Manager daemon x is now available 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:42:40.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:39 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:42:41.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:40 smithi057 ceph-mon[99476]: Found migration_current of "None". Setting to last migration. 2023-10-08T13:42:41.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:40 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:42:41.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:40 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T13:42:41.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:40 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T13:42:41.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:40 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:41.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:40 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:41.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:40 smithi057 ceph-mon[99476]: mgrmap e7: x(active, since 1.00785s) 2023-10-08T13:42:41.245 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-08T13:42:41.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-08T13:42:41.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-08T13:42:41.246 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-08T13:42:41.246 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 5 is available 2023-10-08T13:42:41.246 INFO:teuthology.orchestra.run.smithi057.stdout:Setting orchestrator backend to cephadm... 2023-10-08T13:42:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:41 smithi057 ceph-mon[99476]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-08T13:42:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:41 smithi057 ceph-mon[99476]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-08T13:42:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:41 smithi057 ceph-mon[99476]: [08/Oct/2023:13:42:41] ENGINE Bus STARTING 2023-10-08T13:42:42.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:41 smithi057 ceph-mon[99476]: [08/Oct/2023:13:42:41] ENGINE Serving on http://172.21.15.57:8765 2023-10-08T13:42:42.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:41 smithi057 ceph-mon[99476]: [08/Oct/2023:13:42:41] ENGINE Serving on https://172.21.15.57:7150 2023-10-08T13:42:42.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:41 smithi057 ceph-mon[99476]: [08/Oct/2023:13:42:41] ENGINE Bus STARTED 2023-10-08T13:42:42.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:41 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:42:43.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:42 smithi057 ceph-mon[99476]: mgrmap e8: x(active, since 2s) 2023-10-08T13:42:43.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:42 smithi057 ceph-mon[99476]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:42:43.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:42 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:43.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:42 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:42:43.827 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-08T13:42:43.827 INFO:teuthology.orchestra.run.smithi057.stdout:Generating ssh key... 2023-10-08T13:42:44.064 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:43 smithi057 ceph-mon[99476]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:42:45.227 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-mon[99476]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:42:45.227 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-mon[99476]: Generating ssh key... 2023-10-08T13:42:45.227 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:45.227 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:45.227 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: Generating public/private rsa key pair. 2023-10-08T13:42:45.227 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: Your identification has been saved in /tmp/tmpyghqa1ue/key. 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: Your public key has been saved in /tmp/tmpyghqa1ue/key.pub. 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: The key fingerprint is: 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: SHA256:E0AgzmAIRnRxBiAfMiIBatuhHicfimG+47AU9A78vgs ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: The key's randomart image is: 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: +---[RSA 3072]----+ 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: |^B+++=o | 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: |OB.oo . | 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: |.o+. . | 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: |+ = . . | 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: |.X = S | 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: |=.@ . . | 2023-10-08T13:42:45.228 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: |oE + | 2023-10-08T13:42:45.229 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: |oo+ | 2023-10-08T13:42:45.229 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: |oo.+o | 2023-10-08T13:42:45.229 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:44 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: +----[SHA256]-----+ 2023-10-08T13:42:46.510 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjPaW/0hv3CAvy52fIkmPKhC632/NYnYvu3OenILftA5V0m2NR++n8IKNFgg+GjiNCe2LiGdEmzw+cDPIK0gz3cB3PC4jsDjODguQD+NuQ81kFXITIDAkcuegXTtGQtO8W4XoAh+INt+gwF8/2RV8WIaT6VJbBaZijuqBLdjDn15wWhhEgFFlirRC6MI9s0yOShBjWHDx65vH7Q9gcLzW9o0gaCqSgpSbP0TsrxBrWUeTkTOwE4E+RnbiLQxflQky5XUBEivVoQQuWutxiP3hkXpPKs5uUeaYr/1RxE7zvJeWEfcaXirskAzxtf4gid9g7ofb2RSoCqzEMcytWhvNi+L/5cbQwJ/eT/cDMBlrF2FR7VmVdfBMOCZuIu62pisy/NBiYSTBS+a4gafQ88q+PnuWjaqkX96/mBjLPkALYgFjUgcQOMpKWmAIT0Tfc2XS0kGogr5Qq2djBgB7edXvtVuQt03otvPdckdcLacsZKejsNSGpQXhyaJEM5dmoLw8= ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:42:46.510 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-08T13:42:46.510 INFO:teuthology.orchestra.run.smithi057.stdout:Adding key to root@localhost authorized_keys... 2023-10-08T13:42:46.510 INFO:teuthology.orchestra.run.smithi057.stdout:Adding host smithi057... 2023-10-08T13:42:47.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:46 smithi057 ceph-mon[99476]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:42:48.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:48 smithi057 ceph-mon[99476]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:42:49.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:49 smithi057 ceph-mon[99476]: Deploying cephadm binary to smithi057 2023-10-08T13:42:50.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Added host 'smithi057' with addr '172.21.15.57' 2023-10-08T13:42:50.659 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying unmanaged mon service... 2023-10-08T13:42:51.590 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:51 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:51.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:51 smithi057 ceph-mon[99476]: Added host smithi057 2023-10-08T13:42:51.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:51 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:42:52.001 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-08T13:42:52.002 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying unmanaged mgr service... 2023-10-08T13:42:52.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:52 smithi057 ceph-mon[99476]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:42:52.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:52 smithi057 ceph-mon[99476]: Saving service mon spec with placement count:5 2023-10-08T13:42:52.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:52 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:53.345 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-08T13:42:53.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:53 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:53.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:53 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:53.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:53 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:54.898 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:54 smithi057 ceph-mon[99476]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:42:54.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:54 smithi057 ceph-mon[99476]: Saving service mgr spec with placement count:2 2023-10-08T13:42:54.899 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:54 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/670519806' entity='client.admin' 2023-10-08T13:42:56.232 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling the dashboard module... 2023-10-08T13:42:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:55 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:55 smithi057 ceph-mon[99476]: from='mgr.14120 172.21.15.57:0/3702688372' entity='mgr.x' 2023-10-08T13:42:57.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:56 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2681007621' entity='client.admin' 2023-10-08T13:42:58.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:57 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4053890535' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-08T13:42:58.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:57 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: ignoring --setuser ceph since I am not root 2023-10-08T13:42:58.238 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:57 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: ignoring --setgroup ceph since I am not root 2023-10-08T13:42:58.238 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:58 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:58.139+0000 7fdf6ab72200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T13:42:58.238 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:58 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:58.236+0000 7fdf6ab72200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T13:42:58.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:58 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:58.346+0000 7fdf6ab72200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T13:42:58.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:58 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:58.466+0000 7fdf6ab72200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T13:42:58.927 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:58 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:58.681+0000 7fdf6ab72200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T13:42:59.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:58 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4053890535' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-08T13:42:59.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:58 smithi057 ceph-mon[99476]: mgrmap e9: x(active, since 18s) 2023-10-08T13:42:59.238 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:59 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:59.161+0000 7fdf6ab72200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T13:42:59.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:59 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:59.313+0000 7fdf6ab72200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T13:42:59.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:59 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:59.424+0000 7fdf6ab72200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T13:42:59.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:42:59 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:42:59.545+0000 7fdf6ab72200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T13:42:59.821 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-08T13:42:59.821 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-08T13:42:59.821 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-08T13:42:59.821 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-08T13:42:59.821 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-08T13:42:59.821 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-08T13:42:59.821 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2023-10-08T13:42:59.821 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 9... 2023-10-08T13:43:00.048 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:42:59 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1269306358' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-08T13:43:00.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:00.068+0000 7fdf6ab72200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T13:43:01.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:00.855+0000 7fdf6ab72200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T13:43:01.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:00.954+0000 7fdf6ab72200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T13:43:02.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:01 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:01.929+0000 7fdf6ab72200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:43:02.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:02 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:02.041+0000 7fdf6ab72200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T13:43:02.618 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:02 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:02.264+0000 7fdf6ab72200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:43:02.871 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:02 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:02.617+0000 7fdf6ab72200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T13:43:03.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:02 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:02.870+0000 7fdf6ab72200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T13:43:03.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:03 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:03.053+0000 7fdf6ab72200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T13:43:03.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:03 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:03.386+0000 7fdf6ab72200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T13:43:04.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:04 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:04.211+0000 7fdf6ab72200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T13:43:04.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:04 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:04.369+0000 7fdf6ab72200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T13:43:04.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:04 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:04.617+0000 7fdf6ab72200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T13:43:04.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:04 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:04.755+0000 7fdf6ab72200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T13:43:05.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:05 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:05.030+0000 7fdf6ab72200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T13:43:06.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:43:06.079+0000 7fdf6ab72200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T13:43:06.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: Active manager daemon x restarted 2023-10-08T13:43:06.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: Activating manager daemon x 2023-10-08T13:43:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: osdmap e3: 0 total, 0 up, 0 in 2023-10-08T13:43:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: mgrmap e10: x(active, starting, since 0.0082901s) 2023-10-08T13:43:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:43:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T13:43:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T13:43:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T13:43:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T13:43:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: Manager daemon x is now available 2023-10-08T13:43:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:06 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:07.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:07 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:07.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:07 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T13:43:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:07 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T13:43:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:07 smithi057 ceph-mon[99476]: mgrmap e11: x(active, since 1.01509s) 2023-10-08T13:43:07.574 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2023-10-08T13:43:07.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-08T13:43:07.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-08T13:43:07.575 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2023-10-08T13:43:07.575 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 9 is available 2023-10-08T13:43:07.575 INFO:teuthology.orchestra.run.smithi057.stdout:Generating a dashboard self-signed certificate... 2023-10-08T13:43:09.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:08 smithi057 ceph-mon[99476]: [08/Oct/2023:13:43:07] ENGINE Bus STARTING 2023-10-08T13:43:09.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:08 smithi057 ceph-mon[99476]: [08/Oct/2023:13:43:07] ENGINE Serving on https://172.21.15.57:7150 2023-10-08T13:43:09.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:08 smithi057 ceph-mon[99476]: [08/Oct/2023:13:43:07] ENGINE Serving on http://172.21.15.57:8765 2023-10-08T13:43:09.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:08 smithi057 ceph-mon[99476]: [08/Oct/2023:13:43:07] ENGINE Bus STARTED 2023-10-08T13:43:09.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:08 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:09.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:08 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:09.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:08 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:09.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:08 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:09.169 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-08T13:43:09.169 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial admin user... 2023-10-08T13:43:10.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:09 smithi057 ceph-mon[99476]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:43:10.091 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:09 smithi057 ceph-mon[99476]: mgrmap e12: x(active, since 2s) 2023-10-08T13:43:10.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$SCDHF.hObMLtEfL2EZuIO.DICTvb0N0SuWYf6zp7Y5VwdOu/d1qJ6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1696772590, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-08T13:43:10.712 INFO:teuthology.orchestra.run.smithi057.stdout:Fetching dashboard port number... 2023-10-08T13:43:11.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:11 smithi057 ceph-mon[99476]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:43:11.587 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:11 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:11.955 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 8443 2023-10-08T13:43:11.965 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T13:43:11.965 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2023-10-08T13:43:11.972 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T13:43:11.972 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2023-10-08T13:43:11.973 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2023-10-08T13:43:11.973 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-08T13:43:11.974 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph Dashboard is now available at: 2023-10-08T13:43:11.974 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:11.974 INFO:teuthology.orchestra.run.smithi057.stdout: URL: https://smithi057:8443/ 2023-10-08T13:43:11.974 INFO:teuthology.orchestra.run.smithi057.stdout: User: admin 2023-10-08T13:43:11.974 INFO:teuthology.orchestra.run.smithi057.stdout: Password: vv7m4a5815 2023-10-08T13:43:11.975 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:11.975 INFO:teuthology.orchestra.run.smithi057.stdout:Saving cluster configuration to /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config directory 2023-10-08T13:43:11.975 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling autotune for osd_memory_target 2023-10-08T13:43:12.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:12 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3283520367' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-08T13:43:13.465 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:13 smithi057 ceph-mon[99476]: mgrmap e13: x(active, since 6s) 2023-10-08T13:43:14.661 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-08T13:43:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-08T13:43:14.662 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:14.662 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-08T13:43:14.662 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:Or, if you are only running a single cluster on this host: 2023-10-08T13:43:14.662 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:14.662 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-08T13:43:14.662 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:14.662 INFO:teuthology.orchestra.run.smithi057.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-08T13:43:14.662 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:14.663 INFO:teuthology.orchestra.run.smithi057.stdout: ceph telemetry on 2023-10-08T13:43:14.663 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:For more information see: 2023-10-08T13:43:14.663 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:14.663 INFO:teuthology.orchestra.run.smithi057.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-08T13:43:14.663 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:14.663 INFO:teuthology.orchestra.run.smithi057.stdout:Bootstrap complete. 2023-10-08T13:43:14.699 INFO:tasks.cephadm:Fetching config... 2023-10-08T13:43:14.699 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:43:14.699 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-08T13:43:14.716 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-08T13:43:14.716 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:43:14.716 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-08T13:43:14.776 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-08T13:43:14.776 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:43:14.776 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-08T13:43:14.845 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-08T13:43:14.845 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:43:14.845 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-08T13:43:14.902 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-08T13:43:14.903 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjPaW/0hv3CAvy52fIkmPKhC632/NYnYvu3OenILftA5V0m2NR++n8IKNFgg+GjiNCe2LiGdEmzw+cDPIK0gz3cB3PC4jsDjODguQD+NuQ81kFXITIDAkcuegXTtGQtO8W4XoAh+INt+gwF8/2RV8WIaT6VJbBaZijuqBLdjDn15wWhhEgFFlirRC6MI9s0yOShBjWHDx65vH7Q9gcLzW9o0gaCqSgpSbP0TsrxBrWUeTkTOwE4E+RnbiLQxflQky5XUBEivVoQQuWutxiP3hkXpPKs5uUeaYr/1RxE7zvJeWEfcaXirskAzxtf4gid9g7ofb2RSoCqzEMcytWhvNi+L/5cbQwJ/eT/cDMBlrF2FR7VmVdfBMOCZuIu62pisy/NBiYSTBS+a4gafQ88q+PnuWjaqkX96/mBjLPkALYgFjUgcQOMpKWmAIT0Tfc2XS0kGogr5Qq2djBgB7edXvtVuQt03otvPdckdcLacsZKejsNSGpQXhyaJEM5dmoLw8= ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-08T13:43:14.980 INFO:teuthology.orchestra.run.smithi057.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjPaW/0hv3CAvy52fIkmPKhC632/NYnYvu3OenILftA5V0m2NR++n8IKNFgg+GjiNCe2LiGdEmzw+cDPIK0gz3cB3PC4jsDjODguQD+NuQ81kFXITIDAkcuegXTtGQtO8W4XoAh+INt+gwF8/2RV8WIaT6VJbBaZijuqBLdjDn15wWhhEgFFlirRC6MI9s0yOShBjWHDx65vH7Q9gcLzW9o0gaCqSgpSbP0TsrxBrWUeTkTOwE4E+RnbiLQxflQky5XUBEivVoQQuWutxiP3hkXpPKs5uUeaYr/1RxE7zvJeWEfcaXirskAzxtf4gid9g7ofb2RSoCqzEMcytWhvNi+L/5cbQwJ/eT/cDMBlrF2FR7VmVdfBMOCZuIu62pisy/NBiYSTBS+a4gafQ88q+PnuWjaqkX96/mBjLPkALYgFjUgcQOMpKWmAIT0Tfc2XS0kGogr5Qq2djBgB7edXvtVuQt03otvPdckdcLacsZKejsNSGpQXhyaJEM5dmoLw8= ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:43:14.995 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjPaW/0hv3CAvy52fIkmPKhC632/NYnYvu3OenILftA5V0m2NR++n8IKNFgg+GjiNCe2LiGdEmzw+cDPIK0gz3cB3PC4jsDjODguQD+NuQ81kFXITIDAkcuegXTtGQtO8W4XoAh+INt+gwF8/2RV8WIaT6VJbBaZijuqBLdjDn15wWhhEgFFlirRC6MI9s0yOShBjWHDx65vH7Q9gcLzW9o0gaCqSgpSbP0TsrxBrWUeTkTOwE4E+RnbiLQxflQky5XUBEivVoQQuWutxiP3hkXpPKs5uUeaYr/1RxE7zvJeWEfcaXirskAzxtf4gid9g7ofb2RSoCqzEMcytWhvNi+L/5cbQwJ/eT/cDMBlrF2FR7VmVdfBMOCZuIu62pisy/NBiYSTBS+a4gafQ88q+PnuWjaqkX96/mBjLPkALYgFjUgcQOMpKWmAIT0Tfc2XS0kGogr5Qq2djBgB7edXvtVuQt03otvPdckdcLacsZKejsNSGpQXhyaJEM5dmoLw8= ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-08T13:43:15.043 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjPaW/0hv3CAvy52fIkmPKhC632/NYnYvu3OenILftA5V0m2NR++n8IKNFgg+GjiNCe2LiGdEmzw+cDPIK0gz3cB3PC4jsDjODguQD+NuQ81kFXITIDAkcuegXTtGQtO8W4XoAh+INt+gwF8/2RV8WIaT6VJbBaZijuqBLdjDn15wWhhEgFFlirRC6MI9s0yOShBjWHDx65vH7Q9gcLzW9o0gaCqSgpSbP0TsrxBrWUeTkTOwE4E+RnbiLQxflQky5XUBEivVoQQuWutxiP3hkXpPKs5uUeaYr/1RxE7zvJeWEfcaXirskAzxtf4gid9g7ofb2RSoCqzEMcytWhvNi+L/5cbQwJ/eT/cDMBlrF2FR7VmVdfBMOCZuIu62pisy/NBiYSTBS+a4gafQ88q+PnuWjaqkX96/mBjLPkALYgFjUgcQOMpKWmAIT0Tfc2XS0kGogr5Qq2djBgB7edXvtVuQt03otvPdckdcLacsZKejsNSGpQXhyaJEM5dmoLw8= ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:43:15.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjPaW/0hv3CAvy52fIkmPKhC632/NYnYvu3OenILftA5V0m2NR++n8IKNFgg+GjiNCe2LiGdEmzw+cDPIK0gz3cB3PC4jsDjODguQD+NuQ81kFXITIDAkcuegXTtGQtO8W4XoAh+INt+gwF8/2RV8WIaT6VJbBaZijuqBLdjDn15wWhhEgFFlirRC6MI9s0yOShBjWHDx65vH7Q9gcLzW9o0gaCqSgpSbP0TsrxBrWUeTkTOwE4E+RnbiLQxflQky5XUBEivVoQQuWutxiP3hkXpPKs5uUeaYr/1RxE7zvJeWEfcaXirskAzxtf4gid9g7ofb2RSoCqzEMcytWhvNi+L/5cbQwJ/eT/cDMBlrF2FR7VmVdfBMOCZuIu62pisy/NBiYSTBS+a4gafQ88q+PnuWjaqkX96/mBjLPkALYgFjUgcQOMpKWmAIT0Tfc2XS0kGogr5Qq2djBgB7edXvtVuQt03otvPdckdcLacsZKejsNSGpQXhyaJEM5dmoLw8= ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-08T13:43:15.102 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjPaW/0hv3CAvy52fIkmPKhC632/NYnYvu3OenILftA5V0m2NR++n8IKNFgg+GjiNCe2LiGdEmzw+cDPIK0gz3cB3PC4jsDjODguQD+NuQ81kFXITIDAkcuegXTtGQtO8W4XoAh+INt+gwF8/2RV8WIaT6VJbBaZijuqBLdjDn15wWhhEgFFlirRC6MI9s0yOShBjWHDx65vH7Q9gcLzW9o0gaCqSgpSbP0TsrxBrWUeTkTOwE4E+RnbiLQxflQky5XUBEivVoQQuWutxiP3hkXpPKs5uUeaYr/1RxE7zvJeWEfcaXirskAzxtf4gid9g7ofb2RSoCqzEMcytWhvNi+L/5cbQwJ/eT/cDMBlrF2FR7VmVdfBMOCZuIu62pisy/NBiYSTBS+a4gafQ88q+PnuWjaqkX96/mBjLPkALYgFjUgcQOMpKWmAIT0Tfc2XS0kGogr5Qq2djBgB7edXvtVuQt03otvPdckdcLacsZKejsNSGpQXhyaJEM5dmoLw8= ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:43:15.115 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-08T13:43:15.685 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:43:15.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:15 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1804952258' entity='client.admin' 2023-10-08T13:43:17.715 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:17 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:17.716 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:17 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:17.716 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:17 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:43:17.716 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:17 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:17.716 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:17 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:17.716 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:17 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:17.769 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-08T13:43:17.769 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-08T13:43:18.405 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:43:18.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:18 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1839985899' entity='client.admin' 2023-10-08T13:43:18.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:18 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:18.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:18 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:20.360 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi099 2023-10-08T13:43:20.360 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:43:20.361 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.conf 2023-10-08T13:43:20.381 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:43:20.381 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:43:20.440 INFO:tasks.cephadm:Adding host smithi099 to orchestrator... 2023-10-08T13:43:20.440 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi099 2023-10-08T13:43:21.015 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:43:21.205 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:20 smithi057 ceph-mon[99476]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:43:21.205 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:20 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:21.205 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:20 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:21.205 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:20 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:21.205 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:20 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:43:21.205 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:20 smithi057 ceph-mon[99476]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-08T13:43:22.083 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:21 smithi057 ceph-mon[99476]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:43:22.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:21 smithi057 ceph-mon[99476]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:43:22.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:21 smithi057 ceph-mon[99476]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:43:22.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:21 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:22.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:21 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:22.085 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:21 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:23.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:23 smithi057 ceph-mon[99476]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:43:23.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:23 smithi057 ceph-mon[99476]: Deploying cephadm binary to smithi099 2023-10-08T13:43:25.390 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi099' with addr '172.21.15.99' 2023-10-08T13:43:25.783 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-08T13:43:26.348 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:43:26.729 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:26 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:26.729 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:26 smithi057 ceph-mon[99476]: Added host smithi099 2023-10-08T13:43:26.729 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:26 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:26.729 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:26 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:27.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:27 smithi057 ceph-mon[99476]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:27.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:27 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:27.822 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:27.823 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}] 2023-10-08T13:43:28.214 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi115 2023-10-08T13:43:28.214 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:43:28.214 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.conf 2023-10-08T13:43:28.235 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:43:28.235 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:43:28.293 INFO:tasks.cephadm:Adding host smithi115 to orchestrator... 2023-10-08T13:43:28.294 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi115 2023-10-08T13:43:28.858 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:43:29.062 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:28 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:29.062 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:28 smithi057 ceph-mon[99476]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:43:30.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:29 smithi057 ceph-mon[99476]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:43:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: Deploying cephadm binary to smithi115 2023-10-08T13:43:32.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:32.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:32.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:32.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:43:32.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:32.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:43:32.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:31 smithi057 ceph-mon[99476]: Updating smithi099:/etc/ceph/ceph.conf 2023-10-08T13:43:33.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:32 smithi057 ceph-mon[99476]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:43:33.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:32 smithi057 ceph-mon[99476]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:43:33.256 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi115' with addr '172.21.15.115' 2023-10-08T13:43:33.626 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-08T13:43:33.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:33 smithi057 ceph-mon[99476]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:33 smithi057 ceph-mon[99476]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:43:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:33 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:33 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:33 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:33 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:33 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:33 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:34.202 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:43:34.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:34 smithi057 ceph-mon[99476]: Added host smithi115 2023-10-08T13:43:34.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:34 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:35.662 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:43:35.662 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}] 2023-10-08T13:43:35.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:35 smithi057 ceph-mon[99476]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:35.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:35 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:36.047 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-08T13:43:36.048 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd crush tunables default 2023-10-08T13:43:36.609 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:43:37.071 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:36 smithi057 ceph-mon[99476]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:43:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:37 smithi057 ceph-mon[99476]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:38.860 INFO:teuthology.orchestra.run.smithi057.stderr:adjusted tunables profile to default 2023-10-08T13:43:39.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:38 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/879592778' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-08T13:43:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:38 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:38 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:38 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:38 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:38 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:43:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:38 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:38 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:43:39.276 INFO:tasks.cephadm:Adding mon.a on smithi057 2023-10-08T13:43:39.277 INFO:tasks.cephadm:Adding mon.b on smithi099 2023-10-08T13:43:39.277 INFO:tasks.cephadm:Adding mon.c on smithi115 2023-10-08T13:43:39.277 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch apply mon '3;smithi057:172.21.15.57=a;smithi099:172.21.15.99=b;smithi115:172.21.15.115=c' 2023-10-08T13:43:39.775 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:43:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:39 smithi057 ceph-mon[99476]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:39 smithi057 ceph-mon[99476]: Updating smithi115:/etc/ceph/ceph.conf 2023-10-08T13:43:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:39 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/879592778' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-08T13:43:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:39 smithi057 ceph-mon[99476]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:43:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:39 smithi057 ceph-mon[99476]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:43:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:39 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:39 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:40.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:39 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:41.106 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled mon update... 2023-10-08T13:43:41.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:40 smithi057 ceph-mon[99476]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:43:41.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:40 smithi057 ceph-mon[99476]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:43:41.532 DEBUG:teuthology.orchestra.run.smithi099:mon.b> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.b.service 2023-10-08T13:43:41.534 DEBUG:teuthology.orchestra.run.smithi115:mon.c> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.c.service 2023-10-08T13:43:41.537 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-08T13:43:41.537 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-08T13:43:41.565 INFO:journalctl@ceph.mon.c.smithi115.stdout:-- Logs begin at Sun 2023-10-08 13:28:12 UTC. -- 2023-10-08T13:43:41.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:-- Logs begin at Sun 2023-10-08 13:27:56 UTC. -- 2023-10-08T13:43:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:41 smithi057 ceph-mon[99476]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:41 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:41 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:41 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:42.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:41 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:43:42.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:41 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:42.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:41 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:43:42.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:41 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:42.538 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:43:43.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:42 smithi057 ceph-mon[99476]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi099:172.21.15.99=b;smithi115:172.21.15.115=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:43:43.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:42 smithi057 ceph-mon[99476]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi099:172.21.15.99=b;smithi115:172.21.15.115=c;count:3 2023-10-08T13:43:43.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:42 smithi057 ceph-mon[99476]: Deploying daemon mon.c on smithi115 2023-10-08T13:43:44.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:43 smithi057 ceph-mon[99476]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:43:44.398 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":1,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","modified":"2023-10-08T13:42:04.406237Z","created":"2023-10-08T13:42:04.406237Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-08T13:43:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 1 2023-10-08T13:43:45.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:45 smithi115 ceph-mon[104408]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-08T13:43:45.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:45 smithi115 systemd[1]: Started Ceph mon.c for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:43:45.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:44 smithi057 ceph-mon[99476]: from='client.? 172.21.15.115:0/3879691848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T13:43:45.769 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-08T13:43:45.769 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-08T13:43:46.328 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:43:50.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:43:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: mon.a calling monitor election 2023-10-08T13:43:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: mon.c calling monitor election 2023-10-08T13:43:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T13:43:50.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: monmap e2: 2 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2023-10-08T13:43:50.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: fsmap 2023-10-08T13:43:50.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:43:50.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: mgrmap e13: x(active, since 44s) 2023-10-08T13:43:50.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: overall HEALTH_OK 2023-10-08T13:43:50.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:50.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:50 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:50.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:43:50.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: mon.a calling monitor election 2023-10-08T13:43:50.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:50.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: mon.c calling monitor election 2023-10-08T13:43:50.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:50.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:50.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T13:43:50.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: monmap e2: 2 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2023-10-08T13:43:50.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: fsmap 2023-10-08T13:43:50.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:43:50.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: mgrmap e13: x(active, since 44s) 2023-10-08T13:43:50.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: overall HEALTH_OK 2023-10-08T13:43:50.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:50.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:50 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:50.764 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:43:50.764 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":2,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","modified":"2023-10-08T13:43:45.034996Z","created":"2023-10-08T13:42:04.406237Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-08T13:43:50.766 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 2 2023-10-08T13:43:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:51 smithi115 ceph-mon[104408]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:51.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:51 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:51.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:51 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:43:51.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:51 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:51.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:51 smithi115 ceph-mon[104408]: Deploying daemon mon.b on smithi099 2023-10-08T13:43:51.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:51 smithi115 ceph-mon[104408]: from='client.? 172.21.15.115:0/3553951618' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T13:43:51.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:51 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:51.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:51 smithi057 ceph-mon[99476]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:51.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:51 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:51.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:51 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:43:51.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:51 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:51.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:51 smithi057 ceph-mon[99476]: Deploying daemon mon.b on smithi099 2023-10-08T13:43:51.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:51 smithi057 ceph-mon[99476]: from='client.? 172.21.15.115:0/3553951618' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T13:43:51.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:51 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:52.090 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-08T13:43:52.090 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-08T13:43:52.649 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:43:53.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:52 smithi099 systemd[1]: Starting Ceph mon.b for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:43:53.432 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 podman[103808]: 2023-10-08T13:43:53.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: set uid:gid to 167:167 (ceph:ceph) 2023-10-08T13:43:53.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev), process ceph-mon, pid 8 2023-10-08T13:43:53.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pidfile_write: ignore empty --pid-file 2023-10-08T13:43:53.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: load: jerasure load: lrc 2023-10-08T13:43:53.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: RocksDB version: 7.9.2 2023-10-08T13:43:53.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Git sha 0 2023-10-08T13:43:53.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Compile date 2023-10-07 16:15:44 2023-10-08T13:43:53.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: DB SUMMARY 2023-10-08T13:43:53.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: DB Session ID: Q6SJ2C3F5DO8T68L39AX 2023-10-08T13:43:53.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: CURRENT file: CURRENT 2023-10-08T13:43:53.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: IDENTITY file: IDENTITY 2023-10-08T13:43:53.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-08T13:43:53.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-08T13:43:53.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-08T13:43:53.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.error_if_exists: 0 2023-10-08T13:43:53.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.create_if_missing: 0 2023-10-08T13:43:53.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.paranoid_checks: 1 2023-10-08T13:43:53.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-08T13:43:53.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.env: 0x55572e741da0 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.fs: PosixFileSystem 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.info_log: 0x55572f089640 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_file_opening_threads: 16 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.statistics: (nil) 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.use_fsync: 0 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_log_file_size: 0 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-08T13:43:53.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.keep_log_file_num: 1000 2023-10-08T13:43:53.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.recycle_log_file_num: 0 2023-10-08T13:43:53.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.allow_fallocate: 1 2023-10-08T13:43:53.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.allow_mmap_reads: 0 2023-10-08T13:43:53.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.allow_mmap_writes: 0 2023-10-08T13:43:53.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.use_direct_reads: 0 2023-10-08T13:43:53.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-08T13:43:53.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.create_missing_column_families: 0 2023-10-08T13:43:53.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.db_log_dir: 2023-10-08T13:43:53.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.wal_dir: 2023-10-08T13:43:53.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-08T13:43:53.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-08T13:43:53.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-08T13:43:53.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-08T13:43:53.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-08T13:43:53.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-08T13:43:53.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.advise_random_on_open: 1 2023-10-08T13:43:53.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.db_write_buffer_size: 0 2023-10-08T13:43:53.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.write_buffer_manager: 0x55572f041360 2023-10-08T13:43:53.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-08T13:43:53.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-08T13:43:53.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-08T13:43:53.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.rate_limiter: (nil) 2023-10-08T13:43:53.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-08T13:43:53.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.wal_recovery_mode: 2 2023-10-08T13:43:53.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.enable_thread_tracking: 0 2023-10-08T13:43:53.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.enable_pipelined_write: 0 2023-10-08T13:43:53.779 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.unordered_write: 0 2023-10-08T13:43:53.779 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-08T13:43:53.779 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-08T13:43:53.779 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-08T13:43:53.779 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-08T13:43:53.779 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.row_cache: None 2023-10-08T13:43:53.779 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.wal_filter: None 2023-10-08T13:43:53.779 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-08T13:43:53.779 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.allow_ingest_behind: 0 2023-10-08T13:43:53.780 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.two_write_queues: 0 2023-10-08T13:43:53.780 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.manual_wal_flush: 0 2023-10-08T13:43:53.780 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.wal_compression: 0 2023-10-08T13:43:53.780 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.atomic_flush: 0 2023-10-08T13:43:53.780 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-08T13:43:53.780 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-08T13:43:53.780 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-08T13:43:53.780 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.log_readahead_size: 0 2023-10-08T13:43:53.780 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.best_efforts_recovery: 0 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.allow_data_in_errors: 0 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.db_host_id: __hostname__ 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_background_jobs: 2 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_background_compactions: -1 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_subcompactions: 1 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-08T13:43:53.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-08T13:43:53.782 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-08T13:43:53.782 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_total_wal_size: 0 2023-10-08T13:43:53.782 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-08T13:43:53.782 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-08T13:43:53.782 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-08T13:43:53.782 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-08T13:43:53.782 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_open_files: -1 2023-10-08T13:43:53.782 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bytes_per_sync: 0 2023-10-08T13:43:53.782 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_readahead_size: 0 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_background_flushes: -1 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Compression algorithms supported: 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: kZSTD supported: 0 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: kXpressCompression supported: 0 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: kLZ4HCCompression supported: 1 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: kZlibCompression supported: 1 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: kSnappyCompression supported: 1 2023-10-08T13:43:53.783 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: kLZ4Compression supported: 1 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: kBZip2Compression supported: 0 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.merge_operator: 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_filter: None 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_filter_factory: None 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.sst_partitioner_factory: None 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-08T13:43:53.784 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55572f089740) 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: cache_index_and_filter_blocks: 1 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: pin_top_level_index_and_filter: 1 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_type: 0 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: data_block_index_type: 0 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_shortening: 1 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: checksum: 4 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: no_block_cache: 0 2023-10-08T13:43:53.785 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache: 0x55572f0df1f0 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_name: BinnedLRUCache 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_options: 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: capacity : 536870912 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: num_shard_bits : 4 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: strict_capacity_limit : 0 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: high_pri_pool_ratio: 0.000 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_compressed: (nil) 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: persistent_cache: (nil) 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_size: 4096 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_size_deviation: 10 2023-10-08T13:43:53.786 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_restart_interval: 16 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_block_restart_interval: 1 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: metadata_block_size: 4096 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: partition_filters: 0 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: use_delta_encoding: 1 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: filter_policy: bloomfilter 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: whole_key_filtering: 1 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: verify_compression: 0 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: read_amp_bytes_per_bit: 0 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: format_version: 5 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: enable_index_compression: 1 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_align: 0 2023-10-08T13:43:53.787 INFO:journalctl@ceph.mon.b.smithi099.stdout: max_auto_readahead_size: 262144 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout: prepopulate_block_cache: 0 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout: initial_auto_readahead_size: 8192 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout: num_file_reads_for_auto_readahead: 2 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.write_buffer_size: 33554432 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_write_buffer_number: 2 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compression: NoCompression 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bottommost_compression: Disabled 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.prefix_extractor: nullptr 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.num_levels: 7 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-08T13:43:53.788 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-08T13:43:53.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-08T13:43:53.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compression_opts.level: 32767 2023-10-08T13:43:53.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compression_opts.strategy: 0 2023-10-08T13:43:53.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-08T13:43:53.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-08T13:43:53.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-08T13:43:53.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-08T13:43:53.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compression_opts.enabled: false 2023-10-08T13:43:53.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-08T13:43:53.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-08T13:43:53.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.target_file_size_base: 67108864 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-08T13:43:53.791 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-08T13:43:53.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-08T13:43:53.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-08T13:43:53.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-08T13:43:53.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-08T13:43:53.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 bash[103808]: 0e260681513ebb27b5a45ed19ceb4c99184ad84b1a837b1991ccd9fd2ffb6832 2023-10-08T13:43:53.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.arena_block_size: 1048576 2023-10-08T13:43:53.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-08T13:43:53.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-08T13:43:53.792 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.disable_auto_compactions: 0 2023-10-08T13:43:53.793 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-08T13:43:53.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-08T13:43:53.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-08T13:43:53.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-08T13:43:53.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-08T13:43:53.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-08T13:43:53.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-08T13:43:53.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-08T13:43:53.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-08T13:43:53.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-08T13:43:53.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.table_properties_collectors: 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.inplace_update_support: 0 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.bloom_locality: 0 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.max_successive_merges: 0 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.paranoid_file_checks: 0 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.force_consistency_checks: 1 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.report_bg_io_stats: 0 2023-10-08T13:43:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.ttl: 2592000 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.enable_blob_files: false 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.min_blob_size: 0 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.blob_file_size: 268435456 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-08T13:43:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.blob_file_starting_level: 0 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6378d539-25c6-43de-9def-144e3ac94def 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696772633438215, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696772633438747, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696772633, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6378d539-25c6-43de-9def-144e3ac94def", "db_session_id": "Q6SJ2C3F5DO8T68L39AX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696772633438835, "job": 1, "event": "recovery_finished"} 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-08T13:43:53.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55572f17c000 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: DB pointer 0x55572f168000 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** DB Stats ** 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T13:43:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** Compaction Stats [default] ** 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** Compaction Stats [default] ** 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-08T13:43:53.799 INFO:journalctl@ceph.mon.b.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T13:43:53.800 INFO:journalctl@ceph.mon.b.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-08T13:43:53.801 INFO:journalctl@ceph.mon.b.smithi099.stdout: Block cache BinnedLRUCache@0x55572f0df1f0#8 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-10-08T13:43:53.801 INFO:journalctl@ceph.mon.b.smithi099.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-08T13:43:53.801 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-10-08T13:43:53.802 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-08T13:43:53.802 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-08T13:43:53.802 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: using public_addr v2:172.21.15.99:0/0 -> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] 2023-10-08T13:43:53.802 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: starting mon.b rank -1 at public addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] at bind addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:43:53.802 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(???) e0 preinit fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:43:53.802 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).mds e1 new map 2023-10-08T13:43:53.802 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-08T13:43:53.802 INFO:journalctl@ceph.mon.b.smithi099.stdout: e1 2023-10-08T13:43:53.802 INFO:journalctl@ceph.mon.b.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-08T13:43:53.802 INFO:journalctl@ceph.mon.b.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout: legacy client fscid: -1 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout: No filesystems configured 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T13:43:53.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Deploying cephadm binary to smithi115 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Updating smithi099:/etc/ceph/ceph.conf 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Added host smithi115 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.806 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/879592778' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Updating smithi115:/etc/ceph/ceph.conf 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/879592778' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:43:53.807 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.808 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi099:172.21.15.99=b;smithi115:172.21.15.115=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi099:172.21.15.99=b;smithi115:172.21.15.115=c;count:3 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Deploying daemon mon.c on smithi115 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='client.? 172.21.15.115:0/3879691848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.a calling monitor election 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:53.809 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.c calling monitor election 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: monmap e2: 2 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: fsmap 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mgrmap e13: x(active, since 44s) 2023-10-08T13:43:53.810 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: overall HEALTH_OK 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: Deploying daemon mon.b on smithi099 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='client.? 172.21.15.115:0/3553951618' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 ceph-mon[103849]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-08T13:43:53.811 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:43:53 smithi099 systemd[1]: Started Ceph mon.b for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:43:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:43:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: mon.c calling monitor election 2023-10-08T13:43:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: mon.a calling monitor election 2023-10-08T13:43:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T13:43:58.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2023-10-08T13:43:58.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: fsmap 2023-10-08T13:43:58.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:43:58.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: mgrmap e13: x(active, since 52s) 2023-10-08T13:43:58.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-08T13:43:58.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-08T13:43:58.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-08T13:43:58.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: mon.b (rank 2) addr [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] is down (out of quorum) 2023-10-08T13:43:58.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:58.941 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:58 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:58.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:43:58.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: mon.c calling monitor election 2023-10-08T13:43:58.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: mon.a calling monitor election 2023-10-08T13:43:58.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:43:58.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:58.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:58.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:58.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T13:43:58.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2023-10-08T13:43:58.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: fsmap 2023-10-08T13:43:58.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:43:58.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: mgrmap e13: x(active, since 52s) 2023-10-08T13:43:58.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-08T13:43:58.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-08T13:43:58.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-08T13:43:58.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: mon.b (rank 2) addr [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] is down (out of quorum) 2023-10-08T13:43:58.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:58.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:58 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:59.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:59 smithi115 ceph-mon[104408]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:59.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:59 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:59.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:59 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:59.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:59 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:59.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:43:59 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:43:59.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:59 smithi057 ceph-mon[99476]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:43:59.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:59 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:59.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:59 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:43:59.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:59 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:43:59.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:43:59 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:00.106 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-10-08T13:44:00.106 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":3,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","modified":"2023-10-08T13:43:53.467865Z","created":"2023-10-08T13:42:04.406237Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-08T13:44:00.109 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 3 2023-10-08T13:44:00.508 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-08T13:44:00.509 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-08T13:44:00.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:44:00.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: mon.c calling monitor election 2023-10-08T13:44:00.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: mon.a calling monitor election 2023-10-08T13:44:00.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:00.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:44:00.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:00.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:00.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:00.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:00.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:00.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:00.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T13:44:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2023-10-08T13:44:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: fsmap 2023-10-08T13:44:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:44:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: mgrmap e13: x(active, since 52s) 2023-10-08T13:44:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-08T13:44:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-08T13:44:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-08T13:44:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: mon.b (rank 2) addr [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] is down (out of quorum) 2023-10-08T13:44:00.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:00.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:00.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:00.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:00.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:00.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:00.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:00 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:01.084 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:01.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: mon.b calling monitor election 2023-10-08T13:44:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-08T13:44:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: Updating smithi099:/etc/ceph/ceph.conf 2023-10-08T13:44:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: Updating smithi115:/etc/ceph/ceph.conf 2023-10-08T13:44:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: mon.b calling monitor election 2023-10-08T13:44:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: mon.c calling monitor election 2023-10-08T13:44:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: mon.a calling monitor election 2023-10-08T13:44:01.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-08T13:44:01.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2023-10-08T13:44:01.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: fsmap 2023-10-08T13:44:01.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:44:01.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: mgrmap e13: x(active, since 54s) 2023-10-08T13:44:01.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-08T13:44:01.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: Cluster is now healthy 2023-10-08T13:44:01.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: overall HEALTH_OK 2023-10-08T13:44:01.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: Reconfiguring mon.a (unknown last config time)... 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:44:01.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T13:44:01.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:01.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:01 smithi115 ceph-mon[104408]: Reconfiguring daemon mon.a on smithi057 2023-10-08T13:44:01.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: mon.b calling monitor election 2023-10-08T13:44:01.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:01.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-08T13:44:01.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: Updating smithi099:/etc/ceph/ceph.conf 2023-10-08T13:44:01.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: Updating smithi115:/etc/ceph/ceph.conf 2023-10-08T13:44:01.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:01.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:01.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:01.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: mon.b calling monitor election 2023-10-08T13:44:01.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: mon.c calling monitor election 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: mon.a calling monitor election 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: fsmap 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: mgrmap e13: x(active, since 54s) 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: Cluster is now healthy 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: overall HEALTH_OK 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: Reconfiguring mon.a (unknown last config time)... 2023-10-08T13:44:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:44:01.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T13:44:01.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:01.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:01 smithi057 ceph-mon[99476]: Reconfiguring daemon mon.a on smithi057 2023-10-08T13:44:01.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: mon.b calling monitor election 2023-10-08T13:44:01.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:01.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-08T13:44:01.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: Updating smithi099:/etc/ceph/ceph.conf 2023-10-08T13:44:01.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: Updating smithi115:/etc/ceph/ceph.conf 2023-10-08T13:44:01.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:01.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:01.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:01.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: mon.b calling monitor election 2023-10-08T13:44:01.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: mon.c calling monitor election 2023-10-08T13:44:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: mon.a calling monitor election 2023-10-08T13:44:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-08T13:44:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: monmap e3: 3 mons at {a=[v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0],b=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} removed_ranks: {} 2023-10-08T13:44:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: fsmap 2023-10-08T13:44:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T13:44:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: mgrmap e13: x(active, since 54s) 2023-10-08T13:44:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-08T13:44:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: Cluster is now healthy 2023-10-08T13:44:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: overall HEALTH_OK 2023-10-08T13:44:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:01.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: Reconfiguring mon.a (unknown last config time)... 2023-10-08T13:44:01.526 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:44:01.526 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T13:44:01.526 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:01.526 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:01 smithi099 ceph-mon[103849]: Reconfiguring daemon mon.a on smithi057 2023-10-08T13:44:02.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:02 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:02.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:02 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:02.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:02 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:02.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:02 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:02.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:02 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:02.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:02 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:02.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:02 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:02.467+0000 7fdf241fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-08T13:44:02.858 INFO:teuthology.orchestra.run.smithi057.stdout:# minimal ceph.conf for 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:44:02.858 INFO:teuthology.orchestra.run.smithi057.stdout:[global] 2023-10-08T13:44:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: fsid = 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:44:02.858 INFO:teuthology.orchestra.run.smithi057.stdout: mon_host = [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] 2023-10-08T13:44:03.262 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-08T13:44:03.262 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:44:03.263 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.conf 2023-10-08T13:44:03.293 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:44:03.293 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:03.360 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:44:03.361 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2023-10-08T13:44:03.391 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:44:03.392 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:03.458 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:44:03.459 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2023-10-08T13:44:03.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:03 smithi057 ceph-mon[99476]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:03.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:03 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:03.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:03 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:03.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:03 smithi057 ceph-mon[99476]: Reconfiguring mon.b (monmap changed)... 2023-10-08T13:44:03.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:03 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:44:03.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:03 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T13:44:03.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:03 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:03.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:03 smithi057 ceph-mon[99476]: Reconfiguring daemon mon.b on smithi099 2023-10-08T13:44:03.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:03 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1403294618' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:03.495 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:44:03.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:03.565 INFO:tasks.cephadm:enabling module snap_schedule 2023-10-08T13:44:03.566 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T13:44:03.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:03 smithi099 ceph-mon[103849]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:03.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:03 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:03.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:03 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:03.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:03 smithi099 ceph-mon[103849]: Reconfiguring mon.b (monmap changed)... 2023-10-08T13:44:03.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:03 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:44:03.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:03 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T13:44:03.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:03 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:03.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:03 smithi099 ceph-mon[103849]: Reconfiguring daemon mon.b on smithi099 2023-10-08T13:44:03.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:03 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1403294618' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:03.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:03 smithi115 ceph-mon[104408]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:03.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:03 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:03.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:03 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:03.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:03 smithi115 ceph-mon[104408]: Reconfiguring mon.b (monmap changed)... 2023-10-08T13:44:03.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:03 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:44:03.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:03 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T13:44:03.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:03 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:03.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:03 smithi115 ceph-mon[104408]: Reconfiguring daemon mon.b on smithi099 2023-10-08T13:44:03.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:03 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1403294618' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:04.130 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:04.838 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:04 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:04.838 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:04 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:04.838 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:04 smithi115 ceph-mon[104408]: Reconfiguring mon.c (monmap changed)... 2023-10-08T13:44:04.838 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:04 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:44:04.838 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:04 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T13:44:04.838 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:04 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:04.838 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:04 smithi115 ceph-mon[104408]: Reconfiguring daemon mon.c on smithi115 2023-10-08T13:44:04.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:04 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:04.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:04 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:04.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:04 smithi057 ceph-mon[99476]: Reconfiguring mon.c (monmap changed)... 2023-10-08T13:44:04.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:04 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:44:04.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:04 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T13:44:04.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:04 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:04.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:04 smithi057 ceph-mon[99476]: Reconfiguring daemon mon.c on smithi115 2023-10-08T13:44:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:04 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:04 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:04 smithi099 ceph-mon[103849]: Reconfiguring mon.c (monmap changed)... 2023-10-08T13:44:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:04 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T13:44:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:04 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T13:44:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:04 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:04 smithi099 ceph-mon[103849]: Reconfiguring daemon mon.c on smithi115 2023-10-08T13:44:06.099 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:05 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: ignoring --setuser ceph since I am not root 2023-10-08T13:44:06.099 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:05 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: ignoring --setgroup ceph since I am not root 2023-10-08T13:44:06.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:05 smithi057 ceph-mon[99476]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:06.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:05 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:06.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:05 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:06.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:05 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:06.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:05 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:06.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:05 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:06.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:05 smithi057 ceph-mon[99476]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:06.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:05 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/194602000' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-08T13:44:06.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:05 smithi115 ceph-mon[104408]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:06.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:05 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:06.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:05 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:06.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:05 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:06.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:05 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:06.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:05 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:06.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:05 smithi115 ceph-mon[104408]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:06.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:05 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/194602000' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-08T13:44:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:05 smithi099 ceph-mon[103849]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:05 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:05 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:05 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:05 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:05 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:05 smithi099 ceph-mon[103849]: from='mgr.14152 172.21.15.57:0/1748891377' entity='mgr.x' 2023-10-08T13:44:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:05 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/194602000' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-08T13:44:06.293 INFO:tasks.cephadm:Adding mgr.x on smithi057 2023-10-08T13:44:06.293 INFO:tasks.cephadm:Adding mgr.y on smithi099 2023-10-08T13:44:06.293 INFO:tasks.cephadm:Adding mgr.z on smithi115 2023-10-08T13:44:06.294 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch apply mgr '3;smithi057=x;smithi099=y;smithi115=z' 2023-10-08T13:44:06.350 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:06.097+0000 7fe53e8e7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T13:44:06.351 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:06.194+0000 7fe53e8e7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T13:44:06.351 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:06.305+0000 7fe53e8e7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T13:44:06.626 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:06.417+0000 7fe53e8e7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T13:44:06.858 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:44:06.895 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:06.625+0000 7fe53e8e7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T13:44:07.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:06 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/194602000' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-08T13:44:07.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:06 smithi115 ceph-mon[104408]: mgrmap e14: x(active, since 59s) 2023-10-08T13:44:07.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:07.097+0000 7fe53e8e7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T13:44:07.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:06 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/194602000' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-08T13:44:07.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:06 smithi057 ceph-mon[99476]: mgrmap e14: x(active, since 59s) 2023-10-08T13:44:07.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:06 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/194602000' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-08T13:44:07.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:06 smithi099 ceph-mon[103849]: mgrmap e14: x(active, since 59s) 2023-10-08T13:44:07.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:07.249+0000 7fe53e8e7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T13:44:07.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:07.357+0000 7fe53e8e7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T13:44:07.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:07.473+0000 7fe53e8e7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T13:44:08.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:07.983+0000 7fe53e8e7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T13:44:09.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:08 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:08.763+0000 7fe53e8e7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T13:44:09.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:08 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:08.861+0000 7fe53e8e7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T13:44:10.172 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:09.836+0000 7fe53e8e7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:44:10.172 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:09.948+0000 7fe53e8e7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T13:44:10.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:10.171+0000 7fe53e8e7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:44:10.780 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:10.524+0000 7fe53e8e7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T13:44:11.240 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:10.779+0000 7fe53e8e7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T13:44:11.240 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:10.962+0000 7fe53e8e7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T13:44:11.737 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:11.297+0000 7fe53e8e7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T13:44:12.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:12 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:12.124+0000 7fe53e8e7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T13:44:12.487 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:12 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:12.284+0000 7fe53e8e7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T13:44:12.951 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:12 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:12.533+0000 7fe53e8e7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T13:44:12.951 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:12 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:12.671+0000 7fe53e8e7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T13:44:12.951 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:12 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:12.949+0000 7fe53e8e7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T13:44:14.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: Active manager daemon x restarted 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: Activating manager daemon x 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: osdmap e5: 0 total, 0 up, 0 in 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: mgrmap e15: x(active, starting, since 0.00437311s) 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T13:44:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: Manager daemon x is now available 2023-10-08T13:44:14.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:14.439 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:44:14.010+0000 7fe53e8e7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T13:44:14.439 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: Active manager daemon x restarted 2023-10-08T13:44:14.439 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: Activating manager daemon x 2023-10-08T13:44:14.439 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: osdmap e5: 0 total, 0 up, 0 in 2023-10-08T13:44:14.440 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: mgrmap e15: x(active, starting, since 0.00437311s) 2023-10-08T13:44:14.440 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:44:14.440 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:14.440 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:44:14.440 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T13:44:14.440 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T13:44:14.440 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T13:44:14.440 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T13:44:14.440 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: Manager daemon x is now available 2023-10-08T13:44:14.440 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:14.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: Active manager daemon x restarted 2023-10-08T13:44:14.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: Activating manager daemon x 2023-10-08T13:44:14.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: osdmap e5: 0 total, 0 up, 0 in 2023-10-08T13:44:14.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: mgrmap e15: x(active, starting, since 0.00437311s) 2023-10-08T13:44:14.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:44:14.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:44:14.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:44:14.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T13:44:14.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T13:44:14.445 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T13:44:14.445 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T13:44:14.445 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: Manager daemon x is now available 2023-10-08T13:44:14.445 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:15.040 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled mgr update... 2023-10-08T13:44:15.378 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:15.378 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T13:44:15.378 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T13:44:15.378 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:15 smithi115 ceph-mon[104408]: [08/Oct/2023:13:44:14] ENGINE Bus STARTING 2023-10-08T13:44:15.379 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:15 smithi115 ceph-mon[104408]: [08/Oct/2023:13:44:14] ENGINE Serving on https://172.21.15.57:7150 2023-10-08T13:44:15.379 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:15 smithi115 ceph-mon[104408]: [08/Oct/2023:13:44:14] ENGINE Serving on http://172.21.15.57:8765 2023-10-08T13:44:15.379 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:15 smithi115 ceph-mon[104408]: [08/Oct/2023:13:44:14] ENGINE Bus STARTED 2023-10-08T13:44:15.379 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:15 smithi115 ceph-mon[104408]: mgrmap e16: x(active, since 1.00851s) 2023-10-08T13:44:15.379 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:15.419 DEBUG:teuthology.orchestra.run.smithi099:mgr.y> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.y.service 2023-10-08T13:44:15.422 DEBUG:teuthology.orchestra.run.smithi115:mgr.z> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.z.service 2023-10-08T13:44:15.424 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2023-10-08T13:44:15.425 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set client client_mount_timeout 600 2023-10-08T13:44:15.427 INFO:tasks.cephadm:setting config [client] debug_client = 20 2023-10-08T13:44:15.427 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set client debug_client 20 2023-10-08T13:44:15.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:15.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T13:44:15.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T13:44:15.445 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:15 smithi099 ceph-mon[103849]: [08/Oct/2023:13:44:14] ENGINE Bus STARTING 2023-10-08T13:44:15.445 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:15 smithi099 ceph-mon[103849]: [08/Oct/2023:13:44:14] ENGINE Serving on https://172.21.15.57:7150 2023-10-08T13:44:15.445 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:15 smithi099 ceph-mon[103849]: [08/Oct/2023:13:44:14] ENGINE Serving on http://172.21.15.57:8765 2023-10-08T13:44:15.445 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:15 smithi099 ceph-mon[103849]: [08/Oct/2023:13:44:14] ENGINE Bus STARTED 2023-10-08T13:44:15.445 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:15 smithi099 ceph-mon[103849]: mgrmap e16: x(active, since 1.00851s) 2023-10-08T13:44:15.445 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:15.450 INFO:journalctl@ceph.mgr.y.smithi099.stdout:-- Logs begin at Sun 2023-10-08 13:27:56 UTC. -- 2023-10-08T13:44:15.450 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:15.450 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T13:44:15.451 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T13:44:15.451 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:15 smithi057 ceph-mon[99476]: [08/Oct/2023:13:44:14] ENGINE Bus STARTING 2023-10-08T13:44:15.451 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:15 smithi057 ceph-mon[99476]: [08/Oct/2023:13:44:14] ENGINE Serving on https://172.21.15.57:7150 2023-10-08T13:44:15.451 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:15 smithi057 ceph-mon[99476]: [08/Oct/2023:13:44:14] ENGINE Serving on http://172.21.15.57:8765 2023-10-08T13:44:15.451 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:15 smithi057 ceph-mon[99476]: [08/Oct/2023:13:44:14] ENGINE Bus STARTED 2023-10-08T13:44:15.451 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:15 smithi057 ceph-mon[99476]: mgrmap e16: x(active, since 1.00851s) 2023-10-08T13:44:15.451 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:15.452 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2023-10-08T13:44:15.452 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set client debug_ms 1 2023-10-08T13:44:15.454 INFO:journalctl@ceph.mgr.z.smithi115.stdout:-- Logs begin at Sun 2023-10-08 13:28:12 UTC. -- 2023-10-08T13:44:15.495 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2023-10-08T13:44:15.495 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set client rados_mon_op_timeout 900 2023-10-08T13:44:15.539 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2023-10-08T13:44:15.539 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set client rados_osd_op_timeout 900 2023-10-08T13:44:15.582 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2023-10-08T13:44:15.583 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds debug_mds 20 2023-10-08T13:44:15.626 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2023-10-08T13:44:15.626 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds debug_mds_balancer 20 2023-10-08T13:44:15.688 INFO:tasks.cephadm:setting config [mds] debug_ms = 1 2023-10-08T13:44:15.689 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds debug_ms 1 2023-10-08T13:44:15.736 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2023-10-08T13:44:15.736 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_fragment_size_max 10000 2023-10-08T13:44:15.781 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2023-10-08T13:44:15.781 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_merge_size 5 2023-10-08T13:44:15.826 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2023-10-08T13:44:15.826 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_split_bits 3 2023-10-08T13:44:15.877 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2023-10-08T13:44:15.877 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_split_size 100 2023-10-08T13:44:15.942 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2023-10-08T13:44:15.943 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_debug_frag True 2023-10-08T13:44:15.994 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2023-10-08T13:44:15.995 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_debug_scatterstat True 2023-10-08T13:44:16.043 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2023-10-08T13:44:16.044 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_op_complaint_time 180 2023-10-08T13:44:16.109 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2023-10-08T13:44:16.109 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_verify_scatter True 2023-10-08T13:44:16.153 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2023-10-08T13:44:16.153 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds osd_op_complaint_time 180 2023-10-08T13:44:16.164 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:16.165 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2023-10-08T13:44:16.165 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds rados_mon_op_timeout 900 2023-10-08T13:44:16.208 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2023-10-08T13:44:16.208 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mds rados_osd_op_timeout 900 2023-10-08T13:44:16.261 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2023-10-08T13:44:16.261 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mgr debug_cephsqlite 20 2023-10-08T13:44:16.271 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:16.288 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2023-10-08T13:44:16.288 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T13:44:16.329 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:16.337 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2023-10-08T13:44:16.337 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T13:44:16.359 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:16.376 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2023-10-08T13:44:16.376 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set mon mon_op_complaint_time 120 2023-10-08T13:44:16.448 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2023-10-08T13:44:16.448 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config set osd osd_op_complaint_time 180 2023-10-08T13:44:16.507 DEBUG:tasks.cephadm:set 24 configs 2023-10-08T13:44:16.507 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi057') 2023-10-08T13:44:16.515 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:16.672 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:16.681 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:17.074 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:16 smithi057 ceph-mon[99476]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi099=y;smithi115=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:44:17.074 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:16 smithi057 ceph-mon[99476]: Saving service mgr spec with placement smithi057=x;smithi099=y;smithi115=z;count:3 2023-10-08T13:44:17.074 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.074 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.074 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.074 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.074 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.075 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:16 smithi115 ceph-mon[104408]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi099=y;smithi115=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:44:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:16 smithi115 ceph-mon[104408]: Saving service mgr spec with placement smithi057=x;smithi099=y;smithi115=z;count:3 2023-10-08T13:44:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:16 smithi099 ceph-mon[103849]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi099=y;smithi115=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:44:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:16 smithi099 ceph-mon[103849]: Saving service mgr spec with placement smithi057=x;smithi099=y;smithi115=z;count:3 2023-10-08T13:44:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:17.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.196 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:18 smithi057 ceph-mon[99476]: mgrmap e17: x(active, since 2s) 2023-10-08T13:44:18.196 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.196 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.196 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:44:18.196 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.197 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.197 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:44:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:18 smithi099 ceph-mon[103849]: mgrmap e17: x(active, since 2s) 2023-10-08T13:44:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:44:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:44:18.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:18 smithi115 ceph-mon[104408]: mgrmap e17: x(active, since 2s) 2023-10-08T13:44:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:18 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:18 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:18 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:44:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:18 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:18 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:18 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:44:19.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:44:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-08T13:44:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: Updating smithi099:/etc/ceph/ceph.conf 2023-10-08T13:44:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: Updating smithi115:/etc/ceph/ceph.conf 2023-10-08T13:44:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:19.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:19 smithi057 ceph-mon[99476]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:19.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.727+0000 7f2f50c37700 1 -- 172.21.15.57:0/2482108982 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2f34038b40 msgr2=0x7f2f3403b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:19.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 --2- 172.21.15.57:0/2482108982 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2f34038b40 0x7f2f3403b000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c068a90 tx=0x7f2f3c009310 comp rx=0 tx=0).stop 2023-10-08T13:44:19.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 -- 172.21.15.57:0/2482108982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f4c101330 msgr2=0x7f2f4c19d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:19.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 --2- 172.21.15.57:0/2482108982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f4c101330 0x7f2f4c19d190 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2f44007ee0 tx=0x7f2f4400d3b0 comp rx=0 tx=0).stop 2023-10-08T13:44:19.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 -- 172.21.15.57:0/2482108982 shutdown_connections 2023-10-08T13:44:19.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 --2- 172.21.15.57:0/2482108982 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2f34038b40 0x7f2f3403b000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:19.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 --2- 172.21.15.57:0/2482108982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f4c101330 0x7f2f4c19d190 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:19.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 --2- 172.21.15.57:0/2482108982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f4c0ffc50 0x7f2f4c195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:19.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 --2- 172.21.15.57:0/2482108982 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4c100970 0x7f2f4c196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:19.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 -- 172.21.15.57:0/2482108982 >> 172.21.15.57:0/2482108982 conn(0x7f2f4c0fa450 msgr2=0x7f2f4c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:19.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 -- 172.21.15.57:0/2482108982 shutdown_connections 2023-10-08T13:44:19.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:19.728+0000 7f2f50c37700 1 -- 172.21.15.57:0/2482108982 wait complete. 2023-10-08T13:44:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:44:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-08T13:44:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: Updating smithi099:/etc/ceph/ceph.conf 2023-10-08T13:44:19.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: Updating smithi115:/etc/ceph/ceph.conf 2023-10-08T13:44:19.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:19.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:19.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:19 smithi099 ceph-mon[103849]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:19.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:19.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:19.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:44:19.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:19.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:19.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: Updating smithi057:/etc/ceph/ceph.conf 2023-10-08T13:44:19.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: Updating smithi099:/etc/ceph/ceph.conf 2023-10-08T13:44:19.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: Updating smithi115:/etc/ceph/ceph.conf 2023-10-08T13:44:19.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:19.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:19.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:19 smithi115 ceph-mon[104408]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T13:44:19.988 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:20.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.348+0000 7fa1014ab700 1 -- 172.21.15.57:0/3893483428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0fc1067d0 msgr2=0x7fa0fc106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:20.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.348+0000 7fa1014ab700 1 --2- 172.21.15.57:0/3893483428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0fc1067d0 0x7fa0fc106bb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009990 tx=0x7fa0f0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.348+0000 7fa1014ab700 1 -- 172.21.15.57:0/3893483428 shutdown_connections 2023-10-08T13:44:20.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.348+0000 7fa1014ab700 1 --2- 172.21.15.57:0/3893483428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc0697f0 0x7fa0fc10d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.348+0000 7fa1014ab700 1 --2- 172.21.15.57:0/3893483428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0fc1067d0 0x7fa0fc106bb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.348+0000 7fa1014ab700 1 --2- 172.21.15.57:0/3893483428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0fc068e50 0x7fa0fc0692b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.348+0000 7fa1014ab700 1 -- 172.21.15.57:0/3893483428 >> 172.21.15.57:0/3893483428 conn(0x7fa0fc0766f0 msgr2=0x7fa0fc076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:20.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.348+0000 7fa1014ab700 1 -- 172.21.15.57:0/3893483428 shutdown_connections 2023-10-08T13:44:20.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.348+0000 7fa1014ab700 1 -- 172.21.15.57:0/3893483428 wait complete. 2023-10-08T13:44:20.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.348+0000 7fa1014ab700 1 Processor -- start 2023-10-08T13:44:20.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa1014ab700 1 -- start start 2023-10-08T13:44:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa1014ab700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc068e50 0x7fa0fc1960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:20.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa1014ab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0fc0697f0 0x7fa0fc1965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa1014ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0fc1067d0 0x7fa0fc19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa1014ab700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19afb0 con 0x7fa0fc0697f0 2023-10-08T13:44:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa1014ab700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19b120 con 0x7fa0fc068e50 2023-10-08T13:44:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa1014ab700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19b2e0 con 0x7fa0fc1067d0 2023-10-08T13:44:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa0fa7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0fc0697f0 0x7fa0fc1965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa0fa7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0fc0697f0 0x7fa0fc1965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52738/0 (socket says 172.21.15.57:52738) 2023-10-08T13:44:20.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa0fa7fc700 1 -- 172.21.15.57:0/1687332017 learned_addr learned my addr 172.21.15.57:0/1687332017 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa0fb7fe700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0fc1067d0 0x7fa0fc19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa0faffd700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc068e50 0x7fa0fc1960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa0fa7fc700 1 -- 172.21.15.57:0/1687332017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0fc1067d0 msgr2=0x7fa0fc19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa0fa7fc700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0fc1067d0 0x7fa0fc19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa0fa7fc700 1 -- 172.21.15.57:0/1687332017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc068e50 msgr2=0x7fa0fc1960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:20.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa0fa7fc700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc068e50 0x7fa0fc1960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.349+0000 7fa0fa7fc700 1 -- 172.21.15.57:0/1687332017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00095f0 con 0x7fa0fc0697f0 2023-10-08T13:44:20.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.350+0000 7fa0fa7fc700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0fc0697f0 0x7fa0fc1965f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00d950 tx=0x7fa0ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:20.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.350+0000 7fa0e3fff700 1 -- 172.21.15.57:0/1687332017 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec009940 con 0x7fa0fc0697f0 2023-10-08T13:44:20.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.350+0000 7fa0e3fff700 1 -- 172.21.15.57:0/1687332017 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(22 keys) v1 ==== 987+0+0 (secure 0 0 0) 0x7fa0ec010460 con 0x7fa0fc0697f0 2023-10-08T13:44:20.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.350+0000 7fa1014ab700 1 -- 172.21.15.57:0/1687332017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc19b5c0 con 0x7fa0fc0697f0 2023-10-08T13:44:20.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.350+0000 7fa0e3fff700 1 -- 172.21.15.57:0/1687332017 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00f3c0 con 0x7fa0fc0697f0 2023-10-08T13:44:20.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.350+0000 7fa1014ab700 1 -- 172.21.15.57:0/1687332017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc1a2e20 con 0x7fa0fc0697f0 2023-10-08T13:44:20.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.351+0000 7fa0e3fff700 1 -- 172.21.15.57:0/1687332017 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7fa0ec00f520 con 0x7fa0fc0697f0 2023-10-08T13:44:20.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.351+0000 7fa0e3fff700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0e4038b40 0x7fa0e403b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:20.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.351+0000 7fa0e3fff700 1 -- 172.21.15.57:0/1687332017 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa0ec04b350 con 0x7fa0fc0697f0 2023-10-08T13:44:20.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.351+0000 7fa1014ab700 1 -- 172.21.15.57:0/1687332017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc108d60 con 0x7fa0fc0697f0 2023-10-08T13:44:20.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.351+0000 7fa0faffd700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0e4038b40 0x7fa0e403b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:20.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.351+0000 7fa0faffd700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0e4038b40 0x7fa0e403b000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000b5c0 tx=0x7fa0f0005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:20.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.358+0000 7fa0e3fff700 1 -- 172.21.15.57:0/1687332017 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa0ec019360 con 0x7fa0fc0697f0 2023-10-08T13:44:20.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.560+0000 7fa1014ab700 1 -- 172.21.15.57:0/1687332017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7fa0fc101f80 con 0x7fa0fc0697f0 2023-10-08T13:44:20.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.563+0000 7fa0e3fff700 1 -- 172.21.15.57:0/1687332017 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v11)=0 v11) v1 ==== 123+0+0 (secure 0 0 0) 0x7fa0fc101f80 con 0x7fa0fc0697f0 2023-10-08T13:44:20.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.564+0000 7fa0e3fff700 1 -- 172.21.15.57:0/1687332017 <== mon.0 v2:172.21.15.57:3300/0 8 ==== config(23 keys) v1 ==== 1018+0+0 (secure 0 0 0) 0x7fa0ec019360 con 0x7fa0fc0697f0 2023-10-08T13:44:20.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 -- 172.21.15.57:0/1687332017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0e4038b40 msgr2=0x7fa0e403b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:20.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0e4038b40 0x7fa0e403b000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000b5c0 tx=0x7fa0f0005fd0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 -- 172.21.15.57:0/1687332017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0fc0697f0 msgr2=0x7fa0fc1965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:20.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0fc0697f0 0x7fa0fc1965f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00d950 tx=0x7fa0ec00dc60 comp rx=0 tx=0).stop 2023-10-08T13:44:20.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 -- 172.21.15.57:0/1687332017 shutdown_connections 2023-10-08T13:44:20.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0fc1067d0 0x7fa0fc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0e4038b40 0x7fa0e403b000 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc068e50 0x7fa0fc1960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 --2- 172.21.15.57:0/1687332017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0fc0697f0 0x7fa0fc1965f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:20.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 -- 172.21.15.57:0/1687332017 >> 172.21.15.57:0/1687332017 conn(0x7fa0fc0766f0 msgr2=0x7fa0fc0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:20.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 -- 172.21.15.57:0/1687332017 shutdown_connections 2023-10-08T13:44:20.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:20.566+0000 7fa1014ab700 1 -- 172.21.15.57:0/1687332017 wait complete. 2023-10-08T13:44:20.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:20.866 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:20.866 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:20.866 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:44:20.866 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:44:20.866 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:44:20.866 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.866 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.866 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.866 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.866 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T13:44:20.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' 2023-10-08T13:44:20.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T13:44:20.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T13:44:20.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:20.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: Deploying daemon mgr.z on smithi115 2023-10-08T13:44:20.867 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:20 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1687332017' entity='client.admin' 2023-10-08T13:44:20.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:20.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:20.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:20.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:44:20.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:44:20.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:44:20.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:20.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T13:44:20.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' 2023-10-08T13:44:20.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T13:44:20.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T13:44:20.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:20.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: Deploying daemon mgr.z on smithi115 2023-10-08T13:44:20.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:20 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1687332017' entity='client.admin' 2023-10-08T13:44:21.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:21.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:21.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2023-10-08T13:44:21.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: Updating smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: Updating smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: Updating smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' 2023-10-08T13:44:21.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T13:44:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T13:44:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: Deploying daemon mgr.z on smithi115 2023-10-08T13:44:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:20 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1687332017' entity='client.admin' 2023-10-08T13:44:21.437 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:21 smithi115 systemd[1]: Starting Ceph mgr.z for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:44:21.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.472+0000 7f6a20371700 1 -- 172.21.15.57:0/643153775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a180693d0 msgr2=0x7f6a1810d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:21.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.472+0000 7f6a20371700 1 --2- 172.21.15.57:0/643153775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a180693d0 0x7f6a1810d5e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000b0a0 tx=0x7f6a1000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.472+0000 7f6a20371700 1 -- 172.21.15.57:0/643153775 shutdown_connections 2023-10-08T13:44:21.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.472+0000 7f6a20371700 1 --2- 172.21.15.57:0/643153775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a180693d0 0x7f6a1810d5e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.472+0000 7f6a20371700 1 --2- 172.21.15.57:0/643153775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a18106780 0x7f6a18106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.472+0000 7f6a20371700 1 --2- 172.21.15.57:0/643153775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a18068a10 0x7f6a18068e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.472+0000 7f6a20371700 1 -- 172.21.15.57:0/643153775 >> 172.21.15.57:0/643153775 conn(0x7f6a18076720 msgr2=0x7f6a18076b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:21.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.472+0000 7f6a20371700 1 -- 172.21.15.57:0/643153775 shutdown_connections 2023-10-08T13:44:21.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.473+0000 7f6a20371700 1 -- 172.21.15.57:0/643153775 wait complete. 2023-10-08T13:44:21.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.473+0000 7f6a20371700 1 Processor -- start 2023-10-08T13:44:21.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.473+0000 7f6a20371700 1 -- start start 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.473+0000 7f6a20371700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a18068a10 0x7f6a180ff720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.473+0000 7f6a20371700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a180693d0 0x7f6a180ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.473+0000 7f6a20371700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a18106780 0x7f6a181001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.473+0000 7f6a20371700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a181a4d00 con 0x7f6a18068a10 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.473+0000 7f6a20371700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a181a4e70 con 0x7f6a180693d0 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.473+0000 7f6a20371700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a181a5030 con 0x7f6a18106780 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1e10d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a18068a10 0x7f6a180ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1e10d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a18068a10 0x7f6a180ff720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52748/0 (socket says 172.21.15.57:52748) 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1e10d700 1 -- 172.21.15.57:0/4012822877 learned_addr learned my addr 172.21.15.57:0/4012822877 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1d90c700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a180693d0 0x7f6a180ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:21.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1e90e700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a18106780 0x7f6a181001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1e10d700 1 -- 172.21.15.57:0/4012822877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a18106780 msgr2=0x7f6a181001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1e10d700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a18106780 0x7f6a181001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1e10d700 1 -- 172.21.15.57:0/4012822877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a180693d0 msgr2=0x7f6a180ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1e10d700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a180693d0 0x7f6a180ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1e10d700 1 -- 172.21.15.57:0/4012822877 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a10009d00 con 0x7f6a18068a10 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a1e10d700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a18068a10 0x7f6a180ff720 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400d980 tx=0x7f6a1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a0f7fe700 1 -- 172.21.15.57:0/4012822877 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14004220 con 0x7f6a18068a10 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a0f7fe700 1 -- 172.21.15.57:0/4012822877 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(23 keys) v1 ==== 1018+0+0 (secure 0 0 0) 0x7f6a14007640 con 0x7f6a18068a10 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a20371700 1 -- 172.21.15.57:0/4012822877 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a181a5310 con 0x7f6a18068a10 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a0f7fe700 1 -- 172.21.15.57:0/4012822877 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14005490 con 0x7f6a18068a10 2023-10-08T13:44:21.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.474+0000 7f6a20371700 1 -- 172.21.15.57:0/4012822877 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a181a5810 con 0x7f6a18068a10 2023-10-08T13:44:21.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.475+0000 7f6a0f7fe700 1 -- 172.21.15.57:0/4012822877 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7f6a140055f0 con 0x7f6a18068a10 2023-10-08T13:44:21.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.475+0000 7f6a0f7fe700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a04038b40 0x7f6a0403b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:21.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.475+0000 7f6a0f7fe700 1 -- 172.21.15.57:0/4012822877 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f6a1404b200 con 0x7f6a18068a10 2023-10-08T13:44:21.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.475+0000 7f6a1d90c700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a04038b40 0x7f6a0403b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:21.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.476+0000 7f6a1d90c700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a04038b40 0x7f6a0403b000 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6a18100e50 tx=0x7f6a0800a380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:21.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.476+0000 7f6a20371700 1 -- 172.21.15.57:0/4012822877 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a18101eb0 con 0x7f6a18068a10 2023-10-08T13:44:21.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.481+0000 7f6a0f7fe700 1 -- 172.21.15.57:0/4012822877 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6a14016030 con 0x7f6a18068a10 2023-10-08T13:44:21.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.689+0000 7f6a20371700 1 -- 172.21.15.57:0/4012822877 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) v1 -- 0x7f6a1804e4c0 con 0x7f6a18068a10 2023-10-08T13:44:21.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.698+0000 7f6a0f7fe700 1 -- 172.21.15.57:0/4012822877 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6a1804e4c0 con 0x7f6a18068a10 2023-10-08T13:44:21.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.699+0000 7f6a0f7fe700 1 -- 172.21.15.57:0/4012822877 <== mon.0 v2:172.21.15.57:3300/0 8 ==== config(24 keys) v1 ==== 1049+0+0 (secure 0 0 0) 0x7f6a14022350 con 0x7f6a18068a10 2023-10-08T13:44:21.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 -- 172.21.15.57:0/4012822877 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a04038b40 msgr2=0x7f6a0403b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:21.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a04038b40 0x7f6a0403b000 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6a18100e50 tx=0x7f6a0800a380 comp rx=0 tx=0).stop 2023-10-08T13:44:21.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 -- 172.21.15.57:0/4012822877 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a18068a10 msgr2=0x7f6a180ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:21.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a18068a10 0x7f6a180ff720 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400d980 tx=0x7f6a1400dc90 comp rx=0 tx=0).stop 2023-10-08T13:44:21.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 -- 172.21.15.57:0/4012822877 shutdown_connections 2023-10-08T13:44:21.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a18106780 0x7f6a181001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a18068a10 0x7f6a180ff720 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a04038b40 0x7f6a0403b000 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 --2- 172.21.15.57:0/4012822877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a180693d0 0x7f6a180ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 -- 172.21.15.57:0/4012822877 >> 172.21.15.57:0/4012822877 conn(0x7f6a18076720 msgr2=0x7f6a18104280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:21.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 -- 172.21.15.57:0/4012822877 shutdown_connections 2023-10-08T13:44:21.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.702+0000 7f6a20371700 1 -- 172.21.15.57:0/4012822877 wait complete. 2023-10-08T13:44:21.773 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:21.780 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:21 smithi115 podman[107431]: 2023-10-08T13:44:21.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.840+0000 7f93542fb700 1 -- 172.21.15.57:0/1168597135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934c100390 msgr2=0x7f934c10c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:21.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.840+0000 7f93542fb700 1 --2- 172.21.15.57:0/1168597135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934c100390 0x7f934c10c550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f934400b0a0 tx=0x7f934400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.840+0000 7f93542fb700 1 -- 172.21.15.57:0/1168597135 shutdown_connections 2023-10-08T13:44:21.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.840+0000 7f93542fb700 1 --2- 172.21.15.57:0/1168597135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934c100390 0x7f934c10c550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.840+0000 7f93542fb700 1 --2- 172.21.15.57:0/1168597135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f934c1057e0 0x7f934c105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.840+0000 7f93542fb700 1 --2- 172.21.15.57:0/1168597135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f934c0ff9d0 0x7f934c0ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.840+0000 7f93542fb700 1 -- 172.21.15.57:0/1168597135 >> 172.21.15.57:0/1168597135 conn(0x7f934c0765b0 msgr2=0x7f934c0769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:21.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.840+0000 7f93542fb700 1 -- 172.21.15.57:0/1168597135 shutdown_connections 2023-10-08T13:44:21.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.840+0000 7f93542fb700 1 -- 172.21.15.57:0/1168597135 wait complete. 2023-10-08T13:44:21.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f93542fb700 1 Processor -- start 2023-10-08T13:44:21.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f93542fb700 1 -- start start 2023-10-08T13:44:21.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f93542fb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934c0ff9d0 0x7f934c1119e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:21.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f93542fb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f934c100390 0x7f934c10fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:21.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f93542fb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f934c1057e0 0x7f934c110480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:21.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f93542fb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c110c50 con 0x7f934c0ff9d0 2023-10-08T13:44:21.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f93542fb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1a51e0 con 0x7f934c1057e0 2023-10-08T13:44:21.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f93542fb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1a5320 con 0x7f934c100390 2023-10-08T13:44:21.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f9352097700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934c0ff9d0 0x7f934c1119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:21.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f9352097700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934c0ff9d0 0x7f934c1119e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52762/0 (socket says 172.21.15.57:52762) 2023-10-08T13:44:21.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f9352097700 1 -- 172.21.15.57:0/1996818623 learned_addr learned my addr 172.21.15.57:0/1996818623 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:21.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f9351896700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f934c100390 0x7f934c10fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f9352898700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f934c1057e0 0x7f934c110480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f9352097700 1 -- 172.21.15.57:0/1996818623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f934c100390 msgr2=0x7f934c10fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f9352097700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f934c100390 0x7f934c10fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f9352097700 1 -- 172.21.15.57:0/1996818623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f934c1057e0 msgr2=0x7f934c110480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f9352097700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f934c1057e0 0x7f934c110480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.841+0000 7f9352097700 1 -- 172.21.15.57:0/1996818623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344009d00 con 0x7f934c0ff9d0 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.842+0000 7f9352898700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f934c1057e0 0x7f934c110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.842+0000 7f9352097700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934c0ff9d0 0x7f934c1119e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f934800d9b0 tx=0x7f934800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.842+0000 7f93437fe700 1 -- 172.21.15.57:0/1996818623 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348004220 con 0x7f934c0ff9d0 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.842+0000 7f93437fe700 1 -- 172.21.15.57:0/1996818623 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(24 keys) v1 ==== 1049+0+0 (secure 0 0 0) 0x7f9348007640 con 0x7f934c0ff9d0 2023-10-08T13:44:21.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.842+0000 7f93437fe700 1 -- 172.21.15.57:0/1996818623 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348005500 con 0x7f934c0ff9d0 2023-10-08T13:44:21.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.842+0000 7f93542fb700 1 -- 172.21.15.57:0/1996818623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c1a5600 con 0x7f934c0ff9d0 2023-10-08T13:44:21.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.842+0000 7f93542fb700 1 -- 172.21.15.57:0/1996818623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c1a5ab0 con 0x7f934c0ff9d0 2023-10-08T13:44:21.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.843+0000 7f93437fe700 1 -- 172.21.15.57:0/1996818623 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7f9348005660 con 0x7f934c0ff9d0 2023-10-08T13:44:21.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.843+0000 7f93542fb700 1 -- 172.21.15.57:0/1996818623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c100e90 con 0x7f934c0ff9d0 2023-10-08T13:44:21.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.843+0000 7f93437fe700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9338038b40 0x7f933803b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:21.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.843+0000 7f93437fe700 1 -- 172.21.15.57:0/1996818623 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f934804b240 con 0x7f934c0ff9d0 2023-10-08T13:44:21.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.843+0000 7f9351896700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9338038b40 0x7f933803b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:21.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.843+0000 7f9351896700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9338038b40 0x7f933803b000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f934c111200 tx=0x7f933c0073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:21.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:21.848+0000 7f93437fe700 1 -- 172.21.15.57:0/1996818623 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9348022350 con 0x7f934c0ff9d0 2023-10-08T13:44:22.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.055+0000 7f93542fb700 1 -- 172.21.15.57:0/1996818623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) v1 -- 0x7f934c04e4c0 con 0x7f934c0ff9d0 2023-10-08T13:44:22.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.058+0000 7f93437fe700 1 -- 172.21.15.57:0/1996818623 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v13)=0 v13) v1 ==== 114+0+0 (secure 0 0 0) 0x7f9348016030 con 0x7f934c0ff9d0 2023-10-08T13:44:22.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.058+0000 7f93437fe700 1 -- 172.21.15.57:0/1996818623 <== mon.0 v2:172.21.15.57:3300/0 8 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9348022350 con 0x7f934c0ff9d0 2023-10-08T13:44:22.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 -- 172.21.15.57:0/1996818623 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9338038b40 msgr2=0x7f933803b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9338038b40 0x7f933803b000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f934c111200 tx=0x7f933c0073d0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 -- 172.21.15.57:0/1996818623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934c0ff9d0 msgr2=0x7f934c1119e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934c0ff9d0 0x7f934c1119e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f934800d9b0 tx=0x7f934800dcc0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 -- 172.21.15.57:0/1996818623 shutdown_connections 2023-10-08T13:44:22.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9338038b40 0x7f933803b000 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f934c1057e0 0x7f934c110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f934c0ff9d0 0x7f934c1119e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 --2- 172.21.15.57:0/1996818623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f934c100390 0x7f934c10fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 -- 172.21.15.57:0/1996818623 >> 172.21.15.57:0/1996818623 conn(0x7f934c0765b0 msgr2=0x7f934c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:22.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 -- 172.21.15.57:0/1996818623 shutdown_connections 2023-10-08T13:44:22.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.060+0000 7f93542fb700 1 -- 172.21.15.57:0/1996818623 wait complete. 2023-10-08T13:44:22.073 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:21 smithi115 ceph-mgr[107471]: -- 172.21.15.115:0/4193543429 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1f70411e0 con 0x55f1f7016c00 2023-10-08T13:44:22.073 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:21 smithi115 bash[107431]: 172b5b755eeb2e48a0a3b4cb740ca4f061ae3a91085e82adf580470612527900 2023-10-08T13:44:22.073 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:21 smithi115 systemd[1]: Started Ceph mgr.z for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:44:22.073 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:21 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:21.903+0000 7f1b30d3d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T13:44:22.073 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:21 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:21.983+0000 7f1b30d3d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T13:44:22.073 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:22.072+0000 7f1b30d3d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T13:44:22.335 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:22.165+0000 7f1b30d3d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T13:44:22.416 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi057') 2023-10-08T13:44:22.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.474+0000 7fb28cb2e700 1 -- 172.21.15.57:0/343629903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb288104720 msgr2=0x7fb288104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.474+0000 7fb28cb2e700 1 --2- 172.21.15.57:0/343629903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb288104720 0x7fb288104b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb278009990 tx=0x7fb278009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.474+0000 7fb28cb2e700 1 -- 172.21.15.57:0/343629903 shutdown_connections 2023-10-08T13:44:22.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.474+0000 7fb28cb2e700 1 --2- 172.21.15.57:0/343629903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288100c60 0x7fb28810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.474+0000 7fb28cb2e700 1 --2- 172.21.15.57:0/343629903 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb288104720 0x7fb288104b00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.474+0000 7fb28cb2e700 1 --2- 172.21.15.57:0/343629903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2881002a0 0x7fb288100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.474+0000 7fb28cb2e700 1 -- 172.21.15.57:0/343629903 >> 172.21.15.57:0/343629903 conn(0x7fb288074a10 msgr2=0x7fb288074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:22.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.474+0000 7fb28cb2e700 1 -- 172.21.15.57:0/343629903 shutdown_connections 2023-10-08T13:44:22.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.474+0000 7fb28cb2e700 1 -- 172.21.15.57:0/343629903 wait complete. 2023-10-08T13:44:22.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.475+0000 7fb28cb2e700 1 Processor -- start 2023-10-08T13:44:22.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.475+0000 7fb28cb2e700 1 -- start start 2023-10-08T13:44:22.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb28cb2e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2881002a0 0x7fb288101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:22.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb28cb2e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288100c60 0x7fb288101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:22.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb28cb2e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb288104720 0x7fb288101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:22.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb28cb2e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2881033e0 con 0x7fb288104720 2023-10-08T13:44:22.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb28cb2e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28819ac70 con 0x7fb2881002a0 2023-10-08T13:44:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb28cb2e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28819ae30 con 0x7fb288100c60 2023-10-08T13:44:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb286d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb288104720 0x7fb288101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb285d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288100c60 0x7fb288101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb285d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288100c60 0x7fb288101960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:53590/0 (socket says 172.21.15.57:53590) 2023-10-08T13:44:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb285d9b700 1 -- 172.21.15.57:0/4281413467 learned_addr learned my addr 172.21.15.57:0/4281413467 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb286d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb288104720 0x7fb288101ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52780/0 (socket says 172.21.15.57:52780) 2023-10-08T13:44:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb285d9b700 1 -- 172.21.15.57:0/4281413467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2881002a0 msgr2=0x7fb288101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb28659c700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2881002a0 0x7fb288101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb285d9b700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2881002a0 0x7fb288101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb285d9b700 1 -- 172.21.15.57:0/4281413467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb288104720 msgr2=0x7fb288101ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb285d9b700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb288104720 0x7fb288101ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb285d9b700 1 -- 172.21.15.57:0/4281413467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2780095f0 con 0x7fb288100c60 2023-10-08T13:44:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb286d9d700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb288104720 0x7fb288101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:44:22.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.476+0000 7fb285d9b700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288100c60 0x7fb288101960 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00e9d0 tx=0x7fb27c00ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:22.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.477+0000 7fb2777fe700 1 -- 172.21.15.57:0/4281413467 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c009790 con 0x7fb288100c60 2023-10-08T13:44:22.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.477+0000 7fb2777fe700 1 -- 172.21.15.57:0/4281413467 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb27c00cbd0 con 0x7fb288100c60 2023-10-08T13:44:22.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.477+0000 7fb2777fe700 1 -- 172.21.15.57:0/4281413467 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c018680 con 0x7fb288100c60 2023-10-08T13:44:22.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.477+0000 7fb28cb2e700 1 -- 172.21.15.57:0/4281413467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28819b110 con 0x7fb288100c60 2023-10-08T13:44:22.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.477+0000 7fb28cb2e700 1 -- 172.21.15.57:0/4281413467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28819b610 con 0x7fb288100c60 2023-10-08T13:44:22.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.478+0000 7fb2777fe700 1 -- 172.21.15.57:0/4281413467 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7fb27c0187e0 con 0x7fb288100c60 2023-10-08T13:44:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.478+0000 7fb28cb2e700 1 -- 172.21.15.57:0/4281413467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb288108cb0 con 0x7fb288100c60 2023-10-08T13:44:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.478+0000 7fb2777fe700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb270038b40 0x7fb27003b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.478+0000 7fb2777fe700 1 -- 172.21.15.57:0/4281413467 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb27c014070 con 0x7fb288100c60 2023-10-08T13:44:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.478+0000 7fb28659c700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb270038b40 0x7fb27003b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.479+0000 7fb28659c700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb270038b40 0x7fb27003b000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb27800b2e0 tx=0x7fb278005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:22.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.483+0000 7fb2777fe700 1 -- 172.21.15.57:0/4281413467 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb27c010e40 con 0x7fb288100c60 2023-10-08T13:44:22.686 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:22.334+0000 7f1b30d3d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T13:44:22.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.691+0000 7fb28cb2e700 1 -- 172.21.15.57:0/4281413467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7fb288102900 con 0x7fb288100c60 2023-10-08T13:44:22.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.695+0000 7fb2777fe700 1 -- 172.21.15.57:0/4281413467 <== mon.1 v2:172.21.15.115:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb27c020e90 con 0x7fb288100c60 2023-10-08T13:44:22.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.696+0000 7fb2777fe700 1 -- 172.21.15.57:0/4281413467 <== mon.1 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7fb27c00cd40 con 0x7fb288100c60 2023-10-08T13:44:22.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 -- 172.21.15.57:0/4281413467 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb270038b40 msgr2=0x7fb27003b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb270038b40 0x7fb27003b000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb27800b2e0 tx=0x7fb278005e80 comp rx=0 tx=0).stop 2023-10-08T13:44:22.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 -- 172.21.15.57:0/4281413467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288100c60 msgr2=0x7fb288101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288100c60 0x7fb288101960 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00e9d0 tx=0x7fb27c00ece0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 -- 172.21.15.57:0/4281413467 shutdown_connections 2023-10-08T13:44:22.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb288104720 0x7fb288101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb270038b40 0x7fb27003b000 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2881002a0 0x7fb288101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 --2- 172.21.15.57:0/4281413467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288100c60 0x7fb288101960 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 -- 172.21.15.57:0/4281413467 >> 172.21.15.57:0/4281413467 conn(0x7fb288074a10 msgr2=0x7fb2880ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:22.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 -- 172.21.15.57:0/4281413467 shutdown_connections 2023-10-08T13:44:22.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.699+0000 7fb28cb2e700 1 -- 172.21.15.57:0/4281413467 wait complete. 2023-10-08T13:44:22.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.629+0000 7fab5bc3d700 1 -- 172.21.15.57:0/1386849512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab54068e40 msgr2=0x7fab540692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.629+0000 7fab5bc3d700 1 --2- 172.21.15.57:0/1386849512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab54068e40 0x7fab540692c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab44009990 tx=0x7fab44009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.629+0000 7fab5bc3d700 1 -- 172.21.15.57:0/1386849512 shutdown_connections 2023-10-08T13:44:22.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.629+0000 7fab5bc3d700 1 --2- 172.21.15.57:0/1386849512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab54069800 0x7fab5410d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.629+0000 7fab5bc3d700 1 --2- 172.21.15.57:0/1386849512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab541067d0 0x7fab54106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.629+0000 7fab5bc3d700 1 --2- 172.21.15.57:0/1386849512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab54068e40 0x7fab540692c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.629+0000 7fab5bc3d700 1 -- 172.21.15.57:0/1386849512 >> 172.21.15.57:0/1386849512 conn(0x7fab540764a0 msgr2=0x7fab540768b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:22.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.630+0000 7fab5bc3d700 1 -- 172.21.15.57:0/1386849512 shutdown_connections 2023-10-08T13:44:22.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.630+0000 7fab5bc3d700 1 -- 172.21.15.57:0/1386849512 wait complete. 2023-10-08T13:44:22.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.630+0000 7fab5bc3d700 1 Processor -- start 2023-10-08T13:44:22.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.630+0000 7fab5bc3d700 1 -- start start 2023-10-08T13:44:22.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.630+0000 7fab5bc3d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab54068e40 0x7fab5419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:22.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.630+0000 7fab5bc3d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab54069800 0x7fab5419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:22.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.630+0000 7fab5bc3d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab541067d0 0x7fab54196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:22.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.630+0000 7fab5bc3d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab54197070 con 0x7fab54069800 2023-10-08T13:44:22.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.630+0000 7fab5bc3d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab541971e0 con 0x7fab541067d0 2023-10-08T13:44:22.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.630+0000 7fab5bc3d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab541973a0 con 0x7fab54068e40 2023-10-08T13:44:22.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab591d8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab54069800 0x7fab5419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:22.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab599d9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab54068e40 0x7fab5419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:22.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab591d8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab54069800 0x7fab5419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52792/0 (socket says 172.21.15.57:52792) 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab591d8700 1 -- 172.21.15.57:0/38765521 learned_addr learned my addr 172.21.15.57:0/38765521 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab5a1da700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab541067d0 0x7fab54196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab591d8700 1 -- 172.21.15.57:0/38765521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab54068e40 msgr2=0x7fab5419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab591d8700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab54068e40 0x7fab5419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab591d8700 1 -- 172.21.15.57:0/38765521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab541067d0 msgr2=0x7fab54196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab591d8700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab541067d0 0x7fab54196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab591d8700 1 -- 172.21.15.57:0/38765521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440095f0 con 0x7fab54069800 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab599d9700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab54068e40 0x7fab5419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab5a1da700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab541067d0 0x7fab54196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab591d8700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab54069800 0x7fab5419cdc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab44005910 tx=0x7fab44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:22.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab4affd700 1 -- 172.21.15.57:0/38765521 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4401d070 con 0x7fab54069800 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab5bc3d700 1 -- 172.21.15.57:0/38765521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab54197620 con 0x7fab54069800 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab4affd700 1 -- 172.21.15.57:0/38765521 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fab44022470 con 0x7fab54069800 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab4affd700 1 -- 172.21.15.57:0/38765521 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44017650 con 0x7fab54069800 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.631+0000 7fab5bc3d700 1 -- 172.21.15.57:0/38765521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab541a3870 con 0x7fab54069800 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.632+0000 7fab4affd700 1 -- 172.21.15.57:0/38765521 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7fab4400fac0 con 0x7fab54069800 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.632+0000 7fab5bc3d700 1 -- 172.21.15.57:0/38765521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab54108cf0 con 0x7fab54069800 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.633+0000 7fab4affd700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab40038b40 0x7fab4003b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.633+0000 7fab4affd700 1 -- 172.21.15.57:0/38765521 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fab4404c9a0 con 0x7fab54069800 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.633+0000 7fab599d9700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab40038b40 0x7fab4003b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.633+0000 7fab599d9700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab40038b40 0x7fab4003b000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fab50009fd0 tx=0x7fab50009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:22.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.638+0000 7fab4affd700 1 -- 172.21.15.57:0/38765521 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fab44016580 con 0x7fab54069800 2023-10-08T13:44:22.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.695+0000 7fab4affd700 1 -- 172.21.15.57:0/38765521 <== mon.0 v2:172.21.15.57:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab44016580 con 0x7fab54069800 2023-10-08T13:44:22.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.864+0000 7fab5bc3d700 1 -- 172.21.15.57:0/38765521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7fab3c0051e0 con 0x7fab54069800 2023-10-08T13:44:22.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.867+0000 7fab4affd700 1 -- 172.21.15.57:0/38765521 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v15)=0 v15) v1 ==== 108+0+0 (secure 0 0 0) 0x7fab44027020 con 0x7fab54069800 2023-10-08T13:44:22.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 -- 172.21.15.57:0/38765521 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab40038b40 msgr2=0x7fab4003b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab40038b40 0x7fab4003b000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fab50009fd0 tx=0x7fab50009260 comp rx=0 tx=0).stop 2023-10-08T13:44:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 -- 172.21.15.57:0/38765521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab54069800 msgr2=0x7fab5419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab54069800 0x7fab5419cdc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab44005910 tx=0x7fab44004730 comp rx=0 tx=0).stop 2023-10-08T13:44:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 -- 172.21.15.57:0/38765521 shutdown_connections 2023-10-08T13:44:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab40038b40 0x7fab4003b000 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab541067d0 0x7fab54196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab54068e40 0x7fab5419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 --2- 172.21.15.57:0/38765521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab54069800 0x7fab5419cdc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 -- 172.21.15.57:0/38765521 >> 172.21.15.57:0/38765521 conn(0x7fab540764a0 msgr2=0x7fab540fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.869+0000 7fab5bc3d700 1 -- 172.21.15.57:0/38765521 shutdown_connections 2023-10-08T13:44:22.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:22.870+0000 7fab5bc3d700 1 -- 172.21.15.57:0/38765521 wait complete. 2023-10-08T13:44:22.905 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4012822877' entity='client.admin' 2023-10-08T13:44:22.905 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:22.905 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:22.906 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:22.906 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T13:44:22.906 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T13:44:22.906 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T13:44:22.906 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:22.906 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: Deploying daemon mgr.y on smithi099 2023-10-08T13:44:22.906 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1996818623' entity='client.admin' 2023-10-08T13:44:22.906 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:22 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' 2023-10-08T13:44:22.980 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4012822877' entity='client.admin' 2023-10-08T13:44:22.980 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:22.980 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:22.980 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:22.980 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T13:44:22.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T13:44:22.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T13:44:22.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:22.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: Deploying daemon mgr.y on smithi099 2023-10-08T13:44:22.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1996818623' entity='client.admin' 2023-10-08T13:44:22.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:22 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' 2023-10-08T13:44:23.010 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:22.700+0000 7f1b30d3d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T13:44:23.010 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:22.828+0000 7f1b30d3d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T13:44:23.010 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:22.916+0000 7f1b30d3d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T13:44:23.010 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4012822877' entity='client.admin' 2023-10-08T13:44:23.010 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:23.010 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:23.010 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:23.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T13:44:23.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T13:44:23.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T13:44:23.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:23.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: Deploying daemon mgr.y on smithi099 2023-10-08T13:44:23.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1996818623' entity='client.admin' 2023-10-08T13:44:23.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:22 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' 2023-10-08T13:44:23.016 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi057') 2023-10-08T13:44:23.017 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi057') 2023-10-08T13:44:23.017 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi057') 2023-10-08T13:44:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.040+0000 7f24480bd700 1 -- 172.21.15.57:0/1197805609 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24401058d0 msgr2=0x7f2440105cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.040+0000 7f24480bd700 1 --2- 172.21.15.57:0/1197805609 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24401058d0 0x7f2440105cb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f243c009990 tx=0x7f243c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.041+0000 7f24480bd700 1 -- 172.21.15.57:0/1197805609 shutdown_connections 2023-10-08T13:44:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.041+0000 7f24480bd700 1 --2- 172.21.15.57:0/1197805609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24400ffb30 0x7f244010d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.041+0000 7f24480bd700 1 --2- 172.21.15.57:0/1197805609 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24401058d0 0x7f2440105cb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.041+0000 7f24480bd700 1 --2- 172.21.15.57:0/1197805609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24400ff190 0x7f24400ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.041+0000 7f24480bd700 1 -- 172.21.15.57:0/1197805609 >> 172.21.15.57:0/1197805609 conn(0x7f2440074a10 msgr2=0x7f2440074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:23.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.041+0000 7f24480bd700 1 -- 172.21.15.57:0/1197805609 shutdown_connections 2023-10-08T13:44:23.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.041+0000 7f24480bd700 1 -- 172.21.15.57:0/1197805609 wait complete. 2023-10-08T13:44:23.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.041+0000 7f24480bd700 1 Processor -- start 2023-10-08T13:44:23.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.041+0000 7f24480bd700 1 -- start start 2023-10-08T13:44:23.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f24480bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24400ff190 0x7f244019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:23.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f24480bd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24400ffb30 0x7f244019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:23.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f24480bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24401058d0 0x7f2440196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:23.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f24480bd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401970a0 con 0x7f24400ffb30 2023-10-08T13:44:23.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f24480bd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2440197210 con 0x7f24400ff190 2023-10-08T13:44:23.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f24480bd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401973d0 con 0x7f24401058d0 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f2445658700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24400ffb30 0x7f244019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f2445658700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24400ffb30 0x7f244019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52802/0 (socket says 172.21.15.57:52802) 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f2445658700 1 -- 172.21.15.57:0/3348806981 learned_addr learned my addr 172.21.15.57:0/3348806981 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f2445e59700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24400ff190 0x7f244019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f244665a700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24401058d0 0x7f2440196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f2445658700 1 -- 172.21.15.57:0/3348806981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24401058d0 msgr2=0x7f2440196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f2445658700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24401058d0 0x7f2440196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f2445658700 1 -- 172.21.15.57:0/3348806981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24400ff190 msgr2=0x7f244019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f2445658700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24400ff190 0x7f244019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f2445658700 1 -- 172.21.15.57:0/3348806981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c0095f0 con 0x7f24400ffb30 2023-10-08T13:44:23.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.042+0000 7f2445658700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24400ffb30 0x7f244019ce40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f243000e9a0 tx=0x7f243000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:23.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.043+0000 7f2436ffd700 1 -- 172.21.15.57:0/3348806981 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430009790 con 0x7f24400ffb30 2023-10-08T13:44:23.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.043+0000 7f24480bd700 1 -- 172.21.15.57:0/3348806981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24401976b0 con 0x7f24400ffb30 2023-10-08T13:44:23.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.043+0000 7f2436ffd700 1 -- 172.21.15.57:0/3348806981 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2430010430 con 0x7f24400ffb30 2023-10-08T13:44:23.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.043+0000 7f2436ffd700 1 -- 172.21.15.57:0/3348806981 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430018680 con 0x7f24400ffb30 2023-10-08T13:44:23.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.043+0000 7f24480bd700 1 -- 172.21.15.57:0/3348806981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2440100470 con 0x7f24400ffb30 2023-10-08T13:44:23.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.044+0000 7f2436ffd700 1 -- 172.21.15.57:0/3348806981 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7f24300187e0 con 0x7f24400ffb30 2023-10-08T13:44:23.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.044+0000 7f2436ffd700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f242c038b40 0x7f242c03b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:23.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.044+0000 7f2436ffd700 1 -- 172.21.15.57:0/3348806981 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f2430014070 con 0x7f24400ffb30 2023-10-08T13:44:23.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.044+0000 7f24480bd700 1 -- 172.21.15.57:0/3348806981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2440108d30 con 0x7f24400ffb30 2023-10-08T13:44:23.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.044+0000 7f2445e59700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f242c038b40 0x7f242c03b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:23.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.044+0000 7f2445e59700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f242c038b40 0x7f242c03b000 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f243c006010 tx=0x7f243c020040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:23.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.049+0000 7f2436ffd700 1 -- 172.21.15.57:0/3348806981 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f243001fd30 con 0x7f24400ffb30 2023-10-08T13:44:23.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.293+0000 7f24480bd700 1 -- 172.21.15.57:0/3348806981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7f244004e4f0 con 0x7f24400ffb30 2023-10-08T13:44:23.294 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:23.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.301+0000 7f2436ffd700 1 -- 172.21.15.57:0/3348806981 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v16)=0 v16) v1 ==== 117+0+0 (secure 0 0 0) 0x7f2430029070 con 0x7f24400ffb30 2023-10-08T13:44:23.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.304+0000 7f24480bd700 1 -- 172.21.15.57:0/3348806981 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f242c038b40 msgr2=0x7f242c03b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:23.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.304+0000 7f24480bd700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f242c038b40 0x7f242c03b000 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f243c006010 tx=0x7f243c020040 comp rx=0 tx=0).stop 2023-10-08T13:44:23.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.304+0000 7f24480bd700 1 -- 172.21.15.57:0/3348806981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24400ffb30 msgr2=0x7f244019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:23.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.304+0000 7f24480bd700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24400ffb30 0x7f244019ce40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f243000e9a0 tx=0x7f243000ecb0 comp rx=0 tx=0).stop 2023-10-08T13:44:23.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.305+0000 7f24480bd700 1 -- 172.21.15.57:0/3348806981 shutdown_connections 2023-10-08T13:44:23.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.305+0000 7f24480bd700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24401058d0 0x7f2440196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:23.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.305+0000 7f24480bd700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f242c038b40 0x7f242c03b000 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f243c006010 tx=0x7f243c020040 comp rx=0 tx=0).stop 2023-10-08T13:44:23.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.305+0000 7f24480bd700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24400ff190 0x7f244019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:23.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.305+0000 7f24480bd700 1 --2- 172.21.15.57:0/3348806981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24400ffb30 0x7f244019ce40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:23.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.305+0000 7f24480bd700 1 -- 172.21.15.57:0/3348806981 >> 172.21.15.57:0/3348806981 conn(0x7f2440074a10 msgr2=0x7f24400fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:23.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.305+0000 7f24480bd700 1 -- 172.21.15.57:0/3348806981 shutdown_connections 2023-10-08T13:44:23.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:23.306+0000 7f24480bd700 1 -- 172.21.15.57:0/3348806981 wait complete. 2023-10-08T13:44:23.427 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:23 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:23.009+0000 7f1b30d3d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T13:44:23.686 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:23 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:23.426+0000 7f1b30d3d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T13:44:23.752 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:23 smithi099 systemd[1]: Starting Ceph mgr.y for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:44:23.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:23 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/38765521' entity='client.admin' 2023-10-08T13:44:23.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:23 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3348806981' entity='client.admin' 2023-10-08T13:44:24.022 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:23 smithi099 podman[106215]: 2023-10-08T13:44:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:23 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/38765521' entity='client.admin' 2023-10-08T13:44:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:23 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3348806981' entity='client.admin' 2023-10-08T13:44:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:24.169 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:23 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/38765521' entity='client.admin' 2023-10-08T13:44:24.169 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:23 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3348806981' entity='client.admin' 2023-10-08T13:44:24.170 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:24 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:24.087+0000 7f1b30d3d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T13:44:24.344 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:24 smithi099 ceph-mgr[106256]: -- 172.21.15.99:0/4166055922 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bf7f9f91e0 con 0x55bf7f9cec00 2023-10-08T13:44:24.345 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:24 smithi099 bash[106215]: bd41aa1c9b2f1165b1cb4813ee007f2a9f0c1198d976e7d1c6ce3af5396c6073 2023-10-08T13:44:24.345 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:24 smithi099 systemd[1]: Started Ceph mgr.y for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:44:24.345 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:24.170+0000 7f0d807f5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T13:44:24.345 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:24.254+0000 7f0d807f5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T13:44:24.345 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:24.343+0000 7f0d807f5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T13:44:24.436 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:24 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:24.168+0000 7f1b30d3d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T13:44:24.596 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi057') 2023-10-08T13:44:24.609 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:24.623 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:24.443+0000 7f0d807f5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T13:44:24.696 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi057') 2023-10-08T13:44:24.696 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi057') 2023-10-08T13:44:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 -- 172.21.15.57:0/2942438957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f308c10d300 msgr2=0x7f308c10f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 --2- 172.21.15.57:0/2942438957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f308c10d300 0x7f308c10f6f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f308400b0a0 tx=0x7f308400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 -- 172.21.15.57:0/2942438957 shutdown_connections 2023-10-08T13:44:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 --2- 172.21.15.57:0/2942438957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f308c10d300 0x7f308c10f6f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 --2- 172.21.15.57:0/2942438957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f308c068190 0x7f308c068570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 --2- 172.21.15.57:0/2942438957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f308c068ab0 0x7f308c10cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 -- 172.21.15.57:0/2942438957 >> 172.21.15.57:0/2942438957 conn(0x7f308c075790 msgr2=0x7f308c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:24.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 -- 172.21.15.57:0/2942438957 shutdown_connections 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 -- 172.21.15.57:0/2942438957 wait complete. 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 Processor -- start 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 -- start start 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.766+0000 7f3093e91700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f308c068190 0x7f308c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3093e91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f308c068ab0 0x7f308c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3093e91700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f308c10d300 0x7f308c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3093e91700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c19dde0 con 0x7f308c068190 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3093e91700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c19df50 con 0x7f308c068ab0 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3093e91700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c19e110 con 0x7f308c10d300 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3091c2d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f308c068190 0x7f308c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:24.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3091c2d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f308c068190 0x7f308c198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52814/0 (socket says 172.21.15.57:52814) 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3091c2d700 1 -- 172.21.15.57:0/3456526517 learned_addr learned my addr 172.21.15.57:0/3456526517 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f309142c700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f308c068ab0 0x7f308c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f309242e700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f308c10d300 0x7f308c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3091c2d700 1 -- 172.21.15.57:0/3456526517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f308c10d300 msgr2=0x7f308c19d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3091c2d700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f308c10d300 0x7f308c19d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3091c2d700 1 -- 172.21.15.57:0/3456526517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f308c068ab0 msgr2=0x7f308c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3091c2d700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f308c068ab0 0x7f308c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3091c2d700 1 -- 172.21.15.57:0/3456526517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3084009d00 con 0x7f308c068190 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3091c2d700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f308c068190 0x7f308c198f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f308800c9b0 tx=0x7f308800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3082ffd700 1 -- 172.21.15.57:0/3456526517 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088007b00 con 0x7f308c068190 2023-10-08T13:44:24.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3082ffd700 1 -- 172.21.15.57:0/3456526517 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f308800ce50 con 0x7f308c068190 2023-10-08T13:44:24.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3093e91700 1 -- 172.21.15.57:0/3456526517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c19e3f0 con 0x7f308c068190 2023-10-08T13:44:24.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3082ffd700 1 -- 172.21.15.57:0/3456526517 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088018660 con 0x7f308c068190 2023-10-08T13:44:24.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.767+0000 7f3093e91700 1 -- 172.21.15.57:0/3456526517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c1a5c50 con 0x7f308c068190 2023-10-08T13:44:24.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.768+0000 7f3082ffd700 1 -- 172.21.15.57:0/3456526517 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7f3088004830 con 0x7f308c068190 2023-10-08T13:44:24.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.768+0000 7f3082ffd700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3078038b40 0x7f307803b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:24.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.768+0000 7f3082ffd700 1 -- 172.21.15.57:0/3456526517 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f308804b3e0 con 0x7f308c068190 2023-10-08T13:44:24.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.768+0000 7f3093e91700 1 -- 172.21.15.57:0/3456526517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308c108520 con 0x7f308c068190 2023-10-08T13:44:24.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.768+0000 7f309142c700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3078038b40 0x7f307803b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:24.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.769+0000 7f309142c700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3078038b40 0x7f307803b000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f308c19a440 tx=0x7f307c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:24.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.774+0000 7f3082ffd700 1 -- 172.21.15.57:0/3456526517 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3088004b30 con 0x7f308c068190 2023-10-08T13:44:24.894 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:24.898 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:24.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.984+0000 7f3093e91700 1 -- 172.21.15.57:0/3456526517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7f308c0ffd30 con 0x7f308c068190 2023-10-08T13:44:24.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.988+0000 7f3082ffd700 1 -- 172.21.15.57:0/3456526517 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v17)=0 v17) v1 ==== 106+0+0 (secure 0 0 0) 0x7f308803e030 con 0x7f308c068190 2023-10-08T13:44:24.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.990+0000 7f3093e91700 1 -- 172.21.15.57:0/3456526517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3078038b40 msgr2=0x7f307803b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:24.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.990+0000 7f3093e91700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3078038b40 0x7f307803b000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f308c19a440 tx=0x7f307c00b410 comp rx=0 tx=0).stop 2023-10-08T13:44:24.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.991+0000 7f3093e91700 1 -- 172.21.15.57:0/3456526517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f308c068190 msgr2=0x7f308c198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:24.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.991+0000 7f3093e91700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f308c068190 0x7f308c198f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f308800c9b0 tx=0x7f308800ccc0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.991+0000 7f3093e91700 1 -- 172.21.15.57:0/3456526517 shutdown_connections 2023-10-08T13:44:24.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.991+0000 7f3093e91700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f308c10d300 0x7f308c19d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.991+0000 7f3093e91700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f308c068190 0x7f308c198f30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.991+0000 7f3093e91700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3078038b40 0x7f307803b000 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.991+0000 7f3093e91700 1 --2- 172.21.15.57:0/3456526517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f308c068ab0 0x7f308c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:24.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.991+0000 7f3093e91700 1 -- 172.21.15.57:0/3456526517 >> 172.21.15.57:0/3456526517 conn(0x7f308c075790 msgr2=0x7f308c10e750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:24.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.991+0000 7f3093e91700 1 -- 172.21.15.57:0/3456526517 shutdown_connections 2023-10-08T13:44:24.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:24.991+0000 7f3093e91700 1 -- 172.21.15.57:0/3456526517 wait complete. 2023-10-08T13:44:25.022 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:24.622+0000 7f0d807f5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T13:44:25.032 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:25.032 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:25.097 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:25.098 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:25.148 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:25.226 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:24 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:24.973+0000 7f1b30d3d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:44:25.226 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:25.074+0000 7f1b30d3d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T13:44:25.347 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:25.021+0000 7f0d807f5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T13:44:25.348 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:25.156+0000 7f0d807f5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T13:44:25.348 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:25.250+0000 7f0d807f5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T13:44:25.348 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.348 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.348 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.348 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.348 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:25.348 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3456526517' entity='client.admin' 2023-10-08T13:44:25.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:25.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:25 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3456526517' entity='client.admin' 2023-10-08T13:44:25.550 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:25.255+0000 7f1b30d3d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:44:25.551 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.551 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.551 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.551 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:25.551 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:25.551 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3456526517' entity='client.admin' 2023-10-08T13:44:25.726 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:25.726 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:25.900 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:25.936 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:25.549+0000 7f1b30d3d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T13:44:25.936 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:25.787+0000 7f1b30d3d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T13:44:26.216 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:25.946+0000 7f1b30d3d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T13:44:26.272 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:25.346+0000 7f0d807f5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T13:44:26.272 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:25.796+0000 7f0d807f5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T13:44:26.485 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi057') 2023-10-08T13:44:26.686 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:26.215+0000 7f1b30d3d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T13:44:26.772 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:26.470+0000 7f0d807f5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T13:44:26.772 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:26.551+0000 7f0d807f5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T13:44:26.864 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:27.186 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:26.911+0000 7f1b30d3d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T13:44:27.186 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:27.043+0000 7f1b30d3d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T13:44:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:27.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: Reconfiguring mgr.x (unknown last config time)... 2023-10-08T13:44:27.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T13:44:27.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T13:44:27.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:27.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:26 smithi115 ceph-mon[104408]: Reconfiguring daemon mgr.x on smithi057 2023-10-08T13:44:27.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: Reconfiguring mgr.x (unknown last config time)... 2023-10-08T13:44:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T13:44:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T13:44:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:26 smithi057 ceph-mon[99476]: Reconfiguring daemon mgr.x on smithi057 2023-10-08T13:44:27.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:27.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:27.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:27.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: Reconfiguring mgr.x (unknown last config time)... 2023-10-08T13:44:27.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T13:44:27.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T13:44:27.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:27.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:26 smithi099 ceph-mon[103849]: Reconfiguring daemon mgr.x on smithi057 2023-10-08T13:44:27.385 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:27.564 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:27.235+0000 7f1b30d3d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T13:44:27.564 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:27.347+0000 7f1b30d3d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T13:44:27.578 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:27.315+0000 7f0d807f5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:44:27.578 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:27.406+0000 7f0d807f5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T13:44:27.859 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:27.578+0000 7f0d807f5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T13:44:27.936 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:27.563+0000 7f1b30d3d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T13:44:27.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.880+0000 7fa866397700 1 -- 172.21.15.57:0/2897377277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa860107040 msgr2=0x7fa860107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:27.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.880+0000 7fa866397700 1 --2- 172.21.15.57:0/2897377277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa860107040 0x7fa860107420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa850009990 tx=0x7fa850009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:27.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.880+0000 7fa866397700 1 -- 172.21.15.57:0/2897377277 shutdown_connections 2023-10-08T13:44:27.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.880+0000 7fa866397700 1 --2- 172.21.15.57:0/2897377277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa86010c120 0x7fa860113cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:27.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.880+0000 7fa866397700 1 --2- 172.21.15.57:0/2897377277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa860107040 0x7fa860107420 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:27.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.880+0000 7fa866397700 1 --2- 172.21.15.57:0/2897377277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8601079f0 0x7fa86010ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:27.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.881+0000 7fa866397700 1 -- 172.21.15.57:0/2897377277 >> 172.21.15.57:0/2897377277 conn(0x7fa860073fc0 msgr2=0x7fa8600763e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:27.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.881+0000 7fa866397700 1 -- 172.21.15.57:0/2897377277 shutdown_connections 2023-10-08T13:44:27.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.881+0000 7fa866397700 1 -- 172.21.15.57:0/2897377277 wait complete. 2023-10-08T13:44:27.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.882+0000 7fa866397700 1 Processor -- start 2023-10-08T13:44:27.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.882+0000 7fa866397700 1 -- start start 2023-10-08T13:44:27.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa866397700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa860107040 0x7fa86019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa866397700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8601079f0 0x7fa86019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa866397700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa86010c120 0x7fa8601a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa866397700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8601a2160 con 0x7fa8601079f0 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa866397700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8601a22d0 con 0x7fa86010c120 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa866397700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8601a2490 con 0x7fa860107040 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8601079f0 0x7fa86019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8601079f0 0x7fa86019d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52842/0 (socket says 172.21.15.57:52842) 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa864934700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa86010c120 0x7fa8601a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85f7fe700 1 -- 172.21.15.57:0/1329111525 learned_addr learned my addr 172.21.15.57:0/1329111525 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85ffff700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa860107040 0x7fa86019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:27.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85f7fe700 1 -- 172.21.15.57:0/1329111525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa860107040 msgr2=0x7fa86019d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85f7fe700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa860107040 0x7fa86019d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85f7fe700 1 -- 172.21.15.57:0/1329111525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa86010c120 msgr2=0x7fa8601a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85f7fe700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa86010c120 0x7fa8601a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85f7fe700 1 -- 172.21.15.57:0/1329111525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8500095f0 con 0x7fa8601079f0 2023-10-08T13:44:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85ffff700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa860107040 0x7fa86019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:44:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa864934700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa86010c120 0x7fa8601a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:44:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.883+0000 7fa85f7fe700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8601079f0 0x7fa86019d7f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa85400b500 tx=0x7fa85400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.884+0000 7fa85d7fa700 1 -- 172.21.15.57:0/1329111525 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854011830 con 0x7fa8601079f0 2023-10-08T13:44:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.884+0000 7fa866397700 1 -- 172.21.15.57:0/1329111525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8601a2770 con 0x7fa8601079f0 2023-10-08T13:44:27.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.884+0000 7fa85d7fa700 1 -- 172.21.15.57:0/1329111525 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa854011e50 con 0x7fa8601079f0 2023-10-08T13:44:27.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.884+0000 7fa85d7fa700 1 -- 172.21.15.57:0/1329111525 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854010de0 con 0x7fa8601079f0 2023-10-08T13:44:27.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.884+0000 7fa866397700 1 -- 172.21.15.57:0/1329111525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8601a9fd0 con 0x7fa8601079f0 2023-10-08T13:44:27.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.885+0000 7fa866397700 1 -- 172.21.15.57:0/1329111525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86010f390 con 0x7fa8601079f0 2023-10-08T13:44:27.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.886+0000 7fa85d7fa700 1 -- 172.21.15.57:0/1329111525 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7fa854010480 con 0x7fa8601079f0 2023-10-08T13:44:27.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.886+0000 7fa85d7fa700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa848038b40 0x7fa84803b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:27.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.886+0000 7fa85d7fa700 1 -- 172.21.15.57:0/1329111525 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa85404b4b0 con 0x7fa8601079f0 2023-10-08T13:44:27.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.886+0000 7fa85ffff700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa848038b40 0x7fa84803b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:27.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.887+0000 7fa85ffff700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa848038b40 0x7fa84803b000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa85000b5c0 tx=0x7fa85001a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:27.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.891+0000 7fa85d7fa700 1 -- 172.21.15.57:0/1329111525 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa854022b40 con 0x7fa8601079f0 2023-10-08T13:44:28.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.124+0000 7fa866397700 1 -- 172.21.15.57:0/1329111525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7fa86019e570 con 0x7fa8601079f0 2023-10-08T13:44:28.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.129+0000 7fa85d7fa700 1 -- 172.21.15.57:0/1329111525 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v19)=0 v19) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa854015020 con 0x7fa8601079f0 2023-10-08T13:44:28.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 -- 172.21.15.57:0/1329111525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa848038b40 msgr2=0x7fa84803b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:28.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa848038b40 0x7fa84803b000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa85000b5c0 tx=0x7fa85001a040 comp rx=0 tx=0).stop 2023-10-08T13:44:28.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 -- 172.21.15.57:0/1329111525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8601079f0 msgr2=0x7fa86019d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:28.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8601079f0 0x7fa86019d7f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa85400b500 tx=0x7fa85400b810 comp rx=0 tx=0).stop 2023-10-08T13:44:28.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 -- 172.21.15.57:0/1329111525 shutdown_connections 2023-10-08T13:44:28.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa848038b40 0x7fa84803b000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa86010c120 0x7fa8601a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa860107040 0x7fa86019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 --2- 172.21.15.57:0/1329111525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8601079f0 0x7fa86019d7f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 -- 172.21.15.57:0/1329111525 >> 172.21.15.57:0/1329111525 conn(0x7fa860073fc0 msgr2=0x7fa860075300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:28.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 -- 172.21.15.57:0/1329111525 shutdown_connections 2023-10-08T13:44:28.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.132+0000 7fa866397700 1 -- 172.21.15.57:0/1329111525 wait complete. 2023-10-08T13:44:28.239 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:27.859+0000 7f0d807f5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T13:44:28.239 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:28 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:28.093+0000 7f0d807f5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T13:44:28.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.783+0000 7f3195b08700 1 -- 172.21.15.57:0/1328741381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3190100840 msgr2=0x7f3190100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.783+0000 7f3195b08700 1 --2- 172.21.15.57:0/1328741381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3190100840 0x7f3190100cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3184009990 tx=0x7f3184009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.783+0000 7f3195b08700 1 -- 172.21.15.57:0/1328741381 shutdown_connections 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.783+0000 7f3195b08700 1 --2- 172.21.15.57:0/1328741381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3190101200 0x7f319010d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.783+0000 7f3195b08700 1 --2- 172.21.15.57:0/1328741381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31901056a0 0x7f3190105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.783+0000 7f3195b08700 1 --2- 172.21.15.57:0/1328741381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3190100840 0x7f3190100cc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.783+0000 7f3195b08700 1 -- 172.21.15.57:0/1328741381 >> 172.21.15.57:0/1328741381 conn(0x7f3190077950 msgr2=0x7f3190077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.783+0000 7f3195b08700 1 -- 172.21.15.57:0/1328741381 shutdown_connections 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.783+0000 7f3195b08700 1 -- 172.21.15.57:0/1328741381 wait complete. 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f3195b08700 1 Processor -- start 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f3195b08700 1 -- start start 2023-10-08T13:44:28.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f3195b08700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3190100840 0x7f319019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f3195b08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3190101200 0x7f319019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f3195b08700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31901056a0 0x7f3190196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f3195b08700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3190197190 con 0x7f31901056a0 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f3195b08700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3190197300 con 0x7f3190101200 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f3195b08700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31901974c0 con 0x7f3190100840 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f318effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3190101200 0x7f319019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f318ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31901056a0 0x7f3190196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f318f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3190100840 0x7f319019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f318f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3190100840 0x7f319019c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:53638/0 (socket says 172.21.15.57:53638) 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f318effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3190101200 0x7f319019cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:54254/0 (socket says 172.21.15.57:54254) 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f318f7fe700 1 -- 172.21.15.57:0/3181516812 learned_addr learned my addr 172.21.15.57:0/3181516812 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:28.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.784+0000 7f318ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31901056a0 0x7f3190196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52830/0 (socket says 172.21.15.57:52830) 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f318f7fe700 1 -- 172.21.15.57:0/3181516812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3190101200 msgr2=0x7f319019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f318f7fe700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3190101200 0x7f319019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f318f7fe700 1 -- 172.21.15.57:0/3181516812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31901056a0 msgr2=0x7f3190196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f318f7fe700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31901056a0 0x7f3190196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f318f7fe700 1 -- 172.21.15.57:0/3181516812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31840095f0 con 0x7f3190100840 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f318ffff700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31901056a0 0x7f3190196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f318f7fe700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3190100840 0x7f319019c9f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f318000e9c0 tx=0x7f318000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f318cff9700 1 -- 172.21.15.57:0/3181516812 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800097b0 con 0x7f3190100840 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f3195b08700 1 -- 172.21.15.57:0/3181516812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31901977a0 con 0x7f3190100840 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f318cff9700 1 -- 172.21.15.57:0/3181516812 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f318000ee60 con 0x7f3190100840 2023-10-08T13:44:28.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f318cff9700 1 -- 172.21.15.57:0/3181516812 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800054f0 con 0x7f3190100840 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.785+0000 7f3195b08700 1 -- 172.21.15.57:0/3181516812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31901a39e0 con 0x7f3190100840 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.786+0000 7f318cff9700 1 -- 172.21.15.57:0/3181516812 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7f3180021440 con 0x7f3190100840 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.786+0000 7f3195b08700 1 -- 172.21.15.57:0/3181516812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3190108d10 con 0x7f3190100840 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.786+0000 7f318cff9700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3178038b40 0x7f317803b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.786+0000 7f318cff9700 1 -- 172.21.15.57:0/3181516812 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3180014070 con 0x7f3190100840 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.787+0000 7f318effd700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3178038b40 0x7f317803b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.787+0000 7f318effd700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3178038b40 0x7f317803b000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3190197cf0 tx=0x7f3184005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:27.791+0000 7f318cff9700 1 -- 172.21.15.57:0/3181516812 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f318001f930 con 0x7f3190100840 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.023+0000 7f3195b08700 1 -- 172.21.15.57:0/3181516812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7f3190101ec0 con 0x7f3190100840 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.029+0000 7f318cff9700 1 -- 172.21.15.57:0/3181516812 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v18)=0 v18) v1 ==== 116+0+0 (secure 0 0 0) 0x7f318001f340 con 0x7f3190100840 2023-10-08T13:44:28.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 -- 172.21.15.57:0/3181516812 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3178038b40 msgr2=0x7f317803b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3178038b40 0x7f317803b000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3190197cf0 tx=0x7f3184005fd0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 -- 172.21.15.57:0/3181516812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3190100840 msgr2=0x7f319019c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3190100840 0x7f319019c9f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f318000e9c0 tx=0x7f318000ecd0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 -- 172.21.15.57:0/3181516812 shutdown_connections 2023-10-08T13:44:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31901056a0 0x7f3190196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3190100840 0x7f319019c9f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3178038b40 0x7f317803b000 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 --2- 172.21.15.57:0/3181516812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3190101200 0x7f319019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 -- 172.21.15.57:0/3181516812 >> 172.21.15.57:0/3181516812 conn(0x7f3190077950 msgr2=0x7f31900fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:28.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 -- 172.21.15.57:0/3181516812 shutdown_connections 2023-10-08T13:44:28.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:28.033+0000 7f3195b08700 1 -- 172.21.15.57:0/3181516812 wait complete. 2023-10-08T13:44:28.501 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:28 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:28.239+0000 7f0d807f5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T13:44:28.686 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 13:44:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z[107447]: 2023-10-08T13:44:28.428+0000 7f1b30d3d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T13:44:28.772 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:28 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:28.500+0000 7f0d807f5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T13:44:29.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:29 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' 2023-10-08T13:44:29.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:29 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1329111525' entity='client.admin' 2023-10-08T13:44:29.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:29 smithi057 ceph-mon[99476]: Standby manager daemon z started 2023-10-08T13:44:29.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:29 smithi057 ceph-mon[99476]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-08T13:44:29.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:29 smithi057 ceph-mon[99476]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T13:44:29.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:29 smithi057 ceph-mon[99476]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-08T13:44:29.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:29 smithi057 ceph-mon[99476]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T13:44:29.326 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:29.195+0000 7f0d807f5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T13:44:29.326 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' 2023-10-08T13:44:29.326 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1329111525' entity='client.admin' 2023-10-08T13:44:29.326 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-mon[103849]: Standby manager daemon z started 2023-10-08T13:44:29.327 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-mon[103849]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-08T13:44:29.327 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-mon[103849]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T13:44:29.327 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-mon[103849]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-08T13:44:29.327 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-mon[103849]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T13:44:29.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:29 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' 2023-10-08T13:44:29.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:29 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1329111525' entity='client.admin' 2023-10-08T13:44:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:29 smithi115 ceph-mon[104408]: Standby manager daemon z started 2023-10-08T13:44:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:29 smithi115 ceph-mon[104408]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-08T13:44:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:29 smithi115 ceph-mon[104408]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T13:44:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:29 smithi115 ceph-mon[104408]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-08T13:44:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:29 smithi115 ceph-mon[104408]: from='mgr.? 172.21.15.115:0/1639747378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T13:44:29.632 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:29.326+0000 7f0d807f5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T13:44:29.632 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:29.518+0000 7f0d807f5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T13:44:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.398+0000 7f0911ec9700 1 -- 172.21.15.57:0/4157574665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f090c0ffa10 msgr2=0x7f090c0ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.398+0000 7f0911ec9700 1 --2- 172.21.15.57:0/4157574665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f090c0ffa10 0x7f090c0ffe90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0900009990 tx=0x7f0900009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.398+0000 7f0911ec9700 1 -- 172.21.15.57:0/4157574665 shutdown_connections 2023-10-08T13:44:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.398+0000 7f0911ec9700 1 --2- 172.21.15.57:0/4157574665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f090c1003d0 0x7f090c10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.398+0000 7f0911ec9700 1 --2- 172.21.15.57:0/4157574665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c105760 0x7f090c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.398+0000 7f0911ec9700 1 --2- 172.21.15.57:0/4157574665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f090c0ffa10 0x7f090c0ffe90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.398+0000 7f0911ec9700 1 -- 172.21.15.57:0/4157574665 >> 172.21.15.57:0/4157574665 conn(0x7f090c076490 msgr2=0x7f090c0768a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.398+0000 7f0911ec9700 1 -- 172.21.15.57:0/4157574665 shutdown_connections 2023-10-08T13:44:29.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.399+0000 7f0911ec9700 1 -- 172.21.15.57:0/4157574665 wait complete. 2023-10-08T13:44:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.399+0000 7f0911ec9700 1 Processor -- start 2023-10-08T13:44:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.399+0000 7f0911ec9700 1 -- start start 2023-10-08T13:44:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f0911ec9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c0ffa10 0x7f090c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f0911ec9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f090c1003d0 0x7f090c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f0911ec9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f090c105760 0x7f090c1a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f0911ec9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c198910 con 0x7f090c105760 2023-10-08T13:44:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f0911ec9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c1a3140 con 0x7f090c0ffa10 2023-10-08T13:44:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f0911ec9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c1a3280 con 0x7f090c1003d0 2023-10-08T13:44:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f090c105760 0x7f090c1a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:29.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c0ffa10 0x7f090c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c0ffa10 0x7f090c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:54282/0 (socket says 172.21.15.57:54282) 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f090c105760 0x7f090c1a2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52858/0 (socket says 172.21.15.57:52858) 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090b7fe700 1 -- 172.21.15.57:0/2689486528 learned_addr learned my addr 172.21.15.57:0/2689486528 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090affd700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f090c1003d0 0x7f090c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090b7fe700 1 -- 172.21.15.57:0/2689486528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f090c1003d0 msgr2=0x7f090c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090b7fe700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f090c1003d0 0x7f090c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090b7fe700 1 -- 172.21.15.57:0/2689486528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f090c105760 msgr2=0x7f090c1a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090b7fe700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f090c105760 0x7f090c1a2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090b7fe700 1 -- 172.21.15.57:0/2689486528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09000095f0 con 0x7f090c0ffa10 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090bfff700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f090c105760 0x7f090c1a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f090b7fe700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c0ffa10 0x7f090c197b80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00e9d0 tx=0x7f08fc00ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:29.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.400+0000 7f0908ff9700 1 -- 172.21.15.57:0/2689486528 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc009790 con 0x7f090c0ffa10 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.401+0000 7f0908ff9700 1 -- 172.21.15.57:0/2689486528 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08fc00ee70 con 0x7f090c0ffa10 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.401+0000 7f0911ec9700 1 -- 172.21.15.57:0/2689486528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c1a34e0 con 0x7f090c0ffa10 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.401+0000 7f0908ff9700 1 -- 172.21.15.57:0/2689486528 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc010ea0 con 0x7f090c0ffa10 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.401+0000 7f0911ec9700 1 -- 172.21.15.57:0/2689486528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c1a39e0 con 0x7f090c0ffa10 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.402+0000 7f0911ec9700 1 -- 172.21.15.57:0/2689486528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c107bb0 con 0x7f090c0ffa10 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.402+0000 7f0908ff9700 1 -- 172.21.15.57:0/2689486528 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 90808+0+0 (secure 0 0 0) 0x7f08fc018680 con 0x7f090c0ffa10 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.402+0000 7f0908ff9700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08f406cfc0 0x7f08f406f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.402+0000 7f0908ff9700 1 -- 172.21.15.57:0/2689486528 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f08fc014070 con 0x7f090c0ffa10 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.403+0000 7f090affd700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08f406cfc0 0x7f08f406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.403+0000 7f090affd700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08f406cfc0 0x7f08f406f480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f090c069650 tx=0x7f0900005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:29.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.407+0000 7f0908ff9700 1 -- 172.21.15.57:0/2689486528 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f08fc052100 con 0x7f090c0ffa10 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.615+0000 7f0911ec9700 1 -- 172.21.15.57:0/2689486528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f090c100e10 con 0x7f090c0ffa10 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.620+0000 7f0908ff9700 1 -- 172.21.15.57:0/2689486528 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v20)=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f08fc056980 con 0x7f090c0ffa10 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 -- 172.21.15.57:0/2689486528 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08f406cfc0 msgr2=0x7f08f406f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08f406cfc0 0x7f08f406f480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f090c069650 tx=0x7f0900005fd0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 -- 172.21.15.57:0/2689486528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c0ffa10 msgr2=0x7f090c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c0ffa10 0x7f090c197b80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00e9d0 tx=0x7f08fc00ece0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 -- 172.21.15.57:0/2689486528 shutdown_connections 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f090c105760 0x7f090c1a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08f406cfc0 0x7f08f406f480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c0ffa10 0x7f090c197b80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 --2- 172.21.15.57:0/2689486528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f090c1003d0 0x7f090c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:29.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 -- 172.21.15.57:0/2689486528 >> 172.21.15.57:0/2689486528 conn(0x7f090c076490 msgr2=0x7f090c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:29.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 -- 172.21.15.57:0/2689486528 shutdown_connections 2023-10-08T13:44:29.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:29.622+0000 7f0911ec9700 1 -- 172.21.15.57:0/2689486528 wait complete. 2023-10-08T13:44:30.022 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:29.632+0000 7f0d807f5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T13:44:30.022 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:29.851+0000 7f0d807f5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T13:44:30.223 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi057') 2023-10-08T13:44:30.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.233+0000 7fac27305700 1 -- 172.21.15.57:0/2445778554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20073340 msgr2=0x7fac2010cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:30.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.233+0000 7fac27305700 1 --2- 172.21.15.57:0/2445778554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20073340 0x7fac2010cd10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fac10009990 tx=0x7fac10009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:30.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.233+0000 7fac27305700 1 -- 172.21.15.57:0/2445778554 shutdown_connections 2023-10-08T13:44:30.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.233+0000 7fac27305700 1 --2- 172.21.15.57:0/2445778554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac2010d250 0x7fac2010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.233+0000 7fac27305700 1 --2- 172.21.15.57:0/2445778554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac20072a20 0x7fac20072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.233+0000 7fac27305700 1 --2- 172.21.15.57:0/2445778554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20073340 0x7fac2010cd10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.233+0000 7fac27305700 1 -- 172.21.15.57:0/2445778554 >> 172.21.15.57:0/2445778554 conn(0x7fac20077910 msgr2=0x7fac20077d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.233+0000 7fac27305700 1 -- 172.21.15.57:0/2445778554 shutdown_connections 2023-10-08T13:44:30.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.233+0000 7fac27305700 1 -- 172.21.15.57:0/2445778554 wait complete. 2023-10-08T13:44:30.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.234+0000 7fac27305700 1 Processor -- start 2023-10-08T13:44:30.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.234+0000 7fac27305700 1 -- start start 2023-10-08T13:44:30.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac27305700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac20072a20 0x7fac20198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:30.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac27305700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20073340 0x7fac20199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:30.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac27305700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac2010d250 0x7fac2019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:30.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac27305700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2019dd00 con 0x7fac20073340 2023-10-08T13:44:30.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac27305700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2019de70 con 0x7fac2010d250 2023-10-08T13:44:30.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac27305700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2019e030 con 0x7fac20072a20 2023-10-08T13:44:30.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac248a0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20073340 0x7fac20199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:30.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac248a0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20073340 0x7fac20199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52870/0 (socket says 172.21.15.57:52870) 2023-10-08T13:44:30.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac248a0700 1 -- 172.21.15.57:0/2314490477 learned_addr learned my addr 172.21.15.57:0/2314490477 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:30.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac250a1700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac20072a20 0x7fac20198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:30.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac258a2700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac2010d250 0x7fac2019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:30.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac248a0700 1 -- 172.21.15.57:0/2314490477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac20072a20 msgr2=0x7fac20198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:30.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac248a0700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac20072a20 0x7fac20198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:30.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac248a0700 1 -- 172.21.15.57:0/2314490477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac2010d250 msgr2=0x7fac2019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:30.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac248a0700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac2010d250 0x7fac2019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:30.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac248a0700 1 -- 172.21.15.57:0/2314490477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac100095f0 con 0x7fac20073340 2023-10-08T13:44:30.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.235+0000 7fac248a0700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20073340 0x7fac20199340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:30.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.236+0000 7fac167fc700 1 -- 172.21.15.57:0/2314490477 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1001d070 con 0x7fac20073340 2023-10-08T13:44:30.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.236+0000 7fac167fc700 1 -- 172.21.15.57:0/2314490477 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac10022470 con 0x7fac20073340 2023-10-08T13:44:30.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.236+0000 7fac27305700 1 -- 172.21.15.57:0/2314490477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2019e2b0 con 0x7fac20073340 2023-10-08T13:44:30.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.236+0000 7fac167fc700 1 -- 172.21.15.57:0/2314490477 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10017650 con 0x7fac20073340 2023-10-08T13:44:30.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.236+0000 7fac27305700 1 -- 172.21.15.57:0/2314490477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac201a5b70 con 0x7fac20073340 2023-10-08T13:44:30.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.237+0000 7fac27305700 1 -- 172.21.15.57:0/2314490477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac20108460 con 0x7fac20073340 2023-10-08T13:44:30.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.237+0000 7fac167fc700 1 -- 172.21.15.57:0/2314490477 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) v1 ==== 90808+0+0 (secure 0 0 0) 0x7fac100225e0 con 0x7fac20073340 2023-10-08T13:44:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.238+0000 7fac167fc700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac0c06cfc0 0x7fac0c06f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.238+0000 7fac167fc700 1 -- 172.21.15.57:0/2314490477 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fac1008c4a0 con 0x7fac20073340 2023-10-08T13:44:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.238+0000 7fac250a1700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac0c06cfc0 0x7fac0c06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.238+0000 7fac250a1700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac0c06cfc0 0x7fac0c06f480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fac1c005950 tx=0x7fac1c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.242+0000 7fac167fc700 1 -- 172.21.15.57:0/2314490477 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fac10057980 con 0x7fac20073340 2023-10-08T13:44:30.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:30.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:30 smithi057 ceph-mon[99476]: mgrmap e18: x(active, since 15s), standbys: z 2023-10-08T13:44:30.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-08T13:44:30.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:30 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' 2023-10-08T13:44:30.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:30.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:30 smithi115 ceph-mon[104408]: mgrmap e18: x(active, since 15s), standbys: z 2023-10-08T13:44:30.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-08T13:44:30.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:30 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' 2023-10-08T13:44:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.450+0000 7fac27305700 1 -- 172.21.15.57:0/2314490477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7fac2004e4c0 con 0x7fac20073340 2023-10-08T13:44:30.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.453+0000 7fac167fc700 1 -- 172.21.15.57:0/2314490477 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v21)=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7fac1005afa0 con 0x7fac20073340 2023-10-08T13:44:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 -- 172.21.15.57:0/2314490477 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac0c06cfc0 msgr2=0x7fac0c06f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac0c06cfc0 0x7fac0c06f480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fac1c005950 tx=0x7fac1c00b410 comp rx=0 tx=0).stop 2023-10-08T13:44:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 -- 172.21.15.57:0/2314490477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20073340 msgr2=0x7fac20199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20073340 0x7fac20199340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac10004730 comp rx=0 tx=0).stop 2023-10-08T13:44:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 -- 172.21.15.57:0/2314490477 shutdown_connections 2023-10-08T13:44:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac0c06cfc0 0x7fac0c06f480 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac2010d250 0x7fac2019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac20072a20 0x7fac20198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 --2- 172.21.15.57:0/2314490477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac20073340 0x7fac20199340 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 -- 172.21.15.57:0/2314490477 >> 172.21.15.57:0/2314490477 conn(0x7fac20077910 msgr2=0x7fac200fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 -- 172.21.15.57:0/2314490477 shutdown_connections 2023-10-08T13:44:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:30.455+0000 7fac27305700 1 -- 172.21.15.57:0/2314490477 wait complete. 2023-10-08T13:44:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:30 smithi099 ceph-mon[103849]: mgrmap e18: x(active, since 15s), standbys: z 2023-10-08T13:44:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-08T13:44:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:30 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' 2023-10-08T13:44:31.022 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 13:44:30 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y[106231]: 2023-10-08T13:44:30.705+0000 7f0d807f5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T13:44:31.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:31 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2314490477' entity='client.admin' 2023-10-08T13:44:31.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:31 smithi057 ceph-mon[99476]: Standby manager daemon y started 2023-10-08T13:44:31.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:31 smithi057 ceph-mon[99476]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-08T13:44:31.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:31 smithi057 ceph-mon[99476]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T13:44:31.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:31 smithi057 ceph-mon[99476]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-08T13:44:31.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:31 smithi057 ceph-mon[99476]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T13:44:31.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:31 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2314490477' entity='client.admin' 2023-10-08T13:44:31.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:31 smithi099 ceph-mon[103849]: Standby manager daemon y started 2023-10-08T13:44:31.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:31 smithi099 ceph-mon[103849]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-08T13:44:31.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:31 smithi099 ceph-mon[103849]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T13:44:31.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:31 smithi099 ceph-mon[103849]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-08T13:44:31.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:31 smithi099 ceph-mon[103849]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T13:44:31.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:31 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2314490477' entity='client.admin' 2023-10-08T13:44:31.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:31 smithi115 ceph-mon[104408]: Standby manager daemon y started 2023-10-08T13:44:31.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:31 smithi115 ceph-mon[104408]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-08T13:44:31.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:31 smithi115 ceph-mon[104408]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T13:44:31.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:31 smithi115 ceph-mon[104408]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-08T13:44:31.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:31 smithi115 ceph-mon[104408]: from='mgr.? 172.21.15.99:0/2010257214' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T13:44:32.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.188+0000 7f11cfc49700 1 -- 172.21.15.57:0/1120983474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11c80ff750 msgr2=0x7f11c80ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:32.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.188+0000 7f11cfc49700 1 --2- 172.21.15.57:0/1120983474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11c80ff750 0x7f11c80ffbd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009990 tx=0x7f11b8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:32.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.188+0000 7f11cfc49700 1 -- 172.21.15.57:0/1120983474 shutdown_connections 2023-10-08T13:44:32.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.188+0000 7f11cfc49700 1 --2- 172.21.15.57:0/1120983474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11c8100110 0x7f11c810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.189+0000 7f11cfc49700 1 --2- 172.21.15.57:0/1120983474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c81056b0 0x7f11c8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.189+0000 7f11cfc49700 1 --2- 172.21.15.57:0/1120983474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11c80ff750 0x7f11c80ffbd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.189+0000 7f11cfc49700 1 -- 172.21.15.57:0/1120983474 >> 172.21.15.57:0/1120983474 conn(0x7f11c8075790 msgr2=0x7f11c8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.189+0000 7f11cfc49700 1 -- 172.21.15.57:0/1120983474 shutdown_connections 2023-10-08T13:44:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.189+0000 7f11cfc49700 1 -- 172.21.15.57:0/1120983474 wait complete. 2023-10-08T13:44:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.189+0000 7f11cfc49700 1 Processor -- start 2023-10-08T13:44:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.189+0000 7f11cfc49700 1 -- start start 2023-10-08T13:44:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cfc49700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c80ff750 0x7f11c8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cfc49700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11c8100110 0x7f11c8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cfc49700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11c81056b0 0x7f11c81a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:32.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cfc49700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8198720 con 0x7f11c8100110 2023-10-08T13:44:32.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cfc49700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c81a2f50 con 0x7f11c80ff750 2023-10-08T13:44:32.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cfc49700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c81a3090 con 0x7f11c81056b0 2023-10-08T13:44:32.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cd1e4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11c8100110 0x7f11c8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:32.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cd1e4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11c8100110 0x7f11c8197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52884/0 (socket says 172.21.15.57:52884) 2023-10-08T13:44:32.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cd1e4700 1 -- 172.21.15.57:0/1006317595 learned_addr learned my addr 172.21.15.57:0/1006317595 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:32.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11ce1e6700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11c81056b0 0x7f11c81a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:32.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cd9e5700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c80ff750 0x7f11c8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:32.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cd1e4700 1 -- 172.21.15.57:0/1006317595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11c81056b0 msgr2=0x7f11c81a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:32.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cd1e4700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11c81056b0 0x7f11c81a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:32.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cd1e4700 1 -- 172.21.15.57:0/1006317595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c80ff750 msgr2=0x7f11c8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:32.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cd1e4700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c80ff750 0x7f11c8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:32.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cd1e4700 1 -- 172.21.15.57:0/1006317595 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b80095f0 con 0x7f11c8100110 2023-10-08T13:44:32.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cd1e4700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11c8100110 0x7f11c8197ed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f11b8000c00 tx=0x7f11b800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:32.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11beffd700 1 -- 172.21.15.57:0/1006317595 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b801d070 con 0x7f11c8100110 2023-10-08T13:44:32.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11beffd700 1 -- 172.21.15.57:0/1006317595 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11b8022470 con 0x7f11c8100110 2023-10-08T13:44:32.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11beffd700 1 -- 172.21.15.57:0/1006317595 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8017650 con 0x7f11c8100110 2023-10-08T13:44:32.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cfc49700 1 -- 172.21.15.57:0/1006317595 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c81a3290 con 0x7f11c8100110 2023-10-08T13:44:32.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.190+0000 7f11cfc49700 1 -- 172.21.15.57:0/1006317595 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c81a36a0 con 0x7f11c8100110 2023-10-08T13:44:32.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.192+0000 7f11cfc49700 1 -- 172.21.15.57:0/1006317595 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c8107cd0 con 0x7f11c8100110 2023-10-08T13:44:32.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.192+0000 7f11beffd700 1 -- 172.21.15.57:0/1006317595 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f11b80225e0 con 0x7f11c8100110 2023-10-08T13:44:32.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.193+0000 7f11beffd700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11b40a14c0 0x7f11b40a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:32.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.193+0000 7f11cd9e5700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11b40a14c0 0x7f11b40a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:32.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.194+0000 7f11beffd700 1 -- 172.21.15.57:0/1006317595 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f11b80cc110 con 0x7f11c8100110 2023-10-08T13:44:32.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.194+0000 7f11cd9e5700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11b40a14c0 0x7f11b40a3980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f11c4005fd0 tx=0x7f11c400af20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:32.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.198+0000 7f11beffd700 1 -- 172.21.15.57:0/1006317595 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f11b8097400 con 0x7f11c8100110 2023-10-08T13:44:32.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.402+0000 7f11cfc49700 1 -- 172.21.15.57:0/1006317595 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f11c8068730 con 0x7f11c8100110 2023-10-08T13:44:32.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.406+0000 7f11beffd700 1 -- 172.21.15.57:0/1006317595 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) v1 ==== 121+0+0 (secure 0 0 0) 0x7f11b809aa20 con 0x7f11c8100110 2023-10-08T13:44:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.408+0000 7f11cfc49700 1 -- 172.21.15.57:0/1006317595 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11b40a14c0 msgr2=0x7f11b40a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.408+0000 7f11cfc49700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11b40a14c0 0x7f11b40a3980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f11c4005fd0 tx=0x7f11c400af20 comp rx=0 tx=0).stop 2023-10-08T13:44:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.408+0000 7f11cfc49700 1 -- 172.21.15.57:0/1006317595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11c8100110 msgr2=0x7f11c8197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.408+0000 7f11cfc49700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11c8100110 0x7f11c8197ed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f11b8000c00 tx=0x7f11b800f670 comp rx=0 tx=0).stop 2023-10-08T13:44:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.409+0000 7f11cfc49700 1 -- 172.21.15.57:0/1006317595 shutdown_connections 2023-10-08T13:44:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.409+0000 7f11cfc49700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11c81056b0 0x7f11c81a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.409+0000 7f11cfc49700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11b40a14c0 0x7f11b40a3980 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.409+0000 7f11cfc49700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c80ff750 0x7f11c8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.409+0000 7f11cfc49700 1 --2- 172.21.15.57:0/1006317595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11c8100110 0x7f11c8197ed0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.409+0000 7f11cfc49700 1 -- 172.21.15.57:0/1006317595 >> 172.21.15.57:0/1006317595 conn(0x7f11c8075790 msgr2=0x7f11c80fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:32.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.409+0000 7f11cfc49700 1 -- 172.21.15.57:0/1006317595 shutdown_connections 2023-10-08T13:44:32.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:32.409+0000 7f11cfc49700 1 -- 172.21.15.57:0/1006317595 wait complete. 2023-10-08T13:44:32.643 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:32 smithi057 ceph-mon[99476]: mgrmap e19: x(active, since 17s), standbys: z, y 2023-10-08T13:44:32.643 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-08T13:44:32.643 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:32.644 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:32.644 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:32.644 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:32.644 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:32.644 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:32.644 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:32 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1006317595' entity='client.admin' 2023-10-08T13:44:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:32 smithi099 ceph-mon[103849]: mgrmap e19: x(active, since 17s), standbys: z, y 2023-10-08T13:44:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-08T13:44:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:32 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1006317595' entity='client.admin' 2023-10-08T13:44:32.903 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi057') 2023-10-08T13:44:32.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:32 smithi115 ceph-mon[104408]: mgrmap e19: x(active, since 17s), standbys: z, y 2023-10-08T13:44:32.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-08T13:44:32.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:32.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:32.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:32.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:32.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:32.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:32.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:32 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1006317595' entity='client.admin' 2023-10-08T13:44:33.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.433+0000 7f3a99ec3700 1 -- 172.21.15.57:0/2830145109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a941067a0 msgr2=0x7f3a94106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:33.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.433+0000 7f3a99ec3700 1 --2- 172.21.15.57:0/2830145109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a941067a0 0x7f3a94106b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009990 tx=0x7f3a84009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.433+0000 7f3a99ec3700 1 -- 172.21.15.57:0/2830145109 shutdown_connections 2023-10-08T13:44:33.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.433+0000 7f3a99ec3700 1 --2- 172.21.15.57:0/2830145109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a94068b30 0x7f3a9410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.433+0000 7f3a99ec3700 1 --2- 172.21.15.57:0/2830145109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a941067a0 0x7f3a94106b80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.433+0000 7f3a99ec3700 1 --2- 172.21.15.57:0/2830145109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a94068190 0x7f3a940685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.433+0000 7f3a99ec3700 1 -- 172.21.15.57:0/2830145109 >> 172.21.15.57:0/2830145109 conn(0x7f3a94075790 msgr2=0x7f3a94075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:33.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.433+0000 7f3a99ec3700 1 -- 172.21.15.57:0/2830145109 shutdown_connections 2023-10-08T13:44:33.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.433+0000 7f3a99ec3700 1 -- 172.21.15.57:0/2830145109 wait complete. 2023-10-08T13:44:33.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a99ec3700 1 Processor -- start 2023-10-08T13:44:33.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a99ec3700 1 -- start start 2023-10-08T13:44:33.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a99ec3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a94068190 0x7f3a940ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:33.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a99ec3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a94068b30 0x7f3a940ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:33.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a99ec3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a941067a0 0x7f3a941002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:33.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a99ec3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a941a4fd0 con 0x7f3a941067a0 2023-10-08T13:44:33.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a99ec3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a941a5140 con 0x7f3a94068b30 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a99ec3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a941a5300 con 0x7f3a94068190 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a93fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a941067a0 0x7f3a941002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a93fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a941067a0 0x7f3a941002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52898/0 (socket says 172.21.15.57:52898) 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a93fff700 1 -- 172.21.15.57:0/882638327 learned_addr learned my addr 172.21.15.57:0/882638327 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a937fe700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a94068190 0x7f3a940ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a92ffd700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a94068b30 0x7f3a940ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a93fff700 1 -- 172.21.15.57:0/882638327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a94068190 msgr2=0x7f3a940ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a93fff700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a94068190 0x7f3a940ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.434+0000 7f3a93fff700 1 -- 172.21.15.57:0/882638327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a94068b30 msgr2=0x7f3a940ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.435+0000 7f3a93fff700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a94068b30 0x7f3a940ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.435+0000 7f3a93fff700 1 -- 172.21.15.57:0/882638327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a840095f0 con 0x7f3a941067a0 2023-10-08T13:44:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.435+0000 7f3a93fff700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a941067a0 0x7f3a941002a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c007ee0 tx=0x7f3a8c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.435+0000 7f3a90ff9700 1 -- 172.21.15.57:0/882638327 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c00dd20 con 0x7f3a941067a0 2023-10-08T13:44:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.435+0000 7f3a90ff9700 1 -- 172.21.15.57:0/882638327 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a8c00f040 con 0x7f3a941067a0 2023-10-08T13:44:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.435+0000 7f3a99ec3700 1 -- 172.21.15.57:0/882638327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a941a55e0 con 0x7f3a941067a0 2023-10-08T13:44:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.435+0000 7f3a90ff9700 1 -- 172.21.15.57:0/882638327 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0048a0 con 0x7f3a941067a0 2023-10-08T13:44:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.435+0000 7f3a99ec3700 1 -- 172.21.15.57:0/882638327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a941a5ae0 con 0x7f3a941067a0 2023-10-08T13:44:33.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.436+0000 7f3a99ec3700 1 -- 172.21.15.57:0/882638327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a94108d50 con 0x7f3a941067a0 2023-10-08T13:44:33.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.437+0000 7f3a90ff9700 1 -- 172.21.15.57:0/882638327 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f3a8c003a00 con 0x7f3a941067a0 2023-10-08T13:44:33.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.437+0000 7f3a90ff9700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a7c0a1440 0x7f3a7c0a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:33.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.438+0000 7f3a90ff9700 1 -- 172.21.15.57:0/882638327 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3a8c0ca900 con 0x7f3a941067a0 2023-10-08T13:44:33.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.438+0000 7f3a937fe700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a7c0a1440 0x7f3a7c0a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:33.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.438+0000 7f3a937fe700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a7c0a1440 0x7f3a7c0a3900 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a84006010 tx=0x7f3a8400b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:33.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.443+0000 7f3a90ff9700 1 -- 172.21.15.57:0/882638327 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3a8c04f100 con 0x7f3a941067a0 2023-10-08T13:44:33.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.662+0000 7f3a99ec3700 1 -- 172.21.15.57:0/882638327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) v1 -- 0x7f3a9404e4c0 con 0x7f3a941067a0 2023-10-08T13:44:33.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.668+0000 7f3a90ff9700 1 -- 172.21.15.57:0/882638327 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v23)=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3a8c095b60 con 0x7f3a941067a0 2023-10-08T13:44:33.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 -- 172.21.15.57:0/882638327 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a7c0a1440 msgr2=0x7f3a7c0a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:33.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a7c0a1440 0x7f3a7c0a3900 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a84006010 tx=0x7f3a8400b260 comp rx=0 tx=0).stop 2023-10-08T13:44:33.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 -- 172.21.15.57:0/882638327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a941067a0 msgr2=0x7f3a941002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:33.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a941067a0 0x7f3a941002a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c007ee0 tx=0x7f3a8c00d3b0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 -- 172.21.15.57:0/882638327 shutdown_connections 2023-10-08T13:44:33.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a941067a0 0x7f3a941002a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a94068190 0x7f3a940ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a7c0a1440 0x7f3a7c0a3900 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 --2- 172.21.15.57:0/882638327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a94068b30 0x7f3a940ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:33.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 -- 172.21.15.57:0/882638327 >> 172.21.15.57:0/882638327 conn(0x7f3a94075790 msgr2=0x7f3a940fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:33.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 -- 172.21.15.57:0/882638327 shutdown_connections 2023-10-08T13:44:33.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:33.669+0000 7f3a99ec3700 1 -- 172.21.15.57:0/882638327 wait complete. 2023-10-08T13:44:34.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.063+0000 7ff0eb153700 1 -- 172.21.15.57:0/3535461864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0e41067a0 msgr2=0x7ff0e4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.063+0000 7ff0eb153700 1 --2- 172.21.15.57:0/3535461864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0e41067a0 0x7ff0e4106b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009990 tx=0x7ff0d8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.063+0000 7ff0eb153700 1 -- 172.21.15.57:0/3535461864 shutdown_connections 2023-10-08T13:44:34.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.063+0000 7ff0eb153700 1 --2- 172.21.15.57:0/3535461864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e4068b30 0x7ff0e410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.063+0000 7ff0eb153700 1 --2- 172.21.15.57:0/3535461864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0e41067a0 0x7ff0e4106b80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.063+0000 7ff0eb153700 1 --2- 172.21.15.57:0/3535461864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0e4068190 0x7ff0e40685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.063+0000 7ff0eb153700 1 -- 172.21.15.57:0/3535461864 >> 172.21.15.57:0/3535461864 conn(0x7ff0e4075790 msgr2=0x7ff0e4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:34.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.063+0000 7ff0eb153700 1 -- 172.21.15.57:0/3535461864 shutdown_connections 2023-10-08T13:44:34.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.063+0000 7ff0eb153700 1 -- 172.21.15.57:0/3535461864 wait complete. 2023-10-08T13:44:34.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0eb153700 1 Processor -- start 2023-10-08T13:44:34.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0eb153700 1 -- start start 2023-10-08T13:44:34.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0eb153700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0e4068190 0x7ff0e4196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0eb153700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e4068b30 0x7ff0e4197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0eb153700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0e41067a0 0x7ff0e419b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0eb153700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e419bb40 con 0x7ff0e4068190 2023-10-08T13:44:34.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0eb153700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e419bcb0 con 0x7ff0e4068b30 2023-10-08T13:44:34.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0eb153700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e419be70 con 0x7ff0e41067a0 2023-10-08T13:44:34.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e8eef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0e4068190 0x7ff0e4196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e8eef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0e4068190 0x7ff0e4196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52908/0 (socket says 172.21.15.57:52908) 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e8eef700 1 -- 172.21.15.57:0/3567091931 learned_addr learned my addr 172.21.15.57:0/3567091931 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e3fff700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e4068b30 0x7ff0e4197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e96f0700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0e41067a0 0x7ff0e419b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e8eef700 1 -- 172.21.15.57:0/3567091931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0e41067a0 msgr2=0x7ff0e419b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e8eef700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0e41067a0 0x7ff0e419b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e8eef700 1 -- 172.21.15.57:0/3567091931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e4068b30 msgr2=0x7ff0e4197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e8eef700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e4068b30 0x7ff0e4197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e8eef700 1 -- 172.21.15.57:0/3567091931 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d80095f0 con 0x7ff0e4068190 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.064+0000 7ff0e8eef700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0e4068190 0x7ff0e4196d00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80098e0 tx=0x7ff0d800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.065+0000 7ff0e1ffb700 1 -- 172.21.15.57:0/3567091931 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d801d070 con 0x7ff0e4068190 2023-10-08T13:44:34.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.065+0000 7ff0eb153700 1 -- 172.21.15.57:0/3567091931 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e419c0f0 con 0x7ff0e4068190 2023-10-08T13:44:34.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.065+0000 7ff0e1ffb700 1 -- 172.21.15.57:0/3567091931 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0d800fb90 con 0x7ff0e4068190 2023-10-08T13:44:34.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.065+0000 7ff0e1ffb700 1 -- 172.21.15.57:0/3567091931 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8017650 con 0x7ff0e4068190 2023-10-08T13:44:34.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.065+0000 7ff0eb153700 1 -- 172.21.15.57:0/3567091931 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e41a39d0 con 0x7ff0e4068190 2023-10-08T13:44:34.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.066+0000 7ff0eb153700 1 -- 172.21.15.57:0/3567091931 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e4108d50 con 0x7ff0e4068190 2023-10-08T13:44:34.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.067+0000 7ff0e1ffb700 1 -- 172.21.15.57:0/3567091931 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7ff0d800fd00 con 0x7ff0e4068190 2023-10-08T13:44:34.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.067+0000 7ff0e1ffb700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff0d40a1440 0x7ff0d40a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.068+0000 7ff0e3fff700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff0d40a1440 0x7ff0d40a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.068+0000 7ff0e1ffb700 1 -- 172.21.15.57:0/3567091931 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff0d80cc270 con 0x7ff0e4068190 2023-10-08T13:44:34.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.068+0000 7ff0e3fff700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff0d40a1440 0x7ff0d40a3900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005950 tx=0x7ff0d00058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:34.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.072+0000 7ff0e1ffb700 1 -- 172.21.15.57:0/3567091931 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff0d8097560 con 0x7ff0e4068190 2023-10-08T13:44:34.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.276+0000 7ff0eb153700 1 -- 172.21.15.57:0/3567091931 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7ff0e404e4c0 con 0x7ff0e4068190 2023-10-08T13:44:34.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.281+0000 7ff0e1ffb700 1 -- 172.21.15.57:0/3567091931 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v24)=0 v24) v1 ==== 119+0+0 (secure 0 0 0) 0x7ff0d809ab80 con 0x7ff0e4068190 2023-10-08T13:44:34.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 -- 172.21.15.57:0/3567091931 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff0d40a1440 msgr2=0x7ff0d40a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff0d40a1440 0x7ff0d40a3900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005950 tx=0x7ff0d00058e0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 -- 172.21.15.57:0/3567091931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0e4068190 msgr2=0x7ff0e4196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0e4068190 0x7ff0e4196d00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80098e0 tx=0x7ff0d800f720 comp rx=0 tx=0).stop 2023-10-08T13:44:34.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 -- 172.21.15.57:0/3567091931 shutdown_connections 2023-10-08T13:44:34.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0e41067a0 0x7ff0e419b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff0e4068190 0x7ff0e4196d00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff0d40a1440 0x7ff0d40a3900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 --2- 172.21.15.57:0/3567091931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e4068b30 0x7ff0e4197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 -- 172.21.15.57:0/3567091931 >> 172.21.15.57:0/3567091931 conn(0x7ff0e4075790 msgr2=0x7ff0e40fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:34.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 -- 172.21.15.57:0/3567091931 shutdown_connections 2023-10-08T13:44:34.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.284+0000 7ff0eb153700 1 -- 172.21.15.57:0/3567091931 wait complete. 2023-10-08T13:44:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.310+0000 7f36027fc700 1 -- 172.21.15.57:0/3371521145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35fc0ffa40 msgr2=0x7f35fc0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.310+0000 7f36027fc700 1 --2- 172.21.15.57:0/3371521145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35fc0ffa40 0x7f35fc0ffec0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009990 tx=0x7f35f0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.310+0000 7f36027fc700 1 -- 172.21.15.57:0/3371521145 shutdown_connections 2023-10-08T13:44:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.310+0000 7f36027fc700 1 --2- 172.21.15.57:0/3371521145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc100400 0x7f35fc10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.310+0000 7f36027fc700 1 --2- 172.21.15.57:0/3371521145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35fc1057b0 0x7f35fc105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.310+0000 7f36027fc700 1 --2- 172.21.15.57:0/3371521145 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35fc0ffa40 0x7f35fc0ffec0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.310+0000 7f36027fc700 1 -- 172.21.15.57:0/3371521145 >> 172.21.15.57:0/3371521145 conn(0x7f35fc0764a0 msgr2=0x7f35fc0768b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.310+0000 7f36027fc700 1 -- 172.21.15.57:0/3371521145 shutdown_connections 2023-10-08T13:44:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.310+0000 7f36027fc700 1 -- 172.21.15.57:0/3371521145 wait complete. 2023-10-08T13:44:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.311+0000 7f36027fc700 1 Processor -- start 2023-10-08T13:44:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.311+0000 7f36027fc700 1 -- start start 2023-10-08T13:44:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f36027fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35fc0ffa40 0x7f35fc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f36027fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc100400 0x7f35fc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f36027fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35fc1057b0 0x7f35fc198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f36027fc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1a2db0 con 0x7f35fc1057b0 2023-10-08T13:44:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f36027fc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1a2f20 con 0x7f35fc0ffa40 2023-10-08T13:44:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f36027fc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1a30e0 con 0x7f35fc100400 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f3600d99700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35fc1057b0 0x7f35fc198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f3600d99700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35fc1057b0 0x7f35fc198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52926/0 (socket says 172.21.15.57:52926) 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f3600d99700 1 -- 172.21.15.57:0/1596511929 learned_addr learned my addr 172.21.15.57:0/1596511929 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f35fbfff700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35fc0ffa40 0x7f35fc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f35fb7fe700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc100400 0x7f35fc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f3600d99700 1 -- 172.21.15.57:0/1596511929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc100400 msgr2=0x7f35fc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f3600d99700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc100400 0x7f35fc197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f3600d99700 1 -- 172.21.15.57:0/1596511929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35fc0ffa40 msgr2=0x7f35fc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f3600d99700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35fc0ffa40 0x7f35fc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f3600d99700 1 -- 172.21.15.57:0/1596511929 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00095f0 con 0x7f35fc1057b0 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f35fbfff700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35fc0ffa40 0x7f35fc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:44:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.313+0000 7f3600d99700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35fc1057b0 0x7f35fc198560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f35f4007ee0 tx=0x7f35f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.314+0000 7f35f97fa700 1 -- 172.21.15.57:0/1596511929 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4017070 con 0x7f35fc1057b0 2023-10-08T13:44:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.314+0000 7f35f97fa700 1 -- 172.21.15.57:0/1596511929 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35f400f040 con 0x7f35fc1057b0 2023-10-08T13:44:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.314+0000 7f36027fc700 1 -- 172.21.15.57:0/1596511929 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc1a33c0 con 0x7f35fc1057b0 2023-10-08T13:44:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.314+0000 7f35f97fa700 1 -- 172.21.15.57:0/1596511929 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4005580 con 0x7f35fc1057b0 2023-10-08T13:44:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.314+0000 7f36027fc700 1 -- 172.21.15.57:0/1596511929 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc1a3890 con 0x7f35fc1057b0 2023-10-08T13:44:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.315+0000 7f36027fc700 1 -- 172.21.15.57:0/1596511929 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc107c60 con 0x7f35fc1057b0 2023-10-08T13:44:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.316+0000 7f35f97fa700 1 -- 172.21.15.57:0/1596511929 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f35f400db20 con 0x7f35fc1057b0 2023-10-08T13:44:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.316+0000 7f35f97fa700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f35e40a1540 0x7f35e40a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.317+0000 7f35fbfff700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f35e40a1540 0x7f35e40a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.317+0000 7f35f97fa700 1 -- 172.21.15.57:0/1596511929 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f35f40ca870 con 0x7f35fc1057b0 2023-10-08T13:44:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.317+0000 7f35fbfff700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f35e40a1540 0x7f35e40a3a00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f35ec005950 tx=0x7f35ec00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:34.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.321+0000 7f35f97fa700 1 -- 172.21.15.57:0/1596511929 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f35f4095a30 con 0x7f35fc1057b0 2023-10-08T13:44:34.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.544+0000 7f36027fc700 1 -- 172.21.15.57:0/1596511929 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7f35fc100df0 con 0x7f35fc1057b0 2023-10-08T13:44:34.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.548+0000 7f35f97fa700 1 -- 172.21.15.57:0/1596511929 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v25)=0 v25) v1 ==== 121+0+0 (secure 0 0 0) 0x7f35f4099050 con 0x7f35fc1057b0 2023-10-08T13:44:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 -- 172.21.15.57:0/1596511929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f35e40a1540 msgr2=0x7f35e40a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f35e40a1540 0x7f35e40a3a00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f35ec005950 tx=0x7f35ec00b410 comp rx=0 tx=0).stop 2023-10-08T13:44:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 -- 172.21.15.57:0/1596511929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35fc1057b0 msgr2=0x7f35fc198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35fc1057b0 0x7f35fc198560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f35f4007ee0 tx=0x7f35f400d3b0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 -- 172.21.15.57:0/1596511929 shutdown_connections 2023-10-08T13:44:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f35fc1057b0 0x7f35fc198560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f35e40a1540 0x7f35e40a3a00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35fc0ffa40 0x7f35fc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 --2- 172.21.15.57:0/1596511929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc100400 0x7f35fc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 -- 172.21.15.57:0/1596511929 >> 172.21.15.57:0/1596511929 conn(0x7f35fc0764a0 msgr2=0x7f35fc0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 -- 172.21.15.57:0/1596511929 shutdown_connections 2023-10-08T13:44:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.550+0000 7f36027fc700 1 -- 172.21.15.57:0/1596511929 wait complete. 2023-10-08T13:44:34.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.646+0000 7f1b0b221700 1 -- 172.21.15.57:0/1722278367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b041056a0 msgr2=0x7f1b04105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.646+0000 7f1b0b221700 1 --2- 172.21.15.57:0/1722278367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b041056a0 0x7f1b04105a80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009990 tx=0x7f1af8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.646+0000 7f1b0b221700 1 -- 172.21.15.57:0/1722278367 shutdown_connections 2023-10-08T13:44:34.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.646+0000 7f1b0b221700 1 --2- 172.21.15.57:0/1722278367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b04101200 0x7f1b0410d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.646+0000 7f1b0b221700 1 --2- 172.21.15.57:0/1722278367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b041056a0 0x7f1b04105a80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.646+0000 7f1b0b221700 1 --2- 172.21.15.57:0/1722278367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100840 0x7f1b04100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.646+0000 7f1b0b221700 1 -- 172.21.15.57:0/1722278367 >> 172.21.15.57:0/1722278367 conn(0x7f1b04077950 msgr2=0x7f1b04077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:34.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.646+0000 7f1b0b221700 1 -- 172.21.15.57:0/1722278367 shutdown_connections 2023-10-08T13:44:34.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.646+0000 7f1b0b221700 1 -- 172.21.15.57:0/1722278367 wait complete. 2023-10-08T13:44:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b0b221700 1 Processor -- start 2023-10-08T13:44:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b0b221700 1 -- start start 2023-10-08T13:44:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b0b221700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100840 0x7f1b04072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b0b221700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b04101200 0x7f1b04073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b0b221700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b041056a0 0x7f1b04073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b0b221700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b041a4f70 con 0x7f1b04101200 2023-10-08T13:44:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b0b221700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b041a50e0 con 0x7f1b04100840 2023-10-08T13:44:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b0b221700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b041a52a0 con 0x7f1b041056a0 2023-10-08T13:44:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b03fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b04101200 0x7f1b04073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b03fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b04101200 0x7f1b04073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52938/0 (socket says 172.21.15.57:52938) 2023-10-08T13:44:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.647+0000 7f1b03fff700 1 -- 172.21.15.57:0/2351305917 learned_addr learned my addr 172.21.15.57:0/2351305917 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b08fbd700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100840 0x7f1b04072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b097be700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b041056a0 0x7f1b04073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b03fff700 1 -- 172.21.15.57:0/2351305917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b041056a0 msgr2=0x7f1b04073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b03fff700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b041056a0 0x7f1b04073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b03fff700 1 -- 172.21.15.57:0/2351305917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100840 msgr2=0x7f1b04072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b03fff700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100840 0x7f1b04072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b03fff700 1 -- 172.21.15.57:0/2351305917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af80095f0 con 0x7f1b04101200 2023-10-08T13:44:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b08fbd700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100840 0x7f1b04072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:44:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b03fff700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b04101200 0x7f1b04073030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1af000d980 tx=0x7f1af000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b01ffb700 1 -- 172.21.15.57:0/2351305917 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0004220 con 0x7f1b04101200 2023-10-08T13:44:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b0b221700 1 -- 172.21.15.57:0/2351305917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b041a5580 con 0x7f1b04101200 2023-10-08T13:44:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b01ffb700 1 -- 172.21.15.57:0/2351305917 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1af0007640 con 0x7f1b04101200 2023-10-08T13:44:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.648+0000 7f1b01ffb700 1 -- 172.21.15.57:0/2351305917 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0005690 con 0x7f1b04101200 2023-10-08T13:44:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.649+0000 7f1b0b221700 1 -- 172.21.15.57:0/2351305917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b041a5a80 con 0x7f1b04101200 2023-10-08T13:44:34.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.650+0000 7f1b0b221700 1 -- 172.21.15.57:0/2351305917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b04108d10 con 0x7f1b04101200 2023-10-08T13:44:34.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.651+0000 7f1b01ffb700 1 -- 172.21.15.57:0/2351305917 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f1af0004750 con 0x7f1b04101200 2023-10-08T13:44:34.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.652+0000 7f1b01ffb700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1af40a1440 0x7f1af40a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:34.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.652+0000 7f1b01ffb700 1 -- 172.21.15.57:0/2351305917 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1af00cae80 con 0x7f1b04101200 2023-10-08T13:44:34.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.657+0000 7f1b08fbd700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1af40a1440 0x7f1af40a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:34.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.658+0000 7f1b08fbd700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1af40a1440 0x7f1af40a3900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1af800b2e0 tx=0x7f1af8005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:34.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.658+0000 7f1b01ffb700 1 -- 172.21.15.57:0/2351305917 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1af00960e0 con 0x7f1b04101200 2023-10-08T13:44:34.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:34 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/882638327' entity='client.admin' 2023-10-08T13:44:34.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:34 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:34.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:34 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:34.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:34 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:34.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:34 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:34.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:34 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3567091931' entity='client.admin' 2023-10-08T13:44:34.790 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:34 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1596511929' entity='client.admin' 2023-10-08T13:44:34.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.876+0000 7f1b0b221700 1 -- 172.21.15.57:0/2351305917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7f1b04073fd0 con 0x7f1b04101200 2023-10-08T13:44:34.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.880+0000 7f1b01ffb700 1 -- 172.21.15.57:0/2351305917 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v26)=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1af00160b0 con 0x7f1b04101200 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.882+0000 7f1b0b221700 1 -- 172.21.15.57:0/2351305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1af40a1440 msgr2=0x7f1af40a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.882+0000 7f1b0b221700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1af40a1440 0x7f1af40a3900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1af800b2e0 tx=0x7f1af8005960 comp rx=0 tx=0).stop 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.882+0000 7f1b0b221700 1 -- 172.21.15.57:0/2351305917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b04101200 msgr2=0x7f1b04073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.882+0000 7f1b0b221700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b04101200 0x7f1b04073030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1af000d980 tx=0x7f1af000dc90 comp rx=0 tx=0).stop 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.882+0000 7f1b0b221700 1 -- 172.21.15.57:0/2351305917 shutdown_connections 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.882+0000 7f1b0b221700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b041056a0 0x7f1b04073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.882+0000 7f1b0b221700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1af40a1440 0x7f1af40a3900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.882+0000 7f1b0b221700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100840 0x7f1b04072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.882+0000 7f1b0b221700 1 --2- 172.21.15.57:0/2351305917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b04101200 0x7f1b04073030 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.882+0000 7f1b0b221700 1 -- 172.21.15.57:0/2351305917 >> 172.21.15.57:0/2351305917 conn(0x7f1b04077950 msgr2=0x7f1b040fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.883+0000 7f1b0b221700 1 -- 172.21.15.57:0/2351305917 shutdown_connections 2023-10-08T13:44:34.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:34.883+0000 7f1b0b221700 1 -- 172.21.15.57:0/2351305917 wait complete. 2023-10-08T13:44:34.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:34 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/882638327' entity='client.admin' 2023-10-08T13:44:34.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:34 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:34.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:34 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:34.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:34 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:34.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:34 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:34.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:34 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3567091931' entity='client.admin' 2023-10-08T13:44:34.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:34 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1596511929' entity='client.admin' 2023-10-08T13:44:35.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:34 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/882638327' entity='client.admin' 2023-10-08T13:44:35.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:34 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:35.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:34 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:35.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:34 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:35.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:34 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:35.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:34 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3567091931' entity='client.admin' 2023-10-08T13:44:35.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:34 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1596511929' entity='client.admin' 2023-10-08T13:44:35.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.556+0000 7f5d01052700 1 -- 172.21.15.57:0/3401403978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cfc1057c0 msgr2=0x7f5cfc105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:35.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.556+0000 7f5d01052700 1 --2- 172.21.15.57:0/3401403978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cfc1057c0 0x7f5cfc105ba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009990 tx=0x7f5cec009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.556+0000 7f5d01052700 1 -- 172.21.15.57:0/3401403978 shutdown_connections 2023-10-08T13:44:35.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.556+0000 7f5d01052700 1 --2- 172.21.15.57:0/3401403978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc101260 0x7f5cfc10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.556+0000 7f5d01052700 1 --2- 172.21.15.57:0/3401403978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cfc1057c0 0x7f5cfc105ba0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.556+0000 7f5d01052700 1 --2- 172.21.15.57:0/3401403978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cfc1008a0 0x7f5cfc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.556+0000 7f5d01052700 1 -- 172.21.15.57:0/3401403978 >> 172.21.15.57:0/3401403978 conn(0x7f5cfc0779e0 msgr2=0x7f5cfc077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:35.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.556+0000 7f5d01052700 1 -- 172.21.15.57:0/3401403978 shutdown_connections 2023-10-08T13:44:35.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.556+0000 7f5d01052700 1 -- 172.21.15.57:0/3401403978 wait complete. 2023-10-08T13:44:35.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.557+0000 7f5d01052700 1 Processor -- start 2023-10-08T13:44:35.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.557+0000 7f5d01052700 1 -- start start 2023-10-08T13:44:35.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5d01052700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc1008a0 0x7f5cfc19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:35.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5d01052700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cfc101260 0x7f5cfc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:35.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5d01052700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cfc1057c0 0x7f5cfc196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:35.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5d01052700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc1970a0 con 0x7f5cfc1057c0 2023-10-08T13:44:35.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5d01052700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc197210 con 0x7f5cfc101260 2023-10-08T13:44:35.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5d01052700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc1973d0 con 0x7f5cfc1008a0 2023-10-08T13:44:35.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfb59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cfc1057c0 0x7f5cfc196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:35.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc1008a0 0x7f5cfc19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:35.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfb59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cfc1057c0 0x7f5cfc196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52968/0 (socket says 172.21.15.57:52968) 2023-10-08T13:44:35.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfb59e700 1 -- 172.21.15.57:0/956364118 learned_addr learned my addr 172.21.15.57:0/956364118 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:35.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfa59c700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cfc101260 0x7f5cfc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:35.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfad9d700 1 -- 172.21.15.57:0/956364118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cfc101260 msgr2=0x7f5cfc19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:35.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfad9d700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cfc101260 0x7f5cfc19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfad9d700 1 -- 172.21.15.57:0/956364118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cfc1057c0 msgr2=0x7f5cfc196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:35.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfad9d700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cfc1057c0 0x7f5cfc196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfad9d700 1 -- 172.21.15.57:0/956364118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec0095f0 con 0x7f5cfc1008a0 2023-10-08T13:44:35.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfb59e700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cfc1057c0 0x7f5cfc196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:44:35.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.558+0000 7f5cfad9d700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc1008a0 0x7f5cfc19c900 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cec000c00 tx=0x7f5cec00f700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:35.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.559+0000 7f5ce3fff700 1 -- 172.21.15.57:0/956364118 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec01d070 con 0x7f5cfc1008a0 2023-10-08T13:44:35.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.559+0000 7f5d01052700 1 -- 172.21.15.57:0/956364118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc197650 con 0x7f5cfc1008a0 2023-10-08T13:44:35.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.559+0000 7f5ce3fff700 1 -- 172.21.15.57:0/956364118 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cec022470 con 0x7f5cfc1008a0 2023-10-08T13:44:35.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.559+0000 7f5ce3fff700 1 -- 172.21.15.57:0/956364118 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec017650 con 0x7f5cfc1008a0 2023-10-08T13:44:35.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.559+0000 7f5d01052700 1 -- 172.21.15.57:0/956364118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc1a38f0 con 0x7f5cfc1008a0 2023-10-08T13:44:35.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.560+0000 7f5d01052700 1 -- 172.21.15.57:0/956364118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc108d80 con 0x7f5cfc1008a0 2023-10-08T13:44:35.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.561+0000 7f5ce3fff700 1 -- 172.21.15.57:0/956364118 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f5cec0225e0 con 0x7f5cfc1008a0 2023-10-08T13:44:35.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.561+0000 7f5ce3fff700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ce40a1440 0x7f5ce40a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:35.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.562+0000 7f5cfa59c700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ce40a1440 0x7f5ce40a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:35.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.562+0000 7f5ce3fff700 1 -- 172.21.15.57:0/956364118 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f5cec0cc610 con 0x7f5cfc1008a0 2023-10-08T13:44:35.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.562+0000 7f5cfa59c700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ce40a1440 0x7f5ce40a3900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc197c00 tx=0x7f5cf000a380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:35.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.565+0000 7f5ce3fff700 1 -- 172.21.15.57:0/956364118 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5cec097900 con 0x7f5cfc1008a0 2023-10-08T13:44:35.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.769+0000 7f5d01052700 1 -- 172.21.15.57:0/956364118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7f5cfc101f30 con 0x7f5cfc1008a0 2023-10-08T13:44:35.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.777+0000 7f5ce3fff700 1 -- 172.21.15.57:0/956364118 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v28)=0 v28) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5cec09af20 con 0x7f5cfc1008a0 2023-10-08T13:44:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 -- 172.21.15.57:0/956364118 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ce40a1440 msgr2=0x7f5ce40a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ce40a1440 0x7f5ce40a3900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc197c00 tx=0x7f5cf000a380 comp rx=0 tx=0).stop 2023-10-08T13:44:35.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 -- 172.21.15.57:0/956364118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc1008a0 msgr2=0x7f5cfc19c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc1008a0 0x7f5cfc19c900 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cec000c00 tx=0x7f5cec00f700 comp rx=0 tx=0).stop 2023-10-08T13:44:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 -- 172.21.15.57:0/956364118 shutdown_connections 2023-10-08T13:44:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cfc1057c0 0x7f5cfc196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc1008a0 0x7f5cfc19c900 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ce40a1440 0x7f5ce40a3900 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 --2- 172.21.15.57:0/956364118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cfc101260 0x7f5cfc19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 -- 172.21.15.57:0/956364118 >> 172.21.15.57:0/956364118 conn(0x7f5cfc0779e0 msgr2=0x7f5cfc0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.779+0000 7f5d01052700 1 -- 172.21.15.57:0/956364118 shutdown_connections 2023-10-08T13:44:35.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.780+0000 7f5d01052700 1 -- 172.21.15.57:0/956364118 wait complete. 2023-10-08T13:44:35.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.972+0000 7ff70108d700 1 -- 172.21.15.57:0/1656777740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6fc1056b0 msgr2=0x7ff6fc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:35.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.972+0000 7ff70108d700 1 --2- 172.21.15.57:0/1656777740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6fc1056b0 0x7ff6fc105a90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009990 tx=0x7ff6ec009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.972+0000 7ff70108d700 1 -- 172.21.15.57:0/1656777740 shutdown_connections 2023-10-08T13:44:35.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.972+0000 7ff70108d700 1 --2- 172.21.15.57:0/1656777740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc100110 0x7ff6fc10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.972+0000 7ff70108d700 1 --2- 172.21.15.57:0/1656777740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6fc1056b0 0x7ff6fc105a90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.972+0000 7ff70108d700 1 --2- 172.21.15.57:0/1656777740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6fc0ff750 0x7ff6fc0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.972+0000 7ff70108d700 1 -- 172.21.15.57:0/1656777740 >> 172.21.15.57:0/1656777740 conn(0x7ff6fc075790 msgr2=0x7ff6fc075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:35.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.973+0000 7ff70108d700 1 -- 172.21.15.57:0/1656777740 shutdown_connections 2023-10-08T13:44:35.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.973+0000 7ff70108d700 1 -- 172.21.15.57:0/1656777740 wait complete. 2023-10-08T13:44:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.973+0000 7ff70108d700 1 Processor -- start 2023-10-08T13:44:35.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff70108d700 1 -- start start 2023-10-08T13:44:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff70108d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6fc0ff750 0x7ff6fc1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff70108d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6fc100110 0x7ff6fc197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff70108d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc1056b0 0x7ff6fc1a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff70108d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc198780 con 0x7ff6fc100110 2023-10-08T13:44:35.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff70108d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc1a2f90 con 0x7ff6fc1056b0 2023-10-08T13:44:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff70108d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc1a30d0 con 0x7ff6fc0ff750 2023-10-08T13:44:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff6fa59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6fc100110 0x7ff6fc197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff6fa59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6fc100110 0x7ff6fc197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52978/0 (socket says 172.21.15.57:52978) 2023-10-08T13:44:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff6fa59c700 1 -- 172.21.15.57:0/1512213225 learned_addr learned my addr 172.21.15.57:0/1512213225 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff6fb59e700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc1056b0 0x7ff6fc1a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:35.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.974+0000 7ff6fad9d700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6fc0ff750 0x7ff6fc1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:35.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff6fa59c700 1 -- 172.21.15.57:0/1512213225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6fc0ff750 msgr2=0x7ff6fc1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:35.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff6fa59c700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6fc0ff750 0x7ff6fc1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff6fa59c700 1 -- 172.21.15.57:0/1512213225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc1056b0 msgr2=0x7ff6fc1a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:35.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff6fa59c700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc1056b0 0x7ff6fc1a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:35.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff6fa59c700 1 -- 172.21.15.57:0/1512213225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0095f0 con 0x7ff6fc100110 2023-10-08T13:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff6fad9d700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6fc0ff750 0x7ff6fc1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff6fa59c700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6fc100110 0x7ff6fc197f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000c9b0 tx=0x7ff6f000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff6e3fff700 1 -- 172.21.15.57:0/1512213225 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0007b00 con 0x7ff6fc100110 2023-10-08T13:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff6e3fff700 1 -- 172.21.15.57:0/1512213225 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6f0007c60 con 0x7ff6fc100110 2023-10-08T13:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff70108d700 1 -- 172.21.15.57:0/1512213225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc1a3360 con 0x7ff6fc100110 2023-10-08T13:44:35.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff6e3fff700 1 -- 172.21.15.57:0/1512213225 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0018660 con 0x7ff6fc100110 2023-10-08T13:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.975+0000 7ff70108d700 1 -- 172.21.15.57:0/1512213225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6fc1a3860 con 0x7ff6fc100110 2023-10-08T13:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.976+0000 7ff70108d700 1 -- 172.21.15.57:0/1512213225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc107c60 con 0x7ff6fc100110 2023-10-08T13:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.977+0000 7ff6e3fff700 1 -- 172.21.15.57:0/1512213225 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7ff6f000f450 con 0x7ff6fc100110 2023-10-08T13:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.978+0000 7ff6e3fff700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff6e40a1440 0x7ff6e40a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.978+0000 7ff6fad9d700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff6e40a1440 0x7ff6e40a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:35.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.978+0000 7ff6e3fff700 1 -- 172.21.15.57:0/1512213225 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff6f00cb0d0 con 0x7ff6fc100110 2023-10-08T13:44:35.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.978+0000 7ff6fad9d700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff6e40a1440 0x7ff6e40a3900 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00b5c0 tx=0x7ff6ec005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:35.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.982+0000 7ff6e3fff700 1 -- 172.21.15.57:0/1512213225 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff6f0096330 con 0x7ff6fc100110 2023-10-08T13:44:36.097 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:35 smithi057 ceph-mon[99476]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:36.097 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:35 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2351305917' entity='client.admin' 2023-10-08T13:44:36.097 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:35 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1374748832' entity='client.admin' 2023-10-08T13:44:36.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:36.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:36.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:36.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:36.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:35 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' 2023-10-08T13:44:36.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.184+0000 7ff70108d700 1 -- 172.21.15.57:0/1512213225 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7ff6fc100df0 con 0x7ff6fc100110 2023-10-08T13:44:36.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:35 smithi115 ceph-mon[104408]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:36.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:35 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2351305917' entity='client.admin' 2023-10-08T13:44:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:35 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1374748832' entity='client.admin' 2023-10-08T13:44:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:35 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' 2023-10-08T13:44:36.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.188+0000 7ff6e3fff700 1 -- 172.21.15.57:0/1512213225 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v29)=0 v29) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff6f0099950 con 0x7ff6fc100110 2023-10-08T13:44:36.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 -- 172.21.15.57:0/1512213225 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff6e40a1440 msgr2=0x7ff6e40a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff6e40a1440 0x7ff6e40a3900 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00b5c0 tx=0x7ff6ec005fb0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 -- 172.21.15.57:0/1512213225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6fc100110 msgr2=0x7ff6fc197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6fc100110 0x7ff6fc197f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000c9b0 tx=0x7ff6f000ccc0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 -- 172.21.15.57:0/1512213225 shutdown_connections 2023-10-08T13:44:36.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff6e40a1440 0x7ff6e40a3900 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6fc1056b0 0x7ff6fc1a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6fc0ff750 0x7ff6fc1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 --2- 172.21.15.57:0/1512213225 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6fc100110 0x7ff6fc197f10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 -- 172.21.15.57:0/1512213225 >> 172.21.15.57:0/1512213225 conn(0x7ff6fc075790 msgr2=0x7ff6fc0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:36.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 -- 172.21.15.57:0/1512213225 shutdown_connections 2023-10-08T13:44:36.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.190+0000 7ff70108d700 1 -- 172.21.15.57:0/1512213225 wait complete. 2023-10-08T13:44:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:35 smithi099 ceph-mon[103849]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:35 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2351305917' entity='client.admin' 2023-10-08T13:44:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:35 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1374748832' entity='client.admin' 2023-10-08T13:44:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:36.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:36.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:36.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:36.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:35 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' 2023-10-08T13:44:36.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.427+0000 7f80729db700 1 -- 172.21.15.57:0/2301027132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f806c101200 msgr2=0x7f806c10d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.427+0000 7f80729db700 1 --2- 172.21.15.57:0/2301027132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f806c101200 0x7f806c10d630 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f806400b0a0 tx=0x7f806400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.427+0000 7f80729db700 1 -- 172.21.15.57:0/2301027132 shutdown_connections 2023-10-08T13:44:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.427+0000 7f80729db700 1 --2- 172.21.15.57:0/2301027132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f806c101200 0x7f806c10d630 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.427+0000 7f80729db700 1 --2- 172.21.15.57:0/2301027132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806c1056a0 0x7f806c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.427+0000 7f80729db700 1 --2- 172.21.15.57:0/2301027132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c100840 0x7f806c100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.427+0000 7f80729db700 1 -- 172.21.15.57:0/2301027132 >> 172.21.15.57:0/2301027132 conn(0x7f806c077950 msgr2=0x7f806c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.427+0000 7f80729db700 1 -- 172.21.15.57:0/2301027132 shutdown_connections 2023-10-08T13:44:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.427+0000 7f80729db700 1 -- 172.21.15.57:0/2301027132 wait complete. 2023-10-08T13:44:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f80729db700 1 Processor -- start 2023-10-08T13:44:36.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f80729db700 1 -- start start 2023-10-08T13:44:36.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f80729db700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c100840 0x7f806c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:36.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f80729db700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806c101200 0x7f806c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:36.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f80729db700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f806c1056a0 0x7f806c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:36.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f80729db700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c1a5100 con 0x7f806c1056a0 2023-10-08T13:44:36.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f80729db700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c1a5270 con 0x7f806c101200 2023-10-08T13:44:36.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f80729db700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c1a5430 con 0x7f806c100840 2023-10-08T13:44:36.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f8070f78700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f806c1056a0 0x7f806c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f8070f78700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f806c1056a0 0x7f806c073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52994/0 (socket says 172.21.15.57:52994) 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f8070f78700 1 -- 172.21.15.57:0/402776212 learned_addr learned my addr 172.21.15.57:0/402776212 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f806b7fe700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806c101200 0x7f806c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f806bfff700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c100840 0x7f806c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.428+0000 7f8070f78700 1 -- 172.21.15.57:0/402776212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c100840 msgr2=0x7f806c072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.429+0000 7f8070f78700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c100840 0x7f806c072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.429+0000 7f8070f78700 1 -- 172.21.15.57:0/402776212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806c101200 msgr2=0x7f806c073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.429+0000 7f8070f78700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806c101200 0x7f806c073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.429+0000 7f8070f78700 1 -- 172.21.15.57:0/402776212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8064009d00 con 0x7f806c1056a0 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.429+0000 7f8070f78700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f806c1056a0 0x7f806c073570 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f80640077f0 tx=0x7f80640078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.429+0000 7f80697fa700 1 -- 172.21.15.57:0/402776212 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806400e040 con 0x7f806c1056a0 2023-10-08T13:44:36.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.429+0000 7f80697fa700 1 -- 172.21.15.57:0/402776212 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8064004c60 con 0x7f806c1056a0 2023-10-08T13:44:36.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.429+0000 7f80729db700 1 -- 172.21.15.57:0/402776212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806c1a56b0 con 0x7f806c1056a0 2023-10-08T13:44:36.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.429+0000 7f80697fa700 1 -- 172.21.15.57:0/402776212 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80640053b0 con 0x7f806c1056a0 2023-10-08T13:44:36.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.429+0000 7f80729db700 1 -- 172.21.15.57:0/402776212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806c1a5bb0 con 0x7f806c1056a0 2023-10-08T13:44:36.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.430+0000 7f80729db700 1 -- 172.21.15.57:0/402776212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806c108d10 con 0x7f806c1056a0 2023-10-08T13:44:36.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.431+0000 7f80697fa700 1 -- 172.21.15.57:0/402776212 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f806401ba10 con 0x7f806c1056a0 2023-10-08T13:44:36.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.431+0000 7f80697fa700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f80540a1440 0x7f80540a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:36.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.432+0000 7f806bfff700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f80540a1440 0x7f80540a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:36.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.432+0000 7f80697fa700 1 -- 172.21.15.57:0/402776212 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f80640cc840 con 0x7f806c1056a0 2023-10-08T13:44:36.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.432+0000 7f806bfff700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f80540a1440 0x7f80540a3900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f805c005950 tx=0x7f805c0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:36.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.436+0000 7f80697fa700 1 -- 172.21.15.57:0/402776212 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8064097b30 con 0x7f806c1056a0 2023-10-08T13:44:36.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.082+0000 7f086d394700 1 -- 172.21.15.57:0/3244113779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08681056b0 msgr2=0x7f0868105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.082+0000 7f086d394700 1 --2- 172.21.15.57:0/3244113779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08681056b0 0x7f0868105a90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0858009990 tx=0x7f0858009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.082+0000 7f086d394700 1 -- 172.21.15.57:0/3244113779 shutdown_connections 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.082+0000 7f086d394700 1 --2- 172.21.15.57:0/3244113779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868100110 0x7f086810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.082+0000 7f086d394700 1 --2- 172.21.15.57:0/3244113779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08681056b0 0x7f0868105a90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.082+0000 7f086d394700 1 --2- 172.21.15.57:0/3244113779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08680ff750 0x7f08680ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.082+0000 7f086d394700 1 -- 172.21.15.57:0/3244113779 >> 172.21.15.57:0/3244113779 conn(0x7f0868075790 msgr2=0x7f0868075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.082+0000 7f086d394700 1 -- 172.21.15.57:0/3244113779 shutdown_connections 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.082+0000 7f086d394700 1 -- 172.21.15.57:0/3244113779 wait complete. 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.082+0000 7f086d394700 1 Processor -- start 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f086d394700 1 -- start start 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f086d394700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08680ff750 0x7f0868197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f086d394700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868100110 0x7f0868198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:36.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f086d394700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08681056b0 0x7f0868198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f086d394700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681a2de0 con 0x7f08681056b0 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f086d394700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681a2f50 con 0x7f08680ff750 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f086d394700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681a3110 con 0x7f0868100110 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08677fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08681056b0 0x7f0868198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08677fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08681056b0 0x7f0868198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52954/0 (socket says 172.21.15.57:52954) 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08677fe700 1 -- 172.21.15.57:0/1374748832 learned_addr learned my addr 172.21.15.57:0/1374748832 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f0866ffd700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08680ff750 0x7f0868197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08667fc700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868100110 0x7f0868198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08677fe700 1 -- 172.21.15.57:0/1374748832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868100110 msgr2=0x7f0868198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08677fe700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868100110 0x7f0868198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08677fe700 1 -- 172.21.15.57:0/1374748832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08680ff750 msgr2=0x7f0868197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08677fe700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08680ff750 0x7f0868197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08677fe700 1 -- 172.21.15.57:0/1374748832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580095f0 con 0x7f08681056b0 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08667fc700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868100110 0x7f0868198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f08677fe700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08681056b0 0x7f0868198590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0860007ee0 tx=0x7f086000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.083+0000 7f084ffff700 1 -- 172.21.15.57:0/1374748832 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860017070 con 0x7f08681056b0 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.084+0000 7f084ffff700 1 -- 172.21.15.57:0/1374748832 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f086000f040 con 0x7f08681056b0 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.084+0000 7f086d394700 1 -- 172.21.15.57:0/1374748832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08681a33f0 con 0x7f08681056b0 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.084+0000 7f084ffff700 1 -- 172.21.15.57:0/1374748832 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860005740 con 0x7f08681056b0 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.084+0000 7f086d394700 1 -- 172.21.15.57:0/1374748832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08681a38a0 con 0x7f08681056b0 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.084+0000 7f086d394700 1 -- 172.21.15.57:0/1374748832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0868107cb0 con 0x7f08681056b0 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.086+0000 7f084ffff700 1 -- 172.21.15.57:0/1374748832 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f086000dc60 con 0x7f08681056b0 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.086+0000 7f084ffff700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08500a1440 0x7f08500a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:36.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.086+0000 7f0866ffd700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08500a1440 0x7f08500a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.087+0000 7f084ffff700 1 -- 172.21.15.57:0/1374748832 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f08600caf20 con 0x7f08681056b0 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.087+0000 7f0866ffd700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08500a1440 0x7f08500a3900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f085800b2e0 tx=0x7f0858005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.090+0000 7f084ffff700 1 -- 172.21.15.57:0/1374748832 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0860096180 con 0x7f08681056b0 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.290+0000 7f086d394700 1 -- 172.21.15.57:0/1374748832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f0868068750 con 0x7f08681056b0 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.294+0000 7f084ffff700 1 -- 172.21.15.57:0/1374748832 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v27)=0 v27) v1 ==== 133+0+0 (secure 0 0 0) 0x7f08600997a0 con 0x7f08681056b0 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 -- 172.21.15.57:0/1374748832 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08500a1440 msgr2=0x7f08500a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08500a1440 0x7f08500a3900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f085800b2e0 tx=0x7f0858005fd0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 -- 172.21.15.57:0/1374748832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08681056b0 msgr2=0x7f0868198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08681056b0 0x7f0868198590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0860007ee0 tx=0x7f086000d3b0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 -- 172.21.15.57:0/1374748832 shutdown_connections 2023-10-08T13:44:36.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08681056b0 0x7f0868198590 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08500a1440 0x7f08500a3900 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08680ff750 0x7f0868197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 --2- 172.21.15.57:0/1374748832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868100110 0x7f0868198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 -- 172.21.15.57:0/1374748832 >> 172.21.15.57:0/1374748832 conn(0x7f0868075790 msgr2=0x7f08680fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:36.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 -- 172.21.15.57:0/1374748832 shutdown_connections 2023-10-08T13:44:36.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:35.297+0000 7f086d394700 1 -- 172.21.15.57:0/1374748832 wait complete. 2023-10-08T13:44:36.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.646+0000 7f80729db700 1 -- 172.21.15.57:0/402776212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f806c074020 con 0x7f806c1056a0 2023-10-08T13:44:36.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.654+0000 7f80697fa700 1 -- 172.21.15.57:0/402776212 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f806409b150 con 0x7f806c1056a0 2023-10-08T13:44:36.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 -- 172.21.15.57:0/402776212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f80540a1440 msgr2=0x7f80540a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f80540a1440 0x7f80540a3900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f805c005950 tx=0x7f805c0058e0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 -- 172.21.15.57:0/402776212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f806c1056a0 msgr2=0x7f806c073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:36.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f806c1056a0 0x7f806c073570 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f80640077f0 tx=0x7f80640078d0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 -- 172.21.15.57:0/402776212 shutdown_connections 2023-10-08T13:44:36.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f806c1056a0 0x7f806c073570 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c100840 0x7f806c072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f80540a1440 0x7f80540a3900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 --2- 172.21.15.57:0/402776212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806c101200 0x7f806c073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:36.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 -- 172.21.15.57:0/402776212 >> 172.21.15.57:0/402776212 conn(0x7f806c077950 msgr2=0x7f806c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:36.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 -- 172.21.15.57:0/402776212 shutdown_connections 2023-10-08T13:44:36.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:36.656+0000 7f80729db700 1 -- 172.21.15.57:0/402776212 wait complete. 2023-10-08T13:44:37.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:37 smithi057 ceph-mon[99476]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:37.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:37 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1512213225' entity='client.admin' 2023-10-08T13:44:37.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:37 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/402776212' entity='client.admin' 2023-10-08T13:44:37.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:37.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:37.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:37.449 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:37.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:37 smithi099 ceph-mon[103849]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:37.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:37 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1512213225' entity='client.admin' 2023-10-08T13:44:37.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:37 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/402776212' entity='client.admin' 2023-10-08T13:44:37.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:37.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:37.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:37.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:37.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.540+0000 7f1b873cb700 1 -- 172.21.15.57:0/4121043386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b801000a0 msgr2=0x7f1b8010c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:37.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.540+0000 7f1b873cb700 1 --2- 172.21.15.57:0/4121043386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b801000a0 0x7f1b8010c540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800b0a0 tx=0x7f1b7800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.540+0000 7f1b873cb700 1 -- 172.21.15.57:0/4121043386 shutdown_connections 2023-10-08T13:44:37.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.540+0000 7f1b873cb700 1 --2- 172.21.15.57:0/4121043386 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b801000a0 0x7f1b8010c540 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.540+0000 7f1b873cb700 1 --2- 172.21.15.57:0/4121043386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b801045b0 0x7f1b80104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.540+0000 7f1b873cb700 1 --2- 172.21.15.57:0/4121043386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b800ff6e0 0x7f1b800ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.540+0000 7f1b873cb700 1 -- 172.21.15.57:0/4121043386 >> 172.21.15.57:0/4121043386 conn(0x7f1b800766c0 msgr2=0x7f1b80076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:37.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.540+0000 7f1b873cb700 1 -- 172.21.15.57:0/4121043386 shutdown_connections 2023-10-08T13:44:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.540+0000 7f1b873cb700 1 -- 172.21.15.57:0/4121043386 wait complete. 2023-10-08T13:44:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b873cb700 1 Processor -- start 2023-10-08T13:44:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b873cb700 1 -- start start 2023-10-08T13:44:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b873cb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b800ff6e0 0x7f1b80196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:37.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b873cb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b801000a0 0x7f1b80197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b873cb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b801045b0 0x7f1b8019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b873cb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8019bcd0 con 0x7f1b800ff6e0 2023-10-08T13:44:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b873cb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8019be40 con 0x7f1b801000a0 2023-10-08T13:44:37.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b873cb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8019c000 con 0x7f1b801045b0 2023-10-08T13:44:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85167700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b800ff6e0 0x7f1b80196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85167700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b800ff6e0 0x7f1b80196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53004/0 (socket says 172.21.15.57:53004) 2023-10-08T13:44:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85167700 1 -- 172.21.15.57:0/3035249322 learned_addr learned my addr 172.21.15.57:0/3035249322 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85968700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b801045b0 0x7f1b8019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b84966700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b801000a0 0x7f1b80197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:37.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85167700 1 -- 172.21.15.57:0/3035249322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b801045b0 msgr2=0x7f1b8019b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85167700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b801045b0 0x7f1b8019b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85167700 1 -- 172.21.15.57:0/3035249322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b801000a0 msgr2=0x7f1b80197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:37.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85167700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b801000a0 0x7f1b80197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85167700 1 -- 172.21.15.57:0/3035249322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b78009d00 con 0x7f1b800ff6e0 2023-10-08T13:44:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85968700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b801045b0 0x7f1b8019b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:44:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.541+0000 7f1b85167700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b800ff6e0 0x7f1b80196dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000d980 tx=0x7f1b7000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.542+0000 7f1b767fc700 1 -- 172.21.15.57:0/3035249322 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7000bb60 con 0x7f1b800ff6e0 2023-10-08T13:44:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.542+0000 7f1b767fc700 1 -- 172.21.15.57:0/3035249322 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b70007640 con 0x7f1b800ff6e0 2023-10-08T13:44:37.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.542+0000 7f1b873cb700 1 -- 172.21.15.57:0/3035249322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8019c2e0 con 0x7f1b800ff6e0 2023-10-08T13:44:37.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.542+0000 7f1b767fc700 1 -- 172.21.15.57:0/3035249322 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b70004e90 con 0x7f1b800ff6e0 2023-10-08T13:44:37.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.542+0000 7f1b873cb700 1 -- 172.21.15.57:0/3035249322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b801a3b40 con 0x7f1b800ff6e0 2023-10-08T13:44:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.543+0000 7f1b873cb700 1 -- 172.21.15.57:0/3035249322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b80108c30 con 0x7f1b800ff6e0 2023-10-08T13:44:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.544+0000 7f1b767fc700 1 -- 172.21.15.57:0/3035249322 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f1b7000bcc0 con 0x7f1b800ff6e0 2023-10-08T13:44:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.544+0000 7f1b767fc700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b6c0a14c0 0x7f1b6c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:37.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.545+0000 7f1b84966700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b6c0a14c0 0x7f1b6c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:37.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.545+0000 7f1b767fc700 1 -- 172.21.15.57:0/3035249322 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1b700cb530 con 0x7f1b800ff6e0 2023-10-08T13:44:37.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.545+0000 7f1b84966700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b6c0a14c0 0x7f1b6c0a3980 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c005950 tx=0x7f1b7c0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:37.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.548+0000 7f1b767fc700 1 -- 172.21.15.57:0/3035249322 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1b700966f0 con 0x7f1b800ff6e0 2023-10-08T13:44:37.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:37 smithi115 ceph-mon[104408]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:37.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:37 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1512213225' entity='client.admin' 2023-10-08T13:44:37.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:37 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/402776212' entity='client.admin' 2023-10-08T13:44:37.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:44:37.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:37.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:44:37.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:44:37.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.707+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2585891277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00c40ff190 msgr2=0x7f00c40ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:37.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.707+0000 7f00cc7e8700 1 --2- 172.21.15.57:0/2585891277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00c40ff190 0x7f00c40ff5f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009990 tx=0x7f00b4009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.707+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2585891277 shutdown_connections 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.707+0000 7f00cc7e8700 1 --2- 172.21.15.57:0/2585891277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00c40ffb30 0x7f00c410d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.707+0000 7f00cc7e8700 1 --2- 172.21.15.57:0/2585891277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c41058d0 0x7f00c4105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.707+0000 7f00cc7e8700 1 --2- 172.21.15.57:0/2585891277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00c40ff190 0x7f00c40ff5f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.707+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2585891277 >> 172.21.15.57:0/2585891277 conn(0x7f00c4074a10 msgr2=0x7f00c4074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.707+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2585891277 shutdown_connections 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.707+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2585891277 wait complete. 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.708+0000 7f00cc7e8700 1 Processor -- start 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.708+0000 7f00cc7e8700 1 -- start start 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.708+0000 7f00cc7e8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00c40ff190 0x7f00c419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.708+0000 7f00cc7e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c40ffb30 0x7f00c419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:37.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.708+0000 7f00cc7e8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00c41058d0 0x7f00c4196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.708+0000 7f00cc7e8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c41970b0 con 0x7f00c40ff190 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.708+0000 7f00cc7e8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c4197220 con 0x7f00c41058d0 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.708+0000 7f00cc7e8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c41973e0 con 0x7f00c40ffb30 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.708+0000 7f00ca584700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00c40ff190 0x7f00c419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.708+0000 7f00ca584700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00c40ff190 0x7f00c419c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53018/0 (socket says 172.21.15.57:53018) 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00ca584700 1 -- 172.21.15.57:0/2688853440 learned_addr learned my addr 172.21.15.57:0/2688853440 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00c9d83700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c40ffb30 0x7f00c419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00cad85700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00c41058d0 0x7f00c4196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00ca584700 1 -- 172.21.15.57:0/2688853440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c40ffb30 msgr2=0x7f00c419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00ca584700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c40ffb30 0x7f00c419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00ca584700 1 -- 172.21.15.57:0/2688853440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00c41058d0 msgr2=0x7f00c4196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00ca584700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00c41058d0 0x7f00c4196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00ca584700 1 -- 172.21.15.57:0/2688853440 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b40095f0 con 0x7f00c40ff190 2023-10-08T13:44:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00cad85700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00c41058d0 0x7f00c4196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:44:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00ca584700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00c40ff190 0x7f00c419c910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f00c000d980 tx=0x7f00c000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00bb7fe700 1 -- 172.21.15.57:0/2688853440 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c000de60 con 0x7f00c40ff190 2023-10-08T13:44:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00bb7fe700 1 -- 172.21.15.57:0/2688853440 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00c00044e0 con 0x7f00c40ff190 2023-10-08T13:44:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2688853440 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c41976c0 con 0x7f00c40ff190 2023-10-08T13:44:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00bb7fe700 1 -- 172.21.15.57:0/2688853440 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0003be0 con 0x7f00c40ff190 2023-10-08T13:44:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.709+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2688853440 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c41a3900 con 0x7f00c40ff190 2023-10-08T13:44:37.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.710+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2688853440 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c4108d30 con 0x7f00c40ff190 2023-10-08T13:44:37.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.711+0000 7f00bb7fe700 1 -- 172.21.15.57:0/2688853440 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f00c0009bb0 con 0x7f00c40ff190 2023-10-08T13:44:37.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.712+0000 7f00bb7fe700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00b00a14c0 0x7f00b00a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:37.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.712+0000 7f00c9d83700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00b00a14c0 0x7f00b00a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:37.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.712+0000 7f00bb7fe700 1 -- 172.21.15.57:0/2688853440 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f00c00cadb0 con 0x7f00c40ff190 2023-10-08T13:44:37.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.713+0000 7f00c9d83700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00b00a14c0 0x7f00b00a3980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f00b400b5c0 tx=0x7f00b401a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:37.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.717+0000 7f00bb7fe700 1 -- 172.21.15.57:0/2688853440 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f00c0096010 con 0x7f00c40ff190 2023-10-08T13:44:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.766+0000 7f1b873cb700 1 -- 172.21.15.57:0/3035249322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f1b801980e0 con 0x7f1b800ff6e0 2023-10-08T13:44:37.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.770+0000 7f1b767fc700 1 -- 172.21.15.57:0/3035249322 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v31)=0 v31) v1 ==== 121+0+0 (secure 0 0 0) 0x7f1b700160c0 con 0x7f1b800ff6e0 2023-10-08T13:44:37.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.772+0000 7f1b873cb700 1 -- 172.21.15.57:0/3035249322 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b6c0a14c0 msgr2=0x7f1b6c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:37.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.772+0000 7f1b873cb700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b6c0a14c0 0x7f1b6c0a3980 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c005950 tx=0x7f1b7c0058e0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.772+0000 7f1b873cb700 1 -- 172.21.15.57:0/3035249322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b800ff6e0 msgr2=0x7f1b80196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:37.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.772+0000 7f1b873cb700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b800ff6e0 0x7f1b80196dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000d980 tx=0x7f1b7000dd40 comp rx=0 tx=0).stop 2023-10-08T13:44:37.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.773+0000 7f1b873cb700 1 -- 172.21.15.57:0/3035249322 shutdown_connections 2023-10-08T13:44:37.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.773+0000 7f1b873cb700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b801045b0 0x7f1b8019b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.773+0000 7f1b873cb700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b800ff6e0 0x7f1b80196dd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.773+0000 7f1b873cb700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b6c0a14c0 0x7f1b6c0a3980 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.773+0000 7f1b873cb700 1 --2- 172.21.15.57:0/3035249322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b801000a0 0x7f1b80197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.773+0000 7f1b873cb700 1 -- 172.21.15.57:0/3035249322 >> 172.21.15.57:0/3035249322 conn(0x7f1b800766c0 msgr2=0x7f1b80068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:37.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.773+0000 7f1b873cb700 1 -- 172.21.15.57:0/3035249322 shutdown_connections 2023-10-08T13:44:37.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.773+0000 7f1b873cb700 1 -- 172.21.15.57:0/3035249322 wait complete. 2023-10-08T13:44:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.929+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2688853440 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7f00c4067c30 con 0x7f00c40ff190 2023-10-08T13:44:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.937+0000 7f00bb7fe700 1 -- 172.21.15.57:0/2688853440 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v32)=0 v32) v1 ==== 116+0+0 (secure 0 0 0) 0x7f00c0009940 con 0x7f00c40ff190 2023-10-08T13:44:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.939+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2688853440 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00b00a14c0 msgr2=0x7f00b00a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.939+0000 7f00cc7e8700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00b00a14c0 0x7f00b00a3980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f00b400b5c0 tx=0x7f00b401a040 comp rx=0 tx=0).stop 2023-10-08T13:44:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.939+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2688853440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00c40ff190 msgr2=0x7f00c419c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.939+0000 7f00cc7e8700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00c40ff190 0x7f00c419c910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f00c000d980 tx=0x7f00c000dd40 comp rx=0 tx=0).stop 2023-10-08T13:44:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.939+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2688853440 shutdown_connections 2023-10-08T13:44:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.939+0000 7f00cc7e8700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00b00a14c0 0x7f00b00a3980 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.939+0000 7f00cc7e8700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00c41058d0 0x7f00c4196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.939+0000 7f00cc7e8700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00c40ff190 0x7f00c419c910 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.939+0000 7f00cc7e8700 1 --2- 172.21.15.57:0/2688853440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c40ffb30 0x7f00c419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:37.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.939+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2688853440 >> 172.21.15.57:0/2688853440 conn(0x7f00c4074a10 msgr2=0x7f00c40fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:37.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.940+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2688853440 shutdown_connections 2023-10-08T13:44:37.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:37.940+0000 7f00cc7e8700 1 -- 172.21.15.57:0/2688853440 wait complete. 2023-10-08T13:44:38.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.621+0000 7ff266fc3700 1 -- 172.21.15.57:0/1019876437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2601056b0 msgr2=0x7ff260105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:38.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.621+0000 7ff266fc3700 1 --2- 172.21.15.57:0/1019876437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2601056b0 0x7ff260105a90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff254009990 tx=0x7ff254009ca0 comp rx=0 tx=0).stop 2023-10-08T13:44:38.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.621+0000 7ff266fc3700 1 -- 172.21.15.57:0/1019876437 shutdown_connections 2023-10-08T13:44:38.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.621+0000 7ff266fc3700 1 --2- 172.21.15.57:0/1019876437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff260100110 0x7ff26010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:38.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.621+0000 7ff266fc3700 1 --2- 172.21.15.57:0/1019876437 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2601056b0 0x7ff260105a90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:38.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.621+0000 7ff266fc3700 1 --2- 172.21.15.57:0/1019876437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2600ff750 0x7ff2600ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:38.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.621+0000 7ff266fc3700 1 -- 172.21.15.57:0/1019876437 >> 172.21.15.57:0/1019876437 conn(0x7ff260075790 msgr2=0x7ff260075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:38.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.621+0000 7ff266fc3700 1 -- 172.21.15.57:0/1019876437 shutdown_connections 2023-10-08T13:44:38.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.621+0000 7ff266fc3700 1 -- 172.21.15.57:0/1019876437 wait complete. 2023-10-08T13:44:38.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff266fc3700 1 Processor -- start 2023-10-08T13:44:38.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff266fc3700 1 -- start start 2023-10-08T13:44:38.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff266fc3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2600ff750 0x7ff26019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:38.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff266fc3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff260100110 0x7ff26019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:38.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff266fc3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2601056b0 0x7ff260196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:38.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff266fc3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff260197140 con 0x7ff260100110 2023-10-08T13:44:38.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff266fc3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2601972b0 con 0x7ff2600ff750 2023-10-08T13:44:38.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff266fc3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff260197470 con 0x7ff2601056b0 2023-10-08T13:44:38.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff25ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff260100110 0x7ff26019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:38.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff25ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff260100110 0x7ff26019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53034/0 (socket says 172.21.15.57:53034) 2023-10-08T13:44:38.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff25ffff700 1 -- 172.21.15.57:0/429567591 learned_addr learned my addr 172.21.15.57:0/429567591 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:38.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff264d5f700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2600ff750 0x7ff26019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:38.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff265560700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2601056b0 0x7ff260196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:38.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff25ffff700 1 -- 172.21.15.57:0/429567591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2601056b0 msgr2=0x7ff260196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:38.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff25ffff700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2601056b0 0x7ff260196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:38.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff25ffff700 1 -- 172.21.15.57:0/429567591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2600ff750 msgr2=0x7ff26019c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff25ffff700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2600ff750 0x7ff26019c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff25ffff700 1 -- 172.21.15.57:0/429567591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540095f0 con 0x7ff260100110 2023-10-08T13:44:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff265560700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2601056b0 0x7ff260196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:44:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.622+0000 7ff264d5f700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2600ff750 0x7ff26019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:44:38.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.623+0000 7ff25ffff700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff260100110 0x7ff26019ce90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00d950 tx=0x7ff24c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.623+0000 7ff25dffb700 1 -- 172.21.15.57:0/429567591 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c009b40 con 0x7ff260100110 2023-10-08T13:44:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.623+0000 7ff25dffb700 1 -- 172.21.15.57:0/429567591 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff24c009ca0 con 0x7ff260100110 2023-10-08T13:44:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.623+0000 7ff266fc3700 1 -- 172.21.15.57:0/429567591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff260197750 con 0x7ff260100110 2023-10-08T13:44:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.623+0000 7ff25dffb700 1 -- 172.21.15.57:0/429567591 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00f620 con 0x7ff260100110 2023-10-08T13:44:38.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.623+0000 7ff266fc3700 1 -- 172.21.15.57:0/429567591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2601a3940 con 0x7ff260100110 2023-10-08T13:44:38.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.624+0000 7ff266fc3700 1 -- 172.21.15.57:0/429567591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff260108cc0 con 0x7ff260100110 2023-10-08T13:44:38.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.625+0000 7ff25dffb700 1 -- 172.21.15.57:0/429567591 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7ff24c010460 con 0x7ff260100110 2023-10-08T13:44:38.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.625+0000 7ff25dffb700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2500a1440 0x7ff2500a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:38.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.626+0000 7ff264d5f700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2500a1440 0x7ff2500a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:38.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.626+0000 7ff25dffb700 1 -- 172.21.15.57:0/429567591 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff24c0cb4b0 con 0x7ff260100110 2023-10-08T13:44:38.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.626+0000 7ff264d5f700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2500a1440 0x7ff2500a3900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff25400b5c0 tx=0x7ff25401a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:38.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.630+0000 7ff25dffb700 1 -- 172.21.15.57:0/429567591 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff24c096710 con 0x7ff260100110 2023-10-08T13:44:38.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi057') 2023-10-08T13:44:38.663 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi057') 2023-10-08T13:44:38.663 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi057') 2023-10-08T13:44:38.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.834+0000 7ff266fc3700 1 -- 172.21.15.57:0/429567591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7ff260197f50 con 0x7ff260100110 2023-10-08T13:44:38.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.837+0000 7ff25dffb700 1 -- 172.21.15.57:0/429567591 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v33)=0 v33) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff24c099d30 con 0x7ff260100110 2023-10-08T13:44:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.839+0000 7ff266fc3700 1 -- 172.21.15.57:0/429567591 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2500a1440 msgr2=0x7ff2500a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.839+0000 7ff266fc3700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2500a1440 0x7ff2500a3900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff25400b5c0 tx=0x7ff25401a040 comp rx=0 tx=0).stop 2023-10-08T13:44:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.839+0000 7ff266fc3700 1 -- 172.21.15.57:0/429567591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff260100110 msgr2=0x7ff26019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.839+0000 7ff266fc3700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff260100110 0x7ff26019ce90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00d950 tx=0x7ff24c00dd10 comp rx=0 tx=0).stop 2023-10-08T13:44:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.840+0000 7ff266fc3700 1 -- 172.21.15.57:0/429567591 shutdown_connections 2023-10-08T13:44:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.840+0000 7ff266fc3700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2601056b0 0x7ff260196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.840+0000 7ff266fc3700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2500a1440 0x7ff2500a3900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.840+0000 7ff266fc3700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2600ff750 0x7ff26019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:38.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.840+0000 7ff266fc3700 1 --2- 172.21.15.57:0/429567591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff260100110 0x7ff26019ce90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:38.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.840+0000 7ff266fc3700 1 -- 172.21.15.57:0/429567591 >> 172.21.15.57:0/429567591 conn(0x7ff260075790 msgr2=0x7ff260068570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:38.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.840+0000 7ff266fc3700 1 -- 172.21.15.57:0/429567591 shutdown_connections 2023-10-08T13:44:38.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:38.840+0000 7ff266fc3700 1 -- 172.21.15.57:0/429567591 wait complete. 2023-10-08T13:44:38.978 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:38 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3035249322' entity='client.admin' 2023-10-08T13:44:38.978 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:38 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2688853440' entity='client.admin' 2023-10-08T13:44:39.012 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi057') 2023-10-08T13:44:39.012 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi057') 2023-10-08T13:44:39.012 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi057') 2023-10-08T13:44:39.012 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi057') 2023-10-08T13:44:39.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:38 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3035249322' entity='client.admin' 2023-10-08T13:44:39.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:38 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2688853440' entity='client.admin' 2023-10-08T13:44:39.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:38 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3035249322' entity='client.admin' 2023-10-08T13:44:39.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:38 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2688853440' entity='client.admin' 2023-10-08T13:44:39.244 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi057') 2023-10-08T13:44:39.244 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi057') 2023-10-08T13:44:39.244 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi057') 2023-10-08T13:44:39.244 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi057') 2023-10-08T13:44:39.244 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi057') 2023-10-08T13:44:39.245 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5f25708e-65e0-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi057') 2023-10-08T13:44:39.245 INFO:tasks.cephadm:Deploying OSDs... 2023-10-08T13:44:39.245 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:44:39.245 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T13:44:39.262 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T13:44:39.263 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_1 2023-10-08T13:44:39.321 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T13:44:39.321 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:39.321 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 390792 Links: 1 2023-10-08T13:44:39.321 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:39.321 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:39.321 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-08 13:43:17.297327256 +0000 2023-10-08T13:44:39.322 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-08 13:38:00.787738023 +0000 2023-10-08T13:44:39.322 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-08 13:38:00.787738023 +0000 2023-10-08T13:44:39.322 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-08T13:44:39.322 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T13:44:39.389 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-08T13:44:39.389 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-08T13:44:39.389 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000294645 s, 1.7 MB/s 2023-10-08T13:44:39.390 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T13:44:39.449 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_2 2023-10-08T13:44:39.507 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T13:44:39.508 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:39.508 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 389665 Links: 1 2023-10-08T13:44:39.508 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:39.508 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:39.508 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-08 13:43:17.351326436 +0000 2023-10-08T13:44:39.508 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-08 13:38:01.304730101 +0000 2023-10-08T13:44:39.508 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-08 13:38:01.304730101 +0000 2023-10-08T13:44:39.508 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-08T13:44:39.508 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T13:44:39.574 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-08T13:44:39.575 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-08T13:44:39.575 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000314069 s, 1.6 MB/s 2023-10-08T13:44:39.576 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T13:44:39.634 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_3 2023-10-08T13:44:39.693 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T13:44:39.693 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:39.693 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 394795 Links: 1 2023-10-08T13:44:39.693 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:39.693 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:39.694 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-08 13:43:17.412325511 +0000 2023-10-08T13:44:39.694 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-08 13:38:01.832722011 +0000 2023-10-08T13:44:39.694 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-08 13:38:01.832722011 +0000 2023-10-08T13:44:39.694 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-08T13:44:39.694 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T13:44:39.759 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-08T13:44:39.759 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-08T13:44:39.759 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000303574 s, 1.7 MB/s 2023-10-08T13:44:39.760 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T13:44:39.819 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_4 2023-10-08T13:44:39.877 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T13:44:39.877 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:39.877 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 390985 Links: 1 2023-10-08T13:44:39.877 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:39.877 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:39.877 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-08 13:43:17.486324387 +0000 2023-10-08T13:44:39.877 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-08 13:38:02.347714120 +0000 2023-10-08T13:44:39.877 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-08 13:38:02.347714120 +0000 2023-10-08T13:44:39.877 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-08T13:44:39.878 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T13:44:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-08T13:44:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-08T13:44:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000307731 s, 1.7 MB/s 2023-10-08T13:44:39.944 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T13:44:40.003 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-08T13:44:40.003 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T13:44:40.024 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T13:44:40.024 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2023-10-08T13:44:40.084 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T13:44:40.085 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:40.085 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 391751 Links: 1 2023-10-08T13:44:40.085 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:40.085 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:40.085 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-08 13:44:23.430236051 +0000 2023-10-08T13:44:40.086 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-08 13:37:47.867145293 +0000 2023-10-08T13:44:40.086 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-08 13:37:47.867145293 +0000 2023-10-08T13:44:40.086 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-10-08T13:44:40.086 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T13:44:40.154 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:39 smithi099 ceph-mon[103849]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:40.154 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:39 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/429567591' entity='client.admin' 2023-10-08T13:44:40.156 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-08T13:44:40.156 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-08T13:44:40.156 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000354484 s, 1.4 MB/s 2023-10-08T13:44:40.158 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T13:44:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:39 smithi115 ceph-mon[104408]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:40.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:39 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/429567591' entity='client.admin' 2023-10-08T13:44:40.219 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2023-10-08T13:44:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:39 smithi057 ceph-mon[99476]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:39 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/429567591' entity='client.admin' 2023-10-08T13:44:40.278 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T13:44:40.278 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:40.278 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 387955 Links: 1 2023-10-08T13:44:40.278 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:40.278 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:40.278 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-08 13:44:23.430236051 +0000 2023-10-08T13:44:40.278 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-08 13:37:48.379133695 +0000 2023-10-08T13:44:40.279 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-08 13:37:48.379133695 +0000 2023-10-08T13:44:40.279 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-10-08T13:44:40.279 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T13:44:40.345 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-08T13:44:40.345 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-08T13:44:40.345 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000230989 s, 2.2 MB/s 2023-10-08T13:44:40.346 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T13:44:40.406 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2023-10-08T13:44:40.464 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T13:44:40.464 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:40.465 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 391963 Links: 1 2023-10-08T13:44:40.465 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:40.465 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:40.465 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-08 13:44:23.430236051 +0000 2023-10-08T13:44:40.465 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-08 13:37:48.875122459 +0000 2023-10-08T13:44:40.465 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-08 13:37:48.875122459 +0000 2023-10-08T13:44:40.466 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-10-08T13:44:40.466 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T13:44:40.532 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-08T13:44:40.532 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-08T13:44:40.532 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000232866 s, 2.2 MB/s 2023-10-08T13:44:40.533 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T13:44:40.593 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2023-10-08T13:44:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T13:44:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:40.651 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 392116 Links: 1 2023-10-08T13:44:40.651 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:40.651 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:40.651 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-08 13:44:23.430236051 +0000 2023-10-08T13:44:40.651 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-08 13:37:49.382110974 +0000 2023-10-08T13:44:40.651 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-08 13:37:49.382110974 +0000 2023-10-08T13:44:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2023-10-08T13:44:40.651 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T13:44:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-08T13:44:40.717 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-08T13:44:40.717 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000213245 s, 2.4 MB/s 2023-10-08T13:44:40.718 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T13:44:40.776 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-10-08T13:44:40.776 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T13:44:40.797 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T13:44:40.797 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2023-10-08T13:44:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T13:44:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:40.858 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 386891 Links: 1 2023-10-08T13:44:40.858 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:40.858 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:40.858 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-10-08 13:44:21.164380094 +0000 2023-10-08T13:44:40.859 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-10-08 13:37:49.689573220 +0000 2023-10-08T13:44:40.859 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-10-08 13:37:49.689573220 +0000 2023-10-08T13:44:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2023-10-08T13:44:40.859 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T13:44:40.932 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-10-08T13:44:40.932 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-10-08T13:44:40.932 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.00026858 s, 1.9 MB/s 2023-10-08T13:44:40.933 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T13:44:40.994 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2023-10-08T13:44:41.053 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T13:44:41.054 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:41.054 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 390451 Links: 1 2023-10-08T13:44:41.054 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:41.054 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:41.054 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-10-08 13:44:21.164380094 +0000 2023-10-08T13:44:41.054 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-10-08 13:37:50.183564010 +0000 2023-10-08T13:44:41.054 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-10-08 13:37:50.183564010 +0000 2023-10-08T13:44:41.054 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2023-10-08T13:44:41.055 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T13:44:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-10-08T13:44:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-10-08T13:44:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000221943 s, 2.3 MB/s 2023-10-08T13:44:41.123 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T13:44:41.181 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2023-10-08T13:44:41.239 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T13:44:41.239 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:41.240 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 395271 Links: 1 2023-10-08T13:44:41.240 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:41.240 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:41.240 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-10-08 13:44:21.164380094 +0000 2023-10-08T13:44:41.240 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-10-08 13:37:50.689554577 +0000 2023-10-08T13:44:41.240 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-10-08 13:37:50.689554577 +0000 2023-10-08T13:44:41.240 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2023-10-08T13:44:41.240 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T13:44:41.307 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-10-08T13:44:41.307 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-10-08T13:44:41.307 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000245981 s, 2.1 MB/s 2023-10-08T13:44:41.308 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T13:44:41.367 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2023-10-08T13:44:41.425 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T13:44:41.426 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T13:44:41.426 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 393718 Links: 1 2023-10-08T13:44:41.426 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:44:41.426 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T13:44:41.426 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-10-08 13:44:21.164380094 +0000 2023-10-08T13:44:41.427 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-10-08 13:37:51.197545107 +0000 2023-10-08T13:44:41.427 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-10-08 13:37:51.197545107 +0000 2023-10-08T13:44:41.427 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2023-10-08T13:44:41.427 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T13:44:41.491 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-10-08T13:44:41.491 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-10-08T13:44:41.492 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000198346 s, 2.6 MB/s 2023-10-08T13:44:41.493 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T13:44:41.551 INFO:tasks.cephadm:Deploying osd.0 on smithi057 with /dev/vg_nvme/lv_4... 2023-10-08T13:44:41.551 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-08T13:44:42.131 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:41 smithi115 ceph-mon[104408]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:41 smithi057 ceph-mon[99476]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:42.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:41 smithi099 ceph-mon[103849]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:43.653 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-08T13:44:43.654 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-08T13:44:43.654 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-08T13:44:43.654 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-08T13:44:43.654 INFO:teuthology.orchestra.run.smithi057.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115343 s, 909 MB/s 2023-10-08T13:44:43.654 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-08T13:44:43.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:43 smithi057 ceph-mon[99476]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:44.084 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_4 2023-10-08T13:44:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:43 smithi115 ceph-mon[104408]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:44.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:43 smithi099 ceph-mon[103849]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:44.651 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:44:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.888+0000 7f30791be700 1 -- 172.21.15.57:0/2543985216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30741000a0 msgr2=0x7f307410c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.888+0000 7f30791be700 1 --2- 172.21.15.57:0/2543985216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30741000a0 0x7f307410c540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f306c00b0a0 tx=0x7f306c00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:44:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.889+0000 7f30791be700 1 -- 172.21.15.57:0/2543985216 shutdown_connections 2023-10-08T13:44:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.889+0000 7f30791be700 1 --2- 172.21.15.57:0/2543985216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30741000a0 0x7f307410c540 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.889+0000 7f30791be700 1 --2- 172.21.15.57:0/2543985216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30741045b0 0x7f3074104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.889+0000 7f30791be700 1 --2- 172.21.15.57:0/2543985216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30740ff6e0 0x7f30740ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.889+0000 7f30791be700 1 -- 172.21.15.57:0/2543985216 >> 172.21.15.57:0/2543985216 conn(0x7f30740766c0 msgr2=0x7f3074076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:44:45.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.889+0000 7f30791be700 1 -- 172.21.15.57:0/2543985216 shutdown_connections 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.889+0000 7f30791be700 1 -- 172.21.15.57:0/2543985216 wait complete. 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.889+0000 7f30791be700 1 Processor -- start 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f30791be700 1 -- start start 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f30791be700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30740ff6e0 0x7f3074197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f30791be700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30741000a0 0x7f3074198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f30791be700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30741045b0 0x7f30741a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f30791be700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3074198940 con 0x7f30741000a0 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f30791be700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30741a30d0 con 0x7f30741045b0 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f30791be700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30741a3240 con 0x7f30740ff6e0 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f307259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30741000a0 0x7f3074198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f307259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30741000a0 0x7f3074198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53048/0 (socket says 172.21.15.57:53048) 2023-10-08T13:44:45.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f307259c700 1 -- 172.21.15.57:0/1038142254 learned_addr learned my addr 172.21.15.57:0/1038142254 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f3072d9d700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30740ff6e0 0x7f3074197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f307359e700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30741045b0 0x7f30741a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f307259c700 1 -- 172.21.15.57:0/1038142254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30740ff6e0 msgr2=0x7f3074197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f307259c700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30740ff6e0 0x7f3074197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f307259c700 1 -- 172.21.15.57:0/1038142254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30741045b0 msgr2=0x7f30741a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f307259c700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30741045b0 0x7f30741a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f307259c700 1 -- 172.21.15.57:0/1038142254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c009d00 con 0x7f30741000a0 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.890+0000 7f307259c700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30741000a0 0x7f3074198050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f306800d950 tx=0x7f306800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.891+0000 7f305bfff700 1 -- 172.21.15.57:0/1038142254 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068009940 con 0x7f30741000a0 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.891+0000 7f305bfff700 1 -- 172.21.15.57:0/1038142254 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3068010460 con 0x7f30741000a0 2023-10-08T13:44:45.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.891+0000 7f30791be700 1 -- 172.21.15.57:0/1038142254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30741a34c0 con 0x7f30741000a0 2023-10-08T13:44:45.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.891+0000 7f305bfff700 1 -- 172.21.15.57:0/1038142254 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800f620 con 0x7f30741000a0 2023-10-08T13:44:45.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.891+0000 7f30791be700 1 -- 172.21.15.57:0/1038142254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30741a3a10 con 0x7f30741000a0 2023-10-08T13:44:45.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.892+0000 7f30791be700 1 -- 172.21.15.57:0/1038142254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3074107c20 con 0x7f30741000a0 2023-10-08T13:44:45.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.893+0000 7f305bfff700 1 -- 172.21.15.57:0/1038142254 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f30680108d0 con 0x7f30741000a0 2023-10-08T13:44:45.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.894+0000 7f305bfff700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f305c0a14c0 0x7f305c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:44:45.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.894+0000 7f3072d9d700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f305c0a14c0 0x7f305c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:44:45.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.894+0000 7f305bfff700 1 -- 172.21.15.57:0/1038142254 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f30680caef0 con 0x7f30741000a0 2023-10-08T13:44:45.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.894+0000 7f3072d9d700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f305c0a14c0 0x7f305c0a3980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f306400abd0 tx=0x7f306400a380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:44:45.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:45.897+0000 7f305bfff700 1 -- 172.21.15.57:0/1038142254 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f30680f8100 con 0x7f30741000a0 2023-10-08T13:44:45.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:45 smithi057 ceph-mon[99476]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:46.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:44:46.106+0000 7f30791be700 1 -- 172.21.15.57:0/1038142254 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3074068f90 con 0x7f305c0a14c0 2023-10-08T13:44:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:45 smithi115 ceph-mon[104408]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:45 smithi099 ceph-mon[103849]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:47.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:46 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:44:47.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:46 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:44:47.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:46 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:47.227 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:46 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:44:47.227 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:46 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:44:47.227 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:46 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:47.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:46 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:44:47.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:46 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:44:47.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:46 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:48.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:47 smithi115 ceph-mon[104408]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:48.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:47 smithi115 ceph-mon[104408]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:44:48.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:47 smithi057 ceph-mon[99476]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:48.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:47 smithi057 ceph-mon[99476]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:44:48.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:47 smithi099 ceph-mon[103849]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:48.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:47 smithi099 ceph-mon[103849]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:44:49.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:48 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1843575278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82271e50-2757-4c42-a3ca-a105fbaa1391"}]: dispatch 2023-10-08T13:44:49.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:48 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1843575278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82271e50-2757-4c42-a3ca-a105fbaa1391"}]': finished 2023-10-08T13:44:49.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:48 smithi115 ceph-mon[104408]: osdmap e6: 1 total, 0 up, 1 in 2023-10-08T13:44:49.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:44:49.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:48 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1843575278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82271e50-2757-4c42-a3ca-a105fbaa1391"}]: dispatch 2023-10-08T13:44:49.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:48 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1843575278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82271e50-2757-4c42-a3ca-a105fbaa1391"}]': finished 2023-10-08T13:44:49.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:48 smithi057 ceph-mon[99476]: osdmap e6: 1 total, 0 up, 1 in 2023-10-08T13:44:49.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:44:49.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:48 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1843575278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82271e50-2757-4c42-a3ca-a105fbaa1391"}]: dispatch 2023-10-08T13:44:49.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:48 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1843575278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82271e50-2757-4c42-a3ca-a105fbaa1391"}]': finished 2023-10-08T13:44:49.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:48 smithi099 ceph-mon[103849]: osdmap e6: 1 total, 0 up, 1 in 2023-10-08T13:44:49.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:44:50.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:49 smithi115 ceph-mon[104408]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:50.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:49 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3039511433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:44:50.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:49 smithi057 ceph-mon[99476]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:50.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:49 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3039511433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:44:50.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:49 smithi099 ceph-mon[103849]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:49 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3039511433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:44:52.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:51 smithi115 ceph-mon[104408]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:52.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:51 smithi057 ceph-mon[99476]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:52.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:51 smithi099 ceph-mon[103849]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:53.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:53 smithi057 ceph-mon[99476]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:54.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:53 smithi115 ceph-mon[104408]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:53 smithi099 ceph-mon[103849]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:55.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:55 smithi057 ceph-mon[99476]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:55.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-08T13:44:55.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:56.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:55 smithi115 ceph-mon[104408]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:56.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-08T13:44:56.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:55 smithi099 ceph-mon[103849]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-08T13:44:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:44:57.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:56 smithi115 ceph-mon[104408]: Deploying daemon osd.0 on smithi057 2023-10-08T13:44:57.208 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:56 smithi057 ceph-mon[99476]: Deploying daemon osd.0 on smithi057 2023-10-08T13:44:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:56 smithi099 ceph-mon[103849]: Deploying daemon osd.0 on smithi057 2023-10-08T13:44:58.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:57 smithi115 ceph-mon[104408]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:58.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:57 smithi057 ceph-mon[99476]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:44:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:57 smithi099 ceph-mon[103849]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:00.183 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:59 smithi057 ceph-mon[99476]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:00.183 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:59 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:00.183 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:59 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:00.183 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:44:59 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:00.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:59 smithi115 ceph-mon[104408]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:00.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:59 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:00.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:59 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:00.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:44:59 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:00.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:59 smithi099 ceph-mon[103849]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:00.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:59 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:00.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:59 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:00.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:44:59 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:01.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.958+0000 7f305bfff700 1 -- 172.21.15.57:0/1038142254 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3074068f90 con 0x7f305c0a14c0 2023-10-08T13:45:01.959 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 0 on host 'smithi057' 2023-10-08T13:45:01.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 -- 172.21.15.57:0/1038142254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f305c0a14c0 msgr2=0x7f305c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:01.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f305c0a14c0 0x7f305c0a3980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f306400abd0 tx=0x7f306400a380 comp rx=0 tx=0).stop 2023-10-08T13:45:01.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 -- 172.21.15.57:0/1038142254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30741000a0 msgr2=0x7f3074198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:01.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30741000a0 0x7f3074198050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f306800d950 tx=0x7f306800dc60 comp rx=0 tx=0).stop 2023-10-08T13:45:01.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 -- 172.21.15.57:0/1038142254 shutdown_connections 2023-10-08T13:45:01.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f305c0a14c0 0x7f305c0a3980 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:01.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30741045b0 0x7f30741a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:01.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30740ff6e0 0x7f3074197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:01.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 --2- 172.21.15.57:0/1038142254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f30741000a0 0x7f3074198050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 -- 172.21.15.57:0/1038142254 >> 172.21.15.57:0/1038142254 conn(0x7f30740766c0 msgr2=0x7f30740fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:45:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 -- 172.21.15.57:0/1038142254 shutdown_connections 2023-10-08T13:45:01.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:01.960+0000 7f30791be700 1 -- 172.21.15.57:0/1038142254 wait complete. 2023-10-08T13:45:02.133 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:01 smithi057 ceph-mon[99476]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:02.133 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:01 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:02.133 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:01 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:02.134 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:01 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:02.134 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:01 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:02.134 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:01 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:02.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:01 smithi115 ceph-mon[104408]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:02.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:01 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:02.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:01 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:02.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:01 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:02.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:01 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:02.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:01 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:01 smithi099 ceph-mon[103849]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:01 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:01 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:01 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:01 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:01 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:02.442 DEBUG:teuthology.orchestra.run.smithi057:osd.0> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.0.service 2023-10-08T13:45:02.444 INFO:tasks.cephadm:Deploying osd.1 on smithi099 with /dev/vg_nvme/lv_4... 2023-10-08T13:45:02.445 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-08T13:45:02.474 INFO:journalctl@ceph.osd.0.smithi057.stdout:-- Logs begin at Sun 2023-10-08 13:28:06 UTC. -- 2023-10-08T13:45:03.010 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T13:45:03.212 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:03.213 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:03.213 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:03.253 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:03.253 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:03.253 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:03.487 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:03 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-0[124884]: 2023-10-08T13:45:03.336+0000 7f4f65d32640 -1 osd.0 0 log_to_monitors true 2023-10-08T13:45:04.235 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:03 smithi099 ceph-mon[103849]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:04.235 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:03 smithi099 ceph-mon[103849]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-08T13:45:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:03 smithi057 ceph-mon[99476]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:03 smithi057 ceph-mon[99476]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-08T13:45:04.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:03 smithi115 ceph-mon[104408]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:04.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:03 smithi115 ceph-mon[104408]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-08T13:45:04.671 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-08T13:45:04.671 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-08T13:45:04.671 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-10-08T13:45:04.671 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-10-08T13:45:04.671 INFO:teuthology.orchestra.run.smithi099.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109363 s, 959 MB/s 2023-10-08T13:45:04.671 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-10-08T13:45:05.155 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2023-10-08T13:45:05.237 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:04 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-0[124884]: 2023-10-08T13:45:04.975+0000 7f4f5b3a9700 -1 osd.0 0 waiting for initial osdmap 2023-10-08T13:45:05.237 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:04 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-0[124884]: 2023-10-08T13:45:04.979+0000 7f4f5599b700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:45:05.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:04 smithi057 ceph-mon[99476]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-08T13:45:05.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:04 smithi057 ceph-mon[99476]: osdmap e7: 1 total, 0 up, 1 in 2023-10-08T13:45:05.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:04 smithi057 ceph-mon[99476]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:45:05.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:04 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:05.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:04 smithi057 ceph-mon[99476]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:05.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:04 smithi099 ceph-mon[103849]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-08T13:45:05.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:04 smithi099 ceph-mon[103849]: osdmap e7: 1 total, 0 up, 1 in 2023-10-08T13:45:05.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:04 smithi099 ceph-mon[103849]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:45:05.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:04 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:05.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:04 smithi099 ceph-mon[103849]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:05.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:04 smithi115 ceph-mon[104408]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-08T13:45:05.333 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:04 smithi115 ceph-mon[104408]: osdmap e7: 1 total, 0 up, 1 in 2023-10-08T13:45:05.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:04 smithi115 ceph-mon[104408]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:45:05.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:04 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:05.334 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:04 smithi115 ceph-mon[104408]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:05.707 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T13:45:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:45:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: osdmap e8: 1 total, 0 up, 1 in 2023-10-08T13:45:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi057 2023-10-08T13:45:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:45:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi057 to 17173M 2023-10-08T13:45:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:05 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:06.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:45:06.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: osdmap e8: 1 total, 0 up, 1 in 2023-10-08T13:45:06.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:06.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:06.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi057 2023-10-08T13:45:06.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:45:06.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi057 to 17173M 2023-10-08T13:45:06.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:06.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:06.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:05 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:06.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:45:06.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: osdmap e8: 1 total, 0 up, 1 in 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi057 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi057 to 17173M 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:06.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:06.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:05 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:07.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.069+0000 7fc7758aa700 1 -- 172.21.15.99:0/3295991507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7700ff6e0 msgr2=0x7fc7700ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:07.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.069+0000 7fc7758aa700 1 --2- 172.21.15.99:0/3295991507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7700ff6e0 0x7fc7700ffb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc764009990 tx=0x7fc764009ca0 comp rx=0 tx=0).stop 2023-10-08T13:45:07.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.069+0000 7fc7758aa700 1 -- 172.21.15.99:0/3295991507 shutdown_connections 2023-10-08T13:45:07.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.069+0000 7fc7758aa700 1 --2- 172.21.15.99:0/3295991507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7701000a0 0x7fc77010c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:07.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.069+0000 7fc7758aa700 1 --2- 172.21.15.99:0/3295991507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc770105690 0x7fc770105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:07.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.069+0000 7fc7758aa700 1 --2- 172.21.15.99:0/3295991507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7700ff6e0 0x7fc7700ffb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:07.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.069+0000 7fc7758aa700 1 -- 172.21.15.99:0/3295991507 >> 172.21.15.99:0/3295991507 conn(0x7fc7700766c0 msgr2=0x7fc770076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:45:07.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.069+0000 7fc7758aa700 1 -- 172.21.15.99:0/3295991507 shutdown_connections 2023-10-08T13:45:07.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.069+0000 7fc7758aa700 1 -- 172.21.15.99:0/3295991507 wait complete. 2023-10-08T13:45:07.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.070+0000 7fc7758aa700 1 Processor -- start 2023-10-08T13:45:07.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.070+0000 7fc7758aa700 1 -- start start 2023-10-08T13:45:07.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.070+0000 7fc7758aa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7700ff6e0 0x7fc77019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:07.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.070+0000 7fc7758aa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7701000a0 0x7fc77019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:07.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.070+0000 7fc7758aa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc770105690 0x7fc770196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:07.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.070+0000 7fc7758aa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7701970c0 con 0x7fc7701000a0 2023-10-08T13:45:07.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.070+0000 7fc7758aa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc770197230 con 0x7fc7700ff6e0 2023-10-08T13:45:07.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.070+0000 7fc7758aa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7701973f0 con 0x7fc770105690 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7700ff6e0 0x7fc77019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7700ff6e0 0x7fc77019c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42462/0 (socket says 172.21.15.99:42462) 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76effd700 1 -- 172.21.15.99:0/3240859829 learned_addr learned my addr 172.21.15.99:0/3240859829 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76f7fe700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc770105690 0x7fc770196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76e7fc700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7701000a0 0x7fc77019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76effd700 1 -- 172.21.15.99:0/3240859829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc770105690 msgr2=0x7fc770196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76effd700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc770105690 0x7fc770196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76effd700 1 -- 172.21.15.99:0/3240859829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7701000a0 msgr2=0x7fc77019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76effd700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7701000a0 0x7fc77019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76effd700 1 -- 172.21.15.99:0/3240859829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7640095f0 con 0x7fc7700ff6e0 2023-10-08T13:45:07.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76f7fe700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc770105690 0x7fc770196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:45:07.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc76effd700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7700ff6e0 0x7fc77019c920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc76000d950 tx=0x7fc76000dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:45:07.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc7748a8700 1 -- 172.21.15.99:0/3240859829 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760009b40 con 0x7fc7700ff6e0 2023-10-08T13:45:07.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc7748a8700 1 -- 172.21.15.99:0/3240859829 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc760009ca0 con 0x7fc7700ff6e0 2023-10-08T13:45:07.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc7748a8700 1 -- 172.21.15.99:0/3240859829 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76000f620 con 0x7fc7700ff6e0 2023-10-08T13:45:07.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc7758aa700 1 -- 172.21.15.99:0/3240859829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7701976d0 con 0x7fc7700ff6e0 2023-10-08T13:45:07.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.071+0000 7fc7758aa700 1 -- 172.21.15.99:0/3240859829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7701a3910 con 0x7fc7700ff6e0 2023-10-08T13:45:07.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.072+0000 7fc7758aa700 1 -- 172.21.15.99:0/3240859829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc770108cc0 con 0x7fc7700ff6e0 2023-10-08T13:45:07.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.073+0000 7fc7748a8700 1 -- 172.21.15.99:0/3240859829 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fc760010460 con 0x7fc7700ff6e0 2023-10-08T13:45:07.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.074+0000 7fc7748a8700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7580a14c0 0x7fc7580a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:07.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.074+0000 7fc7748a8700 1 -- 172.21.15.99:0/3240859829 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7fc7600cb5f0 con 0x7fc7700ff6e0 2023-10-08T13:45:07.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.074+0000 7fc76e7fc700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7580a14c0 0x7fc7580a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:07.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.075+0000 7fc76e7fc700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7580a14c0 0x7fc7580a3980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc764005850 tx=0x7fc76401a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:45:07.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.078+0000 7fc7748a8700 1 -- 172.21.15.99:0/3240859829 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc7600965f0 con 0x7fc7700ff6e0 2023-10-08T13:45:07.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:06 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:45:07.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:06 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:45:07.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:06 smithi099 ceph-mon[103849]: osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] boot 2023-10-08T13:45:07.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:06 smithi099 ceph-mon[103849]: osdmap e9: 1 total, 1 up, 1 in 2023-10-08T13:45:07.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:06 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:07.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:06 smithi099 ceph-mon[103849]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:07.273 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:07.272+0000 7fc7758aa700 1 -- 172.21.15.99:0/3240859829 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc770197f20 con 0x7fc7580a14c0 2023-10-08T13:45:07.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:06 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:45:07.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:06 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:45:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:06 smithi115 ceph-mon[104408]: osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] boot 2023-10-08T13:45:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:06 smithi115 ceph-mon[104408]: osdmap e9: 1 total, 1 up, 1 in 2023-10-08T13:45:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:06 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:06 smithi115 ceph-mon[104408]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:07.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:06 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:45:07.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:06 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:45:07.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:06 smithi057 ceph-mon[99476]: osd.0 [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] boot 2023-10-08T13:45:07.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:06 smithi057 ceph-mon[99476]: osdmap e9: 1 total, 1 up, 1 in 2023-10-08T13:45:07.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:06 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T13:45:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:06 smithi057 ceph-mon[99476]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T13:45:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:07 smithi099 ceph-mon[103849]: osdmap e10: 1 total, 1 up, 1 in 2023-10-08T13:45:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:07 smithi099 ceph-mon[103849]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:45:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:45:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:45:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:07 smithi115 ceph-mon[104408]: osdmap e10: 1 total, 1 up, 1 in 2023-10-08T13:45:08.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:07 smithi115 ceph-mon[104408]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:45:08.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:45:08.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:45:08.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:08.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:07 smithi057 ceph-mon[99476]: osdmap e10: 1 total, 1 up, 1 in 2023-10-08T13:45:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:07 smithi057 ceph-mon[99476]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:45:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:45:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:45:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:09.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:09 smithi099 ceph-mon[103849]: pgmap v25: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:09.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:09 smithi115 ceph-mon[104408]: pgmap v25: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:09.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:09 smithi057 ceph-mon[99476]: pgmap v25: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:10 smithi099 ceph-mon[103849]: from='client.? 172.21.15.99:0/3547049408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef769aca-5124-4f13-931f-2520cbbe2144"}]: dispatch 2023-10-08T13:45:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:10 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef769aca-5124-4f13-931f-2520cbbe2144"}]: dispatch 2023-10-08T13:45:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:10 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef769aca-5124-4f13-931f-2520cbbe2144"}]': finished 2023-10-08T13:45:10.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:10 smithi099 ceph-mon[103849]: osdmap e11: 2 total, 1 up, 2 in 2023-10-08T13:45:10.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:10 smithi115 ceph-mon[104408]: from='client.? 172.21.15.99:0/3547049408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef769aca-5124-4f13-931f-2520cbbe2144"}]: dispatch 2023-10-08T13:45:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:10 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef769aca-5124-4f13-931f-2520cbbe2144"}]: dispatch 2023-10-08T13:45:10.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:10 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef769aca-5124-4f13-931f-2520cbbe2144"}]': finished 2023-10-08T13:45:10.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:10 smithi115 ceph-mon[104408]: osdmap e11: 2 total, 1 up, 2 in 2023-10-08T13:45:10.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:10.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:10 smithi057 ceph-mon[99476]: from='client.? 172.21.15.99:0/3547049408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef769aca-5124-4f13-931f-2520cbbe2144"}]: dispatch 2023-10-08T13:45:10.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:10 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef769aca-5124-4f13-931f-2520cbbe2144"}]: dispatch 2023-10-08T13:45:10.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:10 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef769aca-5124-4f13-931f-2520cbbe2144"}]': finished 2023-10-08T13:45:10.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:10 smithi057 ceph-mon[99476]: osdmap e11: 2 total, 1 up, 2 in 2023-10-08T13:45:10.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:11.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:11 smithi099 ceph-mon[103849]: pgmap v27: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:11.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:11 smithi099 ceph-mon[103849]: from='client.? 172.21.15.99:0/2404140941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:45:11.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:11 smithi115 ceph-mon[104408]: pgmap v27: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:11.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:11 smithi115 ceph-mon[104408]: from='client.? 172.21.15.99:0/2404140941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:45:11.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:11 smithi057 ceph-mon[99476]: pgmap v27: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:11.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:11 smithi057 ceph-mon[99476]: from='client.? 172.21.15.99:0/2404140941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:45:13.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:13 smithi099 ceph-mon[103849]: pgmap v28: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:13.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:13 smithi115 ceph-mon[104408]: pgmap v28: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:13.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:13 smithi057 ceph-mon[99476]: pgmap v28: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:15.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:15 smithi115 ceph-mon[104408]: pgmap v29: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:15.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:15 smithi057 ceph-mon[99476]: pgmap v29: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:16.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:15 smithi099 ceph-mon[103849]: pgmap v29: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:16.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-08T13:45:16.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:16.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-08T13:45:16.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:16.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-08T13:45:16.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:17.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:17 smithi115 ceph-mon[104408]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:17.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:17 smithi115 ceph-mon[104408]: Deploying daemon osd.1 on smithi099 2023-10-08T13:45:17.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:17 smithi057 ceph-mon[99476]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:17.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:17 smithi057 ceph-mon[99476]: Deploying daemon osd.1 on smithi099 2023-10-08T13:45:18.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:17 smithi099 ceph-mon[103849]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:18.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:17 smithi099 ceph-mon[103849]: Deploying daemon osd.1 on smithi099 2023-10-08T13:45:19.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:19 smithi115 ceph-mon[104408]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:19.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:19 smithi057 ceph-mon[99476]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:20.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:19 smithi099 ceph-mon[103849]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:21.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:21 smithi115 ceph-mon[104408]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:21.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:21.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:21.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:21.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:21 smithi057 ceph-mon[99476]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:21.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:21.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:21.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:22.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:21 smithi099 ceph-mon[103849]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:22.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:22.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:22.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:23.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:23 smithi115 ceph-mon[104408]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:23.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:23 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:23.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:23 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:23.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:23 smithi057 ceph-mon[99476]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:23.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:23 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:23.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:23 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:23 smithi099 ceph-mon[103849]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:23 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:23 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.144+0000 7fc7748a8700 1 -- 172.21.15.99:0/3240859829 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc770197f20 con 0x7fc7580a14c0 2023-10-08T13:45:24.146 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 1 on host 'smithi099' 2023-10-08T13:45:24.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.146+0000 7fc7758aa700 1 -- 172.21.15.99:0/3240859829 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7580a14c0 msgr2=0x7fc7580a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:24.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.146+0000 7fc7758aa700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7580a14c0 0x7fc7580a3980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc764005850 tx=0x7fc76401a040 comp rx=0 tx=0).stop 2023-10-08T13:45:24.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.146+0000 7fc7758aa700 1 -- 172.21.15.99:0/3240859829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7700ff6e0 msgr2=0x7fc77019c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:24.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.146+0000 7fc7758aa700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7700ff6e0 0x7fc77019c920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc76000d950 tx=0x7fc76000dd10 comp rx=0 tx=0).stop 2023-10-08T13:45:24.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.146+0000 7fc7758aa700 1 -- 172.21.15.99:0/3240859829 shutdown_connections 2023-10-08T13:45:24.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.146+0000 7fc7758aa700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc770105690 0x7fc770196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:24.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.146+0000 7fc7758aa700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7580a14c0 0x7fc7580a3980 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:24.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.146+0000 7fc7758aa700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7700ff6e0 0x7fc77019c920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:24.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.146+0000 7fc7758aa700 1 --2- 172.21.15.99:0/3240859829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7701000a0 0x7fc77019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:24.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.146+0000 7fc7758aa700 1 -- 172.21.15.99:0/3240859829 >> 172.21.15.99:0/3240859829 conn(0x7fc7700766c0 msgr2=0x7fc770069260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:45:24.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.147+0000 7fc7758aa700 1 -- 172.21.15.99:0/3240859829 shutdown_connections 2023-10-08T13:45:24.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:45:24.147+0000 7fc7758aa700 1 -- 172.21.15.99:0/3240859829 wait complete. 2023-10-08T13:45:24.603 DEBUG:teuthology.orchestra.run.smithi099:osd.1> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.1.service 2023-10-08T13:45:24.605 INFO:tasks.cephadm:Deploying osd.2 on smithi115 with /dev/vg_nvme/lv_4... 2023-10-08T13:45:24.606 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-08T13:45:24.634 INFO:journalctl@ceph.osd.1.smithi099.stdout:-- Logs begin at Sun 2023-10-08 13:27:56 UTC. -- 2023-10-08T13:45:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:45:25.384 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:25 smithi115 ceph-mon[104408]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:25.384 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:25.384 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:25.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:25 smithi057 ceph-mon[99476]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:25.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:25.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:25.508 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-1[110129]: 2023-10-08T13:45:25.255+0000 7f6f588c7640 -1 osd.1 0 log_to_monitors true 2023-10-08T13:45:25.508 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:25 smithi099 ceph-mon[103849]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:25.508 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:25.508 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:26.222 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:26 smithi115 ceph-mon[104408]: from='osd.1 [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T13:45:26.222 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:26 smithi115 ceph-mon[104408]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T13:45:26.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:26 smithi057 ceph-mon[99476]: from='osd.1 [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T13:45:26.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:26 smithi057 ceph-mon[99476]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T13:45:26.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:26 smithi099 ceph-mon[103849]: from='osd.1 [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T13:45:26.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:26 smithi099 ceph-mon[103849]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T13:45:26.672 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-08T13:45:26.672 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-08T13:45:26.672 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2023-10-08T13:45:26.672 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2023-10-08T13:45:26.672 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116875 s, 897 MB/s 2023-10-08T13:45:26.672 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2023-10-08T13:45:27.115 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2023-10-08T13:45:27.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:27 smithi115 ceph-mon[104408]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:27.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:27 smithi115 ceph-mon[104408]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-08T13:45:27.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:27 smithi115 ceph-mon[104408]: osdmap e12: 2 total, 1 up, 2 in 2023-10-08T13:45:27.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:27.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:27 smithi115 ceph-mon[104408]: from='osd.1 [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:45:27.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:27 smithi115 ceph-mon[104408]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:45:27.463 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-1[110129]: 2023-10-08T13:45:27.138+0000 7f6f4df3e700 -1 osd.1 0 waiting for initial osdmap 2023-10-08T13:45:27.463 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-1[110129]: 2023-10-08T13:45:27.145+0000 7f6f49d33700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:45:27.463 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:27 smithi099 ceph-mon[103849]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:27.463 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:27 smithi099 ceph-mon[103849]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-08T13:45:27.464 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:27 smithi099 ceph-mon[103849]: osdmap e12: 2 total, 1 up, 2 in 2023-10-08T13:45:27.464 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:27.464 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:27 smithi099 ceph-mon[103849]: from='osd.1 [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:45:27.464 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:27 smithi099 ceph-mon[103849]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:45:27.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:27 smithi057 ceph-mon[99476]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:27.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:27 smithi057 ceph-mon[99476]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-08T13:45:27.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:27 smithi057 ceph-mon[99476]: osdmap e12: 2 total, 1 up, 2 in 2023-10-08T13:45:27.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:27.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:27 smithi057 ceph-mon[99476]: from='osd.1 [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:45:27.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:27 smithi057 ceph-mon[99476]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:45:27.673 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:45:28.402 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:45:28.402 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: osdmap e13: 2 total, 1 up, 2 in 2023-10-08T13:45:28.402 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi099 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi099 to 17168M 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.403 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:28.404 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:28.404 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:28.404 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.404 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:28.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:45:28.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: osdmap e13: 2 total, 1 up, 2 in 2023-10-08T13:45:28.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:28.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:28.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi099 2023-10-08T13:45:28.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:45:28.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi099 to 17168M 2023-10-08T13:45:28.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:28.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:28.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:28.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:28.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:28.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:28.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:45:28.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: osdmap e13: 2 total, 1 up, 2 in 2023-10-08T13:45:28.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:28.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:28.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi099 2023-10-08T13:45:28.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi099 to 17168M 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:28.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.866+0000 7f86ec9b1700 1 -- 172.21.15.115:0/3871077567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86e81002a0 msgr2=0x7f86e8100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:28.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.866+0000 7f86ec9b1700 1 --2- 172.21.15.115:0/3871077567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86e81002a0 0x7f86e8100720 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009990 tx=0x7f86d8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:45:28.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.866+0000 7f86ec9b1700 1 -- 172.21.15.115:0/3871077567 shutdown_connections 2023-10-08T13:45:28.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.866+0000 7f86ec9b1700 1 --2- 172.21.15.115:0/3871077567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e8100c60 0x7f86e810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.866+0000 7f86ec9b1700 1 --2- 172.21.15.115:0/3871077567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86e8104720 0x7f86e8104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.866+0000 7f86ec9b1700 1 --2- 172.21.15.115:0/3871077567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86e81002a0 0x7f86e8100720 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.866+0000 7f86ec9b1700 1 -- 172.21.15.115:0/3871077567 >> 172.21.15.115:0/3871077567 conn(0x7f86e8074a10 msgr2=0x7f86e8074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.866+0000 7f86ec9b1700 1 -- 172.21.15.115:0/3871077567 shutdown_connections 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.866+0000 7f86ec9b1700 1 -- 172.21.15.115:0/3871077567 wait complete. 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86ec9b1700 1 Processor -- start 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86ec9b1700 1 -- start start 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86ec9b1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86e81002a0 0x7f86e8071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86ec9b1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e8100c60 0x7f86e8072380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86ec9b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86e8104720 0x7f86e806afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:28.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86ec9b1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e806b780 con 0x7f86e81002a0 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86ec9b1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e806b8f0 con 0x7f86e8100c60 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86ec9b1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e806bab0 con 0x7f86e8104720 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e6d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86e8104720 0x7f86e806afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e6d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86e8104720 0x7f86e806afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52504/0 (socket says 172.21.15.115:52504) 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e6d9d700 1 -- 172.21.15.115:0/1687072942 learned_addr learned my addr 172.21.15.115:0/1687072942 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e5d9b700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e8100c60 0x7f86e8072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e659c700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86e81002a0 0x7f86e8071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e6d9d700 1 -- 172.21.15.115:0/1687072942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e8100c60 msgr2=0x7f86e8072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e6d9d700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e8100c60 0x7f86e8072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e6d9d700 1 -- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86e81002a0 msgr2=0x7f86e8071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e6d9d700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86e81002a0 0x7f86e8071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:28.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e6d9d700 1 -- 172.21.15.115:0/1687072942 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80095f0 con 0x7f86e8104720 2023-10-08T13:45:28.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.867+0000 7f86e659c700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86e81002a0 0x7f86e8071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:45:28.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.868+0000 7f86e6d9d700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86e8104720 0x7f86e806afb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f86e0007ee0 tx=0x7f86e000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:45:28.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.868+0000 7f86d77fe700 1 -- 172.21.15.115:0/1687072942 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0017070 con 0x7f86e8104720 2023-10-08T13:45:28.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.868+0000 7f86ec9b1700 1 -- 172.21.15.115:0/1687072942 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e806f9a0 con 0x7f86e8104720 2023-10-08T13:45:28.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.868+0000 7f86d77fe700 1 -- 172.21.15.115:0/1687072942 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e000f040 con 0x7f86e8104720 2023-10-08T13:45:28.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.868+0000 7f86d77fe700 1 -- 172.21.15.115:0/1687072942 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0005740 con 0x7f86e8104720 2023-10-08T13:45:28.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.868+0000 7f86ec9b1700 1 -- 172.21.15.115:0/1687072942 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e806fe20 con 0x7f86e8104720 2023-10-08T13:45:28.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.869+0000 7f86ec9b1700 1 -- 172.21.15.115:0/1687072942 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e8070810 con 0x7f86e8104720 2023-10-08T13:45:28.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.870+0000 7f86d77fe700 1 -- 172.21.15.115:0/1687072942 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f86e000dc60 con 0x7f86e8104720 2023-10-08T13:45:28.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.870+0000 7f86d77fe700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f86d00a1440 0x7f86d00a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:28.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.870+0000 7f86d77fe700 1 -- 172.21.15.115:0/1687072942 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2524+0+0 (secure 0 0 0) 0x7f86e00cadd0 con 0x7f86e8104720 2023-10-08T13:45:28.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.871+0000 7f86e659c700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f86d00a1440 0x7f86d00a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:28.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.871+0000 7f86e659c700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f86d00a1440 0x7f86d00a3900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005fd0 tx=0x7f86dc005db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:45:28.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:28.875+0000 7f86d77fe700 1 -- 172.21.15.115:0/1687072942 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f86e8070810 con 0x7f86e8104720 2023-10-08T13:45:29.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:29.070+0000 7f86ec9b1700 1 -- 172.21.15.115:0/1687072942 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f86e804e4c0 con 0x7f86d00a1440 2023-10-08T13:45:29.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:29 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:45:29.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:29 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:45:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:29 smithi115 ceph-mon[104408]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:29 smithi115 ceph-mon[104408]: osd.1 [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] boot 2023-10-08T13:45:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:29 smithi115 ceph-mon[104408]: osdmap e14: 2 total, 2 up, 2 in 2023-10-08T13:45:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:45:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:45:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:29.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:29 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:45:29.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:29 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:45:29.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:29 smithi057 ceph-mon[99476]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:29.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:29 smithi057 ceph-mon[99476]: osd.1 [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] boot 2023-10-08T13:45:29.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:29 smithi057 ceph-mon[99476]: osdmap e14: 2 total, 2 up, 2 in 2023-10-08T13:45:29.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:29.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:45:29.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:45:29.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:29 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:45:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:29 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:45:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:29 smithi099 ceph-mon[103849]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T13:45:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:29 smithi099 ceph-mon[103849]: osd.1 [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] boot 2023-10-08T13:45:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:29 smithi099 ceph-mon[103849]: osdmap e14: 2 total, 2 up, 2 in 2023-10-08T13:45:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T13:45:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:45:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:45:29.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:30.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:30 smithi115 ceph-mon[104408]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:45:30.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:30 smithi115 ceph-mon[104408]: osdmap e15: 2 total, 2 up, 2 in 2023-10-08T13:45:30.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:30 smithi057 ceph-mon[99476]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:45:30.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:30 smithi057 ceph-mon[99476]: osdmap e15: 2 total, 2 up, 2 in 2023-10-08T13:45:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:30 smithi099 ceph-mon[103849]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:45:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:30 smithi099 ceph-mon[103849]: osdmap e15: 2 total, 2 up, 2 in 2023-10-08T13:45:31.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:31 smithi115 ceph-mon[104408]: pgmap v41: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:31.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:31 smithi057 ceph-mon[99476]: pgmap v41: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:31.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:31 smithi099 ceph-mon[103849]: pgmap v41: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:32.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:32 smithi115 ceph-mon[104408]: from='client.? 172.21.15.115:0/789551161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c72122-dbc2-4a3b-9ee3-6c83f989107a"}]: dispatch 2023-10-08T13:45:32.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:32 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c72122-dbc2-4a3b-9ee3-6c83f989107a"}]: dispatch 2023-10-08T13:45:32.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:32 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8c72122-dbc2-4a3b-9ee3-6c83f989107a"}]': finished 2023-10-08T13:45:32.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:32 smithi115 ceph-mon[104408]: osdmap e16: 3 total, 2 up, 3 in 2023-10-08T13:45:32.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:32.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:32 smithi115 ceph-mon[104408]: from='client.? 172.21.15.115:0/3010728930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:45:32.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:32 smithi057 ceph-mon[99476]: from='client.? 172.21.15.115:0/789551161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c72122-dbc2-4a3b-9ee3-6c83f989107a"}]: dispatch 2023-10-08T13:45:32.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:32 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c72122-dbc2-4a3b-9ee3-6c83f989107a"}]: dispatch 2023-10-08T13:45:32.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:32 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8c72122-dbc2-4a3b-9ee3-6c83f989107a"}]': finished 2023-10-08T13:45:32.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:32 smithi057 ceph-mon[99476]: osdmap e16: 3 total, 2 up, 3 in 2023-10-08T13:45:32.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:32.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:32 smithi057 ceph-mon[99476]: from='client.? 172.21.15.115:0/3010728930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:45:32.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:32 smithi099 ceph-mon[103849]: from='client.? 172.21.15.115:0/789551161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c72122-dbc2-4a3b-9ee3-6c83f989107a"}]: dispatch 2023-10-08T13:45:32.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:32 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8c72122-dbc2-4a3b-9ee3-6c83f989107a"}]: dispatch 2023-10-08T13:45:32.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:32 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8c72122-dbc2-4a3b-9ee3-6c83f989107a"}]': finished 2023-10-08T13:45:32.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:32 smithi099 ceph-mon[103849]: osdmap e16: 3 total, 2 up, 3 in 2023-10-08T13:45:32.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:32.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:32 smithi099 ceph-mon[103849]: from='client.? 172.21.15.115:0/3010728930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:45:33.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:33 smithi115 ceph-mon[104408]: pgmap v43: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:33.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:33 smithi057 ceph-mon[99476]: pgmap v43: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:33 smithi099 ceph-mon[103849]: pgmap v43: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:35.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:35 smithi115 ceph-mon[104408]: pgmap v44: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:35.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:35 smithi057 ceph-mon[99476]: pgmap v44: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:36.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:35 smithi099 ceph-mon[103849]: pgmap v44: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:37.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:37 smithi115 ceph-mon[104408]: pgmap v45: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:37.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:37 smithi057 ceph-mon[99476]: pgmap v45: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:38.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:37 smithi099 ceph-mon[103849]: pgmap v45: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:38.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:38 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-08T13:45:38.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:38 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:38.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:38 smithi115 ceph-mon[104408]: Deploying daemon osd.2 on smithi115 2023-10-08T13:45:38.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:38 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-08T13:45:38.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:38 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:38.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:38 smithi057 ceph-mon[99476]: Deploying daemon osd.2 on smithi115 2023-10-08T13:45:39.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:38 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-08T13:45:39.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:38 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:39.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:38 smithi099 ceph-mon[103849]: Deploying daemon osd.2 on smithi115 2023-10-08T13:45:39.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:39 smithi115 ceph-mon[104408]: pgmap v46: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:39.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:39 smithi057 ceph-mon[99476]: pgmap v46: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:40.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:39 smithi099 ceph-mon[103849]: pgmap v46: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:41.947 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:41 smithi115 ceph-mon[104408]: pgmap v47: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:41 smithi057 ceph-mon[99476]: pgmap v47: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:41 smithi099 ceph-mon[103849]: pgmap v47: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:43.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:43 smithi115 ceph-mon[104408]: pgmap v48: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:43.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:43 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:43.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:43 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:43.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:43 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:43.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:43 smithi057 ceph-mon[99476]: pgmap v48: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:43.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:43 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:43.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:43 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:43.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:43 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:43.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:43 smithi099 ceph-mon[103849]: pgmap v48: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:43.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:43 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:43.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:43 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:43.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:43 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:45.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.085+0000 7f86d77fe700 1 -- 172.21.15.115:0/1687072942 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f86e804e4c0 con 0x7f86d00a1440 2023-10-08T13:45:45.087 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 2 on host 'smithi115' 2023-10-08T13:45:45.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 -- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f86d00a1440 msgr2=0x7f86d00a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:45.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f86d00a1440 0x7f86d00a3900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005fd0 tx=0x7f86dc005db0 comp rx=0 tx=0).stop 2023-10-08T13:45:45.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 -- 172.21.15.115:0/1687072942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86e8104720 msgr2=0x7f86e806afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:45.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86e8104720 0x7f86e806afb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f86e0007ee0 tx=0x7f86e000d3b0 comp rx=0 tx=0).stop 2023-10-08T13:45:45.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 -- 172.21.15.115:0/1687072942 shutdown_connections 2023-10-08T13:45:45.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86e8104720 0x7f86e806afb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:45.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86e81002a0 0x7f86e8071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:45.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f86d00a1440 0x7f86d00a3900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:45.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 --2- 172.21.15.115:0/1687072942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e8100c60 0x7f86e8072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:45.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 -- 172.21.15.115:0/1687072942 >> 172.21.15.115:0/1687072942 conn(0x7f86e8074a10 msgr2=0x7f86e80fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:45:45.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 -- 172.21.15.115:0/1687072942 shutdown_connections 2023-10-08T13:45:45.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:45:45.088+0000 7f86ec9b1700 1 -- 172.21.15.115:0/1687072942 wait complete. 2023-10-08T13:45:45.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:45 smithi115 ceph-mon[104408]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:45.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:45 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:45 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.557 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:45 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:45.558 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:45 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:45.558 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:45 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.558 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:45 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.558 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:45 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.558 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:45 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:45.559 DEBUG:teuthology.orchestra.run.smithi115:osd.2> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.2.service 2023-10-08T13:45:45.561 INFO:tasks.cephadm:Deploying osd.3 on smithi057 with /dev/vg_nvme/lv_3... 2023-10-08T13:45:45.561 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-08T13:45:45.591 INFO:journalctl@ceph.osd.2.smithi115.stdout:-- Logs begin at Sun 2023-10-08 13:28:12 UTC. -- 2023-10-08T13:45:45.594 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:45 smithi057 ceph-mon[99476]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:45.594 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:45 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.594 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:45 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.594 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:45 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:45.595 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:45 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:45.595 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:45 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.595 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:45 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.595 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:45 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.595 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:45 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:45.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:45 smithi099 ceph-mon[103849]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:45.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:45 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:45 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:45 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:45.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:45 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:45.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:45 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:45 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:45 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:45.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:45 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:45:46.141 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:45:46.265 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:46 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-2[111508]: 2023-10-08T13:45:46.069+0000 7f852c8b3640 -1 osd.2 0 log_to_monitors true 2023-10-08T13:45:46.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:46 smithi057 ceph-mon[99476]: from='osd.2 [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T13:45:46.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:46 smithi057 ceph-mon[99476]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T13:45:46.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:46 smithi115 ceph-mon[104408]: from='osd.2 [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T13:45:46.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:46 smithi115 ceph-mon[104408]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T13:45:46.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:46 smithi099 ceph-mon[103849]: from='osd.2 [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T13:45:46.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:46 smithi099 ceph-mon[103849]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T13:45:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:47 smithi057 ceph-mon[99476]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:47 smithi057 ceph-mon[99476]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-08T13:45:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:47 smithi057 ceph-mon[99476]: from='osd.2 [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:45:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:47 smithi057 ceph-mon[99476]: osdmap e17: 3 total, 2 up, 3 in 2023-10-08T13:45:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:47 smithi057 ceph-mon[99476]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:45:47.657 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:47 smithi115 ceph-mon[104408]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:47.657 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:47 smithi115 ceph-mon[104408]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-08T13:45:47.658 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:47 smithi115 ceph-mon[104408]: from='osd.2 [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:45:47.658 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:47 smithi115 ceph-mon[104408]: osdmap e17: 3 total, 2 up, 3 in 2023-10-08T13:45:47.658 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:47.658 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:47 smithi115 ceph-mon[104408]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:45:47.658 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:47 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-2[111508]: 2023-10-08T13:45:47.288+0000 7f8521f2a700 -1 osd.2 0 waiting for initial osdmap 2023-10-08T13:45:47.658 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:47 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-2[111508]: 2023-10-08T13:45:47.295+0000 7f851dd1f700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:45:47.687 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-08T13:45:47.687 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-08T13:45:47.687 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-08T13:45:47.687 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-08T13:45:47.688 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117837 s, 890 MB/s 2023-10-08T13:45:47.688 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-08T13:45:47.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:47 smithi099 ceph-mon[103849]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:47.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:47 smithi099 ceph-mon[103849]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-08T13:45:47.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:47 smithi099 ceph-mon[103849]: from='osd.2 [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:45:47.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:47 smithi099 ceph-mon[103849]: osdmap e17: 3 total, 2 up, 3 in 2023-10-08T13:45:47.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:47.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:47 smithi099 ceph-mon[103849]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:45:48.099 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_3 2023-10-08T13:45:48.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:45:48.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: osdmap e18: 3 total, 2 up, 3 in 2023-10-08T13:45:48.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:48.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:45:48.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:48.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:48.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:48.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:45:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: osdmap e18: 3 total, 2 up, 3 in 2023-10-08T13:45:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:45:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:48.708 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:45:48.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:45:48.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: osdmap e18: 3 total, 2 up, 3 in 2023-10-08T13:45:48.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:48.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:45:48.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:48.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:45:48.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:48.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:49.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:49 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:45:49.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:49 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:45:49.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:49 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi115 2023-10-08T13:45:49.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:49 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi115 to 17168M 2023-10-08T13:45:49.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:49 smithi057 ceph-mon[99476]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:49.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:49 smithi057 ceph-mon[99476]: osd.2 [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] boot 2023-10-08T13:45:49.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:49 smithi057 ceph-mon[99476]: osdmap e19: 3 total, 3 up, 3 in 2023-10-08T13:45:49.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:49.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:49 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:45:49.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:49 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:45:49.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:49 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi115 2023-10-08T13:45:49.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:49 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi115 to 17168M 2023-10-08T13:45:49.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:49 smithi115 ceph-mon[104408]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:49.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:49 smithi115 ceph-mon[104408]: osd.2 [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] boot 2023-10-08T13:45:49.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:49 smithi115 ceph-mon[104408]: osdmap e19: 3 total, 3 up, 3 in 2023-10-08T13:45:49.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:49.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:49 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:45:49.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:49 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:45:49.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:49 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi115 2023-10-08T13:45:49.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:49 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi115 to 17168M 2023-10-08T13:45:49.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:49 smithi099 ceph-mon[103849]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T13:45:49.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:49 smithi099 ceph-mon[103849]: osd.2 [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] boot 2023-10-08T13:45:49.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:49 smithi099 ceph-mon[103849]: osdmap e19: 3 total, 3 up, 3 in 2023-10-08T13:45:49.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T13:45:50.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 -- 172.21.15.57:0/2566189211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ad81002a0 msgr2=0x7f5ad8100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:50.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 --2- 172.21.15.57:0/2566189211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ad81002a0 0x7f5ad8100720 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009990 tx=0x7f5ac0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:45:50.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 -- 172.21.15.57:0/2566189211 shutdown_connections 2023-10-08T13:45:50.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 --2- 172.21.15.57:0/2566189211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad8100c60 0x7f5ad810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:50.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 --2- 172.21.15.57:0/2566189211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ad8104720 0x7f5ad8104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:50.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 --2- 172.21.15.57:0/2566189211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ad81002a0 0x7f5ad8100720 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:50.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 -- 172.21.15.57:0/2566189211 >> 172.21.15.57:0/2566189211 conn(0x7f5ad8074a10 msgr2=0x7f5ad8074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:45:50.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 -- 172.21.15.57:0/2566189211 shutdown_connections 2023-10-08T13:45:50.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 -- 172.21.15.57:0/2566189211 wait complete. 2023-10-08T13:45:50.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 Processor -- start 2023-10-08T13:45:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.021+0000 7f5ade7df700 1 -- start start 2023-10-08T13:45:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ade7df700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ad81002a0 0x7f5ad8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ade7df700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ad8100c60 0x7f5ad8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ade7df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad8104720 0x7f5ad81a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ade7df700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad8198820 con 0x7f5ad81002a0 2023-10-08T13:45:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ade7df700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad81a2fe0 con 0x7f5ad8104720 2023-10-08T13:45:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ade7df700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad81a3120 con 0x7f5ad8100c60 2023-10-08T13:45:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ad81002a0 0x7f5ad8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:50.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ad81002a0 0x7f5ad8197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53116/0 (socket says 172.21.15.57:53116) 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad7fff700 1 -- 172.21.15.57:0/2843820565 learned_addr learned my addr 172.21.15.57:0/2843820565 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5acffff700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ad8100c60 0x7f5ad8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5adcd7c700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad8104720 0x7f5ad81a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad7fff700 1 -- 172.21.15.57:0/2843820565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ad8100c60 msgr2=0x7f5ad8197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad7fff700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ad8100c60 0x7f5ad8197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad7fff700 1 -- 172.21.15.57:0/2843820565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad8104720 msgr2=0x7f5ad81a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad7fff700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad8104720 0x7f5ad81a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad7fff700 1 -- 172.21.15.57:0/2843820565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac00095f0 con 0x7f5ad81002a0 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad7fff700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ad81002a0 0x7f5ad8197a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800d980 tx=0x7f5ac800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad5ffb700 1 -- 172.21.15.57:0/2843820565 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac800bb60 con 0x7f5ad81002a0 2023-10-08T13:45:50.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad5ffb700 1 -- 172.21.15.57:0/2843820565 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ac8007640 con 0x7f5ad81002a0 2023-10-08T13:45:50.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ade7df700 1 -- 172.21.15.57:0/2843820565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad81a33a0 con 0x7f5ad81002a0 2023-10-08T13:45:50.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ad5ffb700 1 -- 172.21.15.57:0/2843820565 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8004e90 con 0x7f5ad81002a0 2023-10-08T13:45:50.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.022+0000 7f5ade7df700 1 -- 172.21.15.57:0/2843820565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad81a38f0 con 0x7f5ad81002a0 2023-10-08T13:45:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.024+0000 7f5ade7df700 1 -- 172.21.15.57:0/2843820565 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad8107d10 con 0x7f5ad81002a0 2023-10-08T13:45:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.025+0000 7f5ad5ffb700 1 -- 172.21.15.57:0/2843820565 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f5ac800bcc0 con 0x7f5ad81002a0 2023-10-08T13:45:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.025+0000 7f5ad5ffb700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ac40a14c0 0x7f5ac40a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:45:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.026+0000 7f5acffff700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ac40a14c0 0x7f5ac40a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:45:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.026+0000 7f5ad5ffb700 1 -- 172.21.15.57:0/2843820565 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3066+0+0 (secure 0 0 0) 0x7f5ac80cb5e0 con 0x7f5ad81002a0 2023-10-08T13:45:50.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.026+0000 7f5acffff700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ac40a14c0 0x7f5ac40a3980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0006010 tx=0x7f5ac000b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:45:50.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.030+0000 7f5ad5ffb700 1 -- 172.21.15.57:0/2843820565 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5ac80961a0 con 0x7f5ad81002a0 2023-10-08T13:45:50.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:50.243+0000 7f5ade7df700 1 -- 172.21.15.57:0/2843820565 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ad804e560 con 0x7f5ac40a14c0 2023-10-08T13:45:51.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:51 smithi057 ceph-mon[99476]: osdmap e20: 3 total, 3 up, 3 in 2023-10-08T13:45:51.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:51 smithi057 ceph-mon[99476]: pgmap v56: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:51.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:51 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-08T13:45:51.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:51 smithi057 ceph-mon[99476]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:45:51.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:51 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:45:51.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:51 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:45:51.331 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:51 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:51.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:51 smithi115 ceph-mon[104408]: osdmap e20: 3 total, 3 up, 3 in 2023-10-08T13:45:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:51 smithi115 ceph-mon[104408]: pgmap v56: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:51 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-08T13:45:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:51 smithi115 ceph-mon[104408]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:45:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:51 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:45:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:51 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:45:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:51 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:51 smithi099 ceph-mon[103849]: osdmap e20: 3 total, 3 up, 3 in 2023-10-08T13:45:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:51 smithi099 ceph-mon[103849]: pgmap v56: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:51 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-08T13:45:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:51 smithi099 ceph-mon[103849]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:45:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:51 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:45:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:51 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:45:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:51 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:45:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:52 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-08T13:45:52.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:52 smithi115 ceph-mon[104408]: osdmap e21: 3 total, 3 up, 3 in 2023-10-08T13:45:52.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:52 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T13:45:52.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:52 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-08T13:45:52.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:52 smithi057 ceph-mon[99476]: osdmap e21: 3 total, 3 up, 3 in 2023-10-08T13:45:52.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:52 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T13:45:52.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:52 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-08T13:45:52.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:52 smithi099 ceph-mon[103849]: osdmap e21: 3 total, 3 up, 3 in 2023-10-08T13:45:52.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:52 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T13:45:53.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:53 smithi115 ceph-mon[104408]: pgmap v58: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:53.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-08T13:45:53.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:53 smithi115 ceph-mon[104408]: osdmap e22: 3 total, 3 up, 3 in 2023-10-08T13:45:53.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:53 smithi115 ceph-mon[104408]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T13:45:53.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:53 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3209736124' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c969ed29-1e01-479b-ab6f-00f8b0dde361"}]: dispatch 2023-10-08T13:45:53.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:53 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3209736124' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c969ed29-1e01-479b-ab6f-00f8b0dde361"}]': finished 2023-10-08T13:45:53.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:53 smithi115 ceph-mon[104408]: osdmap e23: 4 total, 3 up, 4 in 2023-10-08T13:45:53.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:45:53.473 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:53 smithi057 sudo[130691]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T13:45:53.474 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:53 smithi057 sudo[130691]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:53.474 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:53 smithi057 sudo[130691]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:53.474 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:53 smithi057 sudo[130691]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:53.474 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:53 smithi057 ceph-mon[99476]: pgmap v58: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:53.474 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-08T13:45:53.474 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:53 smithi057 ceph-mon[99476]: osdmap e22: 3 total, 3 up, 3 in 2023-10-08T13:45:53.474 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:53 smithi057 ceph-mon[99476]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T13:45:53.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:53 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3209736124' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c969ed29-1e01-479b-ab6f-00f8b0dde361"}]: dispatch 2023-10-08T13:45:53.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:53 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3209736124' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c969ed29-1e01-479b-ab6f-00f8b0dde361"}]': finished 2023-10-08T13:45:53.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:53 smithi057 ceph-mon[99476]: osdmap e23: 4 total, 3 up, 4 in 2023-10-08T13:45:53.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:45:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:53 smithi099 ceph-mon[103849]: pgmap v58: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-08T13:45:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:53 smithi099 ceph-mon[103849]: osdmap e22: 3 total, 3 up, 3 in 2023-10-08T13:45:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:53 smithi099 ceph-mon[103849]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T13:45:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:53 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3209736124' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c969ed29-1e01-479b-ab6f-00f8b0dde361"}]: dispatch 2023-10-08T13:45:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:53 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3209736124' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c969ed29-1e01-479b-ab6f-00f8b0dde361"}]': finished 2023-10-08T13:45:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:53 smithi099 ceph-mon[103849]: osdmap e23: 4 total, 3 up, 4 in 2023-10-08T13:45:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:45:53.737 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:53 smithi057 sudo[130723]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T13:45:53.737 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:53 smithi057 sudo[130723]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:53.737 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:53 smithi057 sudo[130723]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:53.738 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 13:45:53 smithi057 sudo[130723]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:53.995 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:53 smithi099 sudo[115568]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T13:45:53.995 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:53 smithi099 sudo[115568]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:53.995 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:53 smithi099 sudo[115568]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:53.995 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:53 smithi099 sudo[115568]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:54.272 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:53 smithi099 sudo[115571]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T13:45:54.272 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:53 smithi099 sudo[115571]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:54.272 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:53 smithi099 sudo[115571]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:54.272 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 13:45:53 smithi099 sudo[115571]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:54.514 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:54.514 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:54 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1044140543' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:45:54.515 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:54.515 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:54 smithi115 sudo[116136]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T13:45:54.515 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:54 smithi115 sudo[116136]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:54.515 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:54 smithi115 sudo[116136]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:54.515 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:54 smithi115 sudo[116136]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:54.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:54.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:54 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1044140543' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:45:54.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:54.936 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:54 smithi115 sudo[116139]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T13:45:54.937 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:54 smithi115 sudo[116139]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:54.937 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:54 smithi115 sudo[116139]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:54.937 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 13:45:54 smithi115 sudo[116139]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:54.985 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:54.986 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:54 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1044140543' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:45:54.986 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:54 smithi057 sudo[131152]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T13:45:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:54 smithi057 sudo[131152]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:55.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:54 smithi057 sudo[131152]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:55.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 sudo[131152]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:45:55.460+0000 7f5ad5ffb700 1 -- 172.21.15.57:0/2843820565 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f5ac8096570 con 0x7f5ad81002a0 2023-10-08T13:45:55.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 sudo[131520]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T13:45:55.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 sudo[131520]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:55.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 sudo[131520]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:55.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 sudo[131520]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:55.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 sudo[116142]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T13:45:55.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 sudo[116142]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:55.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 sudo[116142]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:55.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 sudo[116142]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: pgmap v61: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T13:45:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T13:45:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T13:45:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: Cluster is now healthy 2023-10-08T13:45:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:45:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:45:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:45:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: mgrmap e20: x(active, since 101s), standbys: z, y 2023-10-08T13:45:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:45:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:45:56.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:45:56.435 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 sudo[115574]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T13:45:56.435 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 sudo[115574]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:56.435 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 sudo[115574]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:56.436 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 sudo[115574]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:56.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 sudo[116145]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T13:45:56.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 sudo[116145]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:56.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 sudo[116145]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:56.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 sudo[116145]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: pgmap v61: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: Cluster is now healthy 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: mgrmap e20: x(active, since 101s), standbys: z, y 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:45:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:45:56.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:45:56.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 sudo[115577]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T13:45:56.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 sudo[115577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T13:45:56.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 sudo[115577]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T13:45:56.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 sudo[115577]: pam_unix(sudo:session): session closed for user root 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: pgmap v61: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: Cluster is now healthy 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: mgrmap e20: x(active, since 101s), standbys: z, y 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:45:56.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:45:56.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:45:57.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:56 smithi057 ceph-mon[99476]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T13:45:57.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:56 smithi057 ceph-mon[99476]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T13:45:57.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:56 smithi057 ceph-mon[99476]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T13:45:57.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:56 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:45:57.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:56 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:45:57.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:56 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:45:57.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:56 smithi057 ceph-mon[99476]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T13:45:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T13:45:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T13:45:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T13:45:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:45:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:45:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:45:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:56 smithi099 ceph-mon[103849]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T13:45:57.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:56 smithi115 ceph-mon[104408]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T13:45:57.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:56 smithi115 ceph-mon[104408]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T13:45:57.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:56 smithi115 ceph-mon[104408]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T13:45:57.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:56 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T13:45:57.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:56 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T13:45:57.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:56 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T13:45:57.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:56 smithi115 ceph-mon[104408]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T13:45:58.166 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:57 smithi057 ceph-mon[99476]: pgmap v62: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:57 smithi099 ceph-mon[103849]: pgmap v62: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:45:58.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:57 smithi115 ceph-mon[104408]: pgmap v62: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:46:00.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:59 smithi057 ceph-mon[99476]: pgmap v63: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:46:00.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:59 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-08T13:46:00.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:45:59 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:00.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:59 smithi099 ceph-mon[103849]: pgmap v63: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:46:00.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:59 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-08T13:46:00.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:45:59 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:00.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:59 smithi115 ceph-mon[104408]: pgmap v63: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:46:00.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:59 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-08T13:46:00.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:45:59 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:01.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:00 smithi057 ceph-mon[99476]: Deploying daemon osd.3 on smithi057 2023-10-08T13:46:01.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:00 smithi099 ceph-mon[103849]: Deploying daemon osd.3 on smithi057 2023-10-08T13:46:01.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:00 smithi115 ceph-mon[104408]: Deploying daemon osd.3 on smithi057 2023-10-08T13:46:02.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:01 smithi057 ceph-mon[99476]: pgmap v64: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 196 KiB/s wr, 14 op/s 2023-10-08T13:46:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:01 smithi099 ceph-mon[103849]: pgmap v64: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 196 KiB/s wr, 14 op/s 2023-10-08T13:46:02.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:01 smithi115 ceph-mon[104408]: pgmap v64: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 196 KiB/s wr, 14 op/s 2023-10-08T13:46:03.849 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:03 smithi057 ceph-mon[99476]: pgmap v65: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-10-08T13:46:03.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:03 smithi115 ceph-mon[104408]: pgmap v65: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-10-08T13:46:04.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:03 smithi099 ceph-mon[103849]: pgmap v65: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-10-08T13:46:04.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:04 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:04.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:04 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:04.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:04 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:05.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:04 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:05.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:04 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:05.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:04 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:05.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:04 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:05.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:04 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:05.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:04 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:06.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:05 smithi115 ceph-mon[104408]: pgmap v66: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2023-10-08T13:46:06.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:05 smithi057 ceph-mon[99476]: pgmap v66: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2023-10-08T13:46:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:05 smithi099 ceph-mon[103849]: pgmap v66: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2023-10-08T13:46:06.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.741+0000 7f5ad5ffb700 1 -- 172.21.15.57:0/2843820565 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5ad804e560 con 0x7f5ac40a14c0 2023-10-08T13:46:06.743 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 3 on host 'smithi057' 2023-10-08T13:46:06.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 -- 172.21.15.57:0/2843820565 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ac40a14c0 msgr2=0x7f5ac40a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:06.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ac40a14c0 0x7f5ac40a3980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0006010 tx=0x7f5ac000b260 comp rx=0 tx=0).stop 2023-10-08T13:46:06.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 -- 172.21.15.57:0/2843820565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ad81002a0 msgr2=0x7f5ad8197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:06.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ad81002a0 0x7f5ad8197a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800d980 tx=0x7f5ac800dd40 comp rx=0 tx=0).stop 2023-10-08T13:46:06.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 -- 172.21.15.57:0/2843820565 shutdown_connections 2023-10-08T13:46:06.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ac40a14c0 0x7f5ac40a3980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:06.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad8104720 0x7f5ad81a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:06.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ad81002a0 0x7f5ad8197a20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 --2- 172.21.15.57:0/2843820565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ad8100c60 0x7f5ad8197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 -- 172.21.15.57:0/2843820565 >> 172.21.15.57:0/2843820565 conn(0x7f5ad8074a10 msgr2=0x7f5ad80fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:46:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 -- 172.21.15.57:0/2843820565 shutdown_connections 2023-10-08T13:46:06.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:06.743+0000 7f5ade7df700 1 -- 172.21.15.57:0/2843820565 wait complete. 2023-10-08T13:46:07.117 DEBUG:teuthology.orchestra.run.smithi057:osd.3> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.3.service 2023-10-08T13:46:07.120 INFO:tasks.cephadm:Deploying osd.4 on smithi099 with /dev/vg_nvme/lv_3... 2023-10-08T13:46:07.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-08T13:46:07.156 INFO:journalctl@ceph.osd.3.smithi057.stdout:-- Logs begin at Sun 2023-10-08 13:28:06 UTC. -- 2023-10-08T13:46:07.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:07 smithi115 ceph-mon[104408]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-10-08T13:46:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:07 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:07 smithi057 ceph-mon[99476]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-10-08T13:46:07.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:07 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:07 smithi099 ceph-mon[103849]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-10-08T13:46:07.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:07.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:07.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:07.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:07 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T13:46:07.931 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 08 13:46:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-3[132958]: 2023-10-08T13:46:07.825+0000 7ffb2af5a640 -1 osd.3 0 log_to_monitors true 2023-10-08T13:46:08.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:08 smithi115 ceph-mon[104408]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-08T13:46:08.947 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:08 smithi099 ceph-mon[103849]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-08T13:46:08.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:08 smithi057 ceph-mon[99476]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-08T13:46:09.447 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-08T13:46:09.447 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-08T13:46:09.447 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-10-08T13:46:09.447 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-10-08T13:46:09.447 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111808 s, 938 MB/s 2023-10-08T13:46:09.447 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-10-08T13:46:09.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:09 smithi099 ceph-mon[103849]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-10-08T13:46:09.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:09 smithi099 ceph-mon[103849]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-08T13:46:09.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:09 smithi099 ceph-mon[103849]: osdmap e24: 4 total, 3 up, 4 in 2023-10-08T13:46:09.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:09 smithi099 ceph-mon[103849]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:46:09.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:09 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:09.848 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2023-10-08T13:46:09.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:09 smithi115 ceph-mon[104408]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-10-08T13:46:09.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:09 smithi115 ceph-mon[104408]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-08T13:46:09.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:09 smithi115 ceph-mon[104408]: osdmap e24: 4 total, 3 up, 4 in 2023-10-08T13:46:09.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:09 smithi115 ceph-mon[104408]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:46:09.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:09 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:09.987 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 08 13:46:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-3[132958]: 2023-10-08T13:46:09.676+0000 7ffb205d1700 -1 osd.3 0 waiting for initial osdmap 2023-10-08T13:46:09.988 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 08 13:46:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-3[132958]: 2023-10-08T13:46:09.682+0000 7ffb1a3c2700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:46:09.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:09 smithi057 ceph-mon[99476]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-10-08T13:46:09.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:09 smithi057 ceph-mon[99476]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-08T13:46:09.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:09 smithi057 ceph-mon[99476]: osdmap e24: 4 total, 3 up, 4 in 2023-10-08T13:46:09.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:09 smithi057 ceph-mon[99476]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:46:09.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:09 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:10.407 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T13:46:10.981 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:46:10.981 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: osdmap e25: 4 total, 3 up, 4 in 2023-10-08T13:46:10.981 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:10.981 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:10.981 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi057 2023-10-08T13:46:10.982 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:10.982 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:10.982 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:10.982 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:10.982 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi057 to 8586M 2023-10-08T13:46:10.982 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:10.982 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:10.982 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:10.982 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:10.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:46:10.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: osdmap e25: 4 total, 3 up, 4 in 2023-10-08T13:46:10.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:10.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:10.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi057 2023-10-08T13:46:10.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:10.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:10.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:10.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:10.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi057 to 8586M 2023-10-08T13:46:10.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:10.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:10.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:10.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:11.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:46:11.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: osdmap e25: 4 total, 3 up, 4 in 2023-10-08T13:46:11.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:11.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:11.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi057 2023-10-08T13:46:11.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:11.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:11.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:11.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:11.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi057 to 8586M 2023-10-08T13:46:11.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:11.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:11.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:11.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:11.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.769+0000 7f4094bf0700 1 -- 172.21.15.99:0/2321473204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40900ff190 msgr2=0x7f40900ff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:11.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.769+0000 7f4094bf0700 1 --2- 172.21.15.99:0/2321473204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40900ff190 0x7f40900ff610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4080009990 tx=0x7f4080009ca0 comp rx=0 tx=0).stop 2023-10-08T13:46:11.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.769+0000 7f4094bf0700 1 -- 172.21.15.99:0/2321473204 shutdown_connections 2023-10-08T13:46:11.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.769+0000 7f4094bf0700 1 --2- 172.21.15.99:0/2321473204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40900ffb50 0x7f409010bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:11.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.769+0000 7f4094bf0700 1 --2- 172.21.15.99:0/2321473204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4090103e70 0x7f4090104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:11.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.769+0000 7f4094bf0700 1 --2- 172.21.15.99:0/2321473204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40900ff190 0x7f40900ff610 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:11.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.769+0000 7f4094bf0700 1 -- 172.21.15.99:0/2321473204 >> 172.21.15.99:0/2321473204 conn(0x7f4090074a10 msgr2=0x7f4090074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:46:11.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.769+0000 7f4094bf0700 1 -- 172.21.15.99:0/2321473204 shutdown_connections 2023-10-08T13:46:11.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.769+0000 7f4094bf0700 1 -- 172.21.15.99:0/2321473204 wait complete. 2023-10-08T13:46:11.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f4094bf0700 1 Processor -- start 2023-10-08T13:46:11.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f4094bf0700 1 -- start start 2023-10-08T13:46:11.773 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f4094bf0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40900ff190 0x7f409019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f4094bf0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40900ffb50 0x7f409019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f4094bf0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4090103e70 0x7f4090196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f4094bf0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4090197150 con 0x7f40900ff190 2023-10-08T13:46:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f4094bf0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40901972c0 con 0x7f4090103e70 2023-10-08T13:46:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f4094bf0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4090197480 con 0x7f40900ffb50 2023-10-08T13:46:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f408ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4090103e70 0x7f4090196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f408ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4090103e70 0x7f4090196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42548/0 (socket says 172.21.15.99:42548) 2023-10-08T13:46:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f408ed9d700 1 -- 172.21.15.99:0/912366428 learned_addr learned my addr 172.21.15.99:0/912366428 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T13:46:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f408e59c700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40900ff190 0x7f409019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.770+0000 7f408dd9b700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40900ffb50 0x7f409019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f408ed9d700 1 -- 172.21.15.99:0/912366428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40900ffb50 msgr2=0x7f409019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f408ed9d700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40900ffb50 0x7f409019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f408ed9d700 1 -- 172.21.15.99:0/912366428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40900ff190 msgr2=0x7f409019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f408ed9d700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40900ff190 0x7f409019c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f408ed9d700 1 -- 172.21.15.99:0/912366428 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40800095f0 con 0x7f4090103e70 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f408ed9d700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4090103e70 0x7f4090196af0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f408800b970 tx=0x7f408800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f407f7fe700 1 -- 172.21.15.99:0/912366428 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408800c930 con 0x7f4090103e70 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f407f7fe700 1 -- 172.21.15.99:0/912366428 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4088014440 con 0x7f4090103e70 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f4094bf0700 1 -- 172.21.15.99:0/912366428 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4090197760 con 0x7f4090103e70 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f407f7fe700 1 -- 172.21.15.99:0/912366428 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40880075d0 con 0x7f4090103e70 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.771+0000 7f4094bf0700 1 -- 172.21.15.99:0/912366428 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40901a39a0 con 0x7f4090103e70 2023-10-08T13:46:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.772+0000 7f4094bf0700 1 -- 172.21.15.99:0/912366428 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40901005b0 con 0x7f4090103e70 2023-10-08T13:46:11.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.773+0000 7f407f7fe700 1 -- 172.21.15.99:0/912366428 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f40880145b0 con 0x7f4090103e70 2023-10-08T13:46:11.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.773+0000 7f407f7fe700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f40780a1550 0x7f40780a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:11.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.773+0000 7f407f7fe700 1 -- 172.21.15.99:0/912366428 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3913+0+0 (secure 0 0 0) 0x7f4088006e40 con 0x7f4090103e70 2023-10-08T13:46:11.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.774+0000 7f408e59c700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f40780a1550 0x7f40780a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:11.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.774+0000 7f408e59c700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f40780a1550 0x7f40780a3a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4090077270 tx=0x7f4084009190 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:46:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.778+0000 7f407f7fe700 1 -- 172.21.15.99:0/912366428 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f408804f100 con 0x7f4090103e70 2023-10-08T13:46:11.974 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:11.973+0000 7f4094bf0700 1 -- 172.21.15.99:0/912366428 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4090198020 con 0x7f40780a1550 2023-10-08T13:46:11.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:11 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:46:11.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:11 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:46:11.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:11 smithi057 ceph-mon[99476]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:46:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:11 smithi057 ceph-mon[99476]: osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] boot 2023-10-08T13:46:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:11 smithi057 ceph-mon[99476]: osdmap e26: 4 total, 4 up, 4 in 2023-10-08T13:46:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:11 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:46:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:11 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:46:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:11 smithi099 ceph-mon[103849]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:46:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:11 smithi099 ceph-mon[103849]: osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] boot 2023-10-08T13:46:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:11 smithi099 ceph-mon[103849]: osdmap e26: 4 total, 4 up, 4 in 2023-10-08T13:46:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:12.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:11 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:46:12.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:11 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:46:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:11 smithi115 ceph-mon[104408]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-08T13:46:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:11 smithi115 ceph-mon[104408]: osd.3 [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] boot 2023-10-08T13:46:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:11 smithi115 ceph-mon[104408]: osdmap e26: 4 total, 4 up, 4 in 2023-10-08T13:46:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T13:46:12.951 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:12 smithi099 ceph-mon[103849]: osdmap e27: 4 total, 4 up, 4 in 2023-10-08T13:46:12.951 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:46:12.951 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:46:12.951 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:12.951 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:12 smithi099 ceph-mon[103849]: osdmap e28: 4 total, 4 up, 4 in 2023-10-08T13:46:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:12 smithi057 ceph-mon[99476]: osdmap e27: 4 total, 4 up, 4 in 2023-10-08T13:46:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:46:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:46:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:12 smithi057 ceph-mon[99476]: osdmap e28: 4 total, 4 up, 4 in 2023-10-08T13:46:13.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:12 smithi115 ceph-mon[104408]: osdmap e27: 4 total, 4 up, 4 in 2023-10-08T13:46:13.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:46:13.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:46:13.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:13.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:12 smithi115 ceph-mon[104408]: osdmap e28: 4 total, 4 up, 4 in 2023-10-08T13:46:13.962 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:13 smithi099 ceph-mon[103849]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:46:13.962 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:13 smithi099 ceph-mon[103849]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:13.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:13 smithi057 ceph-mon[99476]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:46:13.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:13 smithi057 ceph-mon[99476]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:14.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:13 smithi115 ceph-mon[104408]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:46:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:13 smithi115 ceph-mon[104408]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:14.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:14 smithi057 ceph-mon[99476]: osdmap e29: 4 total, 4 up, 4 in 2023-10-08T13:46:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.99:0/1357921377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842feff0-faab-4c13-88b1-91355f53e0e2"}]: dispatch 2023-10-08T13:46:14.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:14 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842feff0-faab-4c13-88b1-91355f53e0e2"}]: dispatch 2023-10-08T13:46:14.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:14 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "842feff0-faab-4c13-88b1-91355f53e0e2"}]': finished 2023-10-08T13:46:14.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:14 smithi057 ceph-mon[99476]: osdmap e30: 5 total, 4 up, 5 in 2023-10-08T13:46:14.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:14 smithi099 ceph-mon[103849]: osdmap e29: 4 total, 4 up, 4 in 2023-10-08T13:46:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:14 smithi099 ceph-mon[103849]: from='client.? 172.21.15.99:0/1357921377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842feff0-faab-4c13-88b1-91355f53e0e2"}]: dispatch 2023-10-08T13:46:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:14 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842feff0-faab-4c13-88b1-91355f53e0e2"}]: dispatch 2023-10-08T13:46:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:14 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "842feff0-faab-4c13-88b1-91355f53e0e2"}]': finished 2023-10-08T13:46:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:14 smithi099 ceph-mon[103849]: osdmap e30: 5 total, 4 up, 5 in 2023-10-08T13:46:15.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:15.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:14 smithi115 ceph-mon[104408]: osdmap e29: 4 total, 4 up, 4 in 2023-10-08T13:46:15.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:14 smithi115 ceph-mon[104408]: from='client.? 172.21.15.99:0/1357921377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842feff0-faab-4c13-88b1-91355f53e0e2"}]: dispatch 2023-10-08T13:46:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:14 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842feff0-faab-4c13-88b1-91355f53e0e2"}]: dispatch 2023-10-08T13:46:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:14 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "842feff0-faab-4c13-88b1-91355f53e0e2"}]': finished 2023-10-08T13:46:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:14 smithi115 ceph-mon[104408]: osdmap e30: 5 total, 4 up, 5 in 2023-10-08T13:46:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:15.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:15 smithi057 ceph-mon[99476]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:15.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:15 smithi057 ceph-mon[99476]: from='client.? 172.21.15.99:0/1327726578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:46:16.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:15 smithi099 ceph-mon[103849]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:16.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:15 smithi099 ceph-mon[103849]: from='client.? 172.21.15.99:0/1327726578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:46:16.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:15 smithi115 ceph-mon[104408]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:15 smithi115 ceph-mon[104408]: from='client.? 172.21.15.99:0/1327726578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:46:17.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:17 smithi057 ceph-mon[99476]: pgmap v79: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-10-08T13:46:18.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:17 smithi099 ceph-mon[103849]: pgmap v79: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-10-08T13:46:18.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:17 smithi115 ceph-mon[104408]: pgmap v79: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-10-08T13:46:19.971 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:19 smithi099 ceph-mon[103849]: pgmap v80: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:46:19.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:19 smithi057 ceph-mon[99476]: pgmap v80: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:46:20.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:19 smithi115 ceph-mon[104408]: pgmap v80: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:46:21.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:21 smithi057 ceph-mon[99476]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-08T13:46:21.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-08T13:46:21.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:22.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:21 smithi099 ceph-mon[103849]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-08T13:46:22.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-08T13:46:22.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:22.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:21 smithi115 ceph-mon[104408]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-08T13:46:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-08T13:46:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:23.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:22 smithi099 ceph-mon[103849]: Deploying daemon osd.4 on smithi099 2023-10-08T13:46:23.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:22 smithi115 ceph-mon[104408]: Deploying daemon osd.4 on smithi099 2023-10-08T13:46:23.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:22 smithi057 ceph-mon[99476]: Deploying daemon osd.4 on smithi099 2023-10-08T13:46:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:23 smithi099 ceph-mon[103849]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-08T13:46:24.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:23 smithi115 ceph-mon[104408]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-08T13:46:24.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:23 smithi057 ceph-mon[99476]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-08T13:46:26.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:25 smithi099 ceph-mon[103849]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T13:46:26.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:25 smithi115 ceph-mon[104408]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T13:46:26.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:25 smithi057 ceph-mon[99476]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T13:46:27.337 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:27 smithi099 ceph-mon[103849]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:27.337 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:27.337 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:27.337 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:27.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:27 smithi057 ceph-mon[99476]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:27.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:27.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:27.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:27.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:27 smithi115 ceph-mon[104408]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:27.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:27.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:27.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.243+0000 7f407f7fe700 1 -- 172.21.15.99:0/912366428 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4090198020 con 0x7f40780a1550 2023-10-08T13:46:29.245 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 4 on host 'smithi099' 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.245+0000 7f4094bf0700 1 -- 172.21.15.99:0/912366428 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f40780a1550 msgr2=0x7f40780a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.245+0000 7f4094bf0700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f40780a1550 0x7f40780a3a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4090077270 tx=0x7f4084009190 comp rx=0 tx=0).stop 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.245+0000 7f4094bf0700 1 -- 172.21.15.99:0/912366428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4090103e70 msgr2=0x7f4090196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.245+0000 7f4094bf0700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4090103e70 0x7f4090196af0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f408800b970 tx=0x7f408800bc80 comp rx=0 tx=0).stop 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.245+0000 7f4094bf0700 1 -- 172.21.15.99:0/912366428 shutdown_connections 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.245+0000 7f4094bf0700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f40780a1550 0x7f40780a3a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.245+0000 7f4094bf0700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4090103e70 0x7f4090196af0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.245+0000 7f4094bf0700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f40900ff190 0x7f409019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.245+0000 7f4094bf0700 1 --2- 172.21.15.99:0/912366428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40900ffb50 0x7f409019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.245+0000 7f4094bf0700 1 -- 172.21.15.99:0/912366428 >> 172.21.15.99:0/912366428 conn(0x7f4090074a10 msgr2=0x7f4090102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:46:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.246+0000 7f4094bf0700 1 -- 172.21.15.99:0/912366428 shutdown_connections 2023-10-08T13:46:29.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:46:29.246+0000 7f4094bf0700 1 -- 172.21.15.99:0/912366428 wait complete. 2023-10-08T13:46:29.693 DEBUG:teuthology.orchestra.run.smithi099:osd.4> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.4.service 2023-10-08T13:46:29.695 INFO:tasks.cephadm:Deploying osd.5 on smithi115 with /dev/vg_nvme/lv_3... 2023-10-08T13:46:29.695 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-08T13:46:29.736 INFO:journalctl@ceph.osd.4.smithi099.stdout:-- Logs begin at Sun 2023-10-08 13:27:56 UTC. -- 2023-10-08T13:46:29.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:29 smithi115 ceph-mon[104408]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:29.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:29.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:29.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:29.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:29.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:29.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:29.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:29.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:29 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:29.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:29 smithi057 ceph-mon[99476]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:29.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:29.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:29.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:29.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:29.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:29.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:29.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:29.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:29 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:29 smithi099 ceph-mon[103849]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:29 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:30.275 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:46:30.403 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 13:46:30 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4[118653]: 2023-10-08T13:46:30.139+0000 7f1d8eae8640 -1 osd.4 0 log_to_monitors true 2023-10-08T13:46:30.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:30 smithi099 ceph-mon[103849]: from='osd.4 [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T13:46:30.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:30 smithi099 ceph-mon[103849]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T13:46:30.780 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:30 smithi115 ceph-mon[104408]: from='osd.4 [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T13:46:30.781 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:30 smithi115 ceph-mon[104408]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T13:46:30.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:30 smithi057 ceph-mon[99476]: from='osd.4 [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T13:46:30.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:30 smithi057 ceph-mon[99476]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T13:46:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-08T13:46:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-08T13:46:31.746 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2023-10-08T13:46:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2023-10-08T13:46:31.746 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109456 s, 958 MB/s 2023-10-08T13:46:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2023-10-08T13:46:31.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:31 smithi115 ceph-mon[104408]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:31.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:31 smithi115 ceph-mon[104408]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-08T13:46:31.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:31 smithi115 ceph-mon[104408]: osdmap e31: 5 total, 4 up, 5 in 2023-10-08T13:46:31.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:31 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:31.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:31 smithi115 ceph-mon[104408]: from='osd.4 [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:46:31.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:31 smithi115 ceph-mon[104408]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:46:31.970 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 13:46:31 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4[118653]: 2023-10-08T13:46:31.587+0000 7f1d8415f700 -1 osd.4 0 waiting for initial osdmap 2023-10-08T13:46:31.970 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 13:46:31 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4[118653]: 2023-10-08T13:46:31.594+0000 7f1d7df50700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:46:31.971 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:31 smithi099 ceph-mon[103849]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:31.971 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:31 smithi099 ceph-mon[103849]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-08T13:46:31.971 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:31 smithi099 ceph-mon[103849]: osdmap e31: 5 total, 4 up, 5 in 2023-10-08T13:46:31.971 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:31 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:31.971 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:31 smithi099 ceph-mon[103849]: from='osd.4 [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:46:31.971 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:31 smithi099 ceph-mon[103849]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:46:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:31 smithi057 ceph-mon[99476]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:31 smithi057 ceph-mon[99476]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-08T13:46:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:31 smithi057 ceph-mon[99476]: osdmap e31: 5 total, 4 up, 5 in 2023-10-08T13:46:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:31 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:31 smithi057 ceph-mon[99476]: from='osd.4 [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:46:31.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:31 smithi057 ceph-mon[99476]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:46:32.118 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2023-10-08T13:46:32.680 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:46:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:46:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: osdmap e32: 5 total, 4 up, 5 in 2023-10-08T13:46:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:32.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:32.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:32.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:32.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:32.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:32.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.886 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:46:32.886 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: osdmap e32: 5 total, 4 up, 5 in 2023-10-08T13:46:32.886 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:32.886 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:32.886 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.886 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:32.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:32.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:32.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:32.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:46:32.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: osdmap e32: 5 total, 4 up, 5 in 2023-10-08T13:46:32.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:32.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:32.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:32.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:32.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:32.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:32.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:32.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:33.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.883+0000 7fe0618bb700 1 -- 172.21.15.115:0/15449385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe05c101260 msgr2=0x7fe05c10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:33.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.883+0000 7fe0618bb700 1 --2- 172.21.15.115:0/15449385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe05c101260 0x7fe05c10d650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe05400b0a0 tx=0x7fe05400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:46:33.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.883+0000 7fe0618bb700 1 -- 172.21.15.115:0/15449385 shutdown_connections 2023-10-08T13:46:33.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.883+0000 7fe0618bb700 1 --2- 172.21.15.115:0/15449385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe05c101260 0x7fe05c10d650 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:33.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.883+0000 7fe0618bb700 1 --2- 172.21.15.115:0/15449385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1067f0 0x7fe05c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:33.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.883+0000 7fe0618bb700 1 --2- 172.21.15.115:0/15449385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe05c1008a0 0x7fe05c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:33.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.883+0000 7fe0618bb700 1 -- 172.21.15.115:0/15449385 >> 172.21.15.115:0/15449385 conn(0x7fe05c077910 msgr2=0x7fe05c077d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:46:33.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.883+0000 7fe0618bb700 1 -- 172.21.15.115:0/15449385 shutdown_connections 2023-10-08T13:46:33.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.883+0000 7fe0618bb700 1 -- 172.21.15.115:0/15449385 wait complete. 2023-10-08T13:46:33.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe0618bb700 1 Processor -- start 2023-10-08T13:46:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe0618bb700 1 -- start start 2023-10-08T13:46:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe0618bb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe05c1008a0 0x7fe05c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe0618bb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe05c101260 0x7fe05c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe0618bb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1067f0 0x7fe05c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe0618bb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c1a5130 con 0x7fe05c101260 2023-10-08T13:46:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe0618bb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c1a52a0 con 0x7fe05c1067f0 2023-10-08T13:46:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe0618bb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c1a5460 con 0x7fe05c1008a0 2023-10-08T13:46:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe05c1008a0 0x7fe05c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe05c1008a0 0x7fe05c072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52586/0 (socket says 172.21.15.115:52586) 2023-10-08T13:46:33.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05affd700 1 -- 172.21.15.115:0/2663294028 learned_addr learned my addr 172.21.15.115:0/2663294028 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-10-08T13:46:33.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05a7fc700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe05c101260 0x7fe05c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:33.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05b7fe700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1067f0 0x7fe05c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:33.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05affd700 1 -- 172.21.15.115:0/2663294028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1067f0 msgr2=0x7fe05c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:33.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05affd700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1067f0 0x7fe05c073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:33.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05affd700 1 -- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe05c101260 msgr2=0x7fe05c073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:33.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05affd700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe05c101260 0x7fe05c073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:33.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05affd700 1 -- 172.21.15.115:0/2663294028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe054009d00 con 0x7fe05c1008a0 2023-10-08T13:46:33.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.884+0000 7fe05a7fc700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe05c101260 0x7fe05c073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:46:33.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.885+0000 7fe05affd700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe05c1008a0 0x7fe05c072af0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe05000d950 tx=0x7fe05000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:46:33.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.885+0000 7fe0608b9700 1 -- 172.21.15.115:0/2663294028 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050009b40 con 0x7fe05c1008a0 2023-10-08T13:46:33.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.885+0000 7fe0618bb700 1 -- 172.21.15.115:0/2663294028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05c1a5740 con 0x7fe05c1008a0 2023-10-08T13:46:33.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.885+0000 7fe0608b9700 1 -- 172.21.15.115:0/2663294028 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe050009ca0 con 0x7fe05c1008a0 2023-10-08T13:46:33.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.885+0000 7fe0608b9700 1 -- 172.21.15.115:0/2663294028 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05000f620 con 0x7fe05c1008a0 2023-10-08T13:46:33.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.885+0000 7fe0618bb700 1 -- 172.21.15.115:0/2663294028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05c1a5c40 con 0x7fe05c1008a0 2023-10-08T13:46:33.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.886+0000 7fe0618bb700 1 -- 172.21.15.115:0/2663294028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05c108d30 con 0x7fe05c1008a0 2023-10-08T13:46:33.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.887+0000 7fe0608b9700 1 -- 172.21.15.115:0/2663294028 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe050010460 con 0x7fe05c1008a0 2023-10-08T13:46:33.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.887+0000 7fe0608b9700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0440a14d0 0x7fe0440a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:33.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.887+0000 7fe0608b9700 1 -- 172.21.15.115:0/2663294028 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4316+0+0 (secure 0 0 0) 0x7fe0500cc0d0 con 0x7fe05c1008a0 2023-10-08T13:46:33.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.888+0000 7fe05a7fc700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0440a14d0 0x7fe0440a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:33.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.888+0000 7fe05a7fc700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0440a14d0 0x7fe0440a3990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe04c005950 tx=0x7fe04c0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:46:33.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:33.892+0000 7fe0608b9700 1 -- 172.21.15.115:0/2663294028 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe0500966d0 con 0x7fe05c1008a0 2023-10-08T13:46:33.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:33 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:46:33.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:33 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:46:33.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:33 smithi115 ceph-mon[104408]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:33.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:33 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi099 2023-10-08T13:46:33.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:33 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi099 to 8584M 2023-10-08T13:46:33.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:33 smithi115 ceph-mon[104408]: osd.4 [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] boot 2023-10-08T13:46:33.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:33 smithi115 ceph-mon[104408]: osdmap e33: 5 total, 5 up, 5 in 2023-10-08T13:46:33.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:33.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:33 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:46:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:33 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:46:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:33 smithi057 ceph-mon[99476]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:33 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi099 2023-10-08T13:46:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:33 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi099 to 8584M 2023-10-08T13:46:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:33 smithi057 ceph-mon[99476]: osd.4 [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] boot 2023-10-08T13:46:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:33 smithi057 ceph-mon[99476]: osdmap e33: 5 total, 5 up, 5 in 2023-10-08T13:46:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:33 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:46:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:33 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:46:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:33 smithi099 ceph-mon[103849]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-08T13:46:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:33 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi099 2023-10-08T13:46:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:33 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi099 to 8584M 2023-10-08T13:46:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:33 smithi099 ceph-mon[103849]: osd.4 [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] boot 2023-10-08T13:46:34.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:33 smithi099 ceph-mon[103849]: osdmap e33: 5 total, 5 up, 5 in 2023-10-08T13:46:34.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T13:46:34.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:34.088+0000 7fe0618bb700 1 -- 172.21.15.115:0/2663294028 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe05c073fd0 con 0x7fe0440a14d0 2023-10-08T13:46:34.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:34 smithi115 ceph-mon[104408]: osdmap e34: 5 total, 5 up, 5 in 2023-10-08T13:46:34.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:34 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:46:34.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:34 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:46:34.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:34 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:34.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:34 smithi057 ceph-mon[99476]: osdmap e34: 5 total, 5 up, 5 in 2023-10-08T13:46:34.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:34 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:46:34.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:34 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:46:34.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:34 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:35.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:34 smithi099 ceph-mon[103849]: osdmap e34: 5 total, 5 up, 5 in 2023-10-08T13:46:35.059 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:34 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:46:35.059 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:34 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:46:35.059 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:34 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:35.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:35 smithi115 ceph-mon[104408]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:35.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:35 smithi115 ceph-mon[104408]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:46:35.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:35 smithi057 ceph-mon[99476]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:35.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:35 smithi057 ceph-mon[99476]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:46:36.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:35 smithi099 ceph-mon[103849]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:36.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:35 smithi099 ceph-mon[103849]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:46:36.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:36 smithi057 ceph-mon[99476]: from='client.? 172.21.15.115:0/1578615173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97f91a58-677f-4005-8a2d-66113c9cdda6"}]: dispatch 2023-10-08T13:46:36.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:36 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97f91a58-677f-4005-8a2d-66113c9cdda6"}]: dispatch 2023-10-08T13:46:36.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:36 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97f91a58-677f-4005-8a2d-66113c9cdda6"}]': finished 2023-10-08T13:46:36.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:36 smithi057 ceph-mon[99476]: osdmap e35: 6 total, 5 up, 6 in 2023-10-08T13:46:36.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:36 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:37.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:36 smithi099 ceph-mon[103849]: from='client.? 172.21.15.115:0/1578615173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97f91a58-677f-4005-8a2d-66113c9cdda6"}]: dispatch 2023-10-08T13:46:37.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:36 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97f91a58-677f-4005-8a2d-66113c9cdda6"}]: dispatch 2023-10-08T13:46:37.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:36 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97f91a58-677f-4005-8a2d-66113c9cdda6"}]': finished 2023-10-08T13:46:37.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:36 smithi099 ceph-mon[103849]: osdmap e35: 6 total, 5 up, 6 in 2023-10-08T13:46:37.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:36 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:37.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:36 smithi115 ceph-mon[104408]: from='client.? 172.21.15.115:0/1578615173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97f91a58-677f-4005-8a2d-66113c9cdda6"}]: dispatch 2023-10-08T13:46:37.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:36 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97f91a58-677f-4005-8a2d-66113c9cdda6"}]: dispatch 2023-10-08T13:46:37.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:36 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97f91a58-677f-4005-8a2d-66113c9cdda6"}]': finished 2023-10-08T13:46:37.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:36 smithi115 ceph-mon[104408]: osdmap e35: 6 total, 5 up, 6 in 2023-10-08T13:46:37.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:36 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:37.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:37 smithi057 ceph-mon[99476]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:37.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:37 smithi057 ceph-mon[99476]: from='client.? 172.21.15.115:0/1382244717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:46:38.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:37 smithi099 ceph-mon[103849]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:38.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:37 smithi099 ceph-mon[103849]: from='client.? 172.21.15.115:0/1382244717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:46:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:37 smithi115 ceph-mon[104408]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:37 smithi115 ceph-mon[104408]: from='client.? 172.21.15.115:0/1382244717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:46:39.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:39 smithi057 ceph-mon[99476]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:40.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:39 smithi099 ceph-mon[103849]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:40.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:39 smithi115 ceph-mon[104408]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:41.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:41 smithi057 ceph-mon[99476]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:41 smithi099 ceph-mon[103849]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:42.126 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:41 smithi115 ceph-mon[104408]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:43.844 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:43 smithi115 ceph-mon[104408]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:43.844 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:43 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-08T13:46:43.844 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:43 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:43.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:43 smithi057 ceph-mon[99476]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:43.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:43 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-08T13:46:43.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:43 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:43 smithi099 ceph-mon[103849]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:43 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-08T13:46:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:43 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:44.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:44 smithi057 ceph-mon[99476]: Deploying daemon osd.5 on smithi115 2023-10-08T13:46:45.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:44 smithi099 ceph-mon[103849]: Deploying daemon osd.5 on smithi115 2023-10-08T13:46:45.138 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:44 smithi115 ceph-mon[104408]: Deploying daemon osd.5 on smithi115 2023-10-08T13:46:45.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:45 smithi057 ceph-mon[99476]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:46.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:45 smithi099 ceph-mon[103849]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:46.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:45 smithi115 ceph-mon[104408]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:47.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:47 smithi057 ceph-mon[99476]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:47.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:47.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:47.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:48.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:47 smithi099 ceph-mon[103849]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:48.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:48.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:48.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:47 smithi115 ceph-mon[104408]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:48.183 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:49.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:49 smithi115 ceph-mon[104408]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:49.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:49.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:49.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:49.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:49.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:50.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:49 smithi099 ceph-mon[103849]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:50.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:50.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:50.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:50.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:50.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:50.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.050+0000 7fe0608b9700 1 -- 172.21.15.115:0/2663294028 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe05c073fd0 con 0x7fe0440a14d0 2023-10-08T13:46:50.052 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 5 on host 'smithi115' 2023-10-08T13:46:50.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 -- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0440a14d0 msgr2=0x7fe0440a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:50.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0440a14d0 0x7fe0440a3990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe04c005950 tx=0x7fe04c0058e0 comp rx=0 tx=0).stop 2023-10-08T13:46:50.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 -- 172.21.15.115:0/2663294028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe05c1008a0 msgr2=0x7fe05c072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:50.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe05c1008a0 0x7fe05c072af0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe05000d950 tx=0x7fe05000dd10 comp rx=0 tx=0).stop 2023-10-08T13:46:50.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 -- 172.21.15.115:0/2663294028 shutdown_connections 2023-10-08T13:46:50.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0440a14d0 0x7fe0440a3990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:50.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1067f0 0x7fe05c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:50.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe05c1008a0 0x7fe05c072af0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:50.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 --2- 172.21.15.115:0/2663294028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe05c101260 0x7fe05c073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:50.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 -- 172.21.15.115:0/2663294028 >> 172.21.15.115:0/2663294028 conn(0x7fe05c077910 msgr2=0x7fe05c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:46:50.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 -- 172.21.15.115:0/2663294028 shutdown_connections 2023-10-08T13:46:50.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:46:50.053+0000 7fe0618bb700 1 -- 172.21.15.115:0/2663294028 wait complete. 2023-10-08T13:46:50.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:49 smithi057 ceph-mon[99476]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:50.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:50.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:50.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:50.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:50.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:50.468 DEBUG:teuthology.orchestra.run.smithi115:osd.5> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.5.service 2023-10-08T13:46:50.470 INFO:tasks.cephadm:Deploying osd.6 on smithi057 with /dev/vg_nvme/lv_2... 2023-10-08T13:46:50.471 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-08T13:46:50.502 INFO:journalctl@ceph.osd.5.smithi115.stdout:-- Logs begin at Sun 2023-10-08 13:28:12 UTC. -- 2023-10-08T13:46:51.063 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:46:51.260 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:51 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:51.260 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:51 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:51.260 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:51 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:51.260 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:51 smithi115 ceph-mon[104408]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:51.302 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:51 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:51.303 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:51 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:51.303 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:51 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:51.303 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:51 smithi057 ceph-mon[99476]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:51 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:51 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:51 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:46:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:51 smithi099 ceph-mon[103849]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:51.555 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 13:46:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T13:46:51.257+0000 7f592b44d640 -1 osd.5 0 log_to_monitors true 2023-10-08T13:46:52.086 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:52 smithi057 ceph-mon[99476]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T13:46:52.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:52 smithi115 ceph-mon[104408]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T13:46:52.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:52 smithi099 ceph-mon[103849]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T13:46:52.563 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-08T13:46:52.563 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-08T13:46:52.563 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-08T13:46:52.563 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-08T13:46:52.563 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00686163 s, 1.5 GB/s 2023-10-08T13:46:52.563 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-08T13:46:53.025 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_2 2023-10-08T13:46:53.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:53 smithi057 ceph-mon[99476]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:53.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:53 smithi057 ceph-mon[99476]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-08T13:46:53.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:53 smithi057 ceph-mon[99476]: osdmap e36: 6 total, 5 up, 6 in 2023-10-08T13:46:53.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:53 smithi057 ceph-mon[99476]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:46:53.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:53.437 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 13:46:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T13:46:53.074+0000 7f5920ac4700 -1 osd.5 0 waiting for initial osdmap 2023-10-08T13:46:53.437 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 13:46:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T13:46:53.083+0000 7f59190b2700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:46:53.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:53 smithi115 ceph-mon[104408]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:53.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:53 smithi115 ceph-mon[104408]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-08T13:46:53.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:53 smithi115 ceph-mon[104408]: osdmap e36: 6 total, 5 up, 6 in 2023-10-08T13:46:53.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:53 smithi115 ceph-mon[104408]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:46:53.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:53 smithi099 ceph-mon[103849]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:53 smithi099 ceph-mon[103849]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-08T13:46:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:53 smithi099 ceph-mon[103849]: osdmap e36: 6 total, 5 up, 6 in 2023-10-08T13:46:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:53 smithi099 ceph-mon[103849]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:46:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:53.601 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:46:54.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:46:54.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: osdmap e37: 6 total, 5 up, 6 in 2023-10-08T13:46:54.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:54.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:54.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi115 2023-10-08T13:46:54.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:54.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:54.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi115 to 8584M 2023-10-08T13:46:54.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:54.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:54.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:54.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: osdmap e37: 6 total, 5 up, 6 in 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi115 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi115 to 8584M 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:54.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:54.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:54.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:46:54.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: osdmap e37: 6 total, 5 up, 6 in 2023-10-08T13:46:54.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:54.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:54.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi115 2023-10-08T13:46:54.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:54.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:46:54.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi115 to 8584M 2023-10-08T13:46:54.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:54.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:46:54.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:46:54.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.786+0000 7f0a64f4e700 1 -- 172.21.15.57:0/2538823155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a60105670 msgr2=0x7f0a60105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:54.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.786+0000 7f0a64f4e700 1 --2- 172.21.15.57:0/2538823155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a60105670 0x7f0a60105a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a500099c0 tx=0x7f0a50009cd0 comp rx=0 tx=0).stop 2023-10-08T13:46:54.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.786+0000 7f0a64f4e700 1 -- 172.21.15.57:0/2538823155 shutdown_connections 2023-10-08T13:46:54.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.786+0000 7f0a64f4e700 1 --2- 172.21.15.57:0/2538823155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a60069410 0x7f0a6010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:54.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.786+0000 7f0a64f4e700 1 --2- 172.21.15.57:0/2538823155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a60105670 0x7f0a60105a50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:54.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.786+0000 7f0a64f4e700 1 --2- 172.21.15.57:0/2538823155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a60068a50 0x7f0a60068ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:54.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.786+0000 7f0a64f4e700 1 -- 172.21.15.57:0/2538823155 >> 172.21.15.57:0/2538823155 conn(0x7f0a600766c0 msgr2=0x7f0a60076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:46:54.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.786+0000 7f0a64f4e700 1 -- 172.21.15.57:0/2538823155 shutdown_connections 2023-10-08T13:46:54.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.786+0000 7f0a64f4e700 1 -- 172.21.15.57:0/2538823155 wait complete. 2023-10-08T13:46:54.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.787+0000 7f0a64f4e700 1 Processor -- start 2023-10-08T13:46:54.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.787+0000 7f0a64f4e700 1 -- start start 2023-10-08T13:46:54.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.787+0000 7f0a64f4e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a60068a50 0x7f0a6019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:54.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.787+0000 7f0a64f4e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a60069410 0x7f0a6019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:54.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.787+0000 7f0a64f4e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a60105670 0x7f0a601969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:54.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.787+0000 7f0a64f4e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a60197050 con 0x7f0a60069410 2023-10-08T13:46:54.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.787+0000 7f0a64f4e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a601971c0 con 0x7f0a60068a50 2023-10-08T13:46:54.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.787+0000 7f0a64f4e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a60197380 con 0x7f0a60105670 2023-10-08T13:46:54.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a60069410 0x7f0a6019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:54.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a60069410 0x7f0a6019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53254/0 (socket says 172.21.15.57:53254) 2023-10-08T13:46:54.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a57fff700 1 -- 172.21.15.57:0/4014369771 learned_addr learned my addr 172.21.15.57:0/4014369771 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:46:54.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a5e59c700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a60068a50 0x7f0a6019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:54.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a5ed9d700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a60105670 0x7f0a601969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:54.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a57fff700 1 -- 172.21.15.57:0/4014369771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a60105670 msgr2=0x7f0a601969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a57fff700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a60105670 0x7f0a601969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a57fff700 1 -- 172.21.15.57:0/4014369771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a60068a50 msgr2=0x7f0a6019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a57fff700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a60068a50 0x7f0a6019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a57fff700 1 -- 172.21.15.57:0/4014369771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a500095f0 con 0x7f0a60069410 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a57fff700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a60069410 0x7f0a6019cdf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800c920 tx=0x7f0a4800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a577fe700 1 -- 172.21.15.57:0/4014369771 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48004a30 con 0x7f0a60069410 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a577fe700 1 -- 172.21.15.57:0/4014369771 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a480054e0 con 0x7f0a60069410 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a64f4e700 1 -- 172.21.15.57:0/4014369771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a60197660 con 0x7f0a60069410 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a577fe700 1 -- 172.21.15.57:0/4014369771 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48007bd0 con 0x7f0a60069410 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.788+0000 7f0a64f4e700 1 -- 172.21.15.57:0/4014369771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a601a38a0 con 0x7f0a60069410 2023-10-08T13:46:54.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.789+0000 7f0a64f4e700 1 -- 172.21.15.57:0/4014369771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a60108d30 con 0x7f0a60069410 2023-10-08T13:46:54.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.790+0000 7f0a577fe700 1 -- 172.21.15.57:0/4014369771 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f0a48007d50 con 0x7f0a60069410 2023-10-08T13:46:54.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.791+0000 7f0a577fe700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a4c0a1550 0x7f0a4c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:46:54.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.791+0000 7f0a5e59c700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a4c0a1550 0x7f0a4c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:46:54.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.791+0000 7f0a577fe700 1 -- 172.21.15.57:0/4014369771 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4781+0+0 (secure 0 0 0) 0x7f0a480cbe20 con 0x7f0a60069410 2023-10-08T13:46:54.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.791+0000 7f0a5e59c700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a4c0a1550 0x7f0a4c0a3a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009990 tx=0x7f0a50003680 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:46:54.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:54.795+0000 7f0a577fe700 1 -- 172.21.15.57:0/4014369771 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0a480962d0 con 0x7f0a60069410 2023-10-08T13:46:55.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:46:55.005+0000 7f0a64f4e700 1 -- 172.21.15.57:0/4014369771 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a60197f20 con 0x7f0a4c0a1550 2023-10-08T13:46:55.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:55 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:46:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:55 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:46:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:55 smithi115 ceph-mon[104408]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:55 smithi115 ceph-mon[104408]: osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] boot 2023-10-08T13:46:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:55 smithi115 ceph-mon[104408]: osdmap e38: 6 total, 6 up, 6 in 2023-10-08T13:46:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:46:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:46:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:55.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:55 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:46:55.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:55 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:46:55.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:55 smithi057 ceph-mon[99476]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:55.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:55 smithi057 ceph-mon[99476]: osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] boot 2023-10-08T13:46:55.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:55 smithi057 ceph-mon[99476]: osdmap e38: 6 total, 6 up, 6 in 2023-10-08T13:46:55.591 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:55.592 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:46:55.592 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:46:55.592 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:55.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:55 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:46:55.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:55 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:46:55.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:55 smithi099 ceph-mon[103849]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-08T13:46:55.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:55 smithi099 ceph-mon[103849]: osd.5 [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] boot 2023-10-08T13:46:55.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:55 smithi099 ceph-mon[103849]: osdmap e38: 6 total, 6 up, 6 in 2023-10-08T13:46:55.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T13:46:55.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:46:55.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:46:55.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:46:56.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:56 smithi115 ceph-mon[104408]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:46:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:56 smithi115 ceph-mon[104408]: osdmap e39: 6 total, 6 up, 6 in 2023-10-08T13:46:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:56 smithi115 ceph-mon[104408]: osdmap e40: 6 total, 6 up, 6 in 2023-10-08T13:46:56.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:56 smithi057 ceph-mon[99476]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:46:56.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:56 smithi057 ceph-mon[99476]: osdmap e39: 6 total, 6 up, 6 in 2023-10-08T13:46:56.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:56 smithi057 ceph-mon[99476]: osdmap e40: 6 total, 6 up, 6 in 2023-10-08T13:46:56.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:56 smithi099 ceph-mon[103849]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:46:56.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:56 smithi099 ceph-mon[103849]: osdmap e39: 6 total, 6 up, 6 in 2023-10-08T13:46:56.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:56 smithi099 ceph-mon[103849]: osdmap e40: 6 total, 6 up, 6 in 2023-10-08T13:46:57.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:57 smithi115 ceph-mon[104408]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:46:57.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:57 smithi057 ceph-mon[99476]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:46:57.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:57 smithi099 ceph-mon[103849]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:46:58.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:58 smithi115 ceph-mon[104408]: osdmap e41: 6 total, 6 up, 6 in 2023-10-08T13:46:58.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:58 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1006710140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2240ad-a8f2-49ed-bf62-8fe8eff60156"}]: dispatch 2023-10-08T13:46:58.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:58 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1006710140' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd2240ad-a8f2-49ed-bf62-8fe8eff60156"}]': finished 2023-10-08T13:46:58.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:58 smithi115 ceph-mon[104408]: osdmap e42: 7 total, 6 up, 7 in 2023-10-08T13:46:58.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:46:58.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:58 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1483769536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:46:58.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:58 smithi057 ceph-mon[99476]: osdmap e41: 6 total, 6 up, 6 in 2023-10-08T13:46:58.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:58 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1006710140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2240ad-a8f2-49ed-bf62-8fe8eff60156"}]: dispatch 2023-10-08T13:46:58.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:58 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1006710140' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd2240ad-a8f2-49ed-bf62-8fe8eff60156"}]': finished 2023-10-08T13:46:58.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:58 smithi057 ceph-mon[99476]: osdmap e42: 7 total, 6 up, 7 in 2023-10-08T13:46:58.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:46:58.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:58 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1483769536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:46:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:58 smithi099 ceph-mon[103849]: osdmap e41: 6 total, 6 up, 6 in 2023-10-08T13:46:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:58 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1006710140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2240ad-a8f2-49ed-bf62-8fe8eff60156"}]: dispatch 2023-10-08T13:46:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:58 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1006710140' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd2240ad-a8f2-49ed-bf62-8fe8eff60156"}]': finished 2023-10-08T13:46:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:58 smithi099 ceph-mon[103849]: osdmap e42: 7 total, 6 up, 7 in 2023-10-08T13:46:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:46:58.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:58 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1483769536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:46:59.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:46:59 smithi115 ceph-mon[104408]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:46:59.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:46:59 smithi057 ceph-mon[99476]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:47:00.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:46:59 smithi099 ceph-mon[103849]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:47:01.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:01 smithi115 ceph-mon[104408]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-10-08T13:47:01.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:01 smithi057 ceph-mon[99476]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-10-08T13:47:02.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:01 smithi099 ceph-mon[103849]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-10-08T13:47:03.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:03 smithi115 ceph-mon[104408]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:47:03.956 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:03 smithi057 ceph-mon[99476]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:47:04.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:03 smithi099 ceph-mon[103849]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:47:04.948 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:04 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-08T13:47:04.948 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:04 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:04 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-08T13:47:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:04 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:05.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:04 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-08T13:47:05.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:04 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:05.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:05 smithi057 ceph-mon[99476]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T13:47:05.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:05 smithi057 ceph-mon[99476]: Deploying daemon osd.6 on smithi057 2023-10-08T13:47:06.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:05 smithi099 ceph-mon[103849]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T13:47:06.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:05 smithi099 ceph-mon[103849]: Deploying daemon osd.6 on smithi057 2023-10-08T13:47:06.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:05 smithi115 ceph-mon[104408]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T13:47:06.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:05 smithi115 ceph-mon[104408]: Deploying daemon osd.6 on smithi057 2023-10-08T13:47:07.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:07 smithi057 ceph-mon[99476]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-08T13:47:08.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:07 smithi099 ceph-mon[103849]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-08T13:47:08.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:07 smithi115 ceph-mon[104408]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-08T13:47:09.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:09 smithi057 ceph-mon[99476]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-08T13:47:09.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:09 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:09 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:09.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:09 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:09 smithi099 ceph-mon[103849]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-08T13:47:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:09 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:09 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:09 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:10.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:09 smithi115 ceph-mon[104408]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-08T13:47:10.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:09 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:10.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:09 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:10.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:09 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:11.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.253+0000 7f0a577fe700 1 -- 172.21.15.57:0/4014369771 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0a60197f20 con 0x7f0a4c0a1550 2023-10-08T13:47:11.255 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 6 on host 'smithi057' 2023-10-08T13:47:11.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 -- 172.21.15.57:0/4014369771 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a4c0a1550 msgr2=0x7f0a4c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:11.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a4c0a1550 0x7f0a4c0a3a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009990 tx=0x7f0a50003680 comp rx=0 tx=0).stop 2023-10-08T13:47:11.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 -- 172.21.15.57:0/4014369771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a60069410 msgr2=0x7f0a6019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:11.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a60069410 0x7f0a6019cdf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800c920 tx=0x7f0a4800cc30 comp rx=0 tx=0).stop 2023-10-08T13:47:11.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 -- 172.21.15.57:0/4014369771 shutdown_connections 2023-10-08T13:47:11.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a60105670 0x7f0a601969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:11.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a4c0a1550 0x7f0a4c0a3a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:11.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a60068a50 0x7f0a6019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:11.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 --2- 172.21.15.57:0/4014369771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a60069410 0x7f0a6019cdf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:11.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 -- 172.21.15.57:0/4014369771 >> 172.21.15.57:0/4014369771 conn(0x7f0a600766c0 msgr2=0x7f0a600fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:47:11.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 -- 172.21.15.57:0/4014369771 shutdown_connections 2023-10-08T13:47:11.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:11.256+0000 7f0a64f4e700 1 -- 172.21.15.57:0/4014369771 wait complete. 2023-10-08T13:47:11.582 DEBUG:teuthology.orchestra.run.smithi057:osd.6> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.6.service 2023-10-08T13:47:11.584 INFO:tasks.cephadm:Deploying osd.7 on smithi099 with /dev/vg_nvme/lv_2... 2023-10-08T13:47:11.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-08T13:47:11.614 INFO:journalctl@ceph.osd.6.smithi057.stdout:-- Logs begin at Sun 2023-10-08 13:28:06 UTC. -- 2023-10-08T13:47:11.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:11 smithi057 ceph-mon[99476]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T13:47:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:11 smithi099 ceph-mon[103849]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T13:47:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:12.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:12.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:12.158 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T13:47:12.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:11 smithi115 ceph-mon[104408]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T13:47:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:12.706 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 13:47:12 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T13:47:12.592+0000 7f4fff81c640 -1 osd.6 0 log_to_monitors true 2023-10-08T13:47:12.965 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:12 smithi057 ceph-mon[99476]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-08T13:47:13.021 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:12 smithi099 ceph-mon[103849]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-08T13:47:13.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:12 smithi115 ceph-mon[104408]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-08T13:47:13.811 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-08T13:47:13.811 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-08T13:47:13.811 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-10-08T13:47:13.811 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-10-08T13:47:13.811 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112548 s, 932 MB/s 2023-10-08T13:47:13.811 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-10-08T13:47:13.988 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 13:47:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T13:47:13.720+0000 7f4ff4e93700 -1 osd.6 0 waiting for initial osdmap 2023-10-08T13:47:13.988 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 13:47:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T13:47:13.732+0000 7f4fecc80700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:47:13.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:13 smithi057 ceph-mon[99476]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:47:13.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:13 smithi057 ceph-mon[99476]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-08T13:47:13.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:13 smithi057 ceph-mon[99476]: osdmap e43: 7 total, 6 up, 7 in 2023-10-08T13:47:13.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:13 smithi057 ceph-mon[99476]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:47:13.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:13 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:14.006 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:13 smithi099 ceph-mon[103849]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:47:14.006 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:13 smithi099 ceph-mon[103849]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-08T13:47:14.006 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:13 smithi099 ceph-mon[103849]: osdmap e43: 7 total, 6 up, 7 in 2023-10-08T13:47:14.006 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:13 smithi099 ceph-mon[103849]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:47:14.006 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:13 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:14.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:13 smithi115 ceph-mon[104408]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:47:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:13 smithi115 ceph-mon[104408]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-08T13:47:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:13 smithi115 ceph-mon[104408]: osdmap e43: 7 total, 6 up, 7 in 2023-10-08T13:47:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:13 smithi115 ceph-mon[104408]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:47:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:13 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:14.304 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2023-10-08T13:47:14.861 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T13:47:14.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:47:14.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: osdmap e44: 7 total, 6 up, 7 in 2023-10-08T13:47:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:14.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:47:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: osdmap e44: 7 total, 6 up, 7 in 2023-10-08T13:47:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:15.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:15.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:15.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:15.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:15.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:47:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: osdmap e44: 7 total, 6 up, 7 in 2023-10-08T13:47:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:15.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:15.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:15.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:15.985 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:15 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:47:15.985 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:15 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:47:15.985 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:15 smithi099 ceph-mon[103849]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:47:15.985 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:15 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi057 2023-10-08T13:47:15.985 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:15 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi057 to 5724M 2023-10-08T13:47:15.986 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:15.986 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:15 smithi099 ceph-mon[103849]: osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] boot 2023-10-08T13:47:15.986 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:15 smithi099 ceph-mon[103849]: osdmap e45: 7 total, 7 up, 7 in 2023-10-08T13:47:15.986 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:15.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:15 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:47:15.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:15 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:47:15.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:15 smithi057 ceph-mon[99476]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:47:15.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:15 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi057 2023-10-08T13:47:15.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:15 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi057 to 5724M 2023-10-08T13:47:15.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:15.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:15 smithi057 ceph-mon[99476]: osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] boot 2023-10-08T13:47:15.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:15 smithi057 ceph-mon[99476]: osdmap e45: 7 total, 7 up, 7 in 2023-10-08T13:47:15.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:16.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.153+0000 7f5e1914a700 1 -- 172.21.15.99:0/1423910575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e14104580 msgr2=0x7f5e14104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:16.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.153+0000 7f5e1914a700 1 --2- 172.21.15.99:0/1423910575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e14104580 0x7f5e14104960 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009990 tx=0x7f5e08009ca0 comp rx=0 tx=0).stop 2023-10-08T13:47:16.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.153+0000 7f5e1914a700 1 -- 172.21.15.99:0/1423910575 shutdown_connections 2023-10-08T13:47:16.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.153+0000 7f5e1914a700 1 --2- 172.21.15.99:0/1423910575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e14100070 0x7f5e1410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:16.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.153+0000 7f5e1914a700 1 --2- 172.21.15.99:0/1423910575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e14104580 0x7f5e14104960 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:16.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.153+0000 7f5e1914a700 1 --2- 172.21.15.99:0/1423910575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e140ff6b0 0x7f5e140ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:16.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.153+0000 7f5e1914a700 1 -- 172.21.15.99:0/1423910575 >> 172.21.15.99:0/1423910575 conn(0x7f5e140767f0 msgr2=0x7f5e14076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:47:16.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.153+0000 7f5e1914a700 1 -- 172.21.15.99:0/1423910575 shutdown_connections 2023-10-08T13:47:16.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.153+0000 7f5e1914a700 1 -- 172.21.15.99:0/1423910575 wait complete. 2023-10-08T13:47:16.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e1914a700 1 Processor -- start 2023-10-08T13:47:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e1914a700 1 -- start start 2023-10-08T13:47:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e1914a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e140ff6b0 0x7f5e14195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e1914a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e14100070 0x7f5e14195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e1914a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e14104580 0x7f5e1419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e1914a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1419a930 con 0x7f5e14100070 2023-10-08T13:47:16.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e1914a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1419aaa0 con 0x7f5e140ff6b0 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e1914a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1419ac60 con 0x7f5e14104580 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e12d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e140ff6b0 0x7f5e14195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e12d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e140ff6b0 0x7f5e14195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42650/0 (socket says 172.21.15.99:42650) 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.154+0000 7f5e12d9d700 1 -- 172.21.15.99:0/1048254374 learned_addr learned my addr 172.21.15.99:0/1048254374 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5e1359e700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e14104580 0x7f5e1419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5e1259c700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e14100070 0x7f5e14195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5e12d9d700 1 -- 172.21.15.99:0/1048254374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e14104580 msgr2=0x7f5e1419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5e12d9d700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e14104580 0x7f5e1419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5e12d9d700 1 -- 172.21.15.99:0/1048254374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e14100070 msgr2=0x7f5e14195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5e12d9d700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e14100070 0x7f5e14195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:16.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5e12d9d700 1 -- 172.21.15.99:0/1048254374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e080095f0 con 0x7f5e140ff6b0 2023-10-08T13:47:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5e12d9d700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e140ff6b0 0x7f5e14195a40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5e080098e0 tx=0x7f5e0800f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:47:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5dfbfff700 1 -- 172.21.15.99:0/1048254374 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0801d070 con 0x7f5e140ff6b0 2023-10-08T13:47:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5dfbfff700 1 -- 172.21.15.99:0/1048254374 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e08022470 con 0x7f5e140ff6b0 2023-10-08T13:47:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5e1914a700 1 -- 172.21.15.99:0/1048254374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1419aee0 con 0x7f5e140ff6b0 2023-10-08T13:47:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5dfbfff700 1 -- 172.21.15.99:0/1048254374 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08017650 con 0x7f5e140ff6b0 2023-10-08T13:47:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.155+0000 7f5e1914a700 1 -- 172.21.15.99:0/1048254374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e141a27e0 con 0x7f5e140ff6b0 2023-10-08T13:47:16.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.156+0000 7f5e1914a700 1 -- 172.21.15.99:0/1048254374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e14100da0 con 0x7f5e140ff6b0 2023-10-08T13:47:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.157+0000 7f5dfbfff700 1 -- 172.21.15.99:0/1048254374 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f5e080225e0 con 0x7f5e140ff6b0 2023-10-08T13:47:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.157+0000 7f5dfbfff700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5dfc0a1550 0x7f5dfc0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.158+0000 7f5dfbfff700 1 -- 172.21.15.99:0/1048254374 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5180+0+0 (secure 0 0 0) 0x7f5e080cd140 con 0x7f5e140ff6b0 2023-10-08T13:47:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.158+0000 7f5e1259c700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5dfc0a1550 0x7f5dfc0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:16.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.158+0000 7f5e1259c700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5dfc0a1550 0x7f5dfc0a3a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e04005950 tx=0x7f5e040058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:47:16.163 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.162+0000 7f5dfbfff700 1 -- 172.21.15.99:0/1048254374 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5e08097470 con 0x7f5e140ff6b0 2023-10-08T13:47:16.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:15 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:47:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:15 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:47:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:15 smithi115 ceph-mon[104408]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-08T13:47:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:15 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi057 2023-10-08T13:47:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:15 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi057 to 5724M 2023-10-08T13:47:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:15 smithi115 ceph-mon[104408]: osd.6 [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] boot 2023-10-08T13:47:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:15 smithi115 ceph-mon[104408]: osdmap e45: 7 total, 7 up, 7 in 2023-10-08T13:47:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T13:47:16.360 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:16.359+0000 7f5e1914a700 1 -- 172.21.15.99:0/1048254374 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e14196dc0 con 0x7f5dfc0a1550 2023-10-08T13:47:17.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:16 smithi099 ceph-mon[103849]: osdmap e46: 7 total, 7 up, 7 in 2023-10-08T13:47:17.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:47:17.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:47:17.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:17.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:16 smithi115 ceph-mon[104408]: osdmap e46: 7 total, 7 up, 7 in 2023-10-08T13:47:17.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:47:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:47:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:17.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:16 smithi057 ceph-mon[99476]: osdmap e46: 7 total, 7 up, 7 in 2023-10-08T13:47:17.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:47:17.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:47:17.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:17.891 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:17 smithi099 ceph-mon[103849]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:17.891 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:17 smithi099 ceph-mon[103849]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:47:18.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:17 smithi115 ceph-mon[104408]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:18.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:17 smithi115 ceph-mon[104408]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:47:18.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:17 smithi057 ceph-mon[99476]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:18.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:17 smithi057 ceph-mon[99476]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:47:20.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:19 smithi099 ceph-mon[103849]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:20.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:19 smithi099 ceph-mon[103849]: from='client.? 172.21.15.99:0/2771532880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bc31070-8d5a-4a0c-b14f-5fa855e951de"}]: dispatch 2023-10-08T13:47:20.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:19 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bc31070-8d5a-4a0c-b14f-5fa855e951de"}]: dispatch 2023-10-08T13:47:20.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:19 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bc31070-8d5a-4a0c-b14f-5fa855e951de"}]': finished 2023-10-08T13:47:20.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:19 smithi099 ceph-mon[103849]: osdmap e47: 8 total, 7 up, 8 in 2023-10-08T13:47:20.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:20.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:19 smithi115 ceph-mon[104408]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:19 smithi115 ceph-mon[104408]: from='client.? 172.21.15.99:0/2771532880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bc31070-8d5a-4a0c-b14f-5fa855e951de"}]: dispatch 2023-10-08T13:47:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:19 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bc31070-8d5a-4a0c-b14f-5fa855e951de"}]: dispatch 2023-10-08T13:47:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:19 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bc31070-8d5a-4a0c-b14f-5fa855e951de"}]': finished 2023-10-08T13:47:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:19 smithi115 ceph-mon[104408]: osdmap e47: 8 total, 7 up, 8 in 2023-10-08T13:47:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:20.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:19 smithi057 ceph-mon[99476]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:20.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:19 smithi057 ceph-mon[99476]: from='client.? 172.21.15.99:0/2771532880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bc31070-8d5a-4a0c-b14f-5fa855e951de"}]: dispatch 2023-10-08T13:47:20.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:19 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bc31070-8d5a-4a0c-b14f-5fa855e951de"}]: dispatch 2023-10-08T13:47:20.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:19 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bc31070-8d5a-4a0c-b14f-5fa855e951de"}]': finished 2023-10-08T13:47:20.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:19 smithi057 ceph-mon[99476]: osdmap e47: 8 total, 7 up, 8 in 2023-10-08T13:47:20.439 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:21.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:20 smithi099 ceph-mon[103849]: from='client.? 172.21.15.99:0/3750239523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:47:21.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:20 smithi115 ceph-mon[104408]: from='client.? 172.21.15.99:0/3750239523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:47:21.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:20 smithi057 ceph-mon[99476]: from='client.? 172.21.15.99:0/3750239523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:47:22.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:21 smithi115 ceph-mon[104408]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:22.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:21 smithi057 ceph-mon[99476]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:22.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:21 smithi099 ceph-mon[103849]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:24.163 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:23 smithi099 ceph-mon[103849]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:24.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:23 smithi115 ceph-mon[104408]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:24.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:23 smithi057 ceph-mon[99476]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:26.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:25 smithi115 ceph-mon[104408]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:26.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:25 smithi057 ceph-mon[99476]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:25 smithi099 ceph-mon[103849]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:27.006 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-08T13:47:27.006 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:27.006 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:26 smithi099 ceph-mon[103849]: Deploying daemon osd.7 on smithi099 2023-10-08T13:47:27.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-08T13:47:27.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:26 smithi115 ceph-mon[104408]: Deploying daemon osd.7 on smithi099 2023-10-08T13:47:27.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-08T13:47:27.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:27.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:26 smithi057 ceph-mon[99476]: Deploying daemon osd.7 on smithi099 2023-10-08T13:47:28.009 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:27 smithi099 ceph-mon[103849]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:28.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:27 smithi115 ceph-mon[104408]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:28.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:27 smithi057 ceph-mon[99476]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:30.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:29 smithi115 ceph-mon[104408]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:30.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:29 smithi057 ceph-mon[99476]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:30.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:29 smithi099 ceph-mon[103849]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:32.104 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:31 smithi099 ceph-mon[103849]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:32.105 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:31 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:32.105 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:31 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:32.105 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:31 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:32.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:31 smithi115 ceph-mon[104408]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:32.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:31 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:32.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:31 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:32.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:31 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:31 smithi057 ceph-mon[99476]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:31 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:31 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:31 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:33.787 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 7 on host 'smithi099' 2023-10-08T13:47:33.787 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.785+0000 7f5dfbfff700 1 -- 172.21.15.99:0/1048254374 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5e14196dc0 con 0x7f5dfc0a1550 2023-10-08T13:47:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 -- 172.21.15.99:0/1048254374 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5dfc0a1550 msgr2=0x7f5dfc0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5dfc0a1550 0x7f5dfc0a3a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e04005950 tx=0x7f5e040058e0 comp rx=0 tx=0).stop 2023-10-08T13:47:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 -- 172.21.15.99:0/1048254374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e140ff6b0 msgr2=0x7f5e14195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e140ff6b0 0x7f5e14195a40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5e080098e0 tx=0x7f5e0800f670 comp rx=0 tx=0).stop 2023-10-08T13:47:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 -- 172.21.15.99:0/1048254374 shutdown_connections 2023-10-08T13:47:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e14104580 0x7f5e1419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5dfc0a1550 0x7f5dfc0a3a10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:33.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e140ff6b0 0x7f5e14195a40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:33.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 --2- 172.21.15.99:0/1048254374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e14100070 0x7f5e14195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:33.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 -- 172.21.15.99:0/1048254374 >> 172.21.15.99:0/1048254374 conn(0x7f5e140767f0 msgr2=0x7f5e140fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:47:33.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 -- 172.21.15.99:0/1048254374 shutdown_connections 2023-10-08T13:47:33.790 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:47:33.788+0000 7f5e1914a700 1 -- 172.21.15.99:0/1048254374 wait complete. 2023-10-08T13:47:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:33 smithi099 ceph-mon[103849]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:33 smithi115 ceph-mon[104408]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:34.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:33 smithi057 ceph-mon[99476]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:34.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:34.253 DEBUG:teuthology.orchestra.run.smithi099:osd.7> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.7.service 2023-10-08T13:47:34.255 INFO:tasks.cephadm:Deploying osd.8 on smithi115 with /dev/vg_nvme/lv_2... 2023-10-08T13:47:34.256 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-08T13:47:34.288 INFO:journalctl@ceph.osd.7.smithi099.stdout:-- Logs begin at Sun 2023-10-08 13:27:56 UTC. -- 2023-10-08T13:47:34.825 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:47:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:34 smithi099 ceph-mon[103849]: from='osd.7 [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T13:47:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:34 smithi099 ceph-mon[103849]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T13:47:35.053 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 13:47:34 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T13:47:34.897+0000 7f4eb3c8c640 -1 osd.7 0 log_to_monitors true 2023-10-08T13:47:35.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:34 smithi115 ceph-mon[104408]: from='osd.7 [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T13:47:35.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:34 smithi115 ceph-mon[104408]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T13:47:35.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:34 smithi057 ceph-mon[99476]: from='osd.7 [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T13:47:35.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:34 smithi057 ceph-mon[99476]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T13:47:36.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 13:47:35 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T13:47:35.922+0000 7f4ea9303700 -1 osd.7 0 waiting for initial osdmap 2023-10-08T13:47:36.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 13:47:35 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T13:47:35.930+0000 7f4ea48f7700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:47:36.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:35 smithi099 ceph-mon[103849]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:36.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:35 smithi099 ceph-mon[103849]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-08T13:47:36.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:35 smithi099 ceph-mon[103849]: from='osd.7 [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:47:36.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:35 smithi099 ceph-mon[103849]: osdmap e48: 8 total, 7 up, 8 in 2023-10-08T13:47:36.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:36.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:35 smithi099 ceph-mon[103849]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:47:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:35 smithi115 ceph-mon[104408]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:35 smithi115 ceph-mon[104408]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-08T13:47:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:35 smithi115 ceph-mon[104408]: from='osd.7 [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:47:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:35 smithi115 ceph-mon[104408]: osdmap e48: 8 total, 7 up, 8 in 2023-10-08T13:47:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:36.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:35 smithi115 ceph-mon[104408]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:47:36.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:35 smithi057 ceph-mon[99476]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:36.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:35 smithi057 ceph-mon[99476]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-08T13:47:36.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:35 smithi057 ceph-mon[99476]: from='osd.7 [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:47:36.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:35 smithi057 ceph-mon[99476]: osdmap e48: 8 total, 7 up, 8 in 2023-10-08T13:47:36.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:36.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:35 smithi057 ceph-mon[99476]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:47:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-08T13:47:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-08T13:47:36.312 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2023-10-08T13:47:36.312 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2023-10-08T13:47:36.312 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0108944 s, 962 MB/s 2023-10-08T13:47:36.312 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2023-10-08T13:47:36.782 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2023-10-08T13:47:37.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:36 smithi115 ceph-mon[104408]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:47:37.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:36 smithi115 ceph-mon[104408]: osdmap e49: 8 total, 7 up, 8 in 2023-10-08T13:47:37.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:36 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:37.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:36 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:37.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:36 smithi057 ceph-mon[99476]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:47:37.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:36 smithi057 ceph-mon[99476]: osdmap e49: 8 total, 7 up, 8 in 2023-10-08T13:47:37.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:36 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:37.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:36 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:37.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:36 smithi099 ceph-mon[103849]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:47:37.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:36 smithi099 ceph-mon[103849]: osdmap e49: 8 total, 7 up, 8 in 2023-10-08T13:47:37.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:36 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:37.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:36 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:37.338 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:47:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:47:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:47:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: osd.7 [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] boot 2023-10-08T13:47:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: osdmap e50: 8 total, 8 up, 8 in 2023-10-08T13:47:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:37 smithi115 ceph-mon[104408]: osdmap e51: 8 total, 8 up, 8 in 2023-10-08T13:47:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:47:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: osd.7 [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] boot 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: osdmap e50: 8 total, 8 up, 8 in 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:38.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:38.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:38.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:38.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:38.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:38.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:37 smithi057 ceph-mon[99476]: osdmap e51: 8 total, 8 up, 8 in 2023-10-08T13:47:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:47:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:47:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-08T13:47:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: osd.7 [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] boot 2023-10-08T13:47:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: osdmap e50: 8 total, 8 up, 8 in 2023-10-08T13:47:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:38.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:38.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:37 smithi099 ceph-mon[103849]: osdmap e51: 8 total, 8 up, 8 in 2023-10-08T13:47:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.513+0000 7fcb5948d700 1 -- 172.21.15.115:0/2551964084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb541003b0 msgr2=0x7fcb5410c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.513+0000 7fcb5948d700 1 --2- 172.21.15.115:0/2551964084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb541003b0 0x7fcb5410c510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00b0a0 tx=0x7fcb4c00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:47:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.513+0000 7fcb5948d700 1 -- 172.21.15.115:0/2551964084 shutdown_connections 2023-10-08T13:47:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.513+0000 7fcb5948d700 1 --2- 172.21.15.115:0/2551964084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb541003b0 0x7fcb5410c510 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.513+0000 7fcb5948d700 1 --2- 172.21.15.115:0/2551964084 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb54104680 0x7fcb54104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.513+0000 7fcb5948d700 1 --2- 172.21.15.115:0/2551964084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb540ff9f0 0x7fcb540ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.513+0000 7fcb5948d700 1 -- 172.21.15.115:0/2551964084 >> 172.21.15.115:0/2551964084 conn(0x7fcb54076510 msgr2=0x7fcb54076920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:47:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.513+0000 7fcb5948d700 1 -- 172.21.15.115:0/2551964084 shutdown_connections 2023-10-08T13:47:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.513+0000 7fcb5948d700 1 -- 172.21.15.115:0/2551964084 wait complete. 2023-10-08T13:47:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.514+0000 7fcb5948d700 1 Processor -- start 2023-10-08T13:47:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.514+0000 7fcb5948d700 1 -- start start 2023-10-08T13:47:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.514+0000 7fcb5948d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb540ff9f0 0x7fcb54197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.514+0000 7fcb5948d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb541003b0 0x7fcb54198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.514+0000 7fcb5948d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb54104680 0x7fcb54198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.514+0000 7fcb5948d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541a2e60 con 0x7fcb54104680 2023-10-08T13:47:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.514+0000 7fcb5948d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541a2fd0 con 0x7fcb540ff9f0 2023-10-08T13:47:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.514+0000 7fcb5948d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541a3190 con 0x7fcb541003b0 2023-10-08T13:47:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb527fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb541003b0 0x7fcb54198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb527fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb541003b0 0x7fcb54198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52702/0 (socket says 172.21.15.115:52702) 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb527fc700 1 -- 172.21.15.115:0/831808263 learned_addr learned my addr 172.21.15.115:0/831808263 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb537fe700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb54104680 0x7fcb54198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb52ffd700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb540ff9f0 0x7fcb54197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb527fc700 1 -- 172.21.15.115:0/831808263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb540ff9f0 msgr2=0x7fcb54197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb527fc700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb540ff9f0 0x7fcb54197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb527fc700 1 -- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb54104680 msgr2=0x7fcb54198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb527fc700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb54104680 0x7fcb54198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb527fc700 1 -- 172.21.15.115:0/831808263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c009d00 con 0x7fcb541003b0 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb52ffd700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb540ff9f0 0x7fcb54197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb537fe700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb54104680 0x7fcb54198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:47:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb527fc700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb541003b0 0x7fcb54198090 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800d950 tx=0x7fcb4800dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:47:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb3bfff700 1 -- 172.21.15.115:0/831808263 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48009b40 con 0x7fcb541003b0 2023-10-08T13:47:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb5948d700 1 -- 172.21.15.115:0/831808263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb541a3470 con 0x7fcb541003b0 2023-10-08T13:47:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb3bfff700 1 -- 172.21.15.115:0/831808263 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb48009ca0 con 0x7fcb541003b0 2023-10-08T13:47:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb3bfff700 1 -- 172.21.15.115:0/831808263 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800f620 con 0x7fcb541003b0 2023-10-08T13:47:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.515+0000 7fcb5948d700 1 -- 172.21.15.115:0/831808263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb541a3970 con 0x7fcb541003b0 2023-10-08T13:47:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.516+0000 7fcb5948d700 1 -- 172.21.15.115:0/831808263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb54107bd0 con 0x7fcb541003b0 2023-10-08T13:47:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.517+0000 7fcb3bfff700 1 -- 172.21.15.115:0/831808263 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fcb48010460 con 0x7fcb541003b0 2023-10-08T13:47:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.518+0000 7fcb3bfff700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb3c0a14d0 0x7fcb3c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.518+0000 7fcb3bfff700 1 -- 172.21.15.115:0/831808263 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5641+0+0 (secure 0 0 0) 0x7fcb480cc6b0 con 0x7fcb541003b0 2023-10-08T13:47:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.518+0000 7fcb52ffd700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb3c0a14d0 0x7fcb3c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.519+0000 7fcb52ffd700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb3c0a14d0 0x7fcb3c0a3990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcb44007900 tx=0x7fcb44008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:47:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.521+0000 7fcb3bfff700 1 -- 172.21.15.115:0/831808263 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcb48096780 con 0x7fcb541003b0 2023-10-08T13:47:38.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:38.722+0000 7fcb5948d700 1 -- 172.21.15.115:0/831808263 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb54069410 con 0x7fcb3c0a14d0 2023-10-08T13:47:39.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:38 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi099 2023-10-08T13:47:39.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:38 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi099 to 5722M 2023-10-08T13:47:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:38 smithi057 ceph-mon[99476]: osdmap e52: 8 total, 8 up, 8 in 2023-10-08T13:47:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:38 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:47:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:38 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:47:39.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:38 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:39.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:38 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi099 2023-10-08T13:47:39.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:38 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi099 to 5722M 2023-10-08T13:47:39.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:38 smithi099 ceph-mon[103849]: osdmap e52: 8 total, 8 up, 8 in 2023-10-08T13:47:39.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:38 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:47:39.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:38 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:47:39.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:38 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:39.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:38 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi099 2023-10-08T13:47:39.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:38 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi099 to 5722M 2023-10-08T13:47:39.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:38 smithi115 ceph-mon[104408]: osdmap e52: 8 total, 8 up, 8 in 2023-10-08T13:47:39.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:38 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:47:39.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:38 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:47:39.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:38 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:39 smithi115 ceph-mon[104408]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:39 smithi115 ceph-mon[104408]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:47:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:39 smithi115 ceph-mon[104408]: osdmap e53: 8 total, 8 up, 8 in 2023-10-08T13:47:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:39 smithi057 ceph-mon[99476]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:39 smithi057 ceph-mon[99476]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:47:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:39 smithi057 ceph-mon[99476]: osdmap e53: 8 total, 8 up, 8 in 2023-10-08T13:47:40.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:39 smithi099 ceph-mon[103849]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:40.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:39 smithi099 ceph-mon[103849]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:47:40.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:39 smithi099 ceph-mon[103849]: osdmap e53: 8 total, 8 up, 8 in 2023-10-08T13:47:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:41 smithi057 ceph-mon[99476]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:41 smithi057 ceph-mon[99476]: osdmap e54: 8 total, 8 up, 8 in 2023-10-08T13:47:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:41 smithi057 ceph-mon[99476]: from='client.? 172.21.15.115:0/3349681009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61534853-3ba8-40cc-b62e-7b1c2efdb996"}]: dispatch 2023-10-08T13:47:41.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:41 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61534853-3ba8-40cc-b62e-7b1c2efdb996"}]: dispatch 2023-10-08T13:47:41.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:41 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61534853-3ba8-40cc-b62e-7b1c2efdb996"}]': finished 2023-10-08T13:47:41.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:41 smithi057 ceph-mon[99476]: osdmap e55: 9 total, 8 up, 9 in 2023-10-08T13:47:41.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:41 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:41 smithi099 ceph-mon[103849]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:41 smithi099 ceph-mon[103849]: osdmap e54: 8 total, 8 up, 8 in 2023-10-08T13:47:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:41 smithi099 ceph-mon[103849]: from='client.? 172.21.15.115:0/3349681009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61534853-3ba8-40cc-b62e-7b1c2efdb996"}]: dispatch 2023-10-08T13:47:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:41 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61534853-3ba8-40cc-b62e-7b1c2efdb996"}]: dispatch 2023-10-08T13:47:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:41 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61534853-3ba8-40cc-b62e-7b1c2efdb996"}]': finished 2023-10-08T13:47:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:41 smithi099 ceph-mon[103849]: osdmap e55: 9 total, 8 up, 9 in 2023-10-08T13:47:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:41 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:42.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:41 smithi115 ceph-mon[104408]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:42.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:41 smithi115 ceph-mon[104408]: osdmap e54: 8 total, 8 up, 8 in 2023-10-08T13:47:42.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:41 smithi115 ceph-mon[104408]: from='client.? 172.21.15.115:0/3349681009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61534853-3ba8-40cc-b62e-7b1c2efdb996"}]: dispatch 2023-10-08T13:47:42.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:41 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61534853-3ba8-40cc-b62e-7b1c2efdb996"}]: dispatch 2023-10-08T13:47:42.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:41 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61534853-3ba8-40cc-b62e-7b1c2efdb996"}]': finished 2023-10-08T13:47:42.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:41 smithi115 ceph-mon[104408]: osdmap e55: 9 total, 8 up, 9 in 2023-10-08T13:47:42.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:41 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:42.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:42 smithi057 ceph-mon[99476]: from='client.? 172.21.15.115:0/3486400104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:47:43.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:42 smithi099 ceph-mon[103849]: from='client.? 172.21.15.115:0/3486400104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:47:43.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:42 smithi115 ceph-mon[104408]: from='client.? 172.21.15.115:0/3486400104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:47:43.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:43 smithi057 ceph-mon[99476]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:43 smithi099 ceph-mon[103849]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:43 smithi115 ceph-mon[104408]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:45.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:45 smithi057 ceph-mon[99476]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:46.021 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:45 smithi099 ceph-mon[103849]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:46.069 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:45 smithi115 ceph-mon[104408]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:47.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:47 smithi115 ceph-mon[104408]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:47.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:47 smithi057 ceph-mon[99476]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:48.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:47 smithi099 ceph-mon[103849]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:48.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-08T13:47:48.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:48 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:48.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:48 smithi115 ceph-mon[104408]: Deploying daemon osd.8 on smithi115 2023-10-08T13:47:48.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-08T13:47:48.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:48 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:48.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:48 smithi057 ceph-mon[99476]: Deploying daemon osd.8 on smithi115 2023-10-08T13:47:49.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-08T13:47:49.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:48 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:49.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:48 smithi099 ceph-mon[103849]: Deploying daemon osd.8 on smithi115 2023-10-08T13:47:49.877 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:49 smithi115 ceph-mon[104408]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:49.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:49 smithi057 ceph-mon[99476]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:50.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:49 smithi099 ceph-mon[103849]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:51.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:51 smithi057 ceph-mon[99476]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:52.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:51 smithi115 ceph-mon[104408]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:52.021 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:51 smithi099 ceph-mon[103849]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:53.342 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:53 smithi115 ceph-mon[104408]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:53.343 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:53.343 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:53.343 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:53.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:53 smithi057 ceph-mon[99476]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:53.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:53.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:53.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:53 smithi099 ceph-mon[103849]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.784+0000 7fcb3bfff700 1 -- 172.21.15.115:0/831808263 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcb54069410 con 0x7fcb3c0a14d0 2023-10-08T13:47:54.786 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 8 on host 'smithi115' 2023-10-08T13:47:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 -- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb3c0a14d0 msgr2=0x7fcb3c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb3c0a14d0 0x7fcb3c0a3990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcb44007900 tx=0x7fcb44008040 comp rx=0 tx=0).stop 2023-10-08T13:47:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 -- 172.21.15.115:0/831808263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb541003b0 msgr2=0x7fcb54198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb541003b0 0x7fcb54198090 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800d950 tx=0x7fcb4800dd10 comp rx=0 tx=0).stop 2023-10-08T13:47:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 -- 172.21.15.115:0/831808263 shutdown_connections 2023-10-08T13:47:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb54104680 0x7fcb54198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb3c0a14d0 0x7fcb3c0a3990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb540ff9f0 0x7fcb54197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 --2- 172.21.15.115:0/831808263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb541003b0 0x7fcb54198090 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 -- 172.21.15.115:0/831808263 >> 172.21.15.115:0/831808263 conn(0x7fcb54076510 msgr2=0x7fcb540fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:47:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 -- 172.21.15.115:0/831808263 shutdown_connections 2023-10-08T13:47:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:47:54.787+0000 7fcb5948d700 1 -- 172.21.15.115:0/831808263 wait complete. 2023-10-08T13:47:55.218 DEBUG:teuthology.orchestra.run.smithi115:osd.8> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.8.service 2023-10-08T13:47:55.220 INFO:tasks.cephadm:Deploying osd.9 on smithi057 with /dev/vg_nvme/lv_1... 2023-10-08T13:47:55.220 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-08T13:47:55.252 INFO:journalctl@ceph.osd.8.smithi115.stdout:-- Logs begin at Sun 2023-10-08 13:28:12 UTC. -- 2023-10-08T13:47:55.801 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:47:55.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:55 smithi115 ceph-mon[104408]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:55.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:55.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:55.891 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:55.892 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:55.892 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:55.892 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:55.892 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:55.892 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:55.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:55 smithi057 ceph-mon[99476]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:55.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:55.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:55.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:55.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:55.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:55.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:55.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:55.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:55 smithi099 ceph-mon[103849]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:47:56.186 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 13:47:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T13:47:56.064+0000 7f61d59b3640 -1 osd.8 0 log_to_monitors true 2023-10-08T13:47:56.661 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:56 smithi057 ceph-mon[99476]: from='osd.8 [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T13:47:56.661 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:56 smithi057 ceph-mon[99476]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T13:47:56.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:56 smithi115 ceph-mon[104408]: from='osd.8 [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T13:47:56.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:56 smithi115 ceph-mon[104408]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T13:47:57.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:56 smithi099 ceph-mon[103849]: from='osd.8 [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T13:47:57.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:56 smithi099 ceph-mon[103849]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T13:47:57.352 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-08T13:47:57.353 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-08T13:47:57.353 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2023-10-08T13:47:57.353 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2023-10-08T13:47:57.353 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113093 s, 927 MB/s 2023-10-08T13:47:57.353 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2023-10-08T13:47:57.753 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi057:vg_nvme/lv_1 2023-10-08T13:47:57.936 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 13:47:57 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T13:47:57.657+0000 7f61cb02a700 -1 osd.8 0 waiting for initial osdmap 2023-10-08T13:47:57.936 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 13:47:57 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T13:47:57.667+0000 7f61c6e1f700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:47:57.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:57 smithi115 ceph-mon[104408]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:57.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:57 smithi115 ceph-mon[104408]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-08T13:47:57.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:57 smithi115 ceph-mon[104408]: from='osd.8 [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:47:57.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:57 smithi115 ceph-mon[104408]: osdmap e56: 9 total, 8 up, 9 in 2023-10-08T13:47:57.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:57 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:57.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:57 smithi115 ceph-mon[104408]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:47:57.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:57 smithi057 ceph-mon[99476]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:57.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:57 smithi057 ceph-mon[99476]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-08T13:47:57.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:57 smithi057 ceph-mon[99476]: from='osd.8 [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:47:57.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:57 smithi057 ceph-mon[99476]: osdmap e56: 9 total, 8 up, 9 in 2023-10-08T13:47:57.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:57 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:57.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:57 smithi057 ceph-mon[99476]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:47:58.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:57 smithi099 ceph-mon[103849]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:58.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:57 smithi099 ceph-mon[103849]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-08T13:47:58.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:57 smithi099 ceph-mon[103849]: from='osd.8 [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:47:58.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:57 smithi099 ceph-mon[103849]: osdmap e56: 9 total, 8 up, 9 in 2023-10-08T13:47:58.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:57 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:58.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:57 smithi099 ceph-mon[103849]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:47:58.328 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:47:58.856 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:47:58.856 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: osdmap e57: 9 total, 8 up, 9 in 2023-10-08T13:47:58.856 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:58.856 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: osd.8 [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] boot 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: osdmap e58: 9 total, 9 up, 9 in 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi115 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi115 to 5722M 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:58.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:58.858 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:58.858 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:58.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:47:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: osdmap e57: 9 total, 8 up, 9 in 2023-10-08T13:47:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: osd.8 [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] boot 2023-10-08T13:47:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: osdmap e58: 9 total, 9 up, 9 in 2023-10-08T13:47:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi115 2023-10-08T13:47:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:58.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi115 to 5722M 2023-10-08T13:47:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:58.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:59.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:47:59.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: osdmap e57: 9 total, 8 up, 9 in 2023-10-08T13:47:59.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:59.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:59.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: osd.8 [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] boot 2023-10-08T13:47:59.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: osdmap e58: 9 total, 9 up, 9 in 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi115 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi115 to 5722M 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:47:59.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:47:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.564+0000 7f62868b7700 1 -- 172.21.15.57:0/64309575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6280106800 msgr2=0x7f6280106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.564+0000 7f62868b7700 1 --2- 172.21.15.57:0/64309575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6280106800 0x7f6280106be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6270009990 tx=0x7f6270009ca0 comp rx=0 tx=0).stop 2023-10-08T13:47:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.564+0000 7f62868b7700 1 -- 172.21.15.57:0/64309575 shutdown_connections 2023-10-08T13:47:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.564+0000 7f62868b7700 1 --2- 172.21.15.57:0/64309575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6280069880 0x7f628010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.564+0000 7f62868b7700 1 --2- 172.21.15.57:0/64309575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6280106800 0x7f6280106be0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:59.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.564+0000 7f62868b7700 1 --2- 172.21.15.57:0/64309575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6280068ec0 0x7f6280069340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:59.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.564+0000 7f62868b7700 1 -- 172.21.15.57:0/64309575 >> 172.21.15.57:0/64309575 conn(0x7f62800765e0 msgr2=0x7f62800769f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:47:59.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.564+0000 7f62868b7700 1 -- 172.21.15.57:0/64309575 shutdown_connections 2023-10-08T13:47:59.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.564+0000 7f62868b7700 1 -- 172.21.15.57:0/64309575 wait complete. 2023-10-08T13:47:59.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.565+0000 7f62868b7700 1 Processor -- start 2023-10-08T13:47:59.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.565+0000 7f62868b7700 1 -- start start 2023-10-08T13:47:59.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.565+0000 7f62868b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6280068ec0 0x7f62800ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:59.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.565+0000 7f62868b7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6280069880 0x7f62800fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:59.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.565+0000 7f62868b7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6280106800 0x7f6280100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:59.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.565+0000 7f62868b7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62801a4f90 con 0x7f6280068ec0 2023-10-08T13:47:59.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.565+0000 7f62868b7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62801a5100 con 0x7f6280106800 2023-10-08T13:47:59.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.565+0000 7f62868b7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62801a52c0 con 0x7f6280069880 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6280068ec0 0x7f62800ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6280068ec0 0x7f62800ffaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53384/0 (socket says 172.21.15.57:53384) 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627ffff700 1 -- 172.21.15.57:0/1201787996 learned_addr learned my addr 172.21.15.57:0/1201787996 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627f7fe700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6280069880 0x7f62800fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f6284e54700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6280106800 0x7f6280100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627ffff700 1 -- 172.21.15.57:0/1201787996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6280069880 msgr2=0x7f62800fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627ffff700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6280069880 0x7f62800fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627ffff700 1 -- 172.21.15.57:0/1201787996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6280106800 msgr2=0x7f6280100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627ffff700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6280106800 0x7f6280100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627ffff700 1 -- 172.21.15.57:0/1201787996 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62700095f0 con 0x7f6280068ec0 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627ffff700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6280068ec0 0x7f62800ffaa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f62700098e0 tx=0x7f627000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:47:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627d7fa700 1 -- 172.21.15.57:0/1201787996 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627001d070 con 0x7f6280068ec0 2023-10-08T13:47:59.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627d7fa700 1 -- 172.21.15.57:0/1201787996 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6270022470 con 0x7f6280068ec0 2023-10-08T13:47:59.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f62868b7700 1 -- 172.21.15.57:0/1201787996 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62801a5540 con 0x7f6280068ec0 2023-10-08T13:47:59.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f627d7fa700 1 -- 172.21.15.57:0/1201787996 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270017650 con 0x7f6280068ec0 2023-10-08T13:47:59.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.566+0000 7f62868b7700 1 -- 172.21.15.57:0/1201787996 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62801a59b0 con 0x7f6280068ec0 2023-10-08T13:47:59.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.567+0000 7f62868b7700 1 -- 172.21.15.57:0/1201787996 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6280108d60 con 0x7f6280068ec0 2023-10-08T13:47:59.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.569+0000 7f627d7fa700 1 -- 172.21.15.57:0/1201787996 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f62700225e0 con 0x7f6280068ec0 2023-10-08T13:47:59.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.569+0000 7f627d7fa700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f626c0a1550 0x7f626c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:47:59.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.569+0000 7f627f7fe700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f626c0a1550 0x7f626c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:47:59.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.569+0000 7f627d7fa700 1 -- 172.21.15.57:0/1201787996 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f62700cd7a0 con 0x7f6280068ec0 2023-10-08T13:47:59.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.570+0000 7f627f7fe700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f626c0a1550 0x7f626c0a3a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f62801011d0 tx=0x7f626800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:47:59.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.573+0000 7f627d7fa700 1 -- 172.21.15.57:0/1201787996 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6270097750 con 0x7f6280068ec0 2023-10-08T13:47:59.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:47:59.783+0000 7f62868b7700 1 -- 172.21.15.57:0/1201787996 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6280100f80 con 0x7f626c0a1550 2023-10-08T13:47:59.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:59 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:47:59.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:59 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:47:59.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:59 smithi057 ceph-mon[99476]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:47:59.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:47:59 smithi057 ceph-mon[99476]: osdmap e59: 9 total, 9 up, 9 in 2023-10-08T13:48:00.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:59 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:48:00.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:59 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:48:00.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:59 smithi099 ceph-mon[103849]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:48:00.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:47:59 smithi099 ceph-mon[103849]: osdmap e59: 9 total, 9 up, 9 in 2023-10-08T13:48:00.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:59 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:48:00.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:59 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:48:00.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:59 smithi115 ceph-mon[104408]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-08T13:48:00.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:47:59 smithi115 ceph-mon[104408]: osdmap e59: 9 total, 9 up, 9 in 2023-10-08T13:48:00.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:00 smithi057 ceph-mon[99476]: osdmap e60: 9 total, 9 up, 9 in 2023-10-08T13:48:00.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:00 smithi057 ceph-mon[99476]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:48:00.916 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:00 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:48:00.916 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:00 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:48:00.916 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:00 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:01.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:00 smithi099 ceph-mon[103849]: osdmap e60: 9 total, 9 up, 9 in 2023-10-08T13:48:01.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:00 smithi099 ceph-mon[103849]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:48:01.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:00 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:48:01.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:00 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:48:01.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:00 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:01.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:00 smithi115 ceph-mon[104408]: osdmap e60: 9 total, 9 up, 9 in 2023-10-08T13:48:01.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:00 smithi115 ceph-mon[104408]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:48:01.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:00 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:48:01.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:00 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:48:01.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:00 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:01.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:01 smithi057 ceph-mon[99476]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:01.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:01 smithi057 ceph-mon[99476]: osdmap e61: 9 total, 9 up, 9 in 2023-10-08T13:48:02.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:01 smithi099 ceph-mon[103849]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:02.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:01 smithi099 ceph-mon[103849]: osdmap e61: 9 total, 9 up, 9 in 2023-10-08T13:48:02.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:01 smithi115 ceph-mon[104408]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:02.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:01 smithi115 ceph-mon[104408]: osdmap e61: 9 total, 9 up, 9 in 2023-10-08T13:48:02.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:02 smithi057 ceph-mon[99476]: osdmap e62: 9 total, 9 up, 9 in 2023-10-08T13:48:02.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:02 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1170864913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28533ac1-c3f9-4587-93d0-752be200eeda"}]: dispatch 2023-10-08T13:48:02.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:02 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1170864913' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28533ac1-c3f9-4587-93d0-752be200eeda"}]': finished 2023-10-08T13:48:02.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:02 smithi057 ceph-mon[99476]: osdmap e63: 10 total, 9 up, 10 in 2023-10-08T13:48:02.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:03.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:02 smithi099 ceph-mon[103849]: osdmap e62: 9 total, 9 up, 9 in 2023-10-08T13:48:03.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:02 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1170864913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28533ac1-c3f9-4587-93d0-752be200eeda"}]: dispatch 2023-10-08T13:48:03.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:02 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1170864913' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28533ac1-c3f9-4587-93d0-752be200eeda"}]': finished 2023-10-08T13:48:03.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:02 smithi099 ceph-mon[103849]: osdmap e63: 10 total, 9 up, 10 in 2023-10-08T13:48:03.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:03.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:02 smithi115 ceph-mon[104408]: osdmap e62: 9 total, 9 up, 9 in 2023-10-08T13:48:03.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:02 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1170864913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28533ac1-c3f9-4587-93d0-752be200eeda"}]: dispatch 2023-10-08T13:48:03.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:02 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1170864913' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28533ac1-c3f9-4587-93d0-752be200eeda"}]': finished 2023-10-08T13:48:03.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:02 smithi115 ceph-mon[104408]: osdmap e63: 10 total, 9 up, 10 in 2023-10-08T13:48:03.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:03.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:03 smithi057 ceph-mon[99476]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:03.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:03 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2263163565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:48:04.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:03 smithi099 ceph-mon[103849]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:04.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:03 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2263163565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:48:04.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:03 smithi115 ceph-mon[104408]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:04.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:03 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2263163565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:48:05.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:05 smithi057 ceph-mon[99476]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-10-08T13:48:06.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:05 smithi099 ceph-mon[103849]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-10-08T13:48:06.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:05 smithi115 ceph-mon[104408]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-10-08T13:48:08.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:07 smithi099 ceph-mon[103849]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:48:08.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:07 smithi115 ceph-mon[104408]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:48:08.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:07 smithi057 ceph-mon[99476]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:48:08.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:08 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-08T13:48:08.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:08 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:09.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:08 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-08T13:48:09.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:08 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:09.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:08 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-08T13:48:09.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:08 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:09.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:09 smithi057 ceph-mon[99476]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-08T13:48:09.828 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:09 smithi057 ceph-mon[99476]: Deploying daemon osd.9 on smithi057 2023-10-08T13:48:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:09 smithi099 ceph-mon[103849]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-08T13:48:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:09 smithi099 ceph-mon[103849]: Deploying daemon osd.9 on smithi057 2023-10-08T13:48:10.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:09 smithi115 ceph-mon[104408]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-08T13:48:10.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:09 smithi115 ceph-mon[104408]: Deploying daemon osd.9 on smithi057 2023-10-08T13:48:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:11 smithi057 ceph-mon[99476]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-08T13:48:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:11 smithi099 ceph-mon[103849]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-08T13:48:12.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:11 smithi115 ceph-mon[104408]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-08T13:48:14.061 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:13 smithi057 ceph-mon[99476]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T13:48:14.062 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:13 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:14.062 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:13 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:14.062 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:13 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:13 smithi099 ceph-mon[103849]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T13:48:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:13 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:13 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:13 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:14.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:13 smithi115 ceph-mon[104408]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T13:48:14.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:13 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:14.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:13 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:14.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:13 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:15.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.483+0000 7f627d7fa700 1 -- 172.21.15.57:0/1201787996 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6280100f80 con 0x7f626c0a1550 2023-10-08T13:48:15.484 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 9 on host 'smithi057' 2023-10-08T13:48:15.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.485+0000 7f62868b7700 1 -- 172.21.15.57:0/1201787996 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f626c0a1550 msgr2=0x7f626c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:15.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.485+0000 7f62868b7700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f626c0a1550 0x7f626c0a3a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f62801011d0 tx=0x7f626800b410 comp rx=0 tx=0).stop 2023-10-08T13:48:15.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.485+0000 7f62868b7700 1 -- 172.21.15.57:0/1201787996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6280068ec0 msgr2=0x7f62800ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:15.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.485+0000 7f62868b7700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6280068ec0 0x7f62800ffaa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f62700098e0 tx=0x7f627000f670 comp rx=0 tx=0).stop 2023-10-08T13:48:15.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.485+0000 7f62868b7700 1 -- 172.21.15.57:0/1201787996 shutdown_connections 2023-10-08T13:48:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.485+0000 7f62868b7700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f626c0a1550 0x7f626c0a3a10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.485+0000 7f62868b7700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6280106800 0x7f6280100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.485+0000 7f62868b7700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6280068ec0 0x7f62800ffaa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.485+0000 7f62868b7700 1 --2- 172.21.15.57:0/1201787996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6280069880 0x7f62800fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:15.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.485+0000 7f62868b7700 1 -- 172.21.15.57:0/1201787996 >> 172.21.15.57:0/1201787996 conn(0x7f62800765e0 msgr2=0x7f62800fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:48:15.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.487+0000 7f62868b7700 1 -- 172.21.15.57:0/1201787996 shutdown_connections 2023-10-08T13:48:15.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:48:15.487+0000 7f62868b7700 1 -- 172.21.15.57:0/1201787996 wait complete. 2023-10-08T13:48:15.927 DEBUG:teuthology.orchestra.run.smithi057:osd.9> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.9.service 2023-10-08T13:48:15.929 INFO:tasks.cephadm:Deploying osd.10 on smithi099 with /dev/vg_nvme/lv_1... 2023-10-08T13:48:15.929 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-08T13:48:15.960 INFO:journalctl@ceph.osd.9.smithi057.stdout:-- Logs begin at Sun 2023-10-08 13:28:06 UTC. -- 2023-10-08T13:48:16.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:15 smithi057 ceph-mon[99476]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-08T13:48:16.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:15 smithi099 ceph-mon[103849]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-08T13:48:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:15 smithi115 ceph-mon[104408]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-08T13:48:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:16.504 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T13:48:17.235 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 13:48:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T13:48:16.949+0000 7fd2da081640 -1 osd.9 0 log_to_monitors true 2023-10-08T13:48:17.236 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:16 smithi057 ceph-mon[99476]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-08T13:48:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:16 smithi099 ceph-mon[103849]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-08T13:48:17.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:16 smithi115 ceph-mon[104408]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-08T13:48:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-08T13:48:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-08T13:48:18.203 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2023-10-08T13:48:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2023-10-08T13:48:18.203 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112075 s, 936 MB/s 2023-10-08T13:48:18.203 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2023-10-08T13:48:18.258 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 13:48:17 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T13:48:17.996+0000 7fd2cf6f8700 -1 osd.9 0 waiting for initial osdmap 2023-10-08T13:48:18.258 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T13:48:18.012+0000 7fd2cacec700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:48:18.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:17 smithi057 ceph-mon[99476]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:18.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:17 smithi057 ceph-mon[99476]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-08T13:48:18.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:17 smithi057 ceph-mon[99476]: osdmap e64: 10 total, 9 up, 10 in 2023-10-08T13:48:18.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:17 smithi057 ceph-mon[99476]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:48:18.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:17 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:17 smithi099 ceph-mon[103849]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:17 smithi099 ceph-mon[103849]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-08T13:48:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:17 smithi099 ceph-mon[103849]: osdmap e64: 10 total, 9 up, 10 in 2023-10-08T13:48:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:17 smithi099 ceph-mon[103849]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:48:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:17 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:18.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:17 smithi115 ceph-mon[104408]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:18.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:17 smithi115 ceph-mon[104408]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-08T13:48:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:17 smithi115 ceph-mon[104408]: osdmap e64: 10 total, 9 up, 10 in 2023-10-08T13:48:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:17 smithi115 ceph-mon[104408]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2023-10-08T13:48:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:17 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:18.658 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2023-10-08T13:48:19.218 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T13:48:19.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:48:19.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: osdmap e65: 10 total, 9 up, 10 in 2023-10-08T13:48:19.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:19.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:19.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:19.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:19.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:48:19.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: osdmap e65: 10 total, 9 up, 10 in 2023-10-08T13:48:19.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:19.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:19.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:19.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:19.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:19.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:19.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:19.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2023-10-08T13:48:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: osdmap e65: 10 total, 9 up, 10 in 2023-10-08T13:48:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:19.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:19.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:19.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:19.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:19.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:19.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:19.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:19 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:48:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:19 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:48:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:19 smithi099 ceph-mon[103849]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:19 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi057 2023-10-08T13:48:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:19 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi057 to 4293M 2023-10-08T13:48:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:19 smithi099 ceph-mon[103849]: osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] boot 2023-10-08T13:48:20.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:19 smithi099 ceph-mon[103849]: osdmap e66: 10 total, 10 up, 10 in 2023-10-08T13:48:20.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:20.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:48:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:48:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi057 2023-10-08T13:48:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi057 to 4293M 2023-10-08T13:48:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] boot 2023-10-08T13:48:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: osdmap e66: 10 total, 10 up, 10 in 2023-10-08T13:48:20.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:20.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:19 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:48:20.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:19 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:48:20.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:19 smithi057 ceph-mon[99476]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-08T13:48:20.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:19 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi057 2023-10-08T13:48:20.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:19 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi057 to 4293M 2023-10-08T13:48:20.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:20.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:19 smithi057 ceph-mon[99476]: osd.9 [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] boot 2023-10-08T13:48:20.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:19 smithi057 ceph-mon[99476]: osdmap e66: 10 total, 10 up, 10 in 2023-10-08T13:48:20.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T13:48:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.497+0000 7f57df5a6700 1 -- 172.21.15.99:0/2732349150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57d8101260 msgr2=0x7f57d810d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.497+0000 7f57df5a6700 1 --2- 172.21.15.99:0/2732349150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57d8101260 0x7f57d810d650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f57d000b0a0 tx=0x7f57d000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:48:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.497+0000 7f57df5a6700 1 -- 172.21.15.99:0/2732349150 shutdown_connections 2023-10-08T13:48:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.497+0000 7f57df5a6700 1 --2- 172.21.15.99:0/2732349150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57d8101260 0x7f57d810d650 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:20.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.497+0000 7f57df5a6700 1 --2- 172.21.15.99:0/2732349150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57d81057c0 0x7f57d8105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:20.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.497+0000 7f57df5a6700 1 --2- 172.21.15.99:0/2732349150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57d81008a0 0x7f57d8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:20.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.497+0000 7f57df5a6700 1 -- 172.21.15.99:0/2732349150 >> 172.21.15.99:0/2732349150 conn(0x7f57d80779e0 msgr2=0x7f57d8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:48:20.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.497+0000 7f57df5a6700 1 -- 172.21.15.99:0/2732349150 shutdown_connections 2023-10-08T13:48:20.499 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.497+0000 7f57df5a6700 1 -- 172.21.15.99:0/2732349150 wait complete. 2023-10-08T13:48:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57df5a6700 1 Processor -- start 2023-10-08T13:48:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57df5a6700 1 -- start start 2023-10-08T13:48:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57df5a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57d81008a0 0x7f57d819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:48:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57df5a6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57d8101260 0x7f57d819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:48:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57df5a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57d81057c0 0x7f57d8196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:48:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57df5a6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d81972a0 con 0x7f57d8101260 2023-10-08T13:48:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57df5a6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8197410 con 0x7f57d81008a0 2023-10-08T13:48:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57df5a6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d81975d0 con 0x7f57d81057c0 2023-10-08T13:48:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57dd342700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57d81008a0 0x7f57d819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:48:20.501 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57dd342700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57d81008a0 0x7f57d819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42786/0 (socket says 172.21.15.99:42786) 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57dd342700 1 -- 172.21.15.99:0/3319661660 learned_addr learned my addr 172.21.15.99:0/3319661660 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57dcb41700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57d8101260 0x7f57d819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57ddb43700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57d81057c0 0x7f57d8196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57dd342700 1 -- 172.21.15.99:0/3319661660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57d81057c0 msgr2=0x7f57d8196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57dd342700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57d81057c0 0x7f57d8196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57dd342700 1 -- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57d8101260 msgr2=0x7f57d819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57dd342700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57d8101260 0x7f57d819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.498+0000 7f57dd342700 1 -- 172.21.15.99:0/3319661660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d0009d00 con 0x7f57d81008a0 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.499+0000 7f57dcb41700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57d8101260 0x7f57d819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.499+0000 7f57dd342700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57d81008a0 0x7f57d819ca40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57c800d980 tx=0x7f57c800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:48:20.502 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.499+0000 7f57ce7fc700 1 -- 172.21.15.99:0/3319661660 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c800de60 con 0x7f57d81008a0 2023-10-08T13:48:20.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.499+0000 7f57df5a6700 1 -- 172.21.15.99:0/3319661660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d81978b0 con 0x7f57d81008a0 2023-10-08T13:48:20.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.499+0000 7f57ce7fc700 1 -- 172.21.15.99:0/3319661660 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c80044e0 con 0x7f57d81008a0 2023-10-08T13:48:20.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.499+0000 7f57ce7fc700 1 -- 172.21.15.99:0/3319661660 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8003be0 con 0x7f57d81008a0 2023-10-08T13:48:20.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.499+0000 7f57df5a6700 1 -- 172.21.15.99:0/3319661660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d81a3a10 con 0x7f57d81008a0 2023-10-08T13:48:20.503 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.500+0000 7f57df5a6700 1 -- 172.21.15.99:0/3319661660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d8108da0 con 0x7f57d81008a0 2023-10-08T13:48:20.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.501+0000 7f57ce7fc700 1 -- 172.21.15.99:0/3319661660 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f57c8009bb0 con 0x7f57d81008a0 2023-10-08T13:48:20.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.501+0000 7f57ce7fc700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57c40a14d0 0x7f57c40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:48:20.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.501+0000 7f57ce7fc700 1 -- 172.21.15.99:0/3319661660 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6476+0+0 (secure 0 0 0) 0x7f57c80cc350 con 0x7f57d81008a0 2023-10-08T13:48:20.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.502+0000 7f57dcb41700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57c40a14d0 0x7f57c40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:48:20.504 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.502+0000 7f57dcb41700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57c40a14d0 0x7f57c40a3990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f57d4005950 tx=0x7f57d40058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:48:20.507 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.506+0000 7f57ce7fc700 1 -- 172.21.15.99:0/3319661660 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f57c80960e0 con 0x7f57d81008a0 2023-10-08T13:48:20.702 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:20.701+0000 7f57df5a6700 1 -- 172.21.15.99:0/3319661660 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f57d8197fd0 con 0x7f57c40a14d0 2023-10-08T13:48:21.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:21 smithi099 ceph-mon[103849]: osdmap e67: 10 total, 10 up, 10 in 2023-10-08T13:48:21.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:21 smithi099 ceph-mon[103849]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:21.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:21 smithi099 ceph-mon[103849]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:48:21.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:48:21.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:48:21.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:21.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:21 smithi115 ceph-mon[104408]: osdmap e67: 10 total, 10 up, 10 in 2023-10-08T13:48:21.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:21 smithi115 ceph-mon[104408]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:21.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:21 smithi115 ceph-mon[104408]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:48:21.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:48:21.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:48:21.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:21.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:21 smithi057 ceph-mon[99476]: osdmap e67: 10 total, 10 up, 10 in 2023-10-08T13:48:21.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:21 smithi057 ceph-mon[99476]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:21.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:21 smithi057 ceph-mon[99476]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:48:21.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:48:21.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:48:21.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:23.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:23 smithi057 ceph-mon[99476]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:23.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:23 smithi057 ceph-mon[99476]: from='client.? 172.21.15.99:0/3069415822' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15226d7f-8a7d-45da-8750-adaa98217259"}]: dispatch 2023-10-08T13:48:23.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:23 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15226d7f-8a7d-45da-8750-adaa98217259"}]: dispatch 2023-10-08T13:48:23.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:23 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15226d7f-8a7d-45da-8750-adaa98217259"}]': finished 2023-10-08T13:48:23.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:23 smithi057 ceph-mon[99476]: osdmap e68: 11 total, 10 up, 11 in 2023-10-08T13:48:23.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:23 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:23 smithi099 ceph-mon[103849]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:23 smithi099 ceph-mon[103849]: from='client.? 172.21.15.99:0/3069415822' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15226d7f-8a7d-45da-8750-adaa98217259"}]: dispatch 2023-10-08T13:48:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:23 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15226d7f-8a7d-45da-8750-adaa98217259"}]: dispatch 2023-10-08T13:48:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:23 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15226d7f-8a7d-45da-8750-adaa98217259"}]': finished 2023-10-08T13:48:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:23 smithi099 ceph-mon[103849]: osdmap e68: 11 total, 10 up, 11 in 2023-10-08T13:48:24.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:23 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:24.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:23 smithi115 ceph-mon[104408]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:23 smithi115 ceph-mon[104408]: from='client.? 172.21.15.99:0/3069415822' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15226d7f-8a7d-45da-8750-adaa98217259"}]: dispatch 2023-10-08T13:48:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:23 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15226d7f-8a7d-45da-8750-adaa98217259"}]: dispatch 2023-10-08T13:48:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:23 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15226d7f-8a7d-45da-8750-adaa98217259"}]': finished 2023-10-08T13:48:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:23 smithi115 ceph-mon[104408]: osdmap e68: 11 total, 10 up, 11 in 2023-10-08T13:48:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:23 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:24.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:24 smithi057 ceph-mon[99476]: from='client.? 172.21.15.99:0/2843269157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:48:25.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:24 smithi099 ceph-mon[103849]: from='client.? 172.21.15.99:0/2843269157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:48:25.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:24 smithi115 ceph-mon[104408]: from='client.? 172.21.15.99:0/2843269157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:48:25.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:25 smithi057 ceph-mon[99476]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:26.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:25 smithi099 ceph-mon[103849]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:26.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:25 smithi115 ceph-mon[104408]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:27.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:27 smithi057 ceph-mon[99476]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:28.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:27 smithi099 ceph-mon[103849]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:28.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:27 smithi115 ceph-mon[104408]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:29.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:29 smithi057 ceph-mon[99476]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:29 smithi099 ceph-mon[103849]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:30.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:29 smithi115 ceph-mon[104408]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:30.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-08T13:48:30.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:31.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-08T13:48:31.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:31.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-08T13:48:31.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:32.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:31 smithi099 ceph-mon[103849]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:32.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:31 smithi099 ceph-mon[103849]: Deploying daemon osd.10 on smithi099 2023-10-08T13:48:32.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:31 smithi115 ceph-mon[104408]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:32.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:31 smithi115 ceph-mon[104408]: Deploying daemon osd.10 on smithi099 2023-10-08T13:48:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:31 smithi057 ceph-mon[99476]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:31 smithi057 ceph-mon[99476]: Deploying daemon osd.10 on smithi099 2023-10-08T13:48:34.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:33 smithi099 ceph-mon[103849]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:34.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:33 smithi115 ceph-mon[104408]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:34.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:33 smithi057 ceph-mon[99476]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:36.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:35 smithi057 ceph-mon[99476]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:36.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:36.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:36.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:35 smithi099 ceph-mon[103849]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:36.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:36.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:35 smithi115 ceph-mon[104408]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:36.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:36.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:36.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:38.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.170+0000 7f57ce7fc700 1 -- 172.21.15.99:0/3319661660 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f57d8197fd0 con 0x7f57c40a14d0 2023-10-08T13:48:38.171 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 10 on host 'smithi099' 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.172+0000 7f57df5a6700 1 -- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57c40a14d0 msgr2=0x7f57c40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.172+0000 7f57df5a6700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57c40a14d0 0x7f57c40a3990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f57d4005950 tx=0x7f57d40058e0 comp rx=0 tx=0).stop 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.172+0000 7f57df5a6700 1 -- 172.21.15.99:0/3319661660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57d81008a0 msgr2=0x7f57d819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.172+0000 7f57df5a6700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57d81008a0 0x7f57d819ca40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57c800d980 tx=0x7f57c800dd40 comp rx=0 tx=0).stop 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.172+0000 7f57df5a6700 1 -- 172.21.15.99:0/3319661660 shutdown_connections 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.172+0000 7f57df5a6700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57d81057c0 0x7f57d8196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.172+0000 7f57df5a6700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57c40a14d0 0x7f57c40a3990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.172+0000 7f57df5a6700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57d81008a0 0x7f57d819ca40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.172+0000 7f57df5a6700 1 --2- 172.21.15.99:0/3319661660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57d8101260 0x7f57d819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.172+0000 7f57df5a6700 1 -- 172.21.15.99:0/3319661660 >> 172.21.15.99:0/3319661660 conn(0x7f57d80779e0 msgr2=0x7f57d80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:48:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.173+0000 7f57df5a6700 1 -- 172.21.15.99:0/3319661660 shutdown_connections 2023-10-08T13:48:38.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-08T13:48:38.173+0000 7f57df5a6700 1 -- 172.21.15.99:0/3319661660 wait complete. 2023-10-08T13:48:38.205 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:37 smithi099 ceph-mon[103849]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:38.205 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:38.205 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:37 smithi057 ceph-mon[99476]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:38.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:37 smithi115 ceph-mon[104408]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:38.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:38.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:38.721 DEBUG:teuthology.orchestra.run.smithi099:osd.10> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.10.service 2023-10-08T13:48:38.723 INFO:tasks.cephadm:Deploying osd.11 on smithi115 with /dev/vg_nvme/lv_1... 2023-10-08T13:48:38.723 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-08T13:48:38.752 INFO:journalctl@ceph.osd.10.smithi099.stdout:-- Logs begin at Sun 2023-10-08 13:27:56 UTC. -- 2023-10-08T13:48:39.022 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 13:48:38 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T13:48:38.867+0000 7f823eade640 -1 osd.10 0 log_to_monitors true 2023-10-08T13:48:39.293 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:48:39.411 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:39 smithi099 ceph-mon[103849]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:39.411 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:39 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:39.411 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:39 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:39.411 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:39 smithi099 ceph-mon[103849]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T13:48:39.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:39 smithi115 ceph-mon[104408]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:39.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:39 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:39.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:39 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:39.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:39 smithi115 ceph-mon[104408]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T13:48:39.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:39 smithi057 ceph-mon[99476]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:39.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:39 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:39.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:39 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:39.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:39 smithi057 ceph-mon[99476]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T13:48:40.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:40 smithi115 ceph-mon[104408]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-08T13:48:40.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:40 smithi115 ceph-mon[104408]: osdmap e69: 11 total, 10 up, 11 in 2023-10-08T13:48:40.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:40 smithi115 ceph-mon[104408]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:48:40.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:40 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:40.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:40 smithi057 ceph-mon[99476]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-08T13:48:40.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:40 smithi057 ceph-mon[99476]: osdmap e69: 11 total, 10 up, 11 in 2023-10-08T13:48:40.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:40 smithi057 ceph-mon[99476]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:48:40.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:40 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:40.522 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 13:48:40 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T13:48:40.162+0000 7f8234155700 -1 osd.10 0 waiting for initial osdmap 2023-10-08T13:48:40.522 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 13:48:40 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T13:48:40.178+0000 7f822c743700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:48:40.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:40 smithi099 ceph-mon[103849]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-08T13:48:40.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:40 smithi099 ceph-mon[103849]: osdmap e69: 11 total, 10 up, 11 in 2023-10-08T13:48:40.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:40 smithi099 ceph-mon[103849]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2023-10-08T13:48:40.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:40 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:40.811 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-08T13:48:40.811 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-08T13:48:40.811 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2023-10-08T13:48:40.811 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2023-10-08T13:48:40.811 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0165795 s, 632 MB/s 2023-10-08T13:48:40.812 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2023-10-08T13:48:41.249 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi115:vg_nvme/lv_1 2023-10-08T13:48:41.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:41 smithi099 ceph-mon[103849]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:41.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:41 smithi099 ceph-mon[103849]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:48:41.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:41 smithi099 ceph-mon[103849]: osdmap e70: 11 total, 10 up, 11 in 2023-10-08T13:48:41.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:41 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:41.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:41 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:41.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:41 smithi115 ceph-mon[104408]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:41.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:41 smithi115 ceph-mon[104408]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:48:41.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:41 smithi115 ceph-mon[104408]: osdmap e70: 11 total, 10 up, 11 in 2023-10-08T13:48:41.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:41 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:41.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:41 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:41.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:41 smithi057 ceph-mon[99476]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:41.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:41 smithi057 ceph-mon[99476]: from='osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288]' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2023-10-08T13:48:41.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:41 smithi057 ceph-mon[99476]: osdmap e70: 11 total, 10 up, 11 in 2023-10-08T13:48:41.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:41 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:41.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:41 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:41.805 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:48:42.310 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:48:42.310 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:48:42.310 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:42.310 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] boot 2023-10-08T13:48:42.310 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: osdmap e71: 11 total, 11 up, 11 in 2023-10-08T13:48:42.310 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:42.310 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi099 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi099 to 4292M 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.311 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:42.312 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:42.312 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:42.312 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:48:42.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:48:42.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:42.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] boot 2023-10-08T13:48:42.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: osdmap e71: 11 total, 11 up, 11 in 2023-10-08T13:48:42.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:42.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi099 2023-10-08T13:48:42.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi099 to 4292M 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:42.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:48:42.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:48:42.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:42.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: osd.10 [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] boot 2023-10-08T13:48:42.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: osdmap e71: 11 total, 11 up, 11 in 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi099 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi099 to 4292M 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:42.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:42.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:42.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:42.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:48:42.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:42.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 -- 172.21.15.115:0/3730513085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab500ff190 msgr2=0x7fab500ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:42.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 --2- 172.21.15.115:0/3730513085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab500ff190 0x7fab500ff5f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab40009990 tx=0x7fab40009ca0 comp rx=0 tx=0).stop 2023-10-08T13:48:42.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 -- 172.21.15.115:0/3730513085 shutdown_connections 2023-10-08T13:48:42.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 --2- 172.21.15.115:0/3730513085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab500ffb30 0x7fab5010d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 --2- 172.21.15.115:0/3730513085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab501058d0 0x7fab50105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 --2- 172.21.15.115:0/3730513085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab500ff190 0x7fab500ff5f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 -- 172.21.15.115:0/3730513085 >> 172.21.15.115:0/3730513085 conn(0x7fab50074a10 msgr2=0x7fab50074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 -- 172.21.15.115:0/3730513085 shutdown_connections 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 -- 172.21.15.115:0/3730513085 wait complete. 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 Processor -- start 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.971+0000 7fab583f9700 1 -- start start 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab583f9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab500ff190 0x7fab50198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab583f9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab500ffb30 0x7fab50199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab583f9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab501058d0 0x7fab5019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:48:42.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab583f9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5019ddd0 con 0x7fab500ff190 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab583f9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5019df40 con 0x7fab500ffb30 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab583f9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5019e100 con 0x7fab501058d0 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab56996700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab501058d0 0x7fab5019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab56996700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab501058d0 0x7fab5019d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52850/0 (socket says 172.21.15.115:52850) 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab56996700 1 -- 172.21.15.115:0/2646889206 learned_addr learned my addr 172.21.15.115:0/2646889206 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab55994700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab500ffb30 0x7fab50199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab56195700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab500ff190 0x7fab50198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab56996700 1 -- 172.21.15.115:0/2646889206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab500ffb30 msgr2=0x7fab50199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab56996700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab500ffb30 0x7fab50199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab56996700 1 -- 172.21.15.115:0/2646889206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab500ff190 msgr2=0x7fab50198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab56996700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab500ff190 0x7fab50198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:42.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab56996700 1 -- 172.21.15.115:0/2646889206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400095f0 con 0x7fab501058d0 2023-10-08T13:48:42.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab56996700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab501058d0 0x7fab5019d7c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b9e0 tx=0x7fab4800bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:48:42.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab477fe700 1 -- 172.21.15.115:0/2646889206 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4800c930 con 0x7fab501058d0 2023-10-08T13:48:42.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab583f9700 1 -- 172.21.15.115:0/2646889206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5019e3e0 con 0x7fab501058d0 2023-10-08T13:48:42.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab477fe700 1 -- 172.21.15.115:0/2646889206 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab48014440 con 0x7fab501058d0 2023-10-08T13:48:42.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab477fe700 1 -- 172.21.15.115:0/2646889206 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab480125c0 con 0x7fab501058d0 2023-10-08T13:48:42.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.972+0000 7fab583f9700 1 -- 172.21.15.115:0/2646889206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab501a5c40 con 0x7fab501058d0 2023-10-08T13:48:42.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.973+0000 7fab583f9700 1 -- 172.21.15.115:0/2646889206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab50067c30 con 0x7fab501058d0 2023-10-08T13:48:42.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.975+0000 7fab477fe700 1 -- 172.21.15.115:0/2646889206 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fab480145b0 con 0x7fab501058d0 2023-10-08T13:48:42.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.976+0000 7fab477fe700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab3c0a1550 0x7fab3c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:48:42.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.976+0000 7fab477fe700 1 -- 172.21.15.115:0/2646889206 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6938+0+0 (secure 0 0 0) 0x7fab48006e40 con 0x7fab501058d0 2023-10-08T13:48:42.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.976+0000 7fab56195700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab3c0a1550 0x7fab3c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:48:42.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.976+0000 7fab56195700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab3c0a1550 0x7fab3c0a3a10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab4c005fd0 tx=0x7fab4c005dd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:48:42.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:42.979+0000 7fab477fe700 1 -- 172.21.15.115:0/2646889206 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fab48095f30 con 0x7fab501058d0 2023-10-08T13:48:43.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:43.180+0000 7fab583f9700 1 -- 172.21.15.115:0/2646889206 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fab501107c0 con 0x7fab3c0a1550 2023-10-08T13:48:43.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:43 smithi115 ceph-mon[104408]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:43.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:43 smithi115 ceph-mon[104408]: osdmap e72: 11 total, 11 up, 11 in 2023-10-08T13:48:43.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:43 smithi057 ceph-mon[99476]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:43.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:43 smithi057 ceph-mon[99476]: osdmap e72: 11 total, 11 up, 11 in 2023-10-08T13:48:43.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:43 smithi099 ceph-mon[103849]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-08T13:48:43.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:43 smithi099 ceph-mon[103849]: osdmap e72: 11 total, 11 up, 11 in 2023-10-08T13:48:44.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:44 smithi115 ceph-mon[104408]: osdmap e73: 11 total, 11 up, 11 in 2023-10-08T13:48:44.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:44 smithi115 ceph-mon[104408]: from='client.24529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:48:44.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:44 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:48:44.300 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:44 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:48:44.301 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:44 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:44.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:44 smithi057 ceph-mon[99476]: osdmap e73: 11 total, 11 up, 11 in 2023-10-08T13:48:44.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:44 smithi057 ceph-mon[99476]: from='client.24529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:48:44.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:44 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:48:44.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:44 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:48:44.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:44 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:44.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:44 smithi099 ceph-mon[103849]: osdmap e73: 11 total, 11 up, 11 in 2023-10-08T13:48:44.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:44 smithi099 ceph-mon[103849]: from='client.24529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:48:44.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:44 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T13:48:44.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:44 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T13:48:44.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:44 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:45.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:45 smithi057 ceph-mon[99476]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:48:45.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:45 smithi057 ceph-mon[99476]: osdmap e74: 11 total, 11 up, 11 in 2023-10-08T13:48:45.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:45 smithi099 ceph-mon[103849]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:48:45.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:45 smithi099 ceph-mon[103849]: osdmap e74: 11 total, 11 up, 11 in 2023-10-08T13:48:45.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:45 smithi115 ceph-mon[104408]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:48:45.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:45 smithi115 ceph-mon[104408]: osdmap e74: 11 total, 11 up, 11 in 2023-10-08T13:48:46.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:46 smithi057 ceph-mon[99476]: osdmap e75: 11 total, 11 up, 11 in 2023-10-08T13:48:46.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:46 smithi057 ceph-mon[99476]: from='client.? 172.21.15.115:0/334122016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd75deff-374f-4629-ba93-d7b9e20882a6"}]: dispatch 2023-10-08T13:48:46.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:46 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd75deff-374f-4629-ba93-d7b9e20882a6"}]: dispatch 2023-10-08T13:48:46.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:46 smithi057 ceph-mon[99476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd75deff-374f-4629-ba93-d7b9e20882a6"}]': finished 2023-10-08T13:48:46.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:46 smithi057 ceph-mon[99476]: osdmap e76: 12 total, 11 up, 12 in 2023-10-08T13:48:46.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:46 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:48:46.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:46 smithi057 ceph-mon[99476]: from='client.? 172.21.15.115:0/635372496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:48:46.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:46 smithi099 ceph-mon[103849]: osdmap e75: 11 total, 11 up, 11 in 2023-10-08T13:48:46.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:46 smithi099 ceph-mon[103849]: from='client.? 172.21.15.115:0/334122016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd75deff-374f-4629-ba93-d7b9e20882a6"}]: dispatch 2023-10-08T13:48:46.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:46 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd75deff-374f-4629-ba93-d7b9e20882a6"}]: dispatch 2023-10-08T13:48:46.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:46 smithi099 ceph-mon[103849]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd75deff-374f-4629-ba93-d7b9e20882a6"}]': finished 2023-10-08T13:48:46.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:46 smithi099 ceph-mon[103849]: osdmap e76: 12 total, 11 up, 12 in 2023-10-08T13:48:46.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:46 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:48:46.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:46 smithi099 ceph-mon[103849]: from='client.? 172.21.15.115:0/635372496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:48:46.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:46 smithi115 ceph-mon[104408]: osdmap e75: 11 total, 11 up, 11 in 2023-10-08T13:48:46.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:46 smithi115 ceph-mon[104408]: from='client.? 172.21.15.115:0/334122016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd75deff-374f-4629-ba93-d7b9e20882a6"}]: dispatch 2023-10-08T13:48:46.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:46 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd75deff-374f-4629-ba93-d7b9e20882a6"}]: dispatch 2023-10-08T13:48:46.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:46 smithi115 ceph-mon[104408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd75deff-374f-4629-ba93-d7b9e20882a6"}]': finished 2023-10-08T13:48:46.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:46 smithi115 ceph-mon[104408]: osdmap e76: 12 total, 11 up, 12 in 2023-10-08T13:48:46.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:46 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:48:46.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:46 smithi115 ceph-mon[104408]: from='client.? 172.21.15.115:0/635372496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T13:48:47.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:47 smithi099 ceph-mon[103849]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:48:47.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:47 smithi115 ceph-mon[104408]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:48:47.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:47 smithi057 ceph-mon[99476]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:48:49.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:49 smithi115 ceph-mon[104408]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-10-08T13:48:49.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:49 smithi057 ceph-mon[99476]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-10-08T13:48:50.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:49 smithi099 ceph-mon[103849]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-10-08T13:48:51.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:51 smithi115 ceph-mon[104408]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:48:51.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:51 smithi057 ceph-mon[99476]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:48:52.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:51 smithi099 ceph-mon[103849]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T13:48:52.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:52 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-08T13:48:52.924 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:52 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:52.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:52 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-08T13:48:52.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:52 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:53.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:52 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-08T13:48:53.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:52 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:48:53.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:53 smithi057 ceph-mon[99476]: Deploying daemon osd.11 on smithi115 2023-10-08T13:48:53.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:53 smithi057 ceph-mon[99476]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-10-08T13:48:54.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:53 smithi099 ceph-mon[103849]: Deploying daemon osd.11 on smithi115 2023-10-08T13:48:54.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:53 smithi099 ceph-mon[103849]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-10-08T13:48:54.106 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:53 smithi115 ceph-mon[104408]: Deploying daemon osd.11 on smithi115 2023-10-08T13:48:54.106 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:53 smithi115 ceph-mon[104408]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-10-08T13:48:55.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:55 smithi057 ceph-mon[99476]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-08T13:48:56.021 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:55 smithi099 ceph-mon[103849]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-08T13:48:56.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:55 smithi115 ceph-mon[104408]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-08T13:48:57.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:57 smithi115 ceph-mon[104408]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-08T13:48:57.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:57 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:57.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:57 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:57.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:57 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:57.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:57 smithi057 ceph-mon[99476]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-08T13:48:57.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:57 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:57.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:57 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:57.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:57 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:57.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:57 smithi099 ceph-mon[103849]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-08T13:48:57.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:57 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:57.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:57 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:48:57.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:57 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:48:58.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.942+0000 7fab477fe700 1 -- 172.21.15.115:0/2646889206 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fab501107c0 con 0x7fab3c0a1550 2023-10-08T13:48:58.944 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 11 on host 'smithi115' 2023-10-08T13:48:58.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 -- 172.21.15.115:0/2646889206 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab3c0a1550 msgr2=0x7fab3c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:58.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab3c0a1550 0x7fab3c0a3a10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab4c005fd0 tx=0x7fab4c005dd0 comp rx=0 tx=0).stop 2023-10-08T13:48:58.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 -- 172.21.15.115:0/2646889206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab501058d0 msgr2=0x7fab5019d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:48:58.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab501058d0 0x7fab5019d7c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b9e0 tx=0x7fab4800bcf0 comp rx=0 tx=0).stop 2023-10-08T13:48:58.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 -- 172.21.15.115:0/2646889206 shutdown_connections 2023-10-08T13:48:58.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab501058d0 0x7fab5019d7c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:58.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fab500ff190 0x7fab50198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:58.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fab3c0a1550 0x7fab3c0a3a10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:58.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 --2- 172.21.15.115:0/2646889206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab500ffb30 0x7fab50199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:48:58.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 -- 172.21.15.115:0/2646889206 >> 172.21.15.115:0/2646889206 conn(0x7fab50074a10 msgr2=0x7fab50100540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:48:58.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 -- 172.21.15.115:0/2646889206 shutdown_connections 2023-10-08T13:48:58.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:48:58.944+0000 7fab583f9700 1 -- 172.21.15.115:0/2646889206 wait complete. 2023-10-08T13:48:59.396 DEBUG:teuthology.orchestra.run.smithi115:osd.11> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.11.service 2023-10-08T13:48:59.398 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-08T13:48:59.399 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-08T13:48:59.429 INFO:journalctl@ceph.osd.11.smithi115.stdout:-- Logs begin at Sun 2023-10-08 13:28:12 UTC. -- 2023-10-08T13:48:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:49:00.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:59 smithi115 ceph-mon[104408]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T13:49:00.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:59 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:59 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:59 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:48:59 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.211 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:59 smithi057 ceph-mon[99476]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T13:49:00.212 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:59 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.212 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:59 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.212 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:59 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.212 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:48:59 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:59 smithi099 ceph-mon[103849]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T13:49:00.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:59 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:59 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:59 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:48:59 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:00.537 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 13:49:00 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T13:49:00.229+0000 7f6f82570640 -1 osd.11 0 log_to_monitors true 2023-10-08T13:49:01.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:00 smithi115 ceph-mon[104408]: from='osd.11 [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T13:49:01.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:00 smithi115 ceph-mon[104408]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T13:49:01.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.209+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/465411283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d281056b0 msgr2=0x7f1d28105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.209+0000 7f1d2ccd6700 1 --2- 172.21.15.57:0/465411283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d281056b0 0x7f1d28105a90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009990 tx=0x7f1d1c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.209+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/465411283 shutdown_connections 2023-10-08T13:49:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.209+0000 7f1d2ccd6700 1 --2- 172.21.15.57:0/465411283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d28100110 0x7f1d2810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.209+0000 7f1d2ccd6700 1 --2- 172.21.15.57:0/465411283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d281056b0 0x7f1d28105a90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.209+0000 7f1d2ccd6700 1 --2- 172.21.15.57:0/465411283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d280ff750 0x7f1d280ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.209+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/465411283 >> 172.21.15.57:0/465411283 conn(0x7f1d28075790 msgr2=0x7f1d28075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.209+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/465411283 shutdown_connections 2023-10-08T13:49:01.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.209+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/465411283 wait complete. 2023-10-08T13:49:01.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d2ccd6700 1 Processor -- start 2023-10-08T13:49:01.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d2ccd6700 1 -- start start 2023-10-08T13:49:01.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d2ccd6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d280ff750 0x7f1d28197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:01.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d2ccd6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d28100110 0x7f1d28197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:01.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d2ccd6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d281056b0 0x7f1d281a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:01.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d2ccd6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d28198810 con 0x7f1d28100110 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d2ccd6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d281a2fd0 con 0x7f1d280ff750 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d2ccd6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d281a3110 con 0x7f1d281056b0 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d25d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d28100110 0x7f1d28197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d25d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d28100110 0x7f1d28197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53546/0 (socket says 172.21.15.57:53546) 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d25d9b700 1 -- 172.21.15.57:0/3248830765 learned_addr learned my addr 172.21.15.57:0/3248830765 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d2659c700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d280ff750 0x7f1d28197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.210+0000 7f1d26d9d700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d281056b0 0x7f1d281a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d25d9b700 1 -- 172.21.15.57:0/3248830765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d281056b0 msgr2=0x7f1d281a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d25d9b700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d281056b0 0x7f1d281a2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d25d9b700 1 -- 172.21.15.57:0/3248830765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d280ff750 msgr2=0x7f1d28197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:01.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d25d9b700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d280ff750 0x7f1d28197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:01.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d25d9b700 1 -- 172.21.15.57:0/3248830765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c0095f0 con 0x7f1d28100110 2023-10-08T13:49:01.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d25d9b700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d28100110 0x7f1d28197f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800d980 tx=0x7f1d1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:01.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d177fe700 1 -- 172.21.15.57:0/3248830765 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18004220 con 0x7f1d28100110 2023-10-08T13:49:01.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/3248830765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d281a3390 con 0x7f1d28100110 2023-10-08T13:49:01.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d177fe700 1 -- 172.21.15.57:0/3248830765 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d18007640 con 0x7f1d28100110 2023-10-08T13:49:01.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d177fe700 1 -- 172.21.15.57:0/3248830765 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18005690 con 0x7f1d28100110 2023-10-08T13:49:01.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.211+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/3248830765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d281a3890 con 0x7f1d28100110 2023-10-08T13:49:01.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.212+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/3248830765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d28107c40 con 0x7f1d28100110 2023-10-08T13:49:01.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.214+0000 7f1d177fe700 1 -- 172.21.15.57:0/3248830765 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f1d18004750 con 0x7f1d28100110 2023-10-08T13:49:01.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.214+0000 7f1d177fe700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1d100a14d0 0x7f1d100a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:01.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.215+0000 7f1d2659c700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1d100a14d0 0x7f1d100a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:01.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.215+0000 7f1d177fe700 1 -- 172.21.15.57:0/3248830765 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7046+0+0 (secure 0 0 0) 0x7f1d180ccbb0 con 0x7f1d28100110 2023-10-08T13:49:01.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.215+0000 7f1d2659c700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1d100a14d0 0x7f1d100a3990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0098e0 tx=0x7f1d1c005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:01.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.218+0000 7f1d177fe700 1 -- 172.21.15.57:0/3248830765 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1d18096660 con 0x7f1d28100110 2023-10-08T13:49:01.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:00 smithi057 ceph-mon[99476]: from='osd.11 [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T13:49:01.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:00 smithi057 ceph-mon[99476]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T13:49:01.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:00 smithi099 ceph-mon[103849]: from='osd.11 [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T13:49:01.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:00 smithi099 ceph-mon[103849]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T13:49:01.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.417+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/3248830765 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1d28100df0 con 0x7f1d28100110 2023-10-08T13:49:01.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.418+0000 7f1d177fe700 1 -- 172.21.15.57:0/3248830765 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1d18099c80 con 0x7f1d28100110 2023-10-08T13:49:01.418 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:01.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/3248830765 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1d100a14d0 msgr2=0x7f1d100a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:01.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1d100a14d0 0x7f1d100a3990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0098e0 tx=0x7f1d1c005960 comp rx=0 tx=0).stop 2023-10-08T13:49:01.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/3248830765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d28100110 msgr2=0x7f1d28197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:01.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d28100110 0x7f1d28197f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800d980 tx=0x7f1d1800dc90 comp rx=0 tx=0).stop 2023-10-08T13:49:01.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/3248830765 shutdown_connections 2023-10-08T13:49:01.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d281056b0 0x7f1d281a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:01.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1d100a14d0 0x7f1d100a3990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:01.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d280ff750 0x7f1d28197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:01.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 --2- 172.21.15.57:0/3248830765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d28100110 0x7f1d28197f50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:01.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/3248830765 >> 172.21.15.57:0/3248830765 conn(0x7f1d28075790 msgr2=0x7f1d280fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:01.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/3248830765 shutdown_connections 2023-10-08T13:49:01.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:01.420+0000 7f1d2ccd6700 1 -- 172.21.15.57:0/3248830765 wait complete. 2023-10-08T13:49:01.841 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1696772921,"num_in_osds":12,"osd_in_since":1696772925,"num_remapped_pgs":0} 2023-10-08T13:49:02.108 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:01 smithi115 ceph-mon[104408]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:49:02.108 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:01 smithi115 ceph-mon[104408]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-08T13:49:02.108 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:01 smithi115 ceph-mon[104408]: from='osd.11 [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:49:02.108 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:01 smithi115 ceph-mon[104408]: osdmap e77: 12 total, 11 up, 12 in 2023-10-08T13:49:02.108 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:01 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:02.108 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:01 smithi115 ceph-mon[104408]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:49:02.108 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:01 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3248830765' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T13:49:02.109 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 13:49:01 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T13:49:01.953+0000 7f6f77be7700 -1 osd.11 0 waiting for initial osdmap 2023-10-08T13:49:02.109 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 13:49:01 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T13:49:01.965+0000 7f6f719d8700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T13:49:02.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:01 smithi057 ceph-mon[99476]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:49:02.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:01 smithi057 ceph-mon[99476]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-08T13:49:02.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:01 smithi057 ceph-mon[99476]: from='osd.11 [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:49:02.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:01 smithi057 ceph-mon[99476]: osdmap e77: 12 total, 11 up, 12 in 2023-10-08T13:49:02.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:01 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:02.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:01 smithi057 ceph-mon[99476]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:49:02.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:01 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3248830765' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T13:49:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:01 smithi099 ceph-mon[103849]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:49:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:01 smithi099 ceph-mon[103849]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-08T13:49:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:01 smithi099 ceph-mon[103849]: from='osd.11 [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:49:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:01 smithi099 ceph-mon[103849]: osdmap e77: 12 total, 11 up, 12 in 2023-10-08T13:49:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:01 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:01 smithi099 ceph-mon[103849]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-10-08T13:49:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:01 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3248830765' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T13:49:02.841 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-08T13:49:03.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:49:03.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: osdmap e78: 12 total, 11 up, 12 in 2023-10-08T13:49:03.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:03.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:03.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:02 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:03.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:49:03.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: osdmap e78: 12 total, 11 up, 12 in 2023-10-08T13:49:03.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:03.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:03.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:03.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:02 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:03.415 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:49:03.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-10-08T13:49:03.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: osdmap e78: 12 total, 11 up, 12 in 2023-10-08T13:49:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:49:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:03.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:03.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:03.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:03.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:03.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:03.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:02 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:03 smithi057 ceph-mon[99476]: purged_snaps scrub starts 2023-10-08T13:49:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:03 smithi057 ceph-mon[99476]: purged_snaps scrub ok 2023-10-08T13:49:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:03 smithi057 ceph-mon[99476]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:49:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:03 smithi057 ceph-mon[99476]: Detected new or changed devices on smithi115 2023-10-08T13:49:04.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:03 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi115 to 4292M 2023-10-08T13:49:04.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:03 smithi057 ceph-mon[99476]: osd.11 [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] boot 2023-10-08T13:49:04.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:03 smithi057 ceph-mon[99476]: osdmap e79: 12 total, 12 up, 12 in 2023-10-08T13:49:04.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:03 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:03 smithi099 ceph-mon[103849]: purged_snaps scrub starts 2023-10-08T13:49:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:03 smithi099 ceph-mon[103849]: purged_snaps scrub ok 2023-10-08T13:49:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:03 smithi099 ceph-mon[103849]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:49:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:03 smithi099 ceph-mon[103849]: Detected new or changed devices on smithi115 2023-10-08T13:49:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:03 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi115 to 4292M 2023-10-08T13:49:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:03 smithi099 ceph-mon[103849]: osd.11 [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] boot 2023-10-08T13:49:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:03 smithi099 ceph-mon[103849]: osdmap e79: 12 total, 12 up, 12 in 2023-10-08T13:49:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:03 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:04.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:03 smithi115 ceph-mon[104408]: purged_snaps scrub starts 2023-10-08T13:49:04.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:03 smithi115 ceph-mon[104408]: purged_snaps scrub ok 2023-10-08T13:49:04.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:03 smithi115 ceph-mon[104408]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-08T13:49:04.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:03 smithi115 ceph-mon[104408]: Detected new or changed devices on smithi115 2023-10-08T13:49:04.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:03 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi115 to 4292M 2023-10-08T13:49:04.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:03 smithi115 ceph-mon[104408]: osd.11 [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] boot 2023-10-08T13:49:04.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:03 smithi115 ceph-mon[104408]: osdmap e79: 12 total, 12 up, 12 in 2023-10-08T13:49:04.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:03 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T13:49:04.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.718+0000 7eff37391700 1 -- 172.21.15.57:0/3818758152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff30100c60 msgr2=0x7eff3010c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:04.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.718+0000 7eff37391700 1 --2- 172.21.15.57:0/3818758152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff30100c60 0x7eff3010c5c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7eff2800b0a0 tx=0x7eff2800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.718+0000 7eff37391700 1 -- 172.21.15.57:0/3818758152 shutdown_connections 2023-10-08T13:49:04.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.718+0000 7eff37391700 1 --2- 172.21.15.57:0/3818758152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff30100c60 0x7eff3010c5c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.718+0000 7eff37391700 1 --2- 172.21.15.57:0/3818758152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff30104720 0x7eff30104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.718+0000 7eff37391700 1 --2- 172.21.15.57:0/3818758152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff301002a0 0x7eff30100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.718+0000 7eff37391700 1 -- 172.21.15.57:0/3818758152 >> 172.21.15.57:0/3818758152 conn(0x7eff30074a10 msgr2=0x7eff30074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:04.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.719+0000 7eff37391700 1 -- 172.21.15.57:0/3818758152 shutdown_connections 2023-10-08T13:49:04.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.719+0000 7eff37391700 1 -- 172.21.15.57:0/3818758152 wait complete. 2023-10-08T13:49:04.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.719+0000 7eff37391700 1 Processor -- start 2023-10-08T13:49:04.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.719+0000 7eff37391700 1 -- start start 2023-10-08T13:49:04.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.719+0000 7eff37391700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff301002a0 0x7eff30197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:04.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.719+0000 7eff37391700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff30100c60 0x7eff301980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:04.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.719+0000 7eff37391700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff30104720 0x7eff301a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.719+0000 7eff37391700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff30198930 con 0x7eff30104720 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.719+0000 7eff37391700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff301a3140 con 0x7eff301002a0 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.719+0000 7eff37391700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff301a3280 con 0x7eff30100c60 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3592e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff30104720 0x7eff301a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3592e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff30104720 0x7eff301a2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53566/0 (socket says 172.21.15.57:53566) 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3592e700 1 -- 172.21.15.57:0/1679390399 learned_addr learned my addr 172.21.15.57:0/1679390399 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3492c700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff30100c60 0x7eff301980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3512d700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff301002a0 0x7eff30197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3592e700 1 -- 172.21.15.57:0/1679390399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff30100c60 msgr2=0x7eff301980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3592e700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff30100c60 0x7eff301980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3592e700 1 -- 172.21.15.57:0/1679390399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff301002a0 msgr2=0x7eff30197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:04.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3592e700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff301002a0 0x7eff30197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3592e700 1 -- 172.21.15.57:0/1679390399 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff28009d00 con 0x7eff30104720 2023-10-08T13:49:04.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3512d700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff301002a0 0x7eff30197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:04.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff3592e700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff30104720 0x7eff301a2c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff2800b830 tx=0x7eff280077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:04.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff227fc700 1 -- 172.21.15.57:0/1679390399 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2800e040 con 0x7eff30104720 2023-10-08T13:49:04.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff227fc700 1 -- 172.21.15.57:0/1679390399 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff28004e60 con 0x7eff30104720 2023-10-08T13:49:04.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff37391700 1 -- 172.21.15.57:0/1679390399 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff301a34b0 con 0x7eff30104720 2023-10-08T13:49:04.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff227fc700 1 -- 172.21.15.57:0/1679390399 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280053c0 con 0x7eff30104720 2023-10-08T13:49:04.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.720+0000 7eff37391700 1 -- 172.21.15.57:0/1679390399 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff301a3920 con 0x7eff30104720 2023-10-08T13:49:04.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.721+0000 7eff37391700 1 -- 172.21.15.57:0/1679390399 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff30107d10 con 0x7eff30104720 2023-10-08T13:49:04.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.722+0000 7eff227fc700 1 -- 172.21.15.57:0/1679390399 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7eff2801b450 con 0x7eff30104720 2023-10-08T13:49:04.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.723+0000 7eff227fc700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff1c0a1550 0x7eff1c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:04.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.723+0000 7eff3512d700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff1c0a1550 0x7eff1c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:04.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.723+0000 7eff227fc700 1 -- 172.21.15.57:0/1679390399 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7eff280ce3e0 con 0x7eff30104720 2023-10-08T13:49:04.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.723+0000 7eff3512d700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff1c0a1550 0x7eff1c0a3a10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff2c005fd0 tx=0x7eff2c005db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:04.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.727+0000 7eff227fc700 1 -- 172.21.15.57:0/1679390399 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7eff28097e00 con 0x7eff30104720 2023-10-08T13:49:04.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.927+0000 7eff37391700 1 -- 172.21.15.57:0/1679390399 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7eff300ff6e0 con 0x7eff30104720 2023-10-08T13:49:04.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.927+0000 7eff227fc700 1 -- 172.21.15.57:0/1679390399 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7eff2809b420 con 0x7eff30104720 2023-10-08T13:49:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:04.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 -- 172.21.15.57:0/1679390399 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff1c0a1550 msgr2=0x7eff1c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:04.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff1c0a1550 0x7eff1c0a3a10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff2c005fd0 tx=0x7eff2c005db0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 -- 172.21.15.57:0/1679390399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff30104720 msgr2=0x7eff301a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:04.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff30104720 0x7eff301a2c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff2800b830 tx=0x7eff280077f0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 -- 172.21.15.57:0/1679390399 shutdown_connections 2023-10-08T13:49:04.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff30104720 0x7eff301a2c00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff1c0a1550 0x7eff1c0a3a10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff301002a0 0x7eff30197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 --2- 172.21.15.57:0/1679390399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff30100c60 0x7eff301980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:04.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 -- 172.21.15.57:0/1679390399 >> 172.21.15.57:0/1679390399 conn(0x7eff30074a10 msgr2=0x7eff300fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:04.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 -- 172.21.15.57:0/1679390399 shutdown_connections 2023-10-08T13:49:04.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:04.930+0000 7eff37391700 1 -- 172.21.15.57:0/1679390399 wait complete. 2023-10-08T13:49:05.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:04 smithi057 ceph-mon[99476]: osdmap e80: 12 total, 12 up, 12 in 2023-10-08T13:49:05.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:04 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1679390399' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T13:49:05.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:04 smithi099 ceph-mon[103849]: osdmap e80: 12 total, 12 up, 12 in 2023-10-08T13:49:05.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:04 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1679390399' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T13:49:05.273 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1696772942,"num_in_osds":12,"osd_in_since":1696772925,"num_remapped_pgs":0} 2023-10-08T13:49:05.273 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-08T13:49:05.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:04 smithi115 ceph-mon[104408]: osdmap e80: 12 total, 12 up, 12 in 2023-10-08T13:49:05.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:04 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1679390399' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T13:49:05.840 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:49:06.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:05 smithi057 ceph-mon[99476]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:05 smithi099 ceph-mon[103849]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:06.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:05 smithi115 ceph-mon[104408]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:07.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.040+0000 7fe02de27700 1 -- 172.21.15.57:0/1738879198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0280ff750 msgr2=0x7fe0280ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.040+0000 7fe02de27700 1 --2- 172.21.15.57:0/1738879198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0280ff750 0x7fe0280ffbd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009990 tx=0x7fe01c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.040+0000 7fe02de27700 1 -- 172.21.15.57:0/1738879198 shutdown_connections 2023-10-08T13:49:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.040+0000 7fe02de27700 1 --2- 172.21.15.57:0/1738879198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe028100110 0x7fe02810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.040+0000 7fe02de27700 1 --2- 172.21.15.57:0/1738879198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0281056b0 0x7fe028105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.040+0000 7fe02de27700 1 --2- 172.21.15.57:0/1738879198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0280ff750 0x7fe0280ffbd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.040+0000 7fe02de27700 1 -- 172.21.15.57:0/1738879198 >> 172.21.15.57:0/1738879198 conn(0x7fe028075790 msgr2=0x7fe028075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.040+0000 7fe02de27700 1 -- 172.21.15.57:0/1738879198 shutdown_connections 2023-10-08T13:49:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.040+0000 7fe02de27700 1 -- 172.21.15.57:0/1738879198 wait complete. 2023-10-08T13:49:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.040+0000 7fe02de27700 1 Processor -- start 2023-10-08T13:49:07.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe02de27700 1 -- start start 2023-10-08T13:49:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe02de27700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0280ff750 0x7fe028101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe02de27700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe028100110 0x7fe0281017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe02de27700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0281056b0 0x7fe028101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe02de27700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0281a50c0 con 0x7fe028100110 2023-10-08T13:49:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe02de27700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0281a5230 con 0x7fe0281056b0 2023-10-08T13:49:07.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe02de27700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0281a53f0 con 0x7fe0280ff750 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe026ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe028100110 0x7fe0281017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe026ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe028100110 0x7fe0281017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53580/0 (socket says 172.21.15.57:53580) 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe026ffd700 1 -- 172.21.15.57:0/181583643 learned_addr learned my addr 172.21.15.57:0/181583643 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe027fff700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0281056b0 0x7fe028101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe0277fe700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0280ff750 0x7fe028101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe026ffd700 1 -- 172.21.15.57:0/181583643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0280ff750 msgr2=0x7fe028101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe026ffd700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0280ff750 0x7fe028101280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe026ffd700 1 -- 172.21.15.57:0/181583643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0281056b0 msgr2=0x7fe028101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe026ffd700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0281056b0 0x7fe028101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe026ffd700 1 -- 172.21.15.57:0/181583643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c0095f0 con 0x7fe028100110 2023-10-08T13:49:07.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe027fff700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0281056b0 0x7fe028101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:49:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.041+0000 7fe026ffd700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe028100110 0x7fe0281017c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0098e0 tx=0x7fe01c00f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.042+0000 7fe024ff9700 1 -- 172.21.15.57:0/181583643 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c01d070 con 0x7fe028100110 2023-10-08T13:49:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.042+0000 7fe024ff9700 1 -- 172.21.15.57:0/181583643 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe01c00fb90 con 0x7fe028100110 2023-10-08T13:49:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.042+0000 7fe02de27700 1 -- 172.21.15.57:0/181583643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0281a5670 con 0x7fe028100110 2023-10-08T13:49:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.042+0000 7fe024ff9700 1 -- 172.21.15.57:0/181583643 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c017650 con 0x7fe028100110 2023-10-08T13:49:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.042+0000 7fe02de27700 1 -- 172.21.15.57:0/181583643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0281a5a80 con 0x7fe028100110 2023-10-08T13:49:07.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.043+0000 7fe02de27700 1 -- 172.21.15.57:0/181583643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe028108ce0 con 0x7fe028100110 2023-10-08T13:49:07.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.044+0000 7fe024ff9700 1 -- 172.21.15.57:0/181583643 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe01c00fd00 con 0x7fe028100110 2023-10-08T13:49:07.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.044+0000 7fe024ff9700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0100a14d0 0x7fe0100a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:07.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.045+0000 7fe0277fe700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0100a14d0 0x7fe0100a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:07.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.045+0000 7fe024ff9700 1 -- 172.21.15.57:0/181583643 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fe01c0cdae0 con 0x7fe028100110 2023-10-08T13:49:07.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.045+0000 7fe0277fe700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0100a14d0 0x7fe0100a3990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe018005950 tx=0x7fe0180058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:07.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.049+0000 7fe024ff9700 1 -- 172.21.15.57:0/181583643 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe01c0975a0 con 0x7fe028100110 2023-10-08T13:49:07.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.251+0000 7fe02de27700 1 -- 172.21.15.57:0/181583643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe028102710 con 0x7fe028100110 2023-10-08T13:49:07.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.252+0000 7fe024ff9700 1 -- 172.21.15.57:0/181583643 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18686 (secure 0 0 0) 0x7fe01c09abc0 con 0x7fe028100110 2023-10-08T13:49:07.253 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:07.253 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":80,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:03.966059+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:49:07.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.254+0000 7fe02de27700 1 -- 172.21.15.57:0/181583643 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0100a14d0 msgr2=0x7fe0100a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:07.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.254+0000 7fe02de27700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0100a14d0 0x7fe0100a3990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe018005950 tx=0x7fe0180058e0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.254+0000 7fe02de27700 1 -- 172.21.15.57:0/181583643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe028100110 msgr2=0x7fe0281017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:07.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.254+0000 7fe02de27700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe028100110 0x7fe0281017c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0098e0 tx=0x7fe01c00f720 comp rx=0 tx=0).stop 2023-10-08T13:49:07.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.254+0000 7fe02de27700 1 -- 172.21.15.57:0/181583643 shutdown_connections 2023-10-08T13:49:07.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.254+0000 7fe02de27700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0100a14d0 0x7fe0100a3990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.254+0000 7fe02de27700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0281056b0 0x7fe028101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.254+0000 7fe02de27700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0280ff750 0x7fe028101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.254+0000 7fe02de27700 1 --2- 172.21.15.57:0/181583643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe028100110 0x7fe0281017c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:07.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.254+0000 7fe02de27700 1 -- 172.21.15.57:0/181583643 >> 172.21.15.57:0/181583643 conn(0x7fe028075790 msgr2=0x7fe028068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:07.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.255+0000 7fe02de27700 1 -- 172.21.15.57:0/181583643 shutdown_connections 2023-10-08T13:49:07.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:07.255+0000 7fe02de27700 1 -- 172.21.15.57:0/181583643 wait complete. 2023-10-08T13:49:07.600 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T13:49:07.601 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-08T13:49:08.163 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:49:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:07 smithi099 ceph-mon[103849]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:07 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/181583643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:08.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:07 smithi057 ceph-mon[99476]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:08.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:07 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/181583643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:07 smithi115 ceph-mon[104408]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:07 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/181583643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:09.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.405+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/1300469712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ee4100110 msgr2=0x7f3ee410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:09.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.406+0000 7f3ee8ad2700 1 --2- 172.21.15.57:0/1300469712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ee4100110 0x7f3ee410c580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00b0a0 tx=0x7f3edc00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:09.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.406+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/1300469712 shutdown_connections 2023-10-08T13:49:09.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.406+0000 7f3ee8ad2700 1 --2- 172.21.15.57:0/1300469712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ee4100110 0x7f3ee410c580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:09.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.406+0000 7f3ee8ad2700 1 --2- 172.21.15.57:0/1300469712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ee41056b0 0x7f3ee4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:09.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.406+0000 7f3ee8ad2700 1 --2- 172.21.15.57:0/1300469712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee40ff750 0x7f3ee40ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:09.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.406+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/1300469712 >> 172.21.15.57:0/1300469712 conn(0x7f3ee4075790 msgr2=0x7f3ee4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:09.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.406+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/1300469712 shutdown_connections 2023-10-08T13:49:09.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.406+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/1300469712 wait complete. 2023-10-08T13:49:09.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee8ad2700 1 Processor -- start 2023-10-08T13:49:09.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee8ad2700 1 -- start start 2023-10-08T13:49:09.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee8ad2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ee40ff750 0x7f3ee4197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:09.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee8ad2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee4100110 0x7f3ee41980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:09.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee8ad2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ee41056b0 0x7f3ee41a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:09.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee8ad2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee4198980 con 0x7f3ee41056b0 2023-10-08T13:49:09.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee8ad2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee41a3130 con 0x7f3ee4100110 2023-10-08T13:49:09.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee8ad2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee41a3270 con 0x7f3ee40ff750 2023-10-08T13:49:09.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ee41056b0 0x7f3ee41a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:09.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ee41056b0 0x7f3ee41a2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53600/0 (socket says 172.21.15.57:53600) 2023-10-08T13:49:09.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee2d9d700 1 -- 172.21.15.57:0/396401543 learned_addr learned my addr 172.21.15.57:0/396401543 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:09.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.407+0000 7f3ee259c700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ee40ff750 0x7f3ee4197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:09.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ee1d9b700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee4100110 0x7f3ee41980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:09.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ee2d9d700 1 -- 172.21.15.57:0/396401543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ee40ff750 msgr2=0x7f3ee4197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:09.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ee2d9d700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ee40ff750 0x7f3ee4197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:09.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ee2d9d700 1 -- 172.21.15.57:0/396401543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee4100110 msgr2=0x7f3ee41980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:09.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ee2d9d700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee4100110 0x7f3ee41980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:09.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ee2d9d700 1 -- 172.21.15.57:0/396401543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc009d00 con 0x7f3ee41056b0 2023-10-08T13:49:09.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ee2d9d700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ee41056b0 0x7f3ee41a2bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3edc007820 tx=0x7f3edc007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:09.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ed37fe700 1 -- 172.21.15.57:0/396401543 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00e040 con 0x7f3ee41056b0 2023-10-08T13:49:09.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ed37fe700 1 -- 172.21.15.57:0/396401543 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3edc004c60 con 0x7f3ee41056b0 2023-10-08T13:49:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ed37fe700 1 -- 172.21.15.57:0/396401543 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0052a0 con 0x7f3ee41056b0 2023-10-08T13:49:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/396401543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee41a34f0 con 0x7f3ee41056b0 2023-10-08T13:49:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.408+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/396401543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee41a3910 con 0x7f3ee41056b0 2023-10-08T13:49:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.409+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/396401543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee4107cb0 con 0x7f3ee41056b0 2023-10-08T13:49:09.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.410+0000 7f3ed37fe700 1 -- 172.21.15.57:0/396401543 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f3edc01b450 con 0x7f3ee41056b0 2023-10-08T13:49:09.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.411+0000 7f3ed37fe700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3ecc0a14d0 0x7f3ecc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:09.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.411+0000 7f3ee259c700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3ecc0a14d0 0x7f3ecc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:09.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.411+0000 7f3ed37fe700 1 -- 172.21.15.57:0/396401543 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f3edc0ce4f0 con 0x7f3ee41056b0 2023-10-08T13:49:09.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.412+0000 7f3ee259c700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3ecc0a14d0 0x7f3ecc0a3990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4005950 tx=0x7f3ed400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:09.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.415+0000 7f3ed37fe700 1 -- 172.21.15.57:0/396401543 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3edc097fb0 con 0x7f3ee41056b0 2023-10-08T13:49:09.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.613+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/396401543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ee4068740 con 0x7f3ee41056b0 2023-10-08T13:49:09.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.613+0000 7f3ed37fe700 1 -- 172.21.15.57:0/396401543 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3edc09b5d0 con 0x7f3ee41056b0 2023-10-08T13:49:09.615 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:49:09.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.615+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/396401543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3ecc0a14d0 msgr2=0x7f3ecc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:09.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.615+0000 7f3ee8ad2700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3ecc0a14d0 0x7f3ecc0a3990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4005950 tx=0x7f3ed400b410 comp rx=0 tx=0).stop 2023-10-08T13:49:09.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.615+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/396401543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ee41056b0 msgr2=0x7f3ee41a2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:09.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.615+0000 7f3ee8ad2700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ee41056b0 0x7f3ee41a2bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3edc007820 tx=0x7f3edc007850 comp rx=0 tx=0).stop 2023-10-08T13:49:09.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.615+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/396401543 shutdown_connections 2023-10-08T13:49:09.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.615+0000 7f3ee8ad2700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ee41056b0 0x7f3ee41a2bf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:09.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.615+0000 7f3ee8ad2700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ee40ff750 0x7f3ee4197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:09.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.615+0000 7f3ee8ad2700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3ecc0a14d0 0x7f3ecc0a3990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:09.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.616+0000 7f3ee8ad2700 1 --2- 172.21.15.57:0/396401543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee4100110 0x7f3ee41980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:09.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.616+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/396401543 >> 172.21.15.57:0/396401543 conn(0x7f3ee4075790 msgr2=0x7f3ee40fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:09.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.616+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/396401543 shutdown_connections 2023-10-08T13:49:09.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:09.616+0000 7f3ee8ad2700 1 -- 172.21.15.57:0/396401543 wait complete. 2023-10-08T13:49:10.024 INFO:tasks.cephadm:Adding mds.a on smithi057 2023-10-08T13:49:10.024 INFO:tasks.cephadm:Adding mds.d on smithi057 2023-10-08T13:49:10.024 INFO:tasks.cephadm:Adding mds.g on smithi057 2023-10-08T13:49:10.025 INFO:tasks.cephadm:Adding mds.j on smithi057 2023-10-08T13:49:10.025 INFO:tasks.cephadm:Adding mds.b on smithi099 2023-10-08T13:49:10.025 INFO:tasks.cephadm:Adding mds.e on smithi099 2023-10-08T13:49:10.025 INFO:tasks.cephadm:Adding mds.h on smithi099 2023-10-08T13:49:10.025 INFO:tasks.cephadm:Adding mds.k on smithi099 2023-10-08T13:49:10.025 INFO:tasks.cephadm:Adding mds.c on smithi115 2023-10-08T13:49:10.025 INFO:tasks.cephadm:Adding mds.f on smithi115 2023-10-08T13:49:10.025 INFO:tasks.cephadm:Adding mds.i on smithi115 2023-10-08T13:49:10.025 INFO:tasks.cephadm:Adding mds.l on smithi115 2023-10-08T13:49:10.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph orch apply mds all '12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l' 2023-10-08T13:49:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:10 smithi099 ceph-mon[103849]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:10 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/396401543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:10 smithi115 ceph-mon[104408]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:10.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:10 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/396401543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:10.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:10 smithi057 ceph-mon[99476]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:10.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:10 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/396401543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T13:49:11.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.773+0000 7fd256757700 1 -- 172.21.15.115:0/1895725762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2500ff750 msgr2=0x7fd2500ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:11.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.773+0000 7fd256757700 1 --2- 172.21.15.115:0/1895725762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2500ff750 0x7fd2500ffbd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd244009990 tx=0x7fd244009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.773+0000 7fd256757700 1 -- 172.21.15.115:0/1895725762 shutdown_connections 2023-10-08T13:49:11.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.773+0000 7fd256757700 1 --2- 172.21.15.115:0/1895725762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd250100110 0x7fd25010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.773+0000 7fd256757700 1 --2- 172.21.15.115:0/1895725762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2501056b0 0x7fd250105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.773+0000 7fd256757700 1 --2- 172.21.15.115:0/1895725762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2500ff750 0x7fd2500ffbd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.773+0000 7fd256757700 1 -- 172.21.15.115:0/1895725762 >> 172.21.15.115:0/1895725762 conn(0x7fd250075790 msgr2=0x7fd250075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:11.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.773+0000 7fd256757700 1 -- 172.21.15.115:0/1895725762 shutdown_connections 2023-10-08T13:49:11.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.773+0000 7fd256757700 1 -- 172.21.15.115:0/1895725762 wait complete. 2023-10-08T13:49:11.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd256757700 1 Processor -- start 2023-10-08T13:49:11.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd256757700 1 -- start start 2023-10-08T13:49:11.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd256757700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2500ff750 0x7fd250197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:11.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd256757700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd250100110 0x7fd250198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:11.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd256757700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2501056b0 0x7fd2501985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:11.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd256757700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2501a2e10 con 0x7fd2500ff750 2023-10-08T13:49:11.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd256757700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2501a2f80 con 0x7fd250100110 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd256757700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2501a3140 con 0x7fd2501056b0 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd254cf4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2501056b0 0x7fd2501985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd254cf4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2501056b0 0x7fd2501985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52982/0 (socket says 172.21.15.115:52982) 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd254cf4700 1 -- 172.21.15.115:0/835343187 learned_addr learned my addr 172.21.15.115:0/835343187 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.774+0000 7fd24ffff700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2500ff750 0x7fd250197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd24f7fe700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd250100110 0x7fd250198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd254cf4700 1 -- 172.21.15.115:0/835343187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd250100110 msgr2=0x7fd250198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd254cf4700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd250100110 0x7fd250198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd254cf4700 1 -- 172.21.15.115:0/835343187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2500ff750 msgr2=0x7fd250197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd254cf4700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2500ff750 0x7fd250197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd254cf4700 1 -- 172.21.15.115:0/835343187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440095f0 con 0x7fd2501056b0 2023-10-08T13:49:11.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd24f7fe700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd250100110 0x7fd250198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:49:11.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd254cf4700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2501056b0 0x7fd2501985c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd248007ee0 tx=0x7fd24800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:11.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd24d7fa700 1 -- 172.21.15.115:0/835343187 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248017070 con 0x7fd2501056b0 2023-10-08T13:49:11.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd24d7fa700 1 -- 172.21.15.115:0/835343187 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd24800f040 con 0x7fd2501056b0 2023-10-08T13:49:11.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd256757700 1 -- 172.21.15.115:0/835343187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2501a3420 con 0x7fd2501056b0 2023-10-08T13:49:11.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd24d7fa700 1 -- 172.21.15.115:0/835343187 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248005740 con 0x7fd2501056b0 2023-10-08T13:49:11.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.775+0000 7fd256757700 1 -- 172.21.15.115:0/835343187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2501a38d0 con 0x7fd2501056b0 2023-10-08T13:49:11.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.776+0000 7fd256757700 1 -- 172.21.15.115:0/835343187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd250107cd0 con 0x7fd2501056b0 2023-10-08T13:49:11.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.777+0000 7fd24d7fa700 1 -- 172.21.15.115:0/835343187 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd2480058a0 con 0x7fd2501056b0 2023-10-08T13:49:11.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.777+0000 7fd24d7fa700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd2380a14d0 0x7fd2380a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:11.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.778+0000 7fd24d7fa700 1 -- 172.21.15.115:0/835343187 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fd2480cc240 con 0x7fd2501056b0 2023-10-08T13:49:11.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.778+0000 7fd24ffff700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd2380a14d0 0x7fd2380a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:11.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.778+0000 7fd24ffff700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd2380a14d0 0x7fd2380a3990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd240006fd0 tx=0x7fd240008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:11.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.781+0000 7fd24d7fa700 1 -- 172.21.15.115:0/835343187 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd248095c70 con 0x7fd2501056b0 2023-10-08T13:49:11.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.979+0000 7fd256757700 1 -- 172.21.15.115:0/835343187 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fd25010f6a0 con 0x7fd2380a14d0 2023-10-08T13:49:11.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.988+0000 7fd24d7fa700 1 -- 172.21.15.115:0/835343187 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fd25010f6a0 con 0x7fd2380a14d0 2023-10-08T13:49:11.988 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled mds.all update... 2023-10-08T13:49:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.990+0000 7fd256757700 1 -- 172.21.15.115:0/835343187 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd2380a14d0 msgr2=0x7fd2380a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.990+0000 7fd256757700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd2380a14d0 0x7fd2380a3990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd240006fd0 tx=0x7fd240008040 comp rx=0 tx=0).stop 2023-10-08T13:49:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.990+0000 7fd256757700 1 -- 172.21.15.115:0/835343187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2501056b0 msgr2=0x7fd2501985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.990+0000 7fd256757700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2501056b0 0x7fd2501985c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd248007ee0 tx=0x7fd24800d3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.991+0000 7fd256757700 1 -- 172.21.15.115:0/835343187 shutdown_connections 2023-10-08T13:49:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.991+0000 7fd256757700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2501056b0 0x7fd2501985c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.991+0000 7fd256757700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2500ff750 0x7fd250197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.991+0000 7fd256757700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd2380a14d0 0x7fd2380a3990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.991+0000 7fd256757700 1 --2- 172.21.15.115:0/835343187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd250100110 0x7fd250198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.991+0000 7fd256757700 1 -- 172.21.15.115:0/835343187 >> 172.21.15.115:0/835343187 conn(0x7fd250075790 msgr2=0x7fd2500fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.991+0000 7fd256757700 1 -- 172.21.15.115:0/835343187 shutdown_connections 2023-10-08T13:49:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-10-08T13:49:11.991+0000 7fd256757700 1 -- 172.21.15.115:0/835343187 wait complete. 2023-10-08T13:49:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:12 smithi099 ceph-mon[103849]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:12.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:12 smithi115 ceph-mon[104408]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:12.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:12.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:12.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:12.432 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:12.453 DEBUG:teuthology.orchestra.run.smithi057:mds.a> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.a.service 2023-10-08T13:49:12.456 DEBUG:teuthology.orchestra.run.smithi057:mds.d> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.d.service 2023-10-08T13:49:12.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:12 smithi057 ceph-mon[99476]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:12.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:12.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:12.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:12.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:12.487 DEBUG:teuthology.orchestra.run.smithi057:mds.g> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.g.service 2023-10-08T13:49:12.494 INFO:journalctl@ceph.mds.a.smithi057.stdout:-- Logs begin at Sun 2023-10-08 13:28:06 UTC. -- 2023-10-08T13:49:12.495 DEBUG:teuthology.orchestra.run.smithi057:mds.j> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.j.service 2023-10-08T13:49:12.519 INFO:journalctl@ceph.mds.d.smithi057.stdout:-- Logs begin at Sun 2023-10-08 13:28:06 UTC. -- 2023-10-08T13:49:12.520 DEBUG:teuthology.orchestra.run.smithi099:mds.b> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.b.service 2023-10-08T13:49:12.522 DEBUG:teuthology.orchestra.run.smithi099:mds.e> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.e.service 2023-10-08T13:49:12.526 INFO:journalctl@ceph.mds.g.smithi057.stdout:-- Logs begin at Sun 2023-10-08 13:28:06 UTC. -- 2023-10-08T13:49:12.549 INFO:journalctl@ceph.mds.j.smithi057.stdout:-- Logs begin at Sun 2023-10-08 13:28:06 UTC. -- 2023-10-08T13:49:12.559 INFO:journalctl@ceph.mds.b.smithi099.stdout:-- Logs begin at Sun 2023-10-08 13:27:56 UTC. -- 2023-10-08T13:49:12.561 DEBUG:teuthology.orchestra.run.smithi099:mds.h> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.h.service 2023-10-08T13:49:12.590 INFO:journalctl@ceph.mds.e.smithi099.stdout:-- Logs begin at Sun 2023-10-08 13:27:56 UTC. -- 2023-10-08T13:49:12.591 DEBUG:teuthology.orchestra.run.smithi099:mds.k> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.k.service 2023-10-08T13:49:12.618 INFO:journalctl@ceph.mds.h.smithi099.stdout:-- Logs begin at Sun 2023-10-08 13:27:56 UTC. -- 2023-10-08T13:49:12.620 DEBUG:teuthology.orchestra.run.smithi115:mds.c> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.c.service 2023-10-08T13:49:12.622 DEBUG:teuthology.orchestra.run.smithi115:mds.f> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.f.service 2023-10-08T13:49:12.645 INFO:journalctl@ceph.mds.k.smithi099.stdout:-- Logs begin at Sun 2023-10-08 13:27:56 UTC. -- 2023-10-08T13:49:12.651 INFO:journalctl@ceph.mds.c.smithi115.stdout:-- Logs begin at Sun 2023-10-08 13:28:12 UTC. -- 2023-10-08T13:49:12.652 DEBUG:teuthology.orchestra.run.smithi115:mds.i> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.i.service 2023-10-08T13:49:12.677 INFO:journalctl@ceph.mds.f.smithi115.stdout:-- Logs begin at Sun 2023-10-08 13:28:12 UTC. -- 2023-10-08T13:49:12.678 DEBUG:teuthology.orchestra.run.smithi115:mds.l> sudo journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.l.service 2023-10-08T13:49:12.703 INFO:journalctl@ceph.mds.i.smithi115.stdout:-- Logs begin at Sun 2023-10-08 13:28:12 UTC. -- 2023-10-08T13:49:12.704 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-08T13:49:12.704 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:49:12.731 INFO:journalctl@ceph.mds.l.smithi115.stdout:-- Logs begin at Sun 2023-10-08 13:28:12 UTC. -- 2023-10-08T13:49:12.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.841+0000 7f3490cc6700 1 -- 172.21.15.57:0/2436220996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f348c0fff00 msgr2=0x7f348c10c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:12.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.841+0000 7f3490cc6700 1 --2- 172.21.15.57:0/2436220996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f348c0fff00 0x7f348c10c3c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f348400b0d0 tx=0x7f348400b3e0 comp rx=0 tx=0).stop 2023-10-08T13:49:12.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.841+0000 7f3490cc6700 1 -- 172.21.15.57:0/2436220996 shutdown_connections 2023-10-08T13:49:12.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.841+0000 7f3490cc6700 1 --2- 172.21.15.57:0/2436220996 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f348c0fff00 0x7f348c10c3c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:12.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.841+0000 7f3490cc6700 1 --2- 172.21.15.57:0/2436220996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f348c105600 0x7f348c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:12.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.841+0000 7f3490cc6700 1 --2- 172.21.15.57:0/2436220996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348c0ff540 0x7f348c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:12.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.841+0000 7f3490cc6700 1 -- 172.21.15.57:0/2436220996 >> 172.21.15.57:0/2436220996 conn(0x7f348c0fb030 msgr2=0x7f348c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:12.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.841+0000 7f3490cc6700 1 -- 172.21.15.57:0/2436220996 shutdown_connections 2023-10-08T13:49:12.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.841+0000 7f3490cc6700 1 -- 172.21.15.57:0/2436220996 wait complete. 2023-10-08T13:49:12.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f3490cc6700 1 Processor -- start 2023-10-08T13:49:12.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f3490cc6700 1 -- start start 2023-10-08T13:49:12.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f3490cc6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348c0ff540 0x7f348c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:12.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f3490cc6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f348c0fff00 0x7f348c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:12.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f3490cc6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f348c105600 0x7f348c196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:12.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f3490cc6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c196f80 con 0x7f348c105600 2023-10-08T13:49:12.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f3490cc6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c1970f0 con 0x7f348c0ff540 2023-10-08T13:49:12.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f3490cc6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c1972b0 con 0x7f348c0fff00 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f348ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f348c105600 0x7f348c196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f348ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f348c105600 0x7f348c196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53614/0 (socket says 172.21.15.57:53614) 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f348ad9d700 1 -- 172.21.15.57:0/1659864858 learned_addr learned my addr 172.21.15.57:0/1659864858 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f3489d9b700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f348c0fff00 0x7f348c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f348a59c700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348c0ff540 0x7f348c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f348ad9d700 1 -- 172.21.15.57:0/1659864858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f348c0fff00 msgr2=0x7f348c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f348ad9d700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f348c0fff00 0x7f348c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f348ad9d700 1 -- 172.21.15.57:0/1659864858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348c0ff540 msgr2=0x7f348c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f348ad9d700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348c0ff540 0x7f348c19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.842+0000 7f348ad9d700 1 -- 172.21.15.57:0/1659864858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3484009d00 con 0x7f348c105600 2023-10-08T13:49:12.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.843+0000 7f348ad9d700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f348c105600 0x7f348c196920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f348400d010 tx=0x7f3484009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:12.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.843+0000 7f34837fe700 1 -- 172.21.15.57:0/1659864858 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348400e040 con 0x7f348c105600 2023-10-08T13:49:12.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.843+0000 7f3490cc6700 1 -- 172.21.15.57:0/1659864858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c197530 con 0x7f348c105600 2023-10-08T13:49:12.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.843+0000 7f34837fe700 1 -- 172.21.15.57:0/1659864858 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3484005070 con 0x7f348c105600 2023-10-08T13:49:12.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.843+0000 7f34837fe700 1 -- 172.21.15.57:0/1659864858 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348401b6d0 con 0x7f348c105600 2023-10-08T13:49:12.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.843+0000 7f3490cc6700 1 -- 172.21.15.57:0/1659864858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c1a37e0 con 0x7f348c105600 2023-10-08T13:49:12.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.844+0000 7f3490cc6700 1 -- 172.21.15.57:0/1659864858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348c100c60 con 0x7f348c105600 2023-10-08T13:49:12.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.845+0000 7f34837fe700 1 -- 172.21.15.57:0/1659864858 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f34840054e0 con 0x7f348c105600 2023-10-08T13:49:12.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.845+0000 7f34837fe700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34780a14d0 0x7f34780a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:12.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.846+0000 7f34837fe700 1 -- 172.21.15.57:0/1659864858 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f34840cd9e0 con 0x7f348c105600 2023-10-08T13:49:12.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.846+0000 7f348a59c700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34780a14d0 0x7f34780a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:12.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.846+0000 7f348a59c700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34780a14d0 0x7f34780a3990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f347c00abd0 tx=0x7f347c00a380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:12.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:12.850+0000 7f34837fe700 1 -- 172.21.15.57:0/1659864858 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f34840974a0 con 0x7f348c105600 2023-10-08T13:49:13.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.056+0000 7f3490cc6700 1 -- 172.21.15.57:0/1659864858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f348c067450 con 0x7f348c105600 2023-10-08T13:49:13.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.057+0000 7f34837fe700 1 -- 172.21.15.57:0/1659864858 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18686 (secure 0 0 0) 0x7f348409aac0 con 0x7f348c105600 2023-10-08T13:49:13.058 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:13.059 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":80,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:03.966059+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:49:13.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 -- 172.21.15.57:0/1659864858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34780a14d0 msgr2=0x7f34780a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34780a14d0 0x7f34780a3990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f347c00abd0 tx=0x7f347c00a380 comp rx=0 tx=0).stop 2023-10-08T13:49:13.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 -- 172.21.15.57:0/1659864858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f348c105600 msgr2=0x7f348c196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f348c105600 0x7f348c196920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f348400d010 tx=0x7f3484009440 comp rx=0 tx=0).stop 2023-10-08T13:49:13.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 -- 172.21.15.57:0/1659864858 shutdown_connections 2023-10-08T13:49:13.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f348c105600 0x7f348c196920 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34780a14d0 0x7f34780a3990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f348c0ff540 0x7f348c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 --2- 172.21.15.57:0/1659864858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f348c0fff00 0x7f348c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 -- 172.21.15.57:0/1659864858 >> 172.21.15.57:0/1659864858 conn(0x7f348c0fb030 msgr2=0x7f348c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:13.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 -- 172.21.15.57:0/1659864858 shutdown_connections 2023-10-08T13:49:13.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.059+0000 7f3490cc6700 1 -- 172.21.15.57:0/1659864858 wait complete. 2023-10-08T13:49:13.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T13:49:13.073 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:49:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.202+0000 7f6b6d66a700 1 -- 172.21.15.57:0/4160972348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b68105640 msgr2=0x7f6b68105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.202+0000 7f6b6d66a700 1 --2- 172.21.15.57:0/4160972348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b68105640 0x7f6b68105a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009990 tx=0x7f6b5c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.202+0000 7f6b6d66a700 1 -- 172.21.15.57:0/4160972348 shutdown_connections 2023-10-08T13:49:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.202+0000 7f6b6d66a700 1 --2- 172.21.15.57:0/4160972348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b680ffff0 0x7f6b6810c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.202+0000 7f6b6d66a700 1 --2- 172.21.15.57:0/4160972348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b68105640 0x7f6b68105a20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.202+0000 7f6b6d66a700 1 --2- 172.21.15.57:0/4160972348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b680ff630 0x7f6b680ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.202+0000 7f6b6d66a700 1 -- 172.21.15.57:0/4160972348 >> 172.21.15.57:0/4160972348 conn(0x7f6b680767f0 msgr2=0x7f6b68076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.202+0000 7f6b6d66a700 1 -- 172.21.15.57:0/4160972348 shutdown_connections 2023-10-08T13:49:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.202+0000 7f6b6d66a700 1 -- 172.21.15.57:0/4160972348 wait complete. 2023-10-08T13:49:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b6d66a700 1 Processor -- start 2023-10-08T13:49:13.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b6d66a700 1 -- start start 2023-10-08T13:49:13.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b6d66a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b680ff630 0x7f6b681979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:13.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b6d66a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b680ffff0 0x7f6b68197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b6d66a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b68105640 0x7f6b681a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b6d66a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68198750 con 0x7f6b680ffff0 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b6d66a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b681a2f60 con 0x7f6b680ff630 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b6d66a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b681a30a0 con 0x7f6b68105640 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b667fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b680ffff0 0x7f6b68197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b667fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b680ffff0 0x7f6b68197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53626/0 (socket says 172.21.15.57:53626) 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b667fc700 1 -- 172.21.15.57:0/737672191 learned_addr learned my addr 172.21.15.57:0/737672191 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b677fe700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b68105640 0x7f6b681a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.203+0000 7f6b66ffd700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b680ff630 0x7f6b681979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b667fc700 1 -- 172.21.15.57:0/737672191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b68105640 msgr2=0x7f6b681a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b667fc700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b68105640 0x7f6b681a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b667fc700 1 -- 172.21.15.57:0/737672191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b680ff630 msgr2=0x7f6b681979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b667fc700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b680ff630 0x7f6b681979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b667fc700 1 -- 172.21.15.57:0/737672191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0095f0 con 0x7f6b680ffff0 2023-10-08T13:49:13.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b66ffd700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b680ff630 0x7f6b681979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:49:13.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b667fc700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b680ffff0 0x7f6b68197ee0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800d950 tx=0x7f6b5800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:13.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b4ffff700 1 -- 172.21.15.57:0/737672191 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58009940 con 0x7f6b680ffff0 2023-10-08T13:49:13.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b6d66a700 1 -- 172.21.15.57:0/737672191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b681a3330 con 0x7f6b680ffff0 2023-10-08T13:49:13.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b4ffff700 1 -- 172.21.15.57:0/737672191 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b58010460 con 0x7f6b680ffff0 2023-10-08T13:49:13.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b4ffff700 1 -- 172.21.15.57:0/737672191 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800f620 con 0x7f6b680ffff0 2023-10-08T13:49:13.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.204+0000 7f6b6d66a700 1 -- 172.21.15.57:0/737672191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b681a3840 con 0x7f6b680ffff0 2023-10-08T13:49:13.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.205+0000 7f6b6d66a700 1 -- 172.21.15.57:0/737672191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b68107be0 con 0x7f6b680ffff0 2023-10-08T13:49:13.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.206+0000 7f6b4ffff700 1 -- 172.21.15.57:0/737672191 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6b580108d0 con 0x7f6b680ffff0 2023-10-08T13:49:13.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.207+0000 7f6b4ffff700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6b500a14d0 0x7f6b500a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:13.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.207+0000 7f6b66ffd700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6b500a14d0 0x7f6b500a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.207+0000 7f6b4ffff700 1 -- 172.21.15.57:0/737672191 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f6b580cc8a0 con 0x7f6b680ffff0 2023-10-08T13:49:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.207+0000 7f6b66ffd700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6b500a14d0 0x7f6b500a3990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b5c0 tx=0x7f6b5c005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:13.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.211+0000 7f6b4ffff700 1 -- 172.21.15.57:0/737672191 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6b58096230 con 0x7f6b680ffff0 2023-10-08T13:49:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:13 smithi099 ceph-mon[103849]: from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:49:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:13 smithi099 ceph-mon[103849]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2023-10-08T13:49:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:13 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:13 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:13 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:13 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:13.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:13 smithi115 ceph-mon[104408]: from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:49:13.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:13 smithi115 ceph-mon[104408]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2023-10-08T13:49:13.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:13 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:13.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:13 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:13.328 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:13 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:13.329 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:13 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:13.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.412+0000 7f6b6d66a700 1 -- 172.21.15.57:0/737672191 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b680692f0 con 0x7f6b680ffff0 2023-10-08T13:49:13.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.413+0000 7f6b4ffff700 1 -- 172.21.15.57:0/737672191 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b58099850 con 0x7f6b680ffff0 2023-10-08T13:49:13.414 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:49:13.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 -- 172.21.15.57:0/737672191 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6b500a14d0 msgr2=0x7f6b500a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6b500a14d0 0x7f6b500a3990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b5c0 tx=0x7f6b5c005fb0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 -- 172.21.15.57:0/737672191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b680ffff0 msgr2=0x7f6b68197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b680ffff0 0x7f6b68197ee0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800d950 tx=0x7f6b5800dc60 comp rx=0 tx=0).stop 2023-10-08T13:49:13.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 -- 172.21.15.57:0/737672191 shutdown_connections 2023-10-08T13:49:13.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b68105640 0x7f6b681a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6b500a14d0 0x7f6b500a3990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b680ff630 0x7f6b681979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 --2- 172.21.15.57:0/737672191 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b680ffff0 0x7f6b68197ee0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 -- 172.21.15.57:0/737672191 >> 172.21.15.57:0/737672191 conn(0x7f6b680767f0 msgr2=0x7f6b681022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:13.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 -- 172.21.15.57:0/737672191 shutdown_connections 2023-10-08T13:49:13.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.415+0000 7f6b6d66a700 1 -- 172.21.15.57:0/737672191 wait complete. 2023-10-08T13:49:13.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:13 smithi057 ceph-mon[99476]: from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:49:13.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:13 smithi057 ceph-mon[99476]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2023-10-08T13:49:13.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:13 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:13.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:13 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:13.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:13 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:13.428 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:13 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:13.429 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:13.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.553+0000 7fbf84c45700 1 -- 172.21.15.57:0/3012163339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf800fff10 msgr2=0x7fbf8010c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.553+0000 7fbf84c45700 1 --2- 172.21.15.57:0/3012163339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf800fff10 0x7fbf8010c3d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800b0d0 tx=0x7fbf7800b3e0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.553+0000 7fbf84c45700 1 -- 172.21.15.57:0/3012163339 shutdown_connections 2023-10-08T13:49:13.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.553+0000 7fbf84c45700 1 --2- 172.21.15.57:0/3012163339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf800fff10 0x7fbf8010c3d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.553+0000 7fbf84c45700 1 --2- 172.21.15.57:0/3012163339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf80105610 0x7fbf801059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.553+0000 7fbf84c45700 1 --2- 172.21.15.57:0/3012163339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf800ff550 0x7fbf800ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.554+0000 7fbf84c45700 1 -- 172.21.15.57:0/3012163339 >> 172.21.15.57:0/3012163339 conn(0x7fbf800fb060 msgr2=0x7fbf800fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:13.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.554+0000 7fbf84c45700 1 -- 172.21.15.57:0/3012163339 shutdown_connections 2023-10-08T13:49:13.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.554+0000 7fbf84c45700 1 -- 172.21.15.57:0/3012163339 wait complete. 2023-10-08T13:49:13.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.554+0000 7fbf84c45700 1 Processor -- start 2023-10-08T13:49:13.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.554+0000 7fbf84c45700 1 -- start start 2023-10-08T13:49:13.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf84c45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf800ff550 0x7fbf80197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:13.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf84c45700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf800fff10 0x7fbf80197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:13.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf84c45700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf80105610 0x7fbf801a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:13.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf84c45700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf801987e0 con 0x7fbf80105610 2023-10-08T13:49:13.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf84c45700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf801a2ff0 con 0x7fbf800fff10 2023-10-08T13:49:13.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf84c45700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf801a3130 con 0x7fbf800ff550 2023-10-08T13:49:13.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf80105610 0x7fbf801a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf80105610 0x7fbf801a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53642/0 (socket says 172.21.15.57:53642) 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7ed9d700 1 -- 172.21.15.57:0/4251856223 learned_addr learned my addr 172.21.15.57:0/4251856223 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7dd9b700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf800fff10 0x7fbf80197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7e59c700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf800ff550 0x7fbf80197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7ed9d700 1 -- 172.21.15.57:0/4251856223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf800ff550 msgr2=0x7fbf80197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7ed9d700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf800ff550 0x7fbf80197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7ed9d700 1 -- 172.21.15.57:0/4251856223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf800fff10 msgr2=0x7fbf80197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7ed9d700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf800fff10 0x7fbf80197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7ed9d700 1 -- 172.21.15.57:0/4251856223 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf78009d00 con 0x7fbf80105610 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf7ed9d700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf80105610 0x7fbf801a2ab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbf78007fd0 tx=0x7fbf780094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:13.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf777fe700 1 -- 172.21.15.57:0/4251856223 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7800e040 con 0x7fbf80105610 2023-10-08T13:49:13.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf777fe700 1 -- 172.21.15.57:0/4251856223 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf78005050 con 0x7fbf80105610 2023-10-08T13:49:13.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf777fe700 1 -- 172.21.15.57:0/4251856223 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7801b6b0 con 0x7fbf80105610 2023-10-08T13:49:13.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.555+0000 7fbf84c45700 1 -- 172.21.15.57:0/4251856223 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf801a3360 con 0x7fbf80105610 2023-10-08T13:49:13.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.556+0000 7fbf84c45700 1 -- 172.21.15.57:0/4251856223 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf801a3860 con 0x7fbf80105610 2023-10-08T13:49:13.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.557+0000 7fbf84c45700 1 -- 172.21.15.57:0/4251856223 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf80107ab0 con 0x7fbf80105610 2023-10-08T13:49:13.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.558+0000 7fbf777fe700 1 -- 172.21.15.57:0/4251856223 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fbf78004a50 con 0x7fbf80105610 2023-10-08T13:49:13.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.559+0000 7fbf777fe700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbf6c0a1550 0x7fbf6c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:13.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.559+0000 7fbf7e59c700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbf6c0a1550 0x7fbf6c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:13.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.559+0000 7fbf777fe700 1 -- 172.21.15.57:0/4251856223 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fbf780ce1a0 con 0x7fbf80105610 2023-10-08T13:49:13.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.559+0000 7fbf7e59c700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbf6c0a1550 0x7fbf6c0a3a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf68005950 tx=0x7fbf6800a300 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:13.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.562+0000 7fbf777fe700 1 -- 172.21.15.57:0/4251856223 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbf78097bc0 con 0x7fbf80105610 2023-10-08T13:49:13.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.806+0000 7fbf84c45700 1 -- 172.21.15.57:0/4251856223 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf80069270 con 0x7fbf80105610 2023-10-08T13:49:13.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.807+0000 7fbf777fe700 1 -- 172.21.15.57:0/4251856223 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fbf7809b1e0 con 0x7fbf80105610 2023-10-08T13:49:13.808 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:13.808 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-08T13:49:13.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 -- 172.21.15.57:0/4251856223 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbf6c0a1550 msgr2=0x7fbf6c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbf6c0a1550 0x7fbf6c0a3a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf68005950 tx=0x7fbf6800a300 comp rx=0 tx=0).stop 2023-10-08T13:49:13.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 -- 172.21.15.57:0/4251856223 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf80105610 msgr2=0x7fbf801a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:13.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf80105610 0x7fbf801a2ab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbf78007fd0 tx=0x7fbf780094c0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 -- 172.21.15.57:0/4251856223 shutdown_connections 2023-10-08T13:49:13.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbf80105610 0x7fbf801a2ab0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf800ff550 0x7fbf80197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbf6c0a1550 0x7fbf6c0a3a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 --2- 172.21.15.57:0/4251856223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf800fff10 0x7fbf80197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:13.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 -- 172.21.15.57:0/4251856223 >> 172.21.15.57:0/4251856223 conn(0x7fbf800fb060 msgr2=0x7fbf800fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:13.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 -- 172.21.15.57:0/4251856223 shutdown_connections 2023-10-08T13:49:13.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:13.809+0000 7fbf84c45700 1 -- 172.21.15.57:0/4251856223 wait complete. 2023-10-08T13:49:13.811 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 1 2023-10-08T13:49:13.908 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 13:49:13 smithi115 systemd[1]: Starting Ceph mds.l for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:14.167 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 13:49:13 smithi115 podman[140835]: 2023-10-08T13:49:14.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:14 smithi115 ceph-mon[104408]: Deploying daemon mds.l on smithi115 2023-10-08T13:49:14.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:14 smithi115 ceph-mon[104408]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:14.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:14 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1659864858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:14.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:14 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/737672191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:14.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:14 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4251856223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:14.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:14 smithi057 ceph-mon[99476]: Deploying daemon mds.l on smithi115 2023-10-08T13:49:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:14 smithi057 ceph-mon[99476]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1659864858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/737672191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4251856223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:14 smithi099 ceph-mon[103849]: Deploying daemon mds.l on smithi115 2023-10-08T13:49:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:14 smithi099 ceph-mon[103849]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:14 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1659864858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:14 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/737672191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:14 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4251856223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:14.436 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 13:49:14 smithi115 ceph-mds[140875]: -- 172.21.15.115:0/845022794 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56451788f1e0 con 0x564517865000 2023-10-08T13:49:14.437 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 13:49:14 smithi115 bash[140835]: 59c97729b96f371a34d005e1a129a431ca1463d4b03d5c29e9ebfeaec1e64c55 2023-10-08T13:49:14.437 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 13:49:14 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-l[140851]: starting mds.l at 2023-10-08T13:49:14.437 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 13:49:14 smithi115 systemd[1]: Started Ceph mds.l for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:15.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:15.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6832/3794414355,v1:172.21.15.115:6833/3794414355] up:boot 2023-10-08T13:49:15.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: fsmap 1 up:standby 2023-10-08T13:49:15.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T13:49:15.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:15.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:15.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:15.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:15.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:15.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:15.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:15 smithi099 ceph-mon[103849]: Deploying daemon mds.k on smithi099 2023-10-08T13:49:15.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:15.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6832/3794414355,v1:172.21.15.115:6833/3794414355] up:boot 2023-10-08T13:49:15.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: fsmap 1 up:standby 2023-10-08T13:49:15.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T13:49:15.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:15.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:15.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:15.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:15.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:15.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:15.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:15 smithi057 ceph-mon[99476]: Deploying daemon mds.k on smithi099 2023-10-08T13:49:15.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:15.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6832/3794414355,v1:172.21.15.115:6833/3794414355] up:boot 2023-10-08T13:49:15.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: fsmap 1 up:standby 2023-10-08T13:49:15.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T13:49:15.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:15.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:15.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:15.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:15.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:15.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:15.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:15 smithi115 ceph-mon[104408]: Deploying daemon mds.k on smithi099 2023-10-08T13:49:15.823 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:15.850 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 13:49:15 smithi099 systemd[1]: Starting Ceph mds.k for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:15.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.962+0000 7f719412a700 1 -- 172.21.15.57:0/3186050818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718c0ff540 msgr2=0x7f718c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.962+0000 7f719412a700 1 --2- 172.21.15.57:0/3186050818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718c0ff540 0x7f718c0ff9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f71880099c0 tx=0x7f7188009cd0 comp rx=0 tx=0).stop 2023-10-08T13:49:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.962+0000 7f719412a700 1 -- 172.21.15.57:0/3186050818 shutdown_connections 2023-10-08T13:49:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.962+0000 7f719412a700 1 --2- 172.21.15.57:0/3186050818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c0fff00 0x7f718c10c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.962+0000 7f719412a700 1 --2- 172.21.15.57:0/3186050818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f718c105600 0x7f718c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.962+0000 7f719412a700 1 --2- 172.21.15.57:0/3186050818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718c0ff540 0x7f718c0ff9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.962+0000 7f719412a700 1 -- 172.21.15.57:0/3186050818 >> 172.21.15.57:0/3186050818 conn(0x7f718c0fb030 msgr2=0x7f718c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.962+0000 7f719412a700 1 -- 172.21.15.57:0/3186050818 shutdown_connections 2023-10-08T13:49:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.962+0000 7f719412a700 1 -- 172.21.15.57:0/3186050818 wait complete. 2023-10-08T13:49:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f719412a700 1 Processor -- start 2023-10-08T13:49:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f719412a700 1 -- start start 2023-10-08T13:49:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f719412a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718c0ff540 0x7f718c19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f719412a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f718c0fff00 0x7f718c19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f719412a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c105600 0x7f718c1957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f719412a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c195e30 con 0x7f718c0ff540 2023-10-08T13:49:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f719412a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c195fa0 con 0x7f718c105600 2023-10-08T13:49:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f719412a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c196160 con 0x7f718c0fff00 2023-10-08T13:49:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f7191ec6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718c0ff540 0x7f718c19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f7191ec6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718c0ff540 0x7f718c19b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53652/0 (socket says 172.21.15.57:53652) 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f7191ec6700 1 -- 172.21.15.57:0/3431299209 learned_addr learned my addr 172.21.15.57:0/3431299209 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f71926c7700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c105600 0x7f718c1957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f71916c5700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f718c0fff00 0x7f718c19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f7191ec6700 1 -- 172.21.15.57:0/3431299209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f718c0fff00 msgr2=0x7f718c19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f7191ec6700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f718c0fff00 0x7f718c19bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f7191ec6700 1 -- 172.21.15.57:0/3431299209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c105600 msgr2=0x7f718c1957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f7191ec6700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c105600 0x7f718c1957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f7191ec6700 1 -- 172.21.15.57:0/3431299209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71880095f0 con 0x7f718c0ff540 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.963+0000 7f71926c7700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c105600 0x7f718c1957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.964+0000 7f7191ec6700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718c0ff540 0x7f718c19b690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f717c00d950 tx=0x7f717c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:15.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.964+0000 7f7182ffd700 1 -- 172.21.15.57:0/3431299209 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c009940 con 0x7f718c0ff540 2023-10-08T13:49:15.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.964+0000 7f7182ffd700 1 -- 172.21.15.57:0/3431299209 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f717c010460 con 0x7f718c0ff540 2023-10-08T13:49:15.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.964+0000 7f719412a700 1 -- 172.21.15.57:0/3431299209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c196440 con 0x7f718c0ff540 2023-10-08T13:49:15.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.964+0000 7f7182ffd700 1 -- 172.21.15.57:0/3431299209 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00f620 con 0x7f718c0ff540 2023-10-08T13:49:15.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.964+0000 7f719412a700 1 -- 172.21.15.57:0/3431299209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c1a2680 con 0x7f718c0ff540 2023-10-08T13:49:15.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.965+0000 7f719412a700 1 -- 172.21.15.57:0/3431299209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718c107aa0 con 0x7f718c0ff540 2023-10-08T13:49:15.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.966+0000 7f7182ffd700 1 -- 172.21.15.57:0/3431299209 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f717c0108d0 con 0x7f718c0ff540 2023-10-08T13:49:15.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.966+0000 7f7182ffd700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71780a1550 0x7f71780a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:15.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.967+0000 7f7182ffd700 1 -- 172.21.15.57:0/3431299209 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f717c0ccb00 con 0x7f718c0ff540 2023-10-08T13:49:15.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.967+0000 7f71916c5700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71780a1550 0x7f71780a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:15.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.967+0000 7f71916c5700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71780a1550 0x7f71780a3a10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7188009fd0 tx=0x7f7188019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:15.971+0000 7f7182ffd700 1 -- 172.21.15.57:0/3431299209 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f717c096530 con 0x7f718c0ff540 2023-10-08T13:49:16.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.221+0000 7f719412a700 1 -- 172.21.15.57:0/3431299209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f718c196d20 con 0x7f718c0ff540 2023-10-08T13:49:16.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.222+0000 7f7182ffd700 1 -- 172.21.15.57:0/3431299209 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f717c099b50 con 0x7f718c0ff540 2023-10-08T13:49:16.223 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:16.223 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T13:49:16.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 -- 172.21.15.57:0/3431299209 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71780a1550 msgr2=0x7f71780a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:16.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71780a1550 0x7f71780a3a10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7188009fd0 tx=0x7f7188019040 comp rx=0 tx=0).stop 2023-10-08T13:49:16.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 -- 172.21.15.57:0/3431299209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718c0ff540 msgr2=0x7f718c19b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:16.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718c0ff540 0x7f718c19b690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f717c00d950 tx=0x7f717c00dc60 comp rx=0 tx=0).stop 2023-10-08T13:49:16.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 -- 172.21.15.57:0/3431299209 shutdown_connections 2023-10-08T13:49:16.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71780a1550 0x7f71780a3a10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:16.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c105600 0x7f718c1957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:16.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718c0ff540 0x7f718c19b690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:16.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 --2- 172.21.15.57:0/3431299209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f718c0fff00 0x7f718c19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:16.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 -- 172.21.15.57:0/3431299209 >> 172.21.15.57:0/3431299209 conn(0x7f718c0fb030 msgr2=0x7f718c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:16.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 -- 172.21.15.57:0/3431299209 shutdown_connections 2023-10-08T13:49:16.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:16.224+0000 7f719412a700 1 -- 172.21.15.57:0/3431299209 wait complete. 2023-10-08T13:49:16.227 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 2 2023-10-08T13:49:16.405 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 13:49:16 smithi099 podman[139849]: 2023-10-08T13:49:16.772 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 13:49:16 smithi099 ceph-mds[139889]: -- 172.21.15.99:0/2024846746 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5597fea631e0 con 0x5597fea38800 2023-10-08T13:49:16.772 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 13:49:16 smithi099 bash[139849]: 6dd2a95958155e7810a0b3a61f3ad0ece14c705aed4d3dc4d596f2b9a6e1c6d6 2023-10-08T13:49:16.772 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 13:49:16 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-k[139865]: starting mds.k at 2023-10-08T13:49:16.772 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 13:49:16 smithi099 systemd[1]: Started Ceph mds.k for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:17.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:17 smithi057 ceph-mon[99476]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:17.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:17 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3431299209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:17.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:17 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:17.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:17 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:17.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:17 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:17.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:17 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:17.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:17 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:17.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:17 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:17.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:17 smithi057 ceph-mon[99476]: Deploying daemon mds.j on smithi057 2023-10-08T13:49:17.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:17 smithi099 ceph-mon[103849]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:17.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:17 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3431299209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:17.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:17 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:17.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:17 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:17.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:17 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:17.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:17 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:17.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:17 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:17.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:17 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:17.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:17 smithi099 ceph-mon[103849]: Deploying daemon mds.j on smithi057 2023-10-08T13:49:17.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:17 smithi115 ceph-mon[104408]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:17.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:17 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3431299209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:17.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:17 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:17.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:17 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:17.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:17 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:17.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:17 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:17.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:17 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:17.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:17 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:17.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:17 smithi115 ceph-mon[104408]: Deploying daemon mds.j on smithi057 2023-10-08T13:49:18.238 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:18.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.383+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/337585369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aac101010 msgr2=0x7f3aac10d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:18.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.383+0000 7f3ab2cb5700 1 --2- 172.21.15.57:0/337585369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aac101010 0x7f3aac10d520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400b0a0 tx=0x7f3aa400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.384+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/337585369 shutdown_connections 2023-10-08T13:49:18.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.384+0000 7f3ab2cb5700 1 --2- 172.21.15.57:0/337585369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aac101010 0x7f3aac10d520 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.384+0000 7f3ab2cb5700 1 --2- 172.21.15.57:0/337585369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac106660 0x7f3aac106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.384+0000 7f3ab2cb5700 1 --2- 172.21.15.57:0/337585369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aac100650 0x7f3aac100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.384+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/337585369 >> 172.21.15.57:0/337585369 conn(0x7f3aac0fc160 msgr2=0x7f3aac0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:18.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.384+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/337585369 shutdown_connections 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.384+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/337585369 wait complete. 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.384+0000 7f3ab2cb5700 1 Processor -- start 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.384+0000 7f3ab2cb5700 1 -- start start 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab2cb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aac100650 0x7f3aac072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab2cb5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aac101010 0x7f3aac073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab2cb5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac106660 0x7f3aac073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab2cb5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac1a4d90 con 0x7f3aac100650 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab2cb5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac1a4f00 con 0x7f3aac106660 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab2cb5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac1a50c0 con 0x7f3aac101010 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab0a51700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aac100650 0x7f3aac072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab0a51700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aac100650 0x7f3aac072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53664/0 (socket says 172.21.15.57:53664) 2023-10-08T13:49:18.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab0a51700 1 -- 172.21.15.57:0/1478982718 learned_addr learned my addr 172.21.15.57:0/1478982718 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab1252700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac106660 0x7f3aac073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3aabfff700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aac101010 0x7f3aac073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab0a51700 1 -- 172.21.15.57:0/1478982718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aac101010 msgr2=0x7f3aac073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab0a51700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aac101010 0x7f3aac073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab0a51700 1 -- 172.21.15.57:0/1478982718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac106660 msgr2=0x7f3aac073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab0a51700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac106660 0x7f3aac073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab0a51700 1 -- 172.21.15.57:0/1478982718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4009d00 con 0x7f3aac100650 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3aabfff700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aac101010 0x7f3aac073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab0a51700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aac100650 0x7f3aac072af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000c9b0 tx=0x7f3aa000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3aa9ffb700 1 -- 172.21.15.57:0/1478982718 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0007b00 con 0x7f3aac100650 2023-10-08T13:49:18.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3aa9ffb700 1 -- 172.21.15.57:0/1478982718 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aa0007c60 con 0x7f3aac100650 2023-10-08T13:49:18.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/1478982718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac1a53a0 con 0x7f3aac100650 2023-10-08T13:49:18.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.385+0000 7f3aa9ffb700 1 -- 172.21.15.57:0/1478982718 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0018660 con 0x7f3aac100650 2023-10-08T13:49:18.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.386+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/1478982718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac1a58b0 con 0x7f3aac100650 2023-10-08T13:49:18.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.386+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/1478982718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac108c70 con 0x7f3aac100650 2023-10-08T13:49:18.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.387+0000 7f3aa9ffb700 1 -- 172.21.15.57:0/1478982718 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f3aa000f450 con 0x7f3aac100650 2023-10-08T13:49:18.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.388+0000 7f3aa9ffb700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a9c0a14d0 0x7f3a9c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:18.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.388+0000 7f3aabfff700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a9c0a14d0 0x7f3a9c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:18.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.388+0000 7f3aa9ffb700 1 -- 172.21.15.57:0/1478982718 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f3aa00cc940 con 0x7f3aac100650 2023-10-08T13:49:18.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.389+0000 7f3aabfff700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a9c0a14d0 0x7f3a9c0a3990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a98005fd0 tx=0x7f3a98005dc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:18.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.393+0000 7f3aa9ffb700 1 -- 172.21.15.57:0/1478982718 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3aa00962d0 con 0x7f3aac100650 2023-10-08T13:49:18.418 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 13:49:18 smithi057 systemd[1]: Starting Ceph mds.j for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:18.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:18 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.99:6832/2949059139,v1:172.21.15.99:6833/2949059139] up:boot 2023-10-08T13:49:18.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:18 smithi057 ceph-mon[99476]: fsmap 2 up:standby 2023-10-08T13:49:18.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:18 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T13:49:18.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:18 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.99:6832/2949059139,v1:172.21.15.99:6833/2949059139] up:boot 2023-10-08T13:49:18.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:18 smithi099 ceph-mon[103849]: fsmap 2 up:standby 2023-10-08T13:49:18.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:18 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T13:49:18.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.649+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/1478982718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3aac101d90 con 0x7f3aac100650 2023-10-08T13:49:18.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.649+0000 7f3aa9ffb700 1 -- 172.21.15.57:0/1478982718 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f3aac101d90 con 0x7f3aac100650 2023-10-08T13:49:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:18.651 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T13:49:18.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/1478982718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a9c0a14d0 msgr2=0x7f3a9c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:18.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a9c0a14d0 0x7f3a9c0a3990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a98005fd0 tx=0x7f3a98005dc0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/1478982718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aac100650 msgr2=0x7f3aac072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aac100650 0x7f3aac072af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000c9b0 tx=0x7f3aa000ccc0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/1478982718 shutdown_connections 2023-10-08T13:49:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a9c0a14d0 0x7f3a9c0a3990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac106660 0x7f3aac073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3aac100650 0x7f3aac072af0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 --2- 172.21.15.57:0/1478982718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aac101010 0x7f3aac073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/1478982718 >> 172.21.15.57:0/1478982718 conn(0x7f3aac0fc160 msgr2=0x7f3aac0fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/1478982718 shutdown_connections 2023-10-08T13:49:18.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:18.652+0000 7f3ab2cb5700 1 -- 172.21.15.57:0/1478982718 wait complete. 2023-10-08T13:49:18.655 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 3 2023-10-08T13:49:18.675 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 13:49:18 smithi057 podman[156267]: 2023-10-08T13:49:18.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:18 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.99:6832/2949059139,v1:172.21.15.99:6833/2949059139] up:boot 2023-10-08T13:49:18.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:18 smithi115 ceph-mon[104408]: fsmap 2 up:standby 2023-10-08T13:49:18.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:18 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T13:49:18.987 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 13:49:18 smithi057 bash[156267]: cc004d39c45df2ee100993e569bceec29ab1940258bd0023bf33b8a1d091d640 2023-10-08T13:49:18.987 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 13:49:18 smithi057 ceph-mds[156330]: -- 172.21.15.57:0/2405424274 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a6688411e0 con 0x55a668816c00 2023-10-08T13:49:18.987 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 13:49:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-j[156304]: starting mds.j at 2023-10-08T13:49:18.988 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 13:49:18 smithi057 systemd[1]: Started Ceph mds.j for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:19.469 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:19 smithi057 ceph-mon[99476]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:19.469 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:19 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1478982718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:19.470 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:19.470 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:19.470 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:19.470 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:19.470 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:19.470 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:19.470 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:19 smithi057 ceph-mon[99476]: Deploying daemon mds.d on smithi057 2023-10-08T13:49:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:19 smithi099 ceph-mon[103849]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:19 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1478982718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:19.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:19 smithi099 ceph-mon[103849]: Deploying daemon mds.d on smithi057 2023-10-08T13:49:19.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:19 smithi115 ceph-mon[104408]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:19.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:19 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1478982718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:19.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:19.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:19.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:19.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:19.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:19.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:19.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:19 smithi115 ceph-mon[104408]: Deploying daemon mds.d on smithi057 2023-10-08T13:49:20.476 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 13:49:20 smithi057 systemd[1]: Starting Ceph mds.d for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:20.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:20 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.57:6834/2801838933,v1:172.21.15.57:6835/2801838933] up:boot 2023-10-08T13:49:20.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:20 smithi057 ceph-mon[99476]: fsmap 3 up:standby 2023-10-08T13:49:20.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:20 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-08T13:49:20.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:20 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.57:6834/2801838933,v1:172.21.15.57:6835/2801838933] up:boot 2023-10-08T13:49:20.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:20 smithi099 ceph-mon[103849]: fsmap 3 up:standby 2023-10-08T13:49:20.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:20 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-08T13:49:20.667 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:20.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:20 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.57:6834/2801838933,v1:172.21.15.57:6835/2801838933] up:boot 2023-10-08T13:49:20.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:20 smithi115 ceph-mon[104408]: fsmap 3 up:standby 2023-10-08T13:49:20.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:20 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-08T13:49:20.737 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 13:49:20 smithi057 podman[156629]: 2023-10-08T13:49:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.800+0000 7f920c1e6700 1 -- 172.21.15.57:0/2946045535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92040ff510 msgr2=0x7f920410d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.800+0000 7f920c1e6700 1 --2- 172.21.15.57:0/2946045535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92040ff510 0x7f920410d4f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00b0a0 tx=0x7f91fc00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.800+0000 7f920c1e6700 1 -- 172.21.15.57:0/2946045535 shutdown_connections 2023-10-08T13:49:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.800+0000 7f920c1e6700 1 --2- 172.21.15.57:0/2946045535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92040ff510 0x7f920410d4f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.800+0000 7f920c1e6700 1 --2- 172.21.15.57:0/2946045535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9204106630 0x7f9204106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.800+0000 7f920c1e6700 1 --2- 172.21.15.57:0/2946045535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92040feb50 0x7f92040fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.800+0000 7f920c1e6700 1 -- 172.21.15.57:0/2946045535 >> 172.21.15.57:0/2946045535 conn(0x7f9204075a90 msgr2=0x7f92040fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:20.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.800+0000 7f920c1e6700 1 -- 172.21.15.57:0/2946045535 shutdown_connections 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.800+0000 7f920c1e6700 1 -- 172.21.15.57:0/2946045535 wait complete. 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f920c1e6700 1 Processor -- start 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f920c1e6700 1 -- start start 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f920c1e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92040feb50 0x7f920419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f920c1e6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92040ff510 0x7f920419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f920c1e6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9204106630 0x7f9204196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f920c1e6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204196fb0 con 0x7f92040ff510 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f920c1e6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204197120 con 0x7f9204106630 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f920c1e6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041972e0 con 0x7f92040feb50 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f9209781700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92040ff510 0x7f920419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f9209781700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92040ff510 0x7f920419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53706/0 (socket says 172.21.15.57:53706) 2023-10-08T13:49:20.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f9209781700 1 -- 172.21.15.57:0/3948806269 learned_addr learned my addr 172.21.15.57:0/3948806269 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.801+0000 7f9209f82700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92040feb50 0x7f920419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f920a783700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9204106630 0x7f9204196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f9209781700 1 -- 172.21.15.57:0/3948806269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92040feb50 msgr2=0x7f920419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f9209781700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92040feb50 0x7f920419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f9209781700 1 -- 172.21.15.57:0/3948806269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9204106630 msgr2=0x7f9204196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f9209781700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9204106630 0x7f9204196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f9209781700 1 -- 172.21.15.57:0/3948806269 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc009d00 con 0x7f92040ff510 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f9209781700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92040ff510 0x7f920419cd10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f91f400e9a0 tx=0x7f91f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f91faffd700 1 -- 172.21.15.57:0/3948806269 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4009790 con 0x7f92040ff510 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f920c1e6700 1 -- 172.21.15.57:0/3948806269 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92041975c0 con 0x7f92040ff510 2023-10-08T13:49:20.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f91faffd700 1 -- 172.21.15.57:0/3948806269 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91f400cbd0 con 0x7f92040ff510 2023-10-08T13:49:20.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f91faffd700 1 -- 172.21.15.57:0/3948806269 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4018680 con 0x7f92040ff510 2023-10-08T13:49:20.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.802+0000 7f920c1e6700 1 -- 172.21.15.57:0/3948806269 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92041a37c0 con 0x7f92040ff510 2023-10-08T13:49:20.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.803+0000 7f920c1e6700 1 -- 172.21.15.57:0/3948806269 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9204108c40 con 0x7f92040ff510 2023-10-08T13:49:20.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.804+0000 7f91faffd700 1 -- 172.21.15.57:0/3948806269 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f91f400cd40 con 0x7f92040ff510 2023-10-08T13:49:20.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.805+0000 7f91faffd700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91f00a14d0 0x7f91f00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:20.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.805+0000 7f9209f82700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91f00a14d0 0x7f91f00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:20.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.805+0000 7f91faffd700 1 -- 172.21.15.57:0/3948806269 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f91f4014070 con 0x7f92040ff510 2023-10-08T13:49:20.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.806+0000 7f9209f82700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91f00a14d0 0x7f91f00a3990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9200009be0 tx=0x7f9200009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:20.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:20.811+0000 7f91faffd700 1 -- 172.21.15.57:0/3948806269 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f91f4096910 con 0x7f92040ff510 2023-10-08T13:49:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.085+0000 7f920c1e6700 1 -- 172.21.15.57:0/3948806269 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9204197dc0 con 0x7f92040ff510 2023-10-08T13:49:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.085+0000 7f91faffd700 1 -- 172.21.15.57:0/3948806269 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f91f4099f30 con 0x7f92040ff510 2023-10-08T13:49:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:21.087 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T13:49:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 -- 172.21.15.57:0/3948806269 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91f00a14d0 msgr2=0x7f91f00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91f00a14d0 0x7f91f00a3990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9200009be0 tx=0x7f9200009260 comp rx=0 tx=0).stop 2023-10-08T13:49:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 -- 172.21.15.57:0/3948806269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92040ff510 msgr2=0x7f920419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92040ff510 0x7f920419cd10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f91f400e9a0 tx=0x7f91f400ecb0 comp rx=0 tx=0).stop 2023-10-08T13:49:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 -- 172.21.15.57:0/3948806269 shutdown_connections 2023-10-08T13:49:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91f00a14d0 0x7f91f00a3990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9204106630 0x7f9204196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:21.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92040feb50 0x7f920419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:21.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 --2- 172.21.15.57:0/3948806269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92040ff510 0x7f920419cd10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:21.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 -- 172.21.15.57:0/3948806269 >> 172.21.15.57:0/3948806269 conn(0x7f9204075a90 msgr2=0x7f92040fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:21.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 -- 172.21.15.57:0/3948806269 shutdown_connections 2023-10-08T13:49:21.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:21.088+0000 7f920c1e6700 1 -- 172.21.15.57:0/3948806269 wait complete. 2023-10-08T13:49:21.090 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 4 2023-10-08T13:49:21.100 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 13:49:20 smithi057 ceph-mds[156693]: -- 172.21.15.57:0/684373010 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d1225011e0 con 0x55d1224d6c00 2023-10-08T13:49:21.101 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 13:49:20 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-d[156645]: starting mds.d at 2023-10-08T13:49:21.101 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 13:49:20 smithi057 bash[156629]: fe9a466a34fc0529f6950a896d6fa112d1629e2074a91871f80b233ccc2e8705 2023-10-08T13:49:21.101 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 13:49:20 smithi057 systemd[1]: Started Ceph mds.d for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:21.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:21.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:21.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:21.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:21.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:21.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:21.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:21.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: Deploying daemon mds.e on smithi099 2023-10-08T13:49:21.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3948806269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:21.849 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.57:6836/4079715365,v1:172.21.15.57:6837/4079715365] up:boot 2023-10-08T13:49:21.849 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: fsmap 4 up:standby 2023-10-08T13:49:21.849 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:21 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T13:49:22.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: Deploying daemon mds.e on smithi099 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3948806269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.57:6836/4079715365,v1:172.21.15.57:6837/4079715365] up:boot 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: fsmap 4 up:standby 2023-10-08T13:49:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:21 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T13:49:22.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:22.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: Deploying daemon mds.e on smithi099 2023-10-08T13:49:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3948806269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.57:6836/4079715365,v1:172.21.15.57:6837/4079715365] up:boot 2023-10-08T13:49:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: fsmap 4 up:standby 2023-10-08T13:49:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:21 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T13:49:22.772 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 13:49:22 smithi099 systemd[1]: Starting Ceph mds.e for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:23.103 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:23.158 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 13:49:22 smithi099 podman[140191]: 2023-10-08T13:49:23.158 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 13:49:23 smithi099 ceph-mds[140231]: -- 172.21.15.99:0/2855233215 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56281a58b1e0 con 0x56281a561000 2023-10-08T13:49:23.158 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 13:49:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-e[140207]: starting mds.e at 2023-10-08T13:49:23.159 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 13:49:23 smithi099 bash[140191]: c7bab04cf6cec5e3e76c0e3fd890bd92d911bfdbedda26bac4f8e0f2147f5efd 2023-10-08T13:49:23.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.243+0000 7f5ad1a72700 1 -- 172.21.15.57:0/1528625819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5acc100630 msgr2=0x7f5acc100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:23.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.243+0000 7f5ad1a72700 1 --2- 172.21.15.57:0/1528625819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5acc100630 0x7f5acc100ab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009990 tx=0x7f5ac0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.243+0000 7f5ad1a72700 1 -- 172.21.15.57:0/1528625819 shutdown_connections 2023-10-08T13:49:23.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.243+0000 7f5ad1a72700 1 --2- 172.21.15.57:0/1528625819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5acc100ff0 0x7f5acc10d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.243+0000 7f5ad1a72700 1 --2- 172.21.15.57:0/1528625819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc106640 0x7f5acc106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.243+0000 7f5ad1a72700 1 --2- 172.21.15.57:0/1528625819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5acc100630 0x7f5acc100ab0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.243+0000 7f5ad1a72700 1 -- 172.21.15.57:0/1528625819 >> 172.21.15.57:0/1528625819 conn(0x7f5acc0fc140 msgr2=0x7f5acc0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:23.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.243+0000 7f5ad1a72700 1 -- 172.21.15.57:0/1528625819 shutdown_connections 2023-10-08T13:49:23.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.243+0000 7f5ad1a72700 1 -- 172.21.15.57:0/1528625819 wait complete. 2023-10-08T13:49:23.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.244+0000 7f5ad1a72700 1 Processor -- start 2023-10-08T13:49:23.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.244+0000 7f5ad1a72700 1 -- start start 2023-10-08T13:49:23.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.244+0000 7f5ad1a72700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc100630 0x7f5acc072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:23.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.244+0000 7f5ad1a72700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5acc100ff0 0x7f5acc073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:23.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.244+0000 7f5ad1a72700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5acc106640 0x7f5acc073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:23.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.244+0000 7f5ad1a72700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc1a4e90 con 0x7f5acc100ff0 2023-10-08T13:49:23.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.244+0000 7f5ad1a72700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc1a5000 con 0x7f5acc100630 2023-10-08T13:49:23.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.244+0000 7f5ad1a72700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc1a51c0 con 0x7f5acc106640 2023-10-08T13:49:23.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5acc100ff0 0x7f5acc073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5acc100ff0 0x7f5acc073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53722/0 (socket says 172.21.15.57:53722) 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc100630 0x7f5acc072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acaffd700 1 -- 172.21.15.57:0/363961985 learned_addr learned my addr 172.21.15.57:0/363961985 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acbfff700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5acc106640 0x7f5acc073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acaffd700 1 -- 172.21.15.57:0/363961985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5acc106640 msgr2=0x7f5acc073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acaffd700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5acc106640 0x7f5acc073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acaffd700 1 -- 172.21.15.57:0/363961985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc100630 msgr2=0x7f5acc072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acaffd700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc100630 0x7f5acc072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acaffd700 1 -- 172.21.15.57:0/363961985 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac00095f0 con 0x7f5acc100ff0 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acb7fe700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc100630 0x7f5acc072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5acaffd700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5acc100ff0 0x7f5acc073030 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00095c0 tx=0x7f5ac000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:23.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5ac8ff9700 1 -- 172.21.15.57:0/363961985 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac001d070 con 0x7f5acc100ff0 2023-10-08T13:49:23.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5ac8ff9700 1 -- 172.21.15.57:0/363961985 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ac00054f0 con 0x7f5acc100ff0 2023-10-08T13:49:23.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5ad1a72700 1 -- 172.21.15.57:0/363961985 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc1a5440 con 0x7f5acc100ff0 2023-10-08T13:49:23.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5ac8ff9700 1 -- 172.21.15.57:0/363961985 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0004a90 con 0x7f5acc100ff0 2023-10-08T13:49:23.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.245+0000 7f5ad1a72700 1 -- 172.21.15.57:0/363961985 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc1a5920 con 0x7f5acc100ff0 2023-10-08T13:49:23.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.246+0000 7f5ad1a72700 1 -- 172.21.15.57:0/363961985 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc108c50 con 0x7f5acc100ff0 2023-10-08T13:49:23.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.247+0000 7f5ac8ff9700 1 -- 172.21.15.57:0/363961985 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f5ac000fb80 con 0x7f5acc100ff0 2023-10-08T13:49:23.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.248+0000 7f5ac8ff9700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ab40a14d0 0x7f5ab40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:23.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.248+0000 7f5acb7fe700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ab40a14d0 0x7f5ab40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:23.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.248+0000 7f5ac8ff9700 1 -- 172.21.15.57:0/363961985 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f5ac00cda40 con 0x7f5acc100ff0 2023-10-08T13:49:23.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.249+0000 7f5acb7fe700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ab40a14d0 0x7f5ab40a3990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009550 tx=0x7f5ac40091f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:23.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.252+0000 7f5ac8ff9700 1 -- 172.21.15.57:0/363961985 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5ac0097500 con 0x7f5acc100ff0 2023-10-08T13:49:23.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.510+0000 7f5ad1a72700 1 -- 172.21.15.57:0/363961985 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5acc073fd0 con 0x7f5acc100ff0 2023-10-08T13:49:23.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.511+0000 7f5ac8ff9700 1 -- 172.21.15.57:0/363961985 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f5ac009ab20 con 0x7f5acc100ff0 2023-10-08T13:49:23.512 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:23.512 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T13:49:23.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 -- 172.21.15.57:0/363961985 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ab40a14d0 msgr2=0x7f5ab40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:23.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ab40a14d0 0x7f5ab40a3990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009550 tx=0x7f5ac40091f0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 -- 172.21.15.57:0/363961985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5acc100ff0 msgr2=0x7f5acc073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:23.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5acc100ff0 0x7f5acc073030 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00095c0 tx=0x7f5ac000f690 comp rx=0 tx=0).stop 2023-10-08T13:49:23.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 -- 172.21.15.57:0/363961985 shutdown_connections 2023-10-08T13:49:23.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5acc106640 0x7f5acc073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ab40a14d0 0x7f5ab40a3990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc100630 0x7f5acc072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 --2- 172.21.15.57:0/363961985 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5acc100ff0 0x7f5acc073030 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:23.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 -- 172.21.15.57:0/363961985 >> 172.21.15.57:0/363961985 conn(0x7f5acc0fc140 msgr2=0x7f5acc0fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:23.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 -- 172.21.15.57:0/363961985 shutdown_connections 2023-10-08T13:49:23.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:23.513+0000 7f5ad1a72700 1 -- 172.21.15.57:0/363961985 wait complete. 2023-10-08T13:49:23.516 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 5 2023-10-08T13:49:23.522 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 13:49:23 smithi099 systemd[1]: Started Ceph mds.e for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:24.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:23 smithi115 ceph-mon[104408]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:23 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:23 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:23 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:23 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:23 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:23 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:23 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/363961985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:24.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:23 smithi057 ceph-mon[99476]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:24.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:23 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:24.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:23 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:24.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:23 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:24.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:23 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:24.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:23 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:24.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:23 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:24.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:23 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/363961985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:23 smithi099 ceph-mon[103849]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:23 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:24.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:23 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:24.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:23 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:24.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:23 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:24.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:23 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:24.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:23 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:24.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:23 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/363961985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:24.874 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 13:49:24 smithi115 systemd[1]: Starting Ceph mds.f for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:25.158 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 13:49:25 smithi115 podman[141183]: 2023-10-08T13:49:25.158 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:24 smithi115 ceph-mon[104408]: Deploying daemon mds.f on smithi115 2023-10-08T13:49:25.158 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:24 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.99:6834/2732351731,v1:172.21.15.99:6835/2732351731] up:boot 2023-10-08T13:49:25.158 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:24 smithi115 ceph-mon[104408]: fsmap 5 up:standby 2023-10-08T13:49:25.158 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:24 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T13:49:25.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:24 smithi057 ceph-mon[99476]: Deploying daemon mds.f on smithi115 2023-10-08T13:49:25.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:24 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.99:6834/2732351731,v1:172.21.15.99:6835/2732351731] up:boot 2023-10-08T13:49:25.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:24 smithi057 ceph-mon[99476]: fsmap 5 up:standby 2023-10-08T13:49:25.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:24 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T13:49:25.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:24 smithi099 ceph-mon[103849]: Deploying daemon mds.f on smithi115 2023-10-08T13:49:25.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:24 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.99:6834/2732351731,v1:172.21.15.99:6835/2732351731] up:boot 2023-10-08T13:49:25.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:24 smithi099 ceph-mon[103849]: fsmap 5 up:standby 2023-10-08T13:49:25.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:24 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T13:49:25.528 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:25.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 -- 172.21.15.57:0/2925393360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710100650 msgr2=0x7f8710100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:25.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 --2- 172.21.15.57:0/2925393360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710100650 0x7f8710100ad0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0098f0 tx=0x7f86fc009c00 comp rx=0 tx=0).stop 2023-10-08T13:49:25.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 -- 172.21.15.57:0/2925393360 shutdown_connections 2023-10-08T13:49:25.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 --2- 172.21.15.57:0/2925393360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8710101010 0x7f871010d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:25.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 --2- 172.21.15.57:0/2925393360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710106660 0x7f8710106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:25.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 --2- 172.21.15.57:0/2925393360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710100650 0x7f8710100ad0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:25.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 -- 172.21.15.57:0/2925393360 >> 172.21.15.57:0/2925393360 conn(0x7f87100fc160 msgr2=0x7f87100fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:25.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 -- 172.21.15.57:0/2925393360 shutdown_connections 2023-10-08T13:49:25.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 -- 172.21.15.57:0/2925393360 wait complete. 2023-10-08T13:49:25.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 Processor -- start 2023-10-08T13:49:25.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.666+0000 7f8716ca0700 1 -- start start 2023-10-08T13:49:25.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f8716ca0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710100650 0x7f871019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:25.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f8716ca0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8710101010 0x7f871019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:25.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f8716ca0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710106660 0x7f87101969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:25.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f8716ca0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8710197000 con 0x7f8710101010 2023-10-08T13:49:25.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f8716ca0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8710197170 con 0x7f8710106660 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f8716ca0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8710197330 con 0x7f8710100650 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8710101010 0x7f871019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8710101010 0x7f871019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53736/0 (socket says 172.21.15.57:53736) 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f8714a3c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710100650 0x7f871019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870ffff700 1 -- 172.21.15.57:0/3905843666 learned_addr learned my addr 172.21.15.57:0/3905843666 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f871523d700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710106660 0x7f87101969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870ffff700 1 -- 172.21.15.57:0/3905843666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710100650 msgr2=0x7f871019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870ffff700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710100650 0x7f871019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870ffff700 1 -- 172.21.15.57:0/3905843666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710106660 msgr2=0x7f87101969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870ffff700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710106660 0x7f87101969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870ffff700 1 -- 172.21.15.57:0/3905843666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8708009d00 con 0x7f8710101010 2023-10-08T13:49:25.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f871523d700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710106660 0x7f87101969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:25.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870ffff700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8710101010 0x7f871019cdb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005950 tx=0x7f86fc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:25.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870dffb700 1 -- 172.21.15.57:0/3905843666 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc01c070 con 0x7f8710101010 2023-10-08T13:49:25.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f8716ca0700 1 -- 172.21.15.57:0/3905843666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86fc009520 con 0x7f8710101010 2023-10-08T13:49:25.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.667+0000 7f870dffb700 1 -- 172.21.15.57:0/3905843666 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86fc00bb20 con 0x7f8710101010 2023-10-08T13:49:25.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.668+0000 7f870dffb700 1 -- 172.21.15.57:0/3905843666 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0038e0 con 0x7f8710101010 2023-10-08T13:49:25.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.668+0000 7f8716ca0700 1 -- 172.21.15.57:0/3905843666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87101a3680 con 0x7f8710101010 2023-10-08T13:49:25.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.669+0000 7f8716ca0700 1 -- 172.21.15.57:0/3905843666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8710108c50 con 0x7f8710101010 2023-10-08T13:49:25.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.670+0000 7f870dffb700 1 -- 172.21.15.57:0/3905843666 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f86fc020030 con 0x7f8710101010 2023-10-08T13:49:25.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.670+0000 7f870dffb700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87000a14d0 0x7f87000a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:25.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.670+0000 7f870dffb700 1 -- 172.21.15.57:0/3905843666 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f86fc0cd7e0 con 0x7f8710101010 2023-10-08T13:49:25.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.671+0000 7f8714a3c700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87000a14d0 0x7f87000a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:25.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.671+0000 7f8714a3c700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87000a14d0 0x7f87000a3990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8704009be0 tx=0x7f8704009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:25.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.675+0000 7f870dffb700 1 -- 172.21.15.57:0/3905843666 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f86fc0972a0 con 0x7f8710101010 2023-10-08T13:49:25.686 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 13:49:25 smithi115 ceph-mds[141223]: -- 172.21.15.115:0/2567778665 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e76e2451e0 con 0x55e76e21ac00 2023-10-08T13:49:25.687 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 13:49:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-f[141199]: starting mds.f at 2023-10-08T13:49:25.687 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 13:49:25 smithi115 bash[141183]: 20dc39932fd93e5e841b886bec494e9a8ce5649e5bb8377c82c99ccc20de0cd1 2023-10-08T13:49:25.687 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 13:49:25 smithi115 systemd[1]: Started Ceph mds.f for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:25.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.927+0000 7f8716ca0700 1 -- 172.21.15.57:0/3905843666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8710197e60 con 0x7f8710101010 2023-10-08T13:49:25.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.928+0000 7f870dffb700 1 -- 172.21.15.57:0/3905843666 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5283 (secure 0 0 0) 0x7f86fc09a8c0 con 0x7f8710101010 2023-10-08T13:49:25.929 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:25.929 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24581,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-10-08T13:49:25.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 -- 172.21.15.57:0/3905843666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87000a14d0 msgr2=0x7f87000a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87000a14d0 0x7f87000a3990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8704009be0 tx=0x7f8704009260 comp rx=0 tx=0).stop 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 -- 172.21.15.57:0/3905843666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8710101010 msgr2=0x7f871019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8710101010 0x7f871019cdb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005950 tx=0x7f86fc00fa00 comp rx=0 tx=0).stop 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 -- 172.21.15.57:0/3905843666 shutdown_connections 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87000a14d0 0x7f87000a3990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710106660 0x7f87101969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710100650 0x7f871019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 --2- 172.21.15.57:0/3905843666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8710101010 0x7f871019cdb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 -- 172.21.15.57:0/3905843666 >> 172.21.15.57:0/3905843666 conn(0x7f87100fc160 msgr2=0x7f87100fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.930+0000 7f8716ca0700 1 -- 172.21.15.57:0/3905843666 shutdown_connections 2023-10-08T13:49:25.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:25.931+0000 7f8716ca0700 1 -- 172.21.15.57:0/3905843666 wait complete. 2023-10-08T13:49:25.933 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 7 2023-10-08T13:49:26.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:25 smithi115 ceph-mon[104408]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:26.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:26.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:26.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:26.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:26.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:26.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:26.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:25 smithi057 ceph-mon[99476]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:26.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:26.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:26.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:26.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:26.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:26.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:25 smithi099 ceph-mon[103849]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:26.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:26 smithi115 ceph-mon[104408]: Deploying daemon mds.c on smithi115 2023-10-08T13:49:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:26 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] up:boot 2023-10-08T13:49:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:26 smithi115 ceph-mon[104408]: fsmap 6 up:standby 2023-10-08T13:49:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:26 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T13:49:27.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:26 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3905843666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:27.188 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 13:49:26 smithi115 systemd[1]: Starting Ceph mds.c for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:27.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:26 smithi057 ceph-mon[99476]: Deploying daemon mds.c on smithi115 2023-10-08T13:49:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:26 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] up:boot 2023-10-08T13:49:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:26 smithi057 ceph-mon[99476]: fsmap 6 up:standby 2023-10-08T13:49:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:26 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T13:49:27.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:26 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3905843666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:27.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:26 smithi099 ceph-mon[103849]: Deploying daemon mds.c on smithi115 2023-10-08T13:49:27.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:26 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] up:boot 2023-10-08T13:49:27.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:26 smithi099 ceph-mon[103849]: fsmap 6 up:standby 2023-10-08T13:49:27.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:26 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T13:49:27.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:26 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3905843666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:27.473 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 13:49:27 smithi115 podman[141524]: 2023-10-08T13:49:27.473 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 13:49:27 smithi115 ceph-mds[141564]: -- 172.21.15.115:0/969028898 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560da4f091e0 con 0x560da4ede800 2023-10-08T13:49:27.889 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 13:49:27 smithi115 bash[141524]: e209e02cf4b568a5e7a167c86905d2f4587332079803c0ba338b5937e42a488c 2023-10-08T13:49:27.889 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 13:49:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-c[141540]: starting mds.c at 2023-10-08T13:49:27.889 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 13:49:27 smithi115 systemd[1]: Started Ceph mds.c for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:27.945 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.085+0000 7fc849641700 1 -- 172.21.15.57:0/1501228770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8440ff530 msgr2=0x7fc8440ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.085+0000 7fc849641700 1 --2- 172.21.15.57:0/1501228770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8440ff530 0x7fc8440ff9b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc8380099c0 tx=0x7fc838009cd0 comp rx=0 tx=0).stop 2023-10-08T13:49:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.085+0000 7fc849641700 1 -- 172.21.15.57:0/1501228770 shutdown_connections 2023-10-08T13:49:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.085+0000 7fc849641700 1 --2- 172.21.15.57:0/1501228770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8440ffef0 0x7fc84410c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:28.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.085+0000 7fc849641700 1 --2- 172.21.15.57:0/1501228770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8441055f0 0x7fc8441059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.085+0000 7fc849641700 1 --2- 172.21.15.57:0/1501228770 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8440ff530 0x7fc8440ff9b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.085+0000 7fc849641700 1 -- 172.21.15.57:0/1501228770 >> 172.21.15.57:0/1501228770 conn(0x7fc8440fb040 msgr2=0x7fc8440fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.085+0000 7fc849641700 1 -- 172.21.15.57:0/1501228770 shutdown_connections 2023-10-08T13:49:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.085+0000 7fc849641700 1 -- 172.21.15.57:0/1501228770 wait complete. 2023-10-08T13:49:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.086+0000 7fc849641700 1 Processor -- start 2023-10-08T13:49:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.086+0000 7fc849641700 1 -- start start 2023-10-08T13:49:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.086+0000 7fc849641700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8440ff530 0x7fc844199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.086+0000 7fc849641700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8440ffef0 0x7fc8441995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.086+0000 7fc849641700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8441055f0 0x7fc84419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.086+0000 7fc849641700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84419db80 con 0x7fc8440ffef0 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.086+0000 7fc849641700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84419dcf0 con 0x7fc8440ff530 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.086+0000 7fc849641700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84419deb0 con 0x7fc8441055f0 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc8427fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8440ffef0 0x7fc8441995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc842ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8440ff530 0x7fc844199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc8427fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8440ffef0 0x7fc8441995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53750/0 (socket says 172.21.15.57:53750) 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc8427fc700 1 -- 172.21.15.57:0/1430818579 learned_addr learned my addr 172.21.15.57:0/1430818579 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc8437fe700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8441055f0 0x7fc84419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc8427fc700 1 -- 172.21.15.57:0/1430818579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8441055f0 msgr2=0x7fc84419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc8427fc700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8441055f0 0x7fc84419d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc8427fc700 1 -- 172.21.15.57:0/1430818579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8440ff530 msgr2=0x7fc844199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc8427fc700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8440ff530 0x7fc844199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc8427fc700 1 -- 172.21.15.57:0/1430818579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380095f0 con 0x7fc8440ffef0 2023-10-08T13:49:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc842ffd700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8440ff530 0x7fc844199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:49:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc8427fc700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8440ffef0 0x7fc8441995b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c010 tx=0x7fc83800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc82bfff700 1 -- 172.21.15.57:0/1430818579 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83801c070 con 0x7fc8440ffef0 2023-10-08T13:49:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc82bfff700 1 -- 172.21.15.57:0/1430818579 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc83800fe60 con 0x7fc8440ffef0 2023-10-08T13:49:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc849641700 1 -- 172.21.15.57:0/1430818579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84419e130 con 0x7fc8440ffef0 2023-10-08T13:49:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc82bfff700 1 -- 172.21.15.57:0/1430818579 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838022d80 con 0x7fc8440ffef0 2023-10-08T13:49:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.087+0000 7fc849641700 1 -- 172.21.15.57:0/1430818579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8441a59f0 con 0x7fc8440ffef0 2023-10-08T13:49:28.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.088+0000 7fc849641700 1 -- 172.21.15.57:0/1430818579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84419e400 con 0x7fc8440ffef0 2023-10-08T13:49:28.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.089+0000 7fc82bfff700 1 -- 172.21.15.57:0/1430818579 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc838020070 con 0x7fc8440ffef0 2023-10-08T13:49:28.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.090+0000 7fc82bfff700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc82c0a14d0 0x7fc82c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:28.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.090+0000 7fc82bfff700 1 -- 172.21.15.57:0/1430818579 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fc8380cdf80 con 0x7fc8440ffef0 2023-10-08T13:49:28.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.090+0000 7fc842ffd700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc82c0a14d0 0x7fc82c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:28.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.091+0000 7fc842ffd700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc82c0a14d0 0x7fc82c0a3990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc834005950 tx=0x7fc83400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:28.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.094+0000 7fc82bfff700 1 -- 172.21.15.57:0/1430818579 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc838097a40 con 0x7fc8440ffef0 2023-10-08T13:49:28.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:27 smithi115 ceph-mon[104408]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:28.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:28.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:28.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:28.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:28.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:28.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:28.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:27 smithi057 ceph-mon[99476]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:28.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:28.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:28.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:28.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:28.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:28.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:27 smithi099 ceph-mon[103849]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:28.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.339+0000 7fc849641700 1 -- 172.21.15.57:0/1430818579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc844199f60 con 0x7fc8440ffef0 2023-10-08T13:49:28.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.339+0000 7fc82bfff700 1 -- 172.21.15.57:0/1430818579 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6074 (secure 0 0 0) 0x7fc83809b060 con 0x7fc8440ffef0 2023-10-08T13:49:28.341 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:28.341 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24581,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-08T13:49:28.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 -- 172.21.15.57:0/1430818579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc82c0a14d0 msgr2=0x7fc82c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:28.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc82c0a14d0 0x7fc82c0a3990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc834005950 tx=0x7fc83400b410 comp rx=0 tx=0).stop 2023-10-08T13:49:28.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 -- 172.21.15.57:0/1430818579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8440ffef0 msgr2=0x7fc8441995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:28.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8440ffef0 0x7fc8441995b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c010 tx=0x7fc83800f690 comp rx=0 tx=0).stop 2023-10-08T13:49:28.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 -- 172.21.15.57:0/1430818579 shutdown_connections 2023-10-08T13:49:28.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8441055f0 0x7fc84419d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:28.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc82c0a14d0 0x7fc82c0a3990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:28.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8440ff530 0x7fc844199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:28.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 --2- 172.21.15.57:0/1430818579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8440ffef0 0x7fc8441995b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:28.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 -- 172.21.15.57:0/1430818579 >> 172.21.15.57:0/1430818579 conn(0x7fc8440fb040 msgr2=0x7fc8440fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:28.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 -- 172.21.15.57:0/1430818579 shutdown_connections 2023-10-08T13:49:28.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:28.341+0000 7fc849641700 1 -- 172.21.15.57:0/1430818579 wait complete. 2023-10-08T13:49:28.344 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 8 2023-10-08T13:49:29.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:28 smithi099 ceph-mon[103849]: Deploying daemon mds.h on smithi099 2023-10-08T13:49:29.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:28 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] up:boot 2023-10-08T13:49:29.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:28 smithi099 ceph-mon[103849]: fsmap 7 up:standby 2023-10-08T13:49:29.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T13:49:29.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:28 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1430818579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:29.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:28 smithi115 ceph-mon[104408]: Deploying daemon mds.h on smithi099 2023-10-08T13:49:29.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:28 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] up:boot 2023-10-08T13:49:29.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:28 smithi115 ceph-mon[104408]: fsmap 7 up:standby 2023-10-08T13:49:29.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T13:49:29.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:28 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1430818579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:29.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:28 smithi057 ceph-mon[99476]: Deploying daemon mds.h on smithi099 2023-10-08T13:49:29.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:28 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] up:boot 2023-10-08T13:49:29.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:28 smithi057 ceph-mon[99476]: fsmap 7 up:standby 2023-10-08T13:49:29.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T13:49:29.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:28 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1430818579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:29.522 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 13:49:29 smithi099 systemd[1]: Starting Ceph mds.h for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:29.885 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 13:49:29 smithi099 podman[140529]: 2023-10-08T13:49:30.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:29 smithi115 ceph-mon[104408]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:30.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:29 smithi057 ceph-mon[99476]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:30.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:29 smithi099 ceph-mon[103849]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:30.272 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 13:49:29 smithi099 ceph-mds[140569]: -- 172.21.15.99:0/3718985411 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5612b209f1e0 con 0x5612b2074800 2023-10-08T13:49:30.272 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 13:49:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-h[140545]: starting mds.h at 2023-10-08T13:49:30.272 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 13:49:29 smithi099 bash[140529]: af9d37c2eaebdc0ae1678afa7be4cea6e253a91d29237e04021748d2a566f132 2023-10-08T13:49:30.272 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 13:49:29 smithi099 systemd[1]: Started Ceph mds.h for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:30.355 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.495+0000 7ff60749f700 1 -- 172.21.15.57:0/3003269160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6000fff00 msgr2=0x7ff60010c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.495+0000 7ff60749f700 1 --2- 172.21.15.57:0/3003269160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6000fff00 0x7ff60010c3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800b0d0 tx=0x7ff5f800b3e0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.495+0000 7ff60749f700 1 -- 172.21.15.57:0/3003269160 shutdown_connections 2023-10-08T13:49:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.495+0000 7ff60749f700 1 --2- 172.21.15.57:0/3003269160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6000fff00 0x7ff60010c3c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.495+0000 7ff60749f700 1 --2- 172.21.15.57:0/3003269160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff600105600 0x7ff6001059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.495+0000 7ff60749f700 1 --2- 172.21.15.57:0/3003269160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6000ff540 0x7ff6000ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.495+0000 7ff60749f700 1 -- 172.21.15.57:0/3003269160 >> 172.21.15.57:0/3003269160 conn(0x7ff6000fb030 msgr2=0x7ff6000fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.495+0000 7ff60749f700 1 -- 172.21.15.57:0/3003269160 shutdown_connections 2023-10-08T13:49:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.495+0000 7ff60749f700 1 -- 172.21.15.57:0/3003269160 wait complete. 2023-10-08T13:49:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60749f700 1 Processor -- start 2023-10-08T13:49:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60749f700 1 -- start start 2023-10-08T13:49:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60749f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6000ff540 0x7ff60019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60749f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6000fff00 0x7ff60019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60749f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff600105600 0x7ff600196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60749f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff600197070 con 0x7ff6000ff540 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60749f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001971e0 con 0x7ff6000fff00 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60749f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001973a0 con 0x7ff600105600 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60523b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6000ff540 0x7ff60019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60523b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6000ff540 0x7ff60019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53762/0 (socket says 172.21.15.57:53762) 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff60523b700 1 -- 172.21.15.57:0/1603257313 learned_addr learned my addr 172.21.15.57:0/1603257313 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff604a3a700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6000fff00 0x7ff60019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.496+0000 7ff605a3c700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff600105600 0x7ff600196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff60523b700 1 -- 172.21.15.57:0/1603257313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff600105600 msgr2=0x7ff600196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff60523b700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff600105600 0x7ff600196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff60523b700 1 -- 172.21.15.57:0/1603257313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6000fff00 msgr2=0x7ff60019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff60523b700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6000fff00 0x7ff60019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff60523b700 1 -- 172.21.15.57:0/1603257313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f8009d00 con 0x7ff6000ff540 2023-10-08T13:49:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff60523b700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6000ff540 0x7ff60019c8e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00e9a0 tx=0x7ff5fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff5f27fc700 1 -- 172.21.15.57:0/1603257313 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc009790 con 0x7ff6000ff540 2023-10-08T13:49:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff5f27fc700 1 -- 172.21.15.57:0/1603257313 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5fc010430 con 0x7ff6000ff540 2023-10-08T13:49:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff60749f700 1 -- 172.21.15.57:0/1603257313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff600197680 con 0x7ff6000ff540 2023-10-08T13:49:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff5f27fc700 1 -- 172.21.15.57:0/1603257313 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc018680 con 0x7ff6000ff540 2023-10-08T13:49:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.497+0000 7ff60749f700 1 -- 172.21.15.57:0/1603257313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6001a38d0 con 0x7ff6000ff540 2023-10-08T13:49:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.498+0000 7ff60749f700 1 -- 172.21.15.57:0/1603257313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff600100c60 con 0x7ff6000ff540 2023-10-08T13:49:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.499+0000 7ff5f27fc700 1 -- 172.21.15.57:0/1603257313 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ff5fc0105a0 con 0x7ff6000ff540 2023-10-08T13:49:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.500+0000 7ff5f27fc700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff5ec0a14d0 0x7ff5ec0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.500+0000 7ff604a3a700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff5ec0a14d0 0x7ff5ec0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.500+0000 7ff5f27fc700 1 -- 172.21.15.57:0/1603257313 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7ff5fc014070 con 0x7ff6000ff540 2023-10-08T13:49:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.500+0000 7ff604a3a700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff5ec0a14d0 0x7ff5ec0a3990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff600197bd0 tx=0x7ff5f4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.504+0000 7ff5f27fc700 1 -- 172.21.15.57:0/1603257313 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff5fc0966e0 con 0x7ff6000ff540 2023-10-08T13:49:30.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.751+0000 7ff60749f700 1 -- 172.21.15.57:0/1603257313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff60010f460 con 0x7ff6000ff540 2023-10-08T13:49:30.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.751+0000 7ff5f27fc700 1 -- 172.21.15.57:0/1603257313 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6862 (secure 0 0 0) 0x7ff5fc099d00 con 0x7ff6000ff540 2023-10-08T13:49:30.753 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:30.753 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24581,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24617,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-08T13:49:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 -- 172.21.15.57:0/1603257313 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff5ec0a14d0 msgr2=0x7ff5ec0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff5ec0a14d0 0x7ff5ec0a3990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff600197bd0 tx=0x7ff5f4009260 comp rx=0 tx=0).stop 2023-10-08T13:49:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 -- 172.21.15.57:0/1603257313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6000ff540 msgr2=0x7ff60019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6000ff540 0x7ff60019c8e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00e9a0 tx=0x7ff5fc00ecb0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 -- 172.21.15.57:0/1603257313 shutdown_connections 2023-10-08T13:49:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff600105600 0x7ff600196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6000ff540 0x7ff60019c8e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff5ec0a14d0 0x7ff5ec0a3990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 --2- 172.21.15.57:0/1603257313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6000fff00 0x7ff60019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:30.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 -- 172.21.15.57:0/1603257313 >> 172.21.15.57:0/1603257313 conn(0x7ff6000fb030 msgr2=0x7ff600103780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:30.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 -- 172.21.15.57:0/1603257313 shutdown_connections 2023-10-08T13:49:30.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:30.754+0000 7ff60749f700 1 -- 172.21.15.57:0/1603257313 wait complete. 2023-10-08T13:49:30.756 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 9 2023-10-08T13:49:31.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:30 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] up:boot 2023-10-08T13:49:31.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:30 smithi115 ceph-mon[104408]: fsmap 8 up:standby 2023-10-08T13:49:31.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T13:49:31.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:31.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:31.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:31.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:31.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:31.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:31.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:30 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1603257313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:31.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:30 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] up:boot 2023-10-08T13:49:31.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:30 smithi057 ceph-mon[99476]: fsmap 8 up:standby 2023-10-08T13:49:31.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T13:49:31.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:31.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:31.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:31.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:31.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:31.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:31.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:30 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1603257313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:31.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:30 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] up:boot 2023-10-08T13:49:31.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:30 smithi099 ceph-mon[103849]: fsmap 8 up:standby 2023-10-08T13:49:31.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T13:49:31.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:31.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:31.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:31.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:31.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:31.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:31.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:30 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1603257313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:31.686 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 13:49:31 smithi115 systemd[1]: Starting Ceph mds.i for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:32.055 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:32 smithi115 ceph-mon[104408]: Deploying daemon mds.i on smithi115 2023-10-08T13:49:32.056 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:32 smithi115 ceph-mon[104408]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:32.056 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 13:49:31 smithi115 podman[141861]: 2023-10-08T13:49:32.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:32 smithi099 ceph-mon[103849]: Deploying daemon mds.i on smithi115 2023-10-08T13:49:32.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:32 smithi099 ceph-mon[103849]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:32.436 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 13:49:32 smithi115 bash[141861]: df7671feed7f0375e122292375954860f73387b8dc553e101c76fa5f5d5f4656 2023-10-08T13:49:32.437 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 13:49:32 smithi115 ceph-mds[141901]: -- 172.21.15.115:0/2060006772 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559f1442b1e0 con 0x559f14401000 2023-10-08T13:49:32.437 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 13:49:32 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-i[141877]: starting mds.i at 2023-10-08T13:49:32.437 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 13:49:32 smithi115 systemd[1]: Started Ceph mds.i for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:32.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:32 smithi057 ceph-mon[99476]: Deploying daemon mds.i on smithi115 2023-10-08T13:49:32.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:32 smithi057 ceph-mon[99476]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:32.768 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:32.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.902+0000 7fc05cc60700 1 -- 172.21.15.57:0/257480616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0580ff540 msgr2=0x7fc0580ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.902+0000 7fc05cc60700 1 --2- 172.21.15.57:0/257480616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0580ff540 0x7fc0580ff9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0099c0 tx=0x7fc04c009cd0 comp rx=0 tx=0).stop 2023-10-08T13:49:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.902+0000 7fc05cc60700 1 -- 172.21.15.57:0/257480616 shutdown_connections 2023-10-08T13:49:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.902+0000 7fc05cc60700 1 --2- 172.21.15.57:0/257480616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0580fff00 0x7fc05810c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.902+0000 7fc05cc60700 1 --2- 172.21.15.57:0/257480616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc058105600 0x7fc0581059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.902+0000 7fc05cc60700 1 --2- 172.21.15.57:0/257480616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0580ff540 0x7fc0580ff9c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.902+0000 7fc05cc60700 1 -- 172.21.15.57:0/257480616 >> 172.21.15.57:0/257480616 conn(0x7fc0580fb030 msgr2=0x7fc0580fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.902+0000 7fc05cc60700 1 -- 172.21.15.57:0/257480616 shutdown_connections 2023-10-08T13:49:32.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.902+0000 7fc05cc60700 1 -- 172.21.15.57:0/257480616 wait complete. 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05cc60700 1 Processor -- start 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05cc60700 1 -- start start 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05cc60700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0580ff540 0x7fc058071f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05cc60700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0580fff00 0x7fc058072460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05cc60700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc058105600 0x7fc05806f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05cc60700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05806f950 con 0x7fc0580ff540 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05cc60700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05806fac0 con 0x7fc058105600 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05cc60700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05806fc80 con 0x7fc0580fff00 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0580ff540 0x7fc058071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0580ff540 0x7fc058071f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53774/0 (socket says 172.21.15.57:53774) 2023-10-08T13:49:32.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.903+0000 7fc05659c700 1 -- 172.21.15.57:0/72217764 learned_addr learned my addr 172.21.15.57:0/72217764 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc056d9d700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc058105600 0x7fc05806f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc055d9b700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0580fff00 0x7fc058072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc05659c700 1 -- 172.21.15.57:0/72217764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0580fff00 msgr2=0x7fc058072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc05659c700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0580fff00 0x7fc058072460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc05659c700 1 -- 172.21.15.57:0/72217764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc058105600 msgr2=0x7fc05806f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc05659c700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc058105600 0x7fc05806f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc05659c700 1 -- 172.21.15.57:0/72217764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c0095f0 con 0x7fc0580ff540 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc05659c700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0580ff540 0x7fc058071f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc04800d980 tx=0x7fc04800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc0477fe700 1 -- 172.21.15.57:0/72217764 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800bb60 con 0x7fc0580ff540 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc0477fe700 1 -- 172.21.15.57:0/72217764 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc048007640 con 0x7fc0580ff540 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc05cc60700 1 -- 172.21.15.57:0/72217764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0581add90 con 0x7fc0580ff540 2023-10-08T13:49:32.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc0477fe700 1 -- 172.21.15.57:0/72217764 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048004e90 con 0x7fc0580ff540 2023-10-08T13:49:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.904+0000 7fc05cc60700 1 -- 172.21.15.57:0/72217764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0581ae160 con 0x7fc0580ff540 2023-10-08T13:49:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.905+0000 7fc05cc60700 1 -- 172.21.15.57:0/72217764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc058100c60 con 0x7fc0580ff540 2023-10-08T13:49:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.906+0000 7fc0477fe700 1 -- 172.21.15.57:0/72217764 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc04800bcc0 con 0x7fc0580ff540 2023-10-08T13:49:32.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.907+0000 7fc0477fe700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc0400a14d0 0x7fc0400a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:32.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.907+0000 7fc055d9b700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc0400a14d0 0x7fc0400a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:32.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.907+0000 7fc0477fe700 1 -- 172.21.15.57:0/72217764 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fc0480ccd00 con 0x7fc0580ff540 2023-10-08T13:49:32.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.908+0000 7fc055d9b700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc0400a14d0 0x7fc0400a3990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009fd0 tx=0x7fc04c005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:32.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:32.911+0000 7fc0477fe700 1 -- 172.21.15.57:0/72217764 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc048096710 con 0x7fc0580ff540 2023-10-08T13:49:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.182+0000 7fc05cc60700 1 -- 172.21.15.57:0/72217764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc05806a660 con 0x7fc0580ff540 2023-10-08T13:49:33.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.184+0000 7fc0477fe700 1 -- 172.21.15.57:0/72217764 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7658 (secure 0 0 0) 0x7fc048099d30 con 0x7fc0580ff540 2023-10-08T13:49:33.185 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:33.185 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24581,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24617,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24652,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-08T13:49:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.185+0000 7fc05cc60700 1 -- 172.21.15.57:0/72217764 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc0400a14d0 msgr2=0x7fc0400a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc0400a14d0 0x7fc0400a3990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009fd0 tx=0x7fc04c005fd0 comp rx=0 tx=0).stop 2023-10-08T13:49:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 -- 172.21.15.57:0/72217764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0580ff540 msgr2=0x7fc058071f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0580ff540 0x7fc058071f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc04800d980 tx=0x7fc04800dd40 comp rx=0 tx=0).stop 2023-10-08T13:49:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 -- 172.21.15.57:0/72217764 shutdown_connections 2023-10-08T13:49:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc0400a14d0 0x7fc0400a3990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc058105600 0x7fc05806f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0580ff540 0x7fc058071f20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 --2- 172.21.15.57:0/72217764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0580fff00 0x7fc058072460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 -- 172.21.15.57:0/72217764 >> 172.21.15.57:0/72217764 conn(0x7fc0580fb030 msgr2=0x7fc058067890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 -- 172.21.15.57:0/72217764 shutdown_connections 2023-10-08T13:49:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:33.186+0000 7fc05cc60700 1 -- 172.21.15.57:0/72217764 wait complete. 2023-10-08T13:49:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 10 2023-10-08T13:49:33.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:33.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:33.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:33.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:33.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:33.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:33.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:33.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: Deploying daemon mds.g on smithi057 2023-10-08T13:49:33.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] up:boot 2023-10-08T13:49:33.380 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: fsmap 9 up:standby 2023-10-08T13:49:33.381 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:33 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T13:49:33.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: Deploying daemon mds.g on smithi057 2023-10-08T13:49:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] up:boot 2023-10-08T13:49:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: fsmap 9 up:standby 2023-10-08T13:49:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:33 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T13:49:33.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:33.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:33.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:33.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:33.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:33.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:33.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:33.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: Deploying daemon mds.g on smithi057 2023-10-08T13:49:33.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] up:boot 2023-10-08T13:49:33.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: fsmap 9 up:standby 2023-10-08T13:49:33.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T13:49:33.988 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 08 13:49:33 smithi057 systemd[1]: Starting Ceph mds.g for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:34.334 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:34 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/72217764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:34.334 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 08 13:49:34 smithi057 podman[157238]: 2023-10-08T13:49:34.334 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 08 13:49:34 smithi057 bash[157238]: fa90b16e98211f0995cc7135472cd169448e316a4f77c43bf15804a775a80390 2023-10-08T13:49:34.334 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 08 13:49:34 smithi057 ceph-mds[157278]: -- 172.21.15.57:0/489438864 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6ecb211e0 con 0x55b6ecaf7000 2023-10-08T13:49:34.334 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 08 13:49:34 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-g[157254]: starting mds.g at 2023-10-08T13:49:34.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:34 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/72217764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:34.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:34 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/72217764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:34.737 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 08 13:49:34 smithi057 systemd[1]: Started Ceph mds.g for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:35.200 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:35.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.337+0000 7fb671c47700 1 -- 172.21.15.57:0/3326388381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66c105600 msgr2=0x7fb66c1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:35.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.337+0000 7fb671c47700 1 --2- 172.21.15.57:0/3326388381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66c105600 0x7fb66c1059e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb664009fc0 tx=0x7fb66400b330 comp rx=0 tx=0).stop 2023-10-08T13:49:35.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.337+0000 7fb671c47700 1 -- 172.21.15.57:0/3326388381 shutdown_connections 2023-10-08T13:49:35.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.337+0000 7fb671c47700 1 --2- 172.21.15.57:0/3326388381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb66c0fff00 0x7fb66c10c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:35.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.337+0000 7fb671c47700 1 --2- 172.21.15.57:0/3326388381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66c105600 0x7fb66c1059e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:35.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.337+0000 7fb671c47700 1 --2- 172.21.15.57:0/3326388381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb66c0ff540 0x7fb66c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:35.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.337+0000 7fb671c47700 1 -- 172.21.15.57:0/3326388381 >> 172.21.15.57:0/3326388381 conn(0x7fb66c0fb030 msgr2=0x7fb66c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:35.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.337+0000 7fb671c47700 1 -- 172.21.15.57:0/3326388381 shutdown_connections 2023-10-08T13:49:35.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.337+0000 7fb671c47700 1 -- 172.21.15.57:0/3326388381 wait complete. 2023-10-08T13:49:35.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.338+0000 7fb671c47700 1 Processor -- start 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.338+0000 7fb671c47700 1 -- start start 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.338+0000 7fb671c47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66c0ff540 0x7fb66c1957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.338+0000 7fb671c47700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb66c0fff00 0x7fb66c195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.338+0000 7fb671c47700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb66c105600 0x7fb66c196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.338+0000 7fb671c47700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c1a4f50 con 0x7fb66c105600 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.338+0000 7fb671c47700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c1a50c0 con 0x7fb66c0ff540 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.338+0000 7fb671c47700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c1a5280 con 0x7fb66c0fff00 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb66c105600 0x7fb66c196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66c0ff540 0x7fb66c1957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb66c105600 0x7fb66c196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53806/0 (socket says 172.21.15.57:53806) 2023-10-08T13:49:35.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66c0ff540 0x7fb66c1957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:55228/0 (socket says 172.21.15.57:55228) 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66bfff700 1 -- 172.21.15.57:0/3898255610 learned_addr learned my addr 172.21.15.57:0/3898255610 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66affd700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb66c0fff00 0x7fb66c195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66bfff700 1 -- 172.21.15.57:0/3898255610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb66c0fff00 msgr2=0x7fb66c195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66bfff700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb66c0fff00 0x7fb66c195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66bfff700 1 -- 172.21.15.57:0/3898255610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66c0ff540 msgr2=0x7fb66c1957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66bfff700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66c0ff540 0x7fb66c1957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66bfff700 1 -- 172.21.15.57:0/3898255610 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb660009520 con 0x7fb66c105600 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66b7fe700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66c0ff540 0x7fb66c1957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb66bfff700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb66c105600 0x7fb66c196230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00eb80 tx=0x7fb65c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb668ff9700 1 -- 172.21.15.57:0/3898255610 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c0099f0 con 0x7fb66c105600 2023-10-08T13:49:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb671c47700 1 -- 172.21.15.57:0/3898255610 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c0095f0 con 0x7fb66c105600 2023-10-08T13:49:35.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb668ff9700 1 -- 172.21.15.57:0/3898255610 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb65c0044f0 con 0x7fb66c105600 2023-10-08T13:49:35.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb668ff9700 1 -- 172.21.15.57:0/3898255610 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c003e90 con 0x7fb66c105600 2023-10-08T13:49:35.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.339+0000 7fb671c47700 1 -- 172.21.15.57:0/3898255610 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb664009c40 con 0x7fb66c105600 2023-10-08T13:49:35.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.340+0000 7fb671c47700 1 -- 172.21.15.57:0/3898255610 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66c107a80 con 0x7fb66c105600 2023-10-08T13:49:35.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.341+0000 7fb668ff9700 1 -- 172.21.15.57:0/3898255610 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb65c004000 con 0x7fb66c105600 2023-10-08T13:49:35.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.342+0000 7fb668ff9700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb6540a14d0 0x7fb6540a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:35.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.342+0000 7fb66b7fe700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb6540a14d0 0x7fb6540a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:35.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.342+0000 7fb668ff9700 1 -- 172.21.15.57:0/3898255610 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fb65c014070 con 0x7fb66c105600 2023-10-08T13:49:35.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.343+0000 7fb66b7fe700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb6540a14d0 0x7fb6540a3990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb6640060b0 tx=0x7fb664006000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:35.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.347+0000 7fb668ff9700 1 -- 172.21.15.57:0/3898255610 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb65c096810 con 0x7fb66c105600 2023-10-08T13:49:35.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.599+0000 7fb671c47700 1 -- 172.21.15.57:0/3898255610 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb66c100c60 con 0x7fb66c105600 2023-10-08T13:49:35.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.600+0000 7fb668ff9700 1 -- 172.21.15.57:0/3898255610 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8450 (secure 0 0 0) 0x7fb65c099e30 con 0x7fb66c105600 2023-10-08T13:49:35.601 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:35.601 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24581,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24617,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24652,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-08T13:49:35.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.602+0000 7fb671c47700 1 -- 172.21.15.57:0/3898255610 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb6540a14d0 msgr2=0x7fb6540a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:35.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.602+0000 7fb671c47700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb6540a14d0 0x7fb6540a3990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb6640060b0 tx=0x7fb664006000 comp rx=0 tx=0).stop 2023-10-08T13:49:35.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.602+0000 7fb671c47700 1 -- 172.21.15.57:0/3898255610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb66c105600 msgr2=0x7fb66c196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:35.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.602+0000 7fb671c47700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb66c105600 0x7fb66c196230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00eb80 tx=0x7fb65c00ee90 comp rx=0 tx=0).stop 2023-10-08T13:49:35.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.602+0000 7fb671c47700 1 -- 172.21.15.57:0/3898255610 shutdown_connections 2023-10-08T13:49:35.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.602+0000 7fb671c47700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb66c105600 0x7fb66c196230 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:35.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.602+0000 7fb671c47700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb6540a14d0 0x7fb6540a3990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:35.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.602+0000 7fb671c47700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66c0ff540 0x7fb66c1957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:35.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.602+0000 7fb671c47700 1 --2- 172.21.15.57:0/3898255610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb66c0fff00 0x7fb66c195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:35.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.602+0000 7fb671c47700 1 -- 172.21.15.57:0/3898255610 >> 172.21.15.57:0/3898255610 conn(0x7fb66c0fb030 msgr2=0x7fb66c0fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:35.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.603+0000 7fb671c47700 1 -- 172.21.15.57:0/3898255610 shutdown_connections 2023-10-08T13:49:35.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:35.603+0000 7fb671c47700 1 -- 172.21.15.57:0/3898255610 wait complete. 2023-10-08T13:49:35.605 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 11 2023-10-08T13:49:35.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:35.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:35.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:35.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:35.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:35.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:35.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:35.615 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: Deploying daemon mds.a on smithi057 2023-10-08T13:49:35.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.57:6838/1513163924,v1:172.21.15.57:6839/1513163924] up:boot 2023-10-08T13:49:35.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: fsmap 10 up:standby 2023-10-08T13:49:35.616 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T13:49:35.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:35.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:35.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:35.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:35.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:35.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:35.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:35.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: Deploying daemon mds.a on smithi057 2023-10-08T13:49:35.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.57:6838/1513163924,v1:172.21.15.57:6839/1513163924] up:boot 2023-10-08T13:49:35.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: fsmap 10 up:standby 2023-10-08T13:49:35.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T13:49:35.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:35.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:35.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:35.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:35.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:35.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:35.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:35.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: Deploying daemon mds.a on smithi057 2023-10-08T13:49:35.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.57:6838/1513163924,v1:172.21.15.57:6839/1513163924] up:boot 2023-10-08T13:49:35.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: fsmap 10 up:standby 2023-10-08T13:49:35.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T13:49:35.921 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 13:49:35 smithi057 systemd[1]: Starting Ceph mds.a for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:36.468 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:36 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3898255610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:36.468 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 13:49:36 smithi057 podman[157627]: 2023-10-08T13:49:36.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:36 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3898255610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:36.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:36 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3898255610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:36.987 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 13:49:36 smithi057 bash[157627]: a4099f8c313310b12d0ce5914a09b943563ea2f72c0eb95b3fe2c31b870e0b6d 2023-10-08T13:49:36.987 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 13:49:36 smithi057 ceph-mds[157667]: -- 172.21.15.57:0/85826678 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5639d0a1d1e0 con 0x5639d09f3000 2023-10-08T13:49:36.987 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 13:49:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-a[157643]: starting mds.a at 2023-10-08T13:49:36.988 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 13:49:36 smithi057 systemd[1]: Started Ceph mds.a for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:37.618 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:37.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.762+0000 7f56c40fe700 1 -- 172.21.15.57:0/648494429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56bc0fff00 msgr2=0x7f56bc10c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:37.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.762+0000 7f56c40fe700 1 --2- 172.21.15.57:0/648494429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56bc0fff00 0x7f56bc10c3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009fc0 tx=0x7f56b400b330 comp rx=0 tx=0).stop 2023-10-08T13:49:37.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.762+0000 7f56c40fe700 1 -- 172.21.15.57:0/648494429 shutdown_connections 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.762+0000 7f56c40fe700 1 --2- 172.21.15.57:0/648494429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56bc0fff00 0x7f56bc10c3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.762+0000 7f56c40fe700 1 --2- 172.21.15.57:0/648494429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56bc105600 0x7f56bc1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.762+0000 7f56c40fe700 1 --2- 172.21.15.57:0/648494429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56bc0ff540 0x7f56bc0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.762+0000 7f56c40fe700 1 -- 172.21.15.57:0/648494429 >> 172.21.15.57:0/648494429 conn(0x7f56bc0fb030 msgr2=0x7f56bc0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c40fe700 1 -- 172.21.15.57:0/648494429 shutdown_connections 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c40fe700 1 -- 172.21.15.57:0/648494429 wait complete. 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c40fe700 1 Processor -- start 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c40fe700 1 -- start start 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c40fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56bc0ff540 0x7f56bc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c40fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56bc0fff00 0x7f56bc19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:37.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c40fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56bc105600 0x7f56bc196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c40fe700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc1971f0 con 0x7f56bc105600 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c40fe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc197360 con 0x7f56bc0ff540 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c40fe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc197520 con 0x7f56bc0fff00 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c1e9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56bc0ff540 0x7f56bc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c269b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56bc105600 0x7f56bc196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c1e9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56bc0ff540 0x7f56bc19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:55254/0 (socket says 172.21.15.57:55254) 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.763+0000 7f56c1e9a700 1 -- 172.21.15.57:0/188634412 learned_addr learned my addr 172.21.15.57:0/188634412 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56c1699700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56bc0fff00 0x7f56bc19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56c269b700 1 -- 172.21.15.57:0/188634412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56bc0fff00 msgr2=0x7f56bc19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56c269b700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56bc0fff00 0x7f56bc19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56c269b700 1 -- 172.21.15.57:0/188634412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56bc0ff540 msgr2=0x7f56bc19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:37.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56c269b700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56bc0ff540 0x7f56bc19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56c269b700 1 -- 172.21.15.57:0/188634412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80095f0 con 0x7f56bc105600 2023-10-08T13:49:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56c1e9a700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56bc0ff540 0x7f56bc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:49:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56c269b700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56bc105600 0x7f56bc196a20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f56b4015000 tx=0x7f56b40093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56b2ffd700 1 -- 172.21.15.57:0/188634412 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b400e040 con 0x7f56bc105600 2023-10-08T13:49:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56b2ffd700 1 -- 172.21.15.57:0/188634412 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b401b450 con 0x7f56bc105600 2023-10-08T13:49:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56b2ffd700 1 -- 172.21.15.57:0/188634412 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b40052a0 con 0x7f56bc105600 2023-10-08T13:49:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56c40fe700 1 -- 172.21.15.57:0/188634412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b4009c40 con 0x7f56bc105600 2023-10-08T13:49:37.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.764+0000 7f56c40fe700 1 -- 172.21.15.57:0/188634412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc1a3720 con 0x7f56bc105600 2023-10-08T13:49:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.765+0000 7f56c40fe700 1 -- 172.21.15.57:0/188634412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc100c60 con 0x7f56bc105600 2023-10-08T13:49:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.766+0000 7f56b2ffd700 1 -- 172.21.15.57:0/188634412 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f56b401b8c0 con 0x7f56bc105600 2023-10-08T13:49:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.767+0000 7f56b2ffd700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56a80a14d0 0x7f56a80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.767+0000 7f56c1e9a700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56a80a14d0 0x7f56a80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:37.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.767+0000 7f56b2ffd700 1 -- 172.21.15.57:0/188634412 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f56b40cdea0 con 0x7f56bc105600 2023-10-08T13:49:37.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.768+0000 7f56c1e9a700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56a80a14d0 0x7f56a80a3990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56ac011550 tx=0x7f56ac009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:37.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:37.771+0000 7f56b2ffd700 1 -- 172.21.15.57:0/188634412 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f56b4097960 con 0x7f56bc105600 2023-10-08T13:49:37.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:37.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:37.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:37.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:37.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:37.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:37.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:37.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: Deploying daemon mds.b on smithi099 2023-10-08T13:49:37.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.57:6840/1847373212,v1:172.21.15.57:6841/1847373212] up:boot 2023-10-08T13:49:37.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: fsmap 11 up:standby 2023-10-08T13:49:37.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T13:49:37.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:37.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:37.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:37.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:37.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:37.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:37.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:37.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: Deploying daemon mds.b on smithi099 2023-10-08T13:49:37.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.57:6840/1847373212,v1:172.21.15.57:6841/1847373212] up:boot 2023-10-08T13:49:37.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: fsmap 11 up:standby 2023-10-08T13:49:37.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T13:49:38.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.017+0000 7f56c40fe700 1 -- 172.21.15.57:0/188634412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56bc10f4b0 con 0x7f56bc105600 2023-10-08T13:49:38.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.018+0000 7f56b2ffd700 1 -- 172.21.15.57:0/188634412 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9242 (secure 0 0 0) 0x7f56b409af80 con 0x7f56bc105600 2023-10-08T13:49:38.019 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:38.019 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14790,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24581,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24617,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24652,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-08T13:49:38.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 -- 172.21.15.57:0/188634412 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56a80a14d0 msgr2=0x7f56a80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:38.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56a80a14d0 0x7f56a80a3990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56ac011550 tx=0x7f56ac009260 comp rx=0 tx=0).stop 2023-10-08T13:49:38.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 -- 172.21.15.57:0/188634412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56bc105600 msgr2=0x7f56bc196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:38.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56bc105600 0x7f56bc196a20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f56b4015000 tx=0x7f56b40093b0 comp rx=0 tx=0).stop 2023-10-08T13:49:38.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 -- 172.21.15.57:0/188634412 shutdown_connections 2023-10-08T13:49:38.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56bc105600 0x7f56bc196a20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:38.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56a80a14d0 0x7f56a80a3990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:38.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56bc0ff540 0x7f56bc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:38.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 --2- 172.21.15.57:0/188634412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56bc0fff00 0x7f56bc19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:38.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 -- 172.21.15.57:0/188634412 >> 172.21.15.57:0/188634412 conn(0x7f56bc0fb030 msgr2=0x7f56bc0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:38.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 -- 172.21.15.57:0/188634412 shutdown_connections 2023-10-08T13:49:38.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:38.020+0000 7f56c40fe700 1 -- 172.21.15.57:0/188634412 wait complete. 2023-10-08T13:49:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T13:49:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T13:49:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: Deploying daemon mds.b on smithi099 2023-10-08T13:49:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.57:6840/1847373212,v1:172.21.15.57:6841/1847373212] up:boot 2023-10-08T13:49:38.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: fsmap 11 up:standby 2023-10-08T13:49:38.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T13:49:38.024 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 12 2023-10-08T13:49:38.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:38 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/188634412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:38.773 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 13:49:38 smithi099 systemd[1]: Starting Ceph mds.b for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T13:49:38.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:38 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/188634412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:39.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:38 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/188634412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:39.191 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 13:49:38 smithi099 podman[140868]: 2023-10-08T13:49:39.191 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 13:49:39 smithi099 ceph-mds[140909]: -- 172.21.15.99:0/2504787439 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5620329851e0 con 0x56203295a800 2023-10-08T13:49:39.191 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 13:49:39 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b[140884]: starting mds.b at 2023-10-08T13:49:39.522 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 13:49:39 smithi099 bash[140868]: 4bbe0e163e806b9262bf9aa34ce44318416cddf454e2b4176e53279deabee8ac 2023-10-08T13:49:39.522 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 13:49:39 smithi099 systemd[1]: Started Ceph mds.b for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T13:49:39.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:39 smithi057 ceph-mon[99476]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:39.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:39 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:39.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:39 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:39.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:39 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:39.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:39 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:39.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:39 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:39.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:39 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:40.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:39 smithi099 ceph-mon[103849]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:40.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:39 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:40.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:39 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:40.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:39 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:40.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:39 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:40.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:39 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:40.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:39 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:40.034 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:40.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 -- 172.21.15.57:0/778249097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6860106630 msgr2=0x7f6860106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 --2- 172.21.15.57:0/778249097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6860106630 0x7f6860106a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f685c009990 tx=0x7f685c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 -- 172.21.15.57:0/778249097 shutdown_connections 2023-10-08T13:49:40.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 --2- 172.21.15.57:0/778249097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68600ff510 0x7f686010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 --2- 172.21.15.57:0/778249097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6860106630 0x7f6860106a10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 --2- 172.21.15.57:0/778249097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68600feb50 0x7f68600fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 -- 172.21.15.57:0/778249097 >> 172.21.15.57:0/778249097 conn(0x7f6860075a90 msgr2=0x7f68600fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:40.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 -- 172.21.15.57:0/778249097 shutdown_connections 2023-10-08T13:49:40.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 -- 172.21.15.57:0/778249097 wait complete. 2023-10-08T13:49:40.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 Processor -- start 2023-10-08T13:49:40.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.166+0000 7f68684c5700 1 -- start start 2023-10-08T13:49:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f68684c5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68600feb50 0x7f686019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f68684c5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68600ff510 0x7f686019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f68684c5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6860106630 0x7f68601968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f68684c5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6860196f30 con 0x7f68600ff510 2023-10-08T13:49:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f68684c5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68601970a0 con 0x7f68600feb50 2023-10-08T13:49:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f68684c5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6860197260 con 0x7f6860106630 2023-10-08T13:49:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6865a60700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68600ff510 0x7f686019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6865a60700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68600ff510 0x7f686019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53842/0 (socket says 172.21.15.57:53842) 2023-10-08T13:49:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6865a60700 1 -- 172.21.15.57:0/1780056930 learned_addr learned my addr 172.21.15.57:0/1780056930 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:40.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6866a62700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6860106630 0x7f68601968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6866261700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68600feb50 0x7f686019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6865a60700 1 -- 172.21.15.57:0/1780056930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6860106630 msgr2=0x7f68601968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6865a60700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6860106630 0x7f68601968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6865a60700 1 -- 172.21.15.57:0/1780056930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68600feb50 msgr2=0x7f686019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6865a60700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68600feb50 0x7f686019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6865a60700 1 -- 172.21.15.57:0/1780056930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0095f0 con 0x7f68600ff510 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f6865a60700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68600ff510 0x7f686019cce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f685000d950 tx=0x7f685000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f68577fe700 1 -- 172.21.15.57:0/1780056930 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685000f890 con 0x7f68600ff510 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f68577fe700 1 -- 172.21.15.57:0/1780056930 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6850010460 con 0x7f68600ff510 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f68684c5700 1 -- 172.21.15.57:0/1780056930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6860197540 con 0x7f68600ff510 2023-10-08T13:49:40.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.167+0000 7f68577fe700 1 -- 172.21.15.57:0/1780056930 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685000e610 con 0x7f68600ff510 2023-10-08T13:49:40.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.168+0000 7f68684c5700 1 -- 172.21.15.57:0/1780056930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68601a3790 con 0x7f68600ff510 2023-10-08T13:49:40.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.168+0000 7f68684c5700 1 -- 172.21.15.57:0/1780056930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6860108c20 con 0x7f68600ff510 2023-10-08T13:49:40.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.170+0000 7f68577fe700 1 -- 172.21.15.57:0/1780056930 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f68500108d0 con 0x7f68600ff510 2023-10-08T13:49:40.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.170+0000 7f68577fe700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f684c0a14d0 0x7f684c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.171+0000 7f6866261700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f684c0a14d0 0x7f684c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.171+0000 7f68577fe700 1 -- 172.21.15.57:0/1780056930 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f68500cc760 con 0x7f68600ff510 2023-10-08T13:49:40.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.171+0000 7f6866261700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f684c0a14d0 0x7f684c0a3990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f685c0098e0 tx=0x7f685c005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:40.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.174+0000 7f68577fe700 1 -- 172.21.15.57:0/1780056930 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6850096190 con 0x7f68600ff510 2023-10-08T13:49:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:39 smithi115 ceph-mon[104408]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:39 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:40.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:39 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:40.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:39 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:40.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:39 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:40.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:39 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:40.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:39 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:40.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.430+0000 7f68684c5700 1 -- 172.21.15.57:0/1780056930 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6860101e70 con 0x7f68600ff510 2023-10-08T13:49:40.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.430+0000 7f68577fe700 1 -- 172.21.15.57:0/1780056930 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10031 (secure 0 0 0) 0x7f68500997b0 con 0x7f68600ff510 2023-10-08T13:49:40.431 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14790,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24581,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24617,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24652,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-08T13:49:40.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.432+0000 7f68684c5700 1 -- 172.21.15.57:0/1780056930 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f684c0a14d0 msgr2=0x7f684c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.432+0000 7f68684c5700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f684c0a14d0 0x7f684c0a3990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f685c0098e0 tx=0x7f685c005960 comp rx=0 tx=0).stop 2023-10-08T13:49:40.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.432+0000 7f68684c5700 1 -- 172.21.15.57:0/1780056930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68600ff510 msgr2=0x7f686019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.432+0000 7f68684c5700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68600ff510 0x7f686019cce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f685000d950 tx=0x7f685000dc60 comp rx=0 tx=0).stop 2023-10-08T13:49:40.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.432+0000 7f68684c5700 1 -- 172.21.15.57:0/1780056930 shutdown_connections 2023-10-08T13:49:40.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.432+0000 7f68684c5700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6860106630 0x7f68601968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.432+0000 7f68684c5700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f684c0a14d0 0x7f684c0a3990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.432+0000 7f68684c5700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68600feb50 0x7f686019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.432+0000 7f68684c5700 1 --2- 172.21.15.57:0/1780056930 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68600ff510 0x7f686019cce0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.432+0000 7f68684c5700 1 -- 172.21.15.57:0/1780056930 >> 172.21.15.57:0/1780056930 conn(0x7f6860075a90 msgr2=0x7f68600fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:40.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.433+0000 7f68684c5700 1 -- 172.21.15.57:0/1780056930 shutdown_connections 2023-10-08T13:49:40.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.433+0000 7f68684c5700 1 -- 172.21.15.57:0/1780056930 wait complete. 2023-10-08T13:49:40.435 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2023-10-08T13:49:40.446 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:49:40.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.575+0000 7fb802feb700 1 -- 172.21.15.57:0/456417842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7fc1017f0 msgr2=0x7fb7fc101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.575+0000 7fb802feb700 1 --2- 172.21.15.57:0/456417842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7fc1017f0 0x7fb7fc101bd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009990 tx=0x7fb7f0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.575+0000 7fb802feb700 1 -- 172.21.15.57:0/456417842 shutdown_connections 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.575+0000 7fb802feb700 1 --2- 172.21.15.57:0/456417842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7fc10abc0 0x7fb7fc10d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.575+0000 7fb802feb700 1 --2- 172.21.15.57:0/456417842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7fc1017f0 0x7fb7fc101bd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.575+0000 7fb802feb700 1 --2- 172.21.15.57:0/456417842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7fc102110 0x7fb7fc10a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.575+0000 7fb802feb700 1 -- 172.21.15.57:0/456417842 >> 172.21.15.57:0/456417842 conn(0x7fb7fc0762e0 msgr2=0x7fb7fc0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.575+0000 7fb802feb700 1 -- 172.21.15.57:0/456417842 shutdown_connections 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.575+0000 7fb802feb700 1 -- 172.21.15.57:0/456417842 wait complete. 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb802feb700 1 Processor -- start 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb802feb700 1 -- start start 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb802feb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7fc1017f0 0x7fb7fc0ff440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb802feb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7fc102110 0x7fb7fc0ff980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb802feb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7fc10abc0 0x7fb7fc0ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb802feb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1a4e90 con 0x7fb7fc1017f0 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb802feb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1a5000 con 0x7fb7fc10abc0 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb802feb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1a51c0 con 0x7fb7fc102110 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb800d87700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7fc1017f0 0x7fb7fc0ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb800d87700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7fc1017f0 0x7fb7fc0ff440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53854/0 (socket says 172.21.15.57:53854) 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb800d87700 1 -- 172.21.15.57:0/104713718 learned_addr learned my addr 172.21.15.57:0/104713718 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb7fbfff700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7fc102110 0x7fb7fc0ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb801588700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7fc10abc0 0x7fb7fc0ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb800d87700 1 -- 172.21.15.57:0/104713718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7fc102110 msgr2=0x7fb7fc0ff980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb800d87700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7fc102110 0x7fb7fc0ff980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb800d87700 1 -- 172.21.15.57:0/104713718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7fc10abc0 msgr2=0x7fb7fc0ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb800d87700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7fc10abc0 0x7fb7fc0ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb800d87700 1 -- 172.21.15.57:0/104713718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f00095f0 con 0x7fb7fc1017f0 2023-10-08T13:49:40.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb801588700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7fc10abc0 0x7fb7fc0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:49:40.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb800d87700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7fc1017f0 0x7fb7fc0ff440 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00095c0 tx=0x7fb7f000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:40.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.576+0000 7fb7f9ffb700 1 -- 172.21.15.57:0/104713718 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f001d070 con 0x7fb7fc1017f0 2023-10-08T13:49:40.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.577+0000 7fb7f9ffb700 1 -- 172.21.15.57:0/104713718 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7f00054f0 con 0x7fb7fc1017f0 2023-10-08T13:49:40.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.577+0000 7fb7f9ffb700 1 -- 172.21.15.57:0/104713718 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0004bd0 con 0x7fb7fc1017f0 2023-10-08T13:49:40.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.577+0000 7fb802feb700 1 -- 172.21.15.57:0/104713718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc1a5440 con 0x7fb7fc1017f0 2023-10-08T13:49:40.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.577+0000 7fb802feb700 1 -- 172.21.15.57:0/104713718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc1a58a0 con 0x7fb7fc1017f0 2023-10-08T13:49:40.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.578+0000 7fb802feb700 1 -- 172.21.15.57:0/104713718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc105d40 con 0x7fb7fc1017f0 2023-10-08T13:49:40.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.579+0000 7fb7f9ffb700 1 -- 172.21.15.57:0/104713718 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb7f000fb80 con 0x7fb7fc1017f0 2023-10-08T13:49:40.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.579+0000 7fb7f9ffb700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7ec0a14d0 0x7fb7ec0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.579+0000 7fb7f9ffb700 1 -- 172.21.15.57:0/104713718 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fb7f00cdb60 con 0x7fb7fc1017f0 2023-10-08T13:49:40.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.579+0000 7fb7fbfff700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7ec0a14d0 0x7fb7ec0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.580+0000 7fb7fbfff700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7ec0a14d0 0x7fb7ec0a3990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc100b70 tx=0x7fb7e800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:40.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.583+0000 7fb7f9ffb700 1 -- 172.21.15.57:0/104713718 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb7f0097620 con 0x7fb7fc1017f0 2023-10-08T13:49:40.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.795+0000 7fb802feb700 1 -- 172.21.15.57:0/104713718 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7fc0680b0 con 0x7fb7fc1017f0 2023-10-08T13:49:40.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.796+0000 7fb7f9ffb700 1 -- 172.21.15.57:0/104713718 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18686 (secure 0 0 0) 0x7fb7f00a1070 con 0x7fb7fc1017f0 2023-10-08T13:49:40.798 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:40.798 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":80,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:03.966059+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:49:40.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 -- 172.21.15.57:0/104713718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7ec0a14d0 msgr2=0x7fb7ec0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7ec0a14d0 0x7fb7ec0a3990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc100b70 tx=0x7fb7e800b410 comp rx=0 tx=0).stop 2023-10-08T13:49:40.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 -- 172.21.15.57:0/104713718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7fc1017f0 msgr2=0x7fb7fc0ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7fc1017f0 0x7fb7fc0ff440 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00095c0 tx=0x7fb7f000f690 comp rx=0 tx=0).stop 2023-10-08T13:49:40.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 -- 172.21.15.57:0/104713718 shutdown_connections 2023-10-08T13:49:40.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7ec0a14d0 0x7fb7ec0a3990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7fc10abc0 0x7fb7fc0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7fc1017f0 0x7fb7fc0ff440 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 --2- 172.21.15.57:0/104713718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7fc102110 0x7fb7fc0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 -- 172.21.15.57:0/104713718 >> 172.21.15.57:0/104713718 conn(0x7fb7fc0762e0 msgr2=0x7fb7fc0fbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:40.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 -- 172.21.15.57:0/104713718 shutdown_connections 2023-10-08T13:49:40.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.799+0000 7fb802feb700 1 -- 172.21.15.57:0/104713718 wait complete. 2023-10-08T13:49:40.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T13:49:40.812 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:49:40.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.955+0000 7fac2b763700 1 -- 172.21.15.57:0/1910717950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac241051c0 msgr2=0x7fac24105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.955+0000 7fac2b763700 1 --2- 172.21.15.57:0/1910717950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac241051c0 0x7fac24105640 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fac200099c0 tx=0x7fac20009cd0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.955+0000 7fac2b763700 1 -- 172.21.15.57:0/1910717950 shutdown_connections 2023-10-08T13:49:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.955+0000 7fac2b763700 1 --2- 172.21.15.57:0/1910717950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac24105b80 0x7fac2410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.955+0000 7fac2b763700 1 --2- 172.21.15.57:0/1910717950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac24067ce0 0x7fac240680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.955+0000 7fac2b763700 1 --2- 172.21.15.57:0/1910717950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac241051c0 0x7fac24105640 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.955+0000 7fac2b763700 1 -- 172.21.15.57:0/1910717950 >> 172.21.15.57:0/1910717950 conn(0x7fac24076780 msgr2=0x7fac24076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.955+0000 7fac2b763700 1 -- 172.21.15.57:0/1910717950 shutdown_connections 2023-10-08T13:49:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.955+0000 7fac2b763700 1 -- 172.21.15.57:0/1910717950 wait complete. 2023-10-08T13:49:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac2b763700 1 Processor -- start 2023-10-08T13:49:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac2b763700 1 -- start start 2023-10-08T13:49:40.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac2b763700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac24067ce0 0x7fac2419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac2b763700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241051c0 0x7fac2419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac2b763700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac24105b80 0x7fac24196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac2b763700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac241970a0 con 0x7fac24105b80 2023-10-08T13:49:40.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac2b763700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac24197210 con 0x7fac241051c0 2023-10-08T13:49:40.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac2b763700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac241973d0 con 0x7fac24067ce0 2023-10-08T13:49:40.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac294ff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac24067ce0 0x7fac2419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac29d00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac24105b80 0x7fac24196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac294ff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac24067ce0 0x7fac2419c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:54680/0 (socket says 172.21.15.57:54680) 2023-10-08T13:49:40.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac294ff700 1 -- 172.21.15.57:0/3650617965 learned_addr learned my addr 172.21.15.57:0/3650617965 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac28cfe700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241051c0 0x7fac2419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac294ff700 1 -- 172.21.15.57:0/3650617965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241051c0 msgr2=0x7fac2419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac294ff700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241051c0 0x7fac2419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac294ff700 1 -- 172.21.15.57:0/3650617965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac24105b80 msgr2=0x7fac24196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac294ff700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac24105b80 0x7fac24196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac294ff700 1 -- 172.21.15.57:0/3650617965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c009c40 con 0x7fac24067ce0 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac29d00700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac24105b80 0x7fac24196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.956+0000 7fac294ff700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac24067ce0 0x7fac2419c910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fac1400d950 tx=0x7fac1400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.957+0000 7fac1a7fc700 1 -- 172.21.15.57:0/3650617965 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14009930 con 0x7fac24067ce0 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.957+0000 7fac2b763700 1 -- 172.21.15.57:0/3650617965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac200095f0 con 0x7fac24067ce0 2023-10-08T13:49:40.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.957+0000 7fac1a7fc700 1 -- 172.21.15.57:0/3650617965 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac14010460 con 0x7fac24067ce0 2023-10-08T13:49:40.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.957+0000 7fac1a7fc700 1 -- 172.21.15.57:0/3650617965 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1400f620 con 0x7fac24067ce0 2023-10-08T13:49:40.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.957+0000 7fac2b763700 1 -- 172.21.15.57:0/3650617965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac241a3720 con 0x7fac24067ce0 2023-10-08T13:49:40.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.958+0000 7fac2b763700 1 -- 172.21.15.57:0/3650617965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac24108c70 con 0x7fac24067ce0 2023-10-08T13:49:40.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.959+0000 7fac1a7fc700 1 -- 172.21.15.57:0/3650617965 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fac140108d0 con 0x7fac24067ce0 2023-10-08T13:49:40.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.959+0000 7fac1a7fc700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac100a14d0 0x7fac100a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:40.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.960+0000 7fac28cfe700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac100a14d0 0x7fac100a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:40.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.960+0000 7fac1a7fc700 1 -- 172.21.15.57:0/3650617965 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fac140cc6f0 con 0x7fac24067ce0 2023-10-08T13:49:40.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.960+0000 7fac28cfe700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac100a14d0 0x7fac100a3990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fac20009fd0 tx=0x7fac20019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:40.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:40.963+0000 7fac1a7fc700 1 -- 172.21.15.57:0/3650617965 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fac14096120 con 0x7fac24067ce0 2023-10-08T13:49:41.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:40 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] up:boot 2023-10-08T13:49:41.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:40 smithi099 ceph-mon[103849]: fsmap 12 up:standby 2023-10-08T13:49:41.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:40 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T13:49:41.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:40 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1780056930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:41.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.169+0000 7fac2b763700 1 -- 172.21.15.57:0/3650617965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac24101e70 con 0x7fac24067ce0 2023-10-08T13:49:41.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.170+0000 7fac1a7fc700 1 -- 172.21.15.57:0/3650617965 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac14099740 con 0x7fac24067ce0 2023-10-08T13:49:41.171 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:49:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 -- 172.21.15.57:0/3650617965 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac100a14d0 msgr2=0x7fac100a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac100a14d0 0x7fac100a3990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fac20009fd0 tx=0x7fac20019040 comp rx=0 tx=0).stop 2023-10-08T13:49:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 -- 172.21.15.57:0/3650617965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac24067ce0 msgr2=0x7fac2419c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac24067ce0 0x7fac2419c910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fac1400d950 tx=0x7fac1400dc60 comp rx=0 tx=0).stop 2023-10-08T13:49:41.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 -- 172.21.15.57:0/3650617965 shutdown_connections 2023-10-08T13:49:41.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac24105b80 0x7fac24196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac24067ce0 0x7fac2419c910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fac100a14d0 0x7fac100a3990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 --2- 172.21.15.57:0/3650617965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241051c0 0x7fac2419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 -- 172.21.15.57:0/3650617965 >> 172.21.15.57:0/3650617965 conn(0x7fac24076780 msgr2=0x7fac240fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:41.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 -- 172.21.15.57:0/3650617965 shutdown_connections 2023-10-08T13:49:41.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.173+0000 7fac2b763700 1 -- 172.21.15.57:0/3650617965 wait complete. 2023-10-08T13:49:41.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:40 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] up:boot 2023-10-08T13:49:41.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:40 smithi115 ceph-mon[104408]: fsmap 12 up:standby 2023-10-08T13:49:41.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:40 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T13:49:41.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:40 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1780056930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:41.188 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T13:49:41.188 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:40 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] up:boot 2023-10-08T13:49:41.189 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:40 smithi057 ceph-mon[99476]: fsmap 12 up:standby 2023-10-08T13:49:41.189 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:40 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T13:49:41.189 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:40 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1780056930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:41.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.317+0000 7f6969ab4700 1 -- 172.21.15.57:0/3734853178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69640ff550 msgr2=0x7f69640ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:41.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.317+0000 7f6969ab4700 1 --2- 172.21.15.57:0/3734853178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69640ff550 0x7f69640ff9d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f694c0099c0 tx=0x7f694c009cd0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.317+0000 7f6969ab4700 1 -- 172.21.15.57:0/3734853178 shutdown_connections 2023-10-08T13:49:41.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.317+0000 7f6969ab4700 1 --2- 172.21.15.57:0/3734853178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69640fff10 0x7f696410c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.317+0000 7f6969ab4700 1 --2- 172.21.15.57:0/3734853178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6964105610 0x7f69641059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.317+0000 7f6969ab4700 1 --2- 172.21.15.57:0/3734853178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69640ff550 0x7f69640ff9d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.317+0000 7f6969ab4700 1 -- 172.21.15.57:0/3734853178 >> 172.21.15.57:0/3734853178 conn(0x7f69640fb060 msgr2=0x7f69640fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:41.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.318+0000 7f6969ab4700 1 -- 172.21.15.57:0/3734853178 shutdown_connections 2023-10-08T13:49:41.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.318+0000 7f6969ab4700 1 -- 172.21.15.57:0/3734853178 wait complete. 2023-10-08T13:49:41.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.318+0000 7f6969ab4700 1 Processor -- start 2023-10-08T13:49:41.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6969ab4700 1 -- start start 2023-10-08T13:49:41.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6969ab4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69640ff550 0x7f6964197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:41.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6969ab4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69640fff10 0x7f6964197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:41.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6969ab4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6964105610 0x7f69641a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:41.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6969ab4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6964198740 con 0x7f69640fff10 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6969ab4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69641a2f00 con 0x7f6964105610 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6969ab4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69641a3040 con 0x7f69640ff550 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6962ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69640fff10 0x7f6964197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6962ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69640fff10 0x7f6964197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53884/0 (socket says 172.21.15.57:53884) 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6962ffd700 1 -- 172.21.15.57:0/768327908 learned_addr learned my addr 172.21.15.57:0/768327908 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6963fff700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6964105610 0x7f69641a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f69637fe700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69640ff550 0x7f6964197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6962ffd700 1 -- 172.21.15.57:0/768327908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69640ff550 msgr2=0x7f6964197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.319+0000 7f6962ffd700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69640ff550 0x7f6964197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.320+0000 7f6962ffd700 1 -- 172.21.15.57:0/768327908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6964105610 msgr2=0x7f69641a29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:41.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.320+0000 7f6962ffd700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6964105610 0x7f69641a29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.320+0000 7f6962ffd700 1 -- 172.21.15.57:0/768327908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0095f0 con 0x7f69640fff10 2023-10-08T13:49:41.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.320+0000 7f69637fe700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69640ff550 0x7f6964197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:41.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.320+0000 7f6962ffd700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69640fff10 0x7f6964197e80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f694c006010 tx=0x7f694c00b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:41.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.320+0000 7f6960ff9700 1 -- 172.21.15.57:0/768327908 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c01d070 con 0x7f69640fff10 2023-10-08T13:49:41.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.320+0000 7f6960ff9700 1 -- 172.21.15.57:0/768327908 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f694c003b70 con 0x7f69640fff10 2023-10-08T13:49:41.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.320+0000 7f6969ab4700 1 -- 172.21.15.57:0/768327908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69641a32c0 con 0x7f69640fff10 2023-10-08T13:49:41.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.320+0000 7f6960ff9700 1 -- 172.21.15.57:0/768327908 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c00fc30 con 0x7f69640fff10 2023-10-08T13:49:41.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.320+0000 7f6969ab4700 1 -- 172.21.15.57:0/768327908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69641a36e0 con 0x7f69640fff10 2023-10-08T13:49:41.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.321+0000 7f6969ab4700 1 -- 172.21.15.57:0/768327908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6964107ab0 con 0x7f69640fff10 2023-10-08T13:49:41.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.322+0000 7f6960ff9700 1 -- 172.21.15.57:0/768327908 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f694c003ce0 con 0x7f69640fff10 2023-10-08T13:49:41.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.323+0000 7f6960ff9700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69500a14d0 0x7f69500a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:41.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.323+0000 7f6960ff9700 1 -- 172.21.15.57:0/768327908 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f694c0cde20 con 0x7f69640fff10 2023-10-08T13:49:41.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.323+0000 7f69637fe700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69500a14d0 0x7f69500a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:41.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.324+0000 7f69637fe700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69500a14d0 0x7f69500a3990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f69540093a0 tx=0x7f695400b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:41.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.327+0000 7f6960ff9700 1 -- 172.21.15.57:0/768327908 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f694c0978e0 con 0x7f69640fff10 2023-10-08T13:49:41.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.541+0000 7f6969ab4700 1 -- 172.21.15.57:0/768327908 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6964100c70 con 0x7f69640fff10 2023-10-08T13:49:41.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.541+0000 7f6960ff9700 1 -- 172.21.15.57:0/768327908 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f694c09af00 con 0x7f69640fff10 2023-10-08T13:49:41.543 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:41.543 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-10-08T13:49:41.543 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T13:49:41.543 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2023-10-08T13:49:41.543 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2023-10-08T13:49:41.543 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:49:41.543 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-10-08T13:49:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 -- 172.21.15.57:0/768327908 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69500a14d0 msgr2=0x7f69500a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69500a14d0 0x7f69500a3990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f69540093a0 tx=0x7f695400b040 comp rx=0 tx=0).stop 2023-10-08T13:49:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 -- 172.21.15.57:0/768327908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69640fff10 msgr2=0x7f6964197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69640fff10 0x7f6964197e80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f694c006010 tx=0x7f694c00b9c0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 -- 172.21.15.57:0/768327908 shutdown_connections 2023-10-08T13:49:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69500a14d0 0x7f69500a3990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6964105610 0x7f69641a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69640ff550 0x7f6964197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 --2- 172.21.15.57:0/768327908 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f69640fff10 0x7f6964197e80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 -- 172.21.15.57:0/768327908 >> 172.21.15.57:0/768327908 conn(0x7f69640fb060 msgr2=0x7f69640fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 -- 172.21.15.57:0/768327908 shutdown_connections 2023-10-08T13:49:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.543+0000 7f6969ab4700 1 -- 172.21.15.57:0/768327908 wait complete. 2023-10-08T13:49:41.557 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T13:49:41.557 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T13:49:41.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.697+0000 7f1fa62eb700 1 -- 172.21.15.57:0/79219149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fa0105630 msgr2=0x7f1fa0105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:41.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.697+0000 7f1fa62eb700 1 --2- 172.21.15.57:0/79219149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fa0105630 0x7f1fa0105a10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009990 tx=0x7f1f90009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.697+0000 7f1fa62eb700 1 -- 172.21.15.57:0/79219149 shutdown_connections 2023-10-08T13:49:41.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.697+0000 7f1fa62eb700 1 --2- 172.21.15.57:0/79219149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa00ffff0 0x7f1fa010c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.697+0000 7f1fa62eb700 1 --2- 172.21.15.57:0/79219149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fa0105630 0x7f1fa0105a10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.697+0000 7f1fa62eb700 1 --2- 172.21.15.57:0/79219149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa00ff630 0x7f1fa00ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.697+0000 7f1fa62eb700 1 -- 172.21.15.57:0/79219149 >> 172.21.15.57:0/79219149 conn(0x7f1fa0076790 msgr2=0x7f1fa0076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:41.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.697+0000 7f1fa62eb700 1 -- 172.21.15.57:0/79219149 shutdown_connections 2023-10-08T13:49:41.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.697+0000 7f1fa62eb700 1 -- 172.21.15.57:0/79219149 wait complete. 2023-10-08T13:49:41.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1fa62eb700 1 Processor -- start 2023-10-08T13:49:41.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1fa62eb700 1 -- start start 2023-10-08T13:49:41.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1fa62eb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fa00ff630 0x7f1fa00fc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:41.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1fa62eb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa00ffff0 0x7f1fa00fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:41.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1fa62eb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa0105630 0x7f1fa00fd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:41.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1fa62eb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa01a4ee0 con 0x7f1fa00ff630 2023-10-08T13:49:41.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1fa62eb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa01a5050 con 0x7f1fa0105630 2023-10-08T13:49:41.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1fa62eb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa01a5210 con 0x7f1fa00ffff0 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1f9ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fa00ff630 0x7f1fa00fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1f9ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fa00ff630 0x7f1fa00fc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53896/0 (socket says 172.21.15.57:53896) 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1f9ffff700 1 -- 172.21.15.57:0/1677812134 learned_addr learned my addr 172.21.15.57:0/1677812134 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1f9f7fe700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa00ffff0 0x7f1fa00fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.698+0000 7f1fa4888700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa0105630 0x7f1fa00fd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1f9ffff700 1 -- 172.21.15.57:0/1677812134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa00ffff0 msgr2=0x7f1fa00fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1f9ffff700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa00ffff0 0x7f1fa00fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1f9ffff700 1 -- 172.21.15.57:0/1677812134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa0105630 msgr2=0x7f1fa00fd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1f9ffff700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa0105630 0x7f1fa00fd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1f9ffff700 1 -- 172.21.15.57:0/1677812134 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900095f0 con 0x7f1fa00ff630 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1f9ffff700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fa00ff630 0x7f1fa00fc640 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1f90000c00 tx=0x7f1f90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:41.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1f9d7fa700 1 -- 172.21.15.57:0/1677812134 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9001d070 con 0x7f1fa00ff630 2023-10-08T13:49:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1f9d7fa700 1 -- 172.21.15.57:0/1677812134 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f90022470 con 0x7f1fa00ff630 2023-10-08T13:49:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1fa62eb700 1 -- 172.21.15.57:0/1677812134 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa01a5490 con 0x7f1fa00ff630 2023-10-08T13:49:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1f9d7fa700 1 -- 172.21.15.57:0/1677812134 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90017650 con 0x7f1fa00ff630 2023-10-08T13:49:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.699+0000 7f1fa62eb700 1 -- 172.21.15.57:0/1677812134 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa01a59c0 con 0x7f1fa00ff630 2023-10-08T13:49:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.700+0000 7f1fa62eb700 1 -- 172.21.15.57:0/1677812134 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa0100d60 con 0x7f1fa00ff630 2023-10-08T13:49:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.701+0000 7f1f9d7fa700 1 -- 172.21.15.57:0/1677812134 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f1f900225e0 con 0x7f1fa00ff630 2023-10-08T13:49:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.702+0000 7f1f9d7fa700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1f880a1550 0x7f1f880a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.702+0000 7f1f9d7fa700 1 -- 172.21.15.57:0/1677812134 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f1f900cda20 con 0x7f1fa00ff630 2023-10-08T13:49:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.702+0000 7f1f9f7fe700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1f880a1550 0x7f1f880a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.702+0000 7f1f9f7fe700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1f880a1550 0x7f1f880a3a10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00fdd70 tx=0x7f1f94009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.706+0000 7f1f9d7fa700 1 -- 172.21.15.57:0/1677812134 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1f900974e0 con 0x7f1fa00ff630 2023-10-08T13:49:41.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:41.914+0000 7f1fa62eb700 1 -- 172.21.15.57:0/1677812134 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1fa00fdb50 con 0x7f1fa00ff630 2023-10-08T13:49:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:41 smithi057 ceph-mon[99476]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:41 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/104713718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:41 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3650617965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:41.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:41 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/768327908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-08T13:49:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:41 smithi099 ceph-mon[103849]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:41 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/104713718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:41 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3650617965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:41 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/768327908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-08T13:49:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:41 smithi115 ceph-mon[104408]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:41 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/104713718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:42.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:41 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3650617965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:42.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:41 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/768327908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-08T13:49:42.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.751+0000 7f1f9d7fa700 1 -- 172.21.15.57:0/1677812134 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1f9009ab00 con 0x7f1fa00ff630 2023-10-08T13:49:42.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.754+0000 7f1fa62eb700 1 -- 172.21.15.57:0/1677812134 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1f880a1550 msgr2=0x7f1f880a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:42.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.754+0000 7f1fa62eb700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1f880a1550 0x7f1f880a3a10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00fdd70 tx=0x7f1f94009260 comp rx=0 tx=0).stop 2023-10-08T13:49:42.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.754+0000 7f1fa62eb700 1 -- 172.21.15.57:0/1677812134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fa00ff630 msgr2=0x7f1fa00fc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:42.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.754+0000 7f1fa62eb700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fa00ff630 0x7f1fa00fc640 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1f90000c00 tx=0x7f1f90004730 comp rx=0 tx=0).stop 2023-10-08T13:49:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.755+0000 7f1fa62eb700 1 -- 172.21.15.57:0/1677812134 shutdown_connections 2023-10-08T13:49:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.755+0000 7f1fa62eb700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1f880a1550 0x7f1f880a3a10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.755+0000 7f1fa62eb700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa0105630 0x7f1fa00fd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.755+0000 7f1fa62eb700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fa00ff630 0x7f1fa00fc640 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:42.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.755+0000 7f1fa62eb700 1 --2- 172.21.15.57:0/1677812134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa00ffff0 0x7f1fa00fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.755+0000 7f1fa62eb700 1 -- 172.21.15.57:0/1677812134 >> 172.21.15.57:0/1677812134 conn(0x7f1fa0076790 msgr2=0x7f1fa00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.755+0000 7f1fa62eb700 1 -- 172.21.15.57:0/1677812134 shutdown_connections 2023-10-08T13:49:42.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.755+0000 7f1fa62eb700 1 -- 172.21.15.57:0/1677812134 wait complete. 2023-10-08T13:49:42.759 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2023-10-08T13:49:42.772 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-08T13:49:42.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.910+0000 7f11e85be700 1 -- 172.21.15.57:0/3388299800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e0101100 msgr2=0x7f11e010d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:42.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.910+0000 7f11e85be700 1 --2- 172.21.15.57:0/3388299800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e0101100 0x7f11e010d610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f11d800b0a0 tx=0x7f11d800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:42.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.910+0000 7f11e85be700 1 -- 172.21.15.57:0/3388299800 shutdown_connections 2023-10-08T13:49:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.910+0000 7f11e85be700 1 --2- 172.21.15.57:0/3388299800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e0101100 0x7f11e010d610 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.910+0000 7f11e85be700 1 --2- 172.21.15.57:0/3388299800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e0106750 0x7f11e0106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.910+0000 7f11e85be700 1 --2- 172.21.15.57:0/3388299800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0100740 0x7f11e0100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.910+0000 7f11e85be700 1 -- 172.21.15.57:0/3388299800 >> 172.21.15.57:0/3388299800 conn(0x7f11e00fc230 msgr2=0x7f11e00fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.910+0000 7f11e85be700 1 -- 172.21.15.57:0/3388299800 shutdown_connections 2023-10-08T13:49:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.910+0000 7f11e85be700 1 -- 172.21.15.57:0/3388299800 wait complete. 2023-10-08T13:49:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e85be700 1 Processor -- start 2023-10-08T13:49:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e85be700 1 -- start start 2023-10-08T13:49:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e85be700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0100740 0x7f11e0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:42.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e85be700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e0101100 0x7f11e0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e85be700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e0106750 0x7f11e0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e85be700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01a4ed0 con 0x7f11e0106750 2023-10-08T13:49:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e85be700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01a5040 con 0x7f11e0100740 2023-10-08T13:49:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e85be700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01a5200 con 0x7f11e0101100 2023-10-08T13:49:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e6b5b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e0106750 0x7f11e0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e5b59700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e0101100 0x7f11e0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e6b5b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e0106750 0x7f11e0073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53914/0 (socket says 172.21.15.57:53914) 2023-10-08T13:49:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e6b5b700 1 -- 172.21.15.57:0/107256836 learned_addr learned my addr 172.21.15.57:0/107256836 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e635a700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0100740 0x7f11e0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:42.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e6b5b700 1 -- 172.21.15.57:0/107256836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e0101100 msgr2=0x7f11e0073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:42.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.911+0000 7f11e6b5b700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e0101100 0x7f11e0073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:42.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.912+0000 7f11e6b5b700 1 -- 172.21.15.57:0/107256836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0100740 msgr2=0x7f11e0072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:42.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.912+0000 7f11e6b5b700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0100740 0x7f11e0072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:42.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.912+0000 7f11e6b5b700 1 -- 172.21.15.57:0/107256836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d8009d00 con 0x7f11e0106750 2023-10-08T13:49:42.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.912+0000 7f11e5b59700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e0101100 0x7f11e0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:49:42.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.912+0000 7f11e6b5b700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e0106750 0x7f11e0073570 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f11d800b070 tx=0x7f11d8008ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:42.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.912+0000 7f11d77fe700 1 -- 172.21.15.57:0/107256836 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d800e040 con 0x7f11e0106750 2023-10-08T13:49:42.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.912+0000 7f11d77fe700 1 -- 172.21.15.57:0/107256836 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11d801d470 con 0x7f11e0106750 2023-10-08T13:49:42.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.912+0000 7f11e85be700 1 -- 172.21.15.57:0/107256836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e01a5480 con 0x7f11e0106750 2023-10-08T13:49:42.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.912+0000 7f11d77fe700 1 -- 172.21.15.57:0/107256836 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d80052a0 con 0x7f11e0106750 2023-10-08T13:49:42.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.912+0000 7f11e85be700 1 -- 172.21.15.57:0/107256836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e01a5900 con 0x7f11e0106750 2023-10-08T13:49:42.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.913+0000 7f11e85be700 1 -- 172.21.15.57:0/107256836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e0108d40 con 0x7f11e0106750 2023-10-08T13:49:42.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.914+0000 7f11d77fe700 1 -- 172.21.15.57:0/107256836 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f11d801d5e0 con 0x7f11e0106750 2023-10-08T13:49:42.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.915+0000 7f11d77fe700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a14d0 0x7f11cc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:42.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.915+0000 7f11e635a700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a14d0 0x7f11cc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:42.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.915+0000 7f11d77fe700 1 -- 172.21.15.57:0/107256836 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7712+0+0 (secure 0 0 0) 0x7f11d80cddb0 con 0x7f11e0106750 2023-10-08T13:49:42.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.915+0000 7f11e635a700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a14d0 0x7f11cc0a3990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009be0 tx=0x7f11dc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:42.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:42.920+0000 7f11d77fe700 1 -- 172.21.15.57:0/107256836 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f11d8097680 con 0x7f11e0106750 2023-10-08T13:49:43.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:42 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1677812134' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-08T13:49:43.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:42 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.144+0000 7f11e85be700 1 -- 172.21.15.57:0/107256836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f11e0073fd0 con 0x7f11e0106750 2023-10-08T13:49:43.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:42 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1677812134' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-08T13:49:43.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:42 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:42 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1677812134' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-08T13:49:43.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:42 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:43.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.758+0000 7f11d77fe700 1 -- 172.21.15.57:0/107256836 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7f11d809aca0 con 0x7f11e0106750 2023-10-08T13:49:43.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 -- 172.21.15.57:0/107256836 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a14d0 msgr2=0x7f11cc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:43.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a14d0 0x7f11cc0a3990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009be0 tx=0x7f11dc009260 comp rx=0 tx=0).stop 2023-10-08T13:49:43.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 -- 172.21.15.57:0/107256836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e0106750 msgr2=0x7f11e0073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:43.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e0106750 0x7f11e0073570 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f11d800b070 tx=0x7f11d8008ff0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 -- 172.21.15.57:0/107256836 shutdown_connections 2023-10-08T13:49:43.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e0106750 0x7f11e0073570 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a14d0 0x7f11cc0a3990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0100740 0x7f11e0072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 --2- 172.21.15.57:0/107256836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e0101100 0x7f11e0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 -- 172.21.15.57:0/107256836 >> 172.21.15.57:0/107256836 conn(0x7f11e00fc230 msgr2=0x7f11e00fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:43.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 -- 172.21.15.57:0/107256836 shutdown_connections 2023-10-08T13:49:43.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.761+0000 7f11e85be700 1 -- 172.21.15.57:0/107256836 wait complete. 2023-10-08T13:49:43.770 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2023-10-08T13:49:43.790 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T13:49:43.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.920+0000 7f7792486700 1 -- 172.21.15.57:0/1392910757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f778c0ff640 msgr2=0x7f778c0ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:43.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.920+0000 7f7792486700 1 --2- 172.21.15.57:0/1392910757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f778c0ff640 0x7f778c0ffac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7774009990 tx=0x7f7774009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.920+0000 7f7792486700 1 -- 172.21.15.57:0/1392910757 shutdown_connections 2023-10-08T13:49:43.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.920+0000 7f7792486700 1 --2- 172.21.15.57:0/1392910757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f778c100000 0x7f778c10c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.920+0000 7f7792486700 1 --2- 172.21.15.57:0/1392910757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f778c105650 0x7f778c105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.920+0000 7f7792486700 1 --2- 172.21.15.57:0/1392910757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f778c0ff640 0x7f778c0ffac0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.920+0000 7f7792486700 1 -- 172.21.15.57:0/1392910757 >> 172.21.15.57:0/1392910757 conn(0x7f778c0767c0 msgr2=0x7f778c076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:43.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.921+0000 7f7792486700 1 -- 172.21.15.57:0/1392910757 shutdown_connections 2023-10-08T13:49:43.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.921+0000 7f7792486700 1 -- 172.21.15.57:0/1392910757 wait complete. 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.921+0000 7f7792486700 1 Processor -- start 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.921+0000 7f7792486700 1 -- start start 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.921+0000 7f7792486700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f778c0ff640 0x7f778c1065b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.921+0000 7f7792486700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f778c100000 0x7f778c106af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7792486700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f778c105650 0x7f778c107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7792486700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c1a4f30 con 0x7f778c105650 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7792486700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c1a50a0 con 0x7f778c100000 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7792486700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c1a5260 con 0x7f778c0ff640 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7790a23700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f778c105650 0x7f778c107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f778bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f778c0ff640 0x7f778c1065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7790a23700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f778c105650 0x7f778c107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53930/0 (socket says 172.21.15.57:53930) 2023-10-08T13:49:43.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7790a23700 1 -- 172.21.15.57:0/1666145406 learned_addr learned my addr 172.21.15.57:0/1666145406 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f778b7fe700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f778c100000 0x7f778c106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7790a23700 1 -- 172.21.15.57:0/1666145406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f778c0ff640 msgr2=0x7f778c1065b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7790a23700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f778c0ff640 0x7f778c1065b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7790a23700 1 -- 172.21.15.57:0/1666145406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f778c100000 msgr2=0x7f778c106af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7790a23700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f778c100000 0x7f778c106af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7790a23700 1 -- 172.21.15.57:0/1666145406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740095f0 con 0x7f778c105650 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7790a23700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f778c105650 0x7f778c107030 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f778400ef30 tx=0x7f778400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f77897fa700 1 -- 172.21.15.57:0/1666145406 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784018070 con 0x7f778c105650 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f77897fa700 1 -- 172.21.15.57:0/1666145406 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f778400f040 con 0x7f778c105650 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7792486700 1 -- 172.21.15.57:0/1666145406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778c1a54e0 con 0x7f778c105650 2023-10-08T13:49:43.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f77897fa700 1 -- 172.21.15.57:0/1666145406 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778400cd00 con 0x7f778c105650 2023-10-08T13:49:43.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.922+0000 7f7792486700 1 -- 172.21.15.57:0/1666145406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c1a59c0 con 0x7f778c105650 2023-10-08T13:49:43.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.923+0000 7f7792486700 1 -- 172.21.15.57:0/1666145406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778c100d80 con 0x7f778c105650 2023-10-08T13:49:43.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.925+0000 7f77897fa700 1 -- 172.21.15.57:0/1666145406 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7784003a00 con 0x7f778c105650 2023-10-08T13:49:43.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.925+0000 7f77897fa700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77780a14d0 0x7f77780a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:43.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.925+0000 7f77897fa700 1 -- 172.21.15.57:0/1666145406 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8094+0+0 (secure 0 0 0) 0x7f77840cd140 con 0x7f778c105650 2023-10-08T13:49:43.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.925+0000 7f778bfff700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77780a14d0 0x7f77780a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:43.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.926+0000 7f778bfff700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77780a14d0 0x7f77780a3990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f777c009be0 tx=0x7f777c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:43.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:43.929+0000 7f77897fa700 1 -- 172.21.15.57:0/1666145406 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7784096910 con 0x7f778c105650 2023-10-08T13:49:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:43 smithi099 ceph-mon[103849]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:43 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1677812134' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-08T13:49:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:43 smithi099 ceph-mon[103849]: osdmap e81: 12 total, 12 up, 12 in 2023-10-08T13:49:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:43 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:43 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:43 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:43 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/107256836' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T13:49:44.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:43 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:44.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:44.185+0000 7f7792486700 1 -- 172.21.15.57:0/1666145406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f778c10f560 con 0x7f778c105650 2023-10-08T13:49:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:43 smithi115 ceph-mon[104408]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:43 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1677812134' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-08T13:49:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:43 smithi115 ceph-mon[104408]: osdmap e81: 12 total, 12 up, 12 in 2023-10-08T13:49:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:43 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:43 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:44.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:43 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:44.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:43 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/107256836' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T13:49:44.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:43 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:44.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:43 smithi057 ceph-mon[99476]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:44.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:43 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1677812134' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-08T13:49:44.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:43 smithi057 ceph-mon[99476]: osdmap e81: 12 total, 12 up, 12 in 2023-10-08T13:49:44.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:43 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:44.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:43 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:44.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:43 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:44.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:43 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/107256836' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T13:49:44.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:43 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:49:45.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.112+0000 7f77897fa700 1 -- 172.21.15.57:0/1666145406 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7784099f30 con 0x7f778c105650 2023-10-08T13:49:45.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 -- 172.21.15.57:0/1666145406 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77780a14d0 msgr2=0x7f77780a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:45.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77780a14d0 0x7f77780a3990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f777c009be0 tx=0x7f777c009260 comp rx=0 tx=0).stop 2023-10-08T13:49:45.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 -- 172.21.15.57:0/1666145406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f778c105650 msgr2=0x7f778c107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:45.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f778c105650 0x7f778c107030 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f778400ef30 tx=0x7f778400c3a0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 -- 172.21.15.57:0/1666145406 shutdown_connections 2023-10-08T13:49:45.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f778c105650 0x7f778c107030 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f778c0ff640 0x7f778c1065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77780a14d0 0x7f77780a3990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 --2- 172.21.15.57:0/1666145406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f778c100000 0x7f778c106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 -- 172.21.15.57:0/1666145406 >> 172.21.15.57:0/1666145406 conn(0x7f778c0767c0 msgr2=0x7f778c0fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:45.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 -- 172.21.15.57:0/1666145406 shutdown_connections 2023-10-08T13:49:45.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.115+0000 7f7792486700 1 -- 172.21.15.57:0/1666145406 wait complete. 2023-10-08T13:49:45.117 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 2 and data pool 3 2023-10-08T13:49:45.132 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:44 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/107256836' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-08T13:49:45.132 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:44 smithi057 ceph-mon[99476]: osdmap e82: 12 total, 12 up, 12 in 2023-10-08T13:49:45.132 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:44 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:45.132 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:44 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1666145406' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-08T13:49:45.132 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:44 smithi057 ceph-mon[99476]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-08T13:49:45.133 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-08T13:49:45.133 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-08T13:49:45.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:44 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/107256836' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-08T13:49:45.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:44 smithi115 ceph-mon[104408]: osdmap e82: 12 total, 12 up, 12 in 2023-10-08T13:49:45.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:44 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:45.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:44 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1666145406' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-08T13:49:45.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:44 smithi115 ceph-mon[104408]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-08T13:49:45.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.268+0000 7f1c432a6700 1 -- 172.21.15.57:0/1252496398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c3c106730 msgr2=0x7f1c3c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:45.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.268+0000 7f1c432a6700 1 --2- 172.21.15.57:0/1252496398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c3c106730 0x7f1c3c106b10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009990 tx=0x7f1c38009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.268+0000 7f1c432a6700 1 -- 172.21.15.57:0/1252496398 shutdown_connections 2023-10-08T13:49:45.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.268+0000 7f1c432a6700 1 --2- 172.21.15.57:0/1252496398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c3c1010e0 0x7f1c3c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.268+0000 7f1c432a6700 1 --2- 172.21.15.57:0/1252496398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c3c106730 0x7f1c3c106b10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.268+0000 7f1c432a6700 1 --2- 172.21.15.57:0/1252496398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c100720 0x7f1c3c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.268+0000 7f1c432a6700 1 -- 172.21.15.57:0/1252496398 >> 172.21.15.57:0/1252496398 conn(0x7f1c3c0fc230 msgr2=0x7f1c3c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:45.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.268+0000 7f1c432a6700 1 -- 172.21.15.57:0/1252496398 shutdown_connections 2023-10-08T13:49:45.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.268+0000 7f1c432a6700 1 -- 172.21.15.57:0/1252496398 wait complete. 2023-10-08T13:49:45.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c432a6700 1 Processor -- start 2023-10-08T13:49:45.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c432a6700 1 -- start start 2023-10-08T13:49:45.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c432a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c3c100720 0x7f1c3c109ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:45.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c432a6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c3c1010e0 0x7f1c3c10a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:45.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c432a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c106730 0x7f1c3c1a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c432a6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c1a9390 con 0x7f1c3c1010e0 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c432a6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c1a94d0 con 0x7f1c3c106730 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c432a6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c1a9660 con 0x7f1c3c100720 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c40841700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c3c1010e0 0x7f1c3c10a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c40841700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c3c1010e0 0x7f1c3c10a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53946/0 (socket says 172.21.15.57:53946) 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c40841700 1 -- 172.21.15.57:0/101468673 learned_addr learned my addr 172.21.15.57:0/101468673 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c41042700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c3c100720 0x7f1c3c109ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.269+0000 7f1c41843700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c106730 0x7f1c3c1a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c40841700 1 -- 172.21.15.57:0/101468673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c3c100720 msgr2=0x7f1c3c109ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c40841700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c3c100720 0x7f1c3c109ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c40841700 1 -- 172.21.15.57:0/101468673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c106730 msgr2=0x7f1c3c1a8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:45.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c40841700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c106730 0x7f1c3c1a8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c40841700 1 -- 172.21.15.57:0/101468673 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c380095f0 con 0x7f1c3c1010e0 2023-10-08T13:49:45.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c41843700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c106730 0x7f1c3c1a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:49:45.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c40841700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c3c1010e0 0x7f1c3c10a400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00d950 tx=0x7f1c2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:45.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c327fc700 1 -- 172.21.15.57:0/101468673 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c00f890 con 0x7f1c3c1010e0 2023-10-08T13:49:45.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c432a6700 1 -- 172.21.15.57:0/101468673 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3c1a9940 con 0x7f1c3c1010e0 2023-10-08T13:49:45.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c327fc700 1 -- 172.21.15.57:0/101468673 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c2c010460 con 0x7f1c3c1010e0 2023-10-08T13:49:45.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c327fc700 1 -- 172.21.15.57:0/101468673 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c00e610 con 0x7f1c3c1010e0 2023-10-08T13:49:45.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.270+0000 7f1c432a6700 1 -- 172.21.15.57:0/101468673 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3c1a9e50 con 0x7f1c3c1010e0 2023-10-08T13:49:45.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.271+0000 7f1c432a6700 1 -- 172.21.15.57:0/101468673 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3c101e60 con 0x7f1c3c1010e0 2023-10-08T13:49:45.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:44 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/107256836' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-08T13:49:45.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:44 smithi099 ceph-mon[103849]: osdmap e82: 12 total, 12 up, 12 in 2023-10-08T13:49:45.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:44 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:45.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:44 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1666145406' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-08T13:49:45.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:44 smithi099 ceph-mon[103849]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-08T13:49:45.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.273+0000 7f1c327fc700 1 -- 172.21.15.57:0/101468673 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f1c2c0108d0 con 0x7f1c3c1010e0 2023-10-08T13:49:45.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.274+0000 7f1c327fc700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c280a1550 0x7f1c280a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:45.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.274+0000 7f1c41042700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c280a1550 0x7f1c280a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:45.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.274+0000 7f1c327fc700 1 -- 172.21.15.57:0/101468673 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f1c2c0ccb20 con 0x7f1c3c1010e0 2023-10-08T13:49:45.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.275+0000 7f1c41042700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c280a1550 0x7f1c280a3a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800b5c0 tx=0x7f1c38005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:45.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.277+0000 7f1c327fc700 1 -- 172.21.15.57:0/101468673 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1c2c096140 con 0x7f1c3c1010e0 2023-10-08T13:49:45.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.495+0000 7f1c432a6700 1 -- 172.21.15.57:0/101468673 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f1c3c066ca0 con 0x7f1c3c1010e0 2023-10-08T13:49:45.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.765+0000 7f1c327fc700 1 -- 172.21.15.57:0/101468673 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7f1c2c0160e0 con 0x7f1c3c1010e0 2023-10-08T13:49:45.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 -- 172.21.15.57:0/101468673 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c280a1550 msgr2=0x7f1c280a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:45.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c280a1550 0x7f1c280a3a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800b5c0 tx=0x7f1c38005960 comp rx=0 tx=0).stop 2023-10-08T13:49:45.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 -- 172.21.15.57:0/101468673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c3c1010e0 msgr2=0x7f1c3c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:45.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c3c1010e0 0x7f1c3c10a400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00d950 tx=0x7f1c2c00dc60 comp rx=0 tx=0).stop 2023-10-08T13:49:45.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 -- 172.21.15.57:0/101468673 shutdown_connections 2023-10-08T13:49:45.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c280a1550 0x7f1c280a3a10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c106730 0x7f1c3c1a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c3c100720 0x7f1c3c109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 --2- 172.21.15.57:0/101468673 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c3c1010e0 0x7f1c3c10a400 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 -- 172.21.15.57:0/101468673 >> 172.21.15.57:0/101468673 conn(0x7f1c3c0fc230 msgr2=0x7f1c3c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:45.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 -- 172.21.15.57:0/101468673 shutdown_connections 2023-10-08T13:49:45.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.768+0000 7f1c432a6700 1 -- 172.21.15.57:0/101468673 wait complete. 2023-10-08T13:49:45.782 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T13:49:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.925+0000 7f215b5cd700 1 -- 172.21.15.57:0/1135135712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21540ff570 msgr2=0x7f21540ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:45.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.925+0000 7f215b5cd700 1 --2- 172.21.15.57:0/1135135712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21540ff570 0x7f21540ff9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f21440099c0 tx=0x7f2144009cd0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.925+0000 7f215b5cd700 1 -- 172.21.15.57:0/1135135712 shutdown_connections 2023-10-08T13:49:45.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.925+0000 7f215b5cd700 1 --2- 172.21.15.57:0/1135135712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21540fff30 0x7f215410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.925+0000 7f215b5cd700 1 --2- 172.21.15.57:0/1135135712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2154105630 0x7f2154105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.925+0000 7f215b5cd700 1 --2- 172.21.15.57:0/1135135712 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21540ff570 0x7f21540ff9f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.925+0000 7f215b5cd700 1 -- 172.21.15.57:0/1135135712 >> 172.21.15.57:0/1135135712 conn(0x7f21540fb020 msgr2=0x7f21540fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:45.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.925+0000 7f215b5cd700 1 -- 172.21.15.57:0/1135135712 shutdown_connections 2023-10-08T13:49:45.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.925+0000 7f215b5cd700 1 -- 172.21.15.57:0/1135135712 wait complete. 2023-10-08T13:49:45.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.925+0000 7f215b5cd700 1 Processor -- start 2023-10-08T13:49:45.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f215b5cd700 1 -- start start 2023-10-08T13:49:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f215b5cd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21540ff570 0x7f2154196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f215b5cd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21540fff30 0x7f2154196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f215b5cd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2154105630 0x7f215419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f215b5cd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215419b900 con 0x7f21540ff570 2023-10-08T13:49:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f215b5cd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215419ba70 con 0x7f2154105630 2023-10-08T13:49:45.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f215b5cd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215419bc30 con 0x7f21540fff30 2023-10-08T13:49:45.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2159369700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21540ff570 0x7f2154196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:45.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2159369700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21540ff570 0x7f2154196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53958/0 (socket says 172.21.15.57:53958) 2023-10-08T13:49:45.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2159369700 1 -- 172.21.15.57:0/1289350368 learned_addr learned my addr 172.21.15.57:0/1289350368 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:45.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2158b68700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21540fff30 0x7f2154196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:45.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2159b6a700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2154105630 0x7f215419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:45.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2159369700 1 -- 172.21.15.57:0/1289350368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21540fff30 msgr2=0x7f2154196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:45.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2159369700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21540fff30 0x7f2154196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2159369700 1 -- 172.21.15.57:0/1289350368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2154105630 msgr2=0x7f215419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:45.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2159369700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2154105630 0x7f215419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:45.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2159369700 1 -- 172.21.15.57:0/1289350368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21440095f0 con 0x7f21540ff570 2023-10-08T13:49:45.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.926+0000 7f2159369700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21540ff570 0x7f2154196a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f215000e9a0 tx=0x7f215000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:45.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.927+0000 7f214a7fc700 1 -- 172.21.15.57:0/1289350368 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150009790 con 0x7f21540ff570 2023-10-08T13:49:45.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.927+0000 7f214a7fc700 1 -- 172.21.15.57:0/1289350368 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2150010430 con 0x7f21540ff570 2023-10-08T13:49:45.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.927+0000 7f215b5cd700 1 -- 172.21.15.57:0/1289350368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215419bf10 con 0x7f21540ff570 2023-10-08T13:49:45.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.927+0000 7f214a7fc700 1 -- 172.21.15.57:0/1289350368 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150018680 con 0x7f21540ff570 2023-10-08T13:49:45.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.927+0000 7f215b5cd700 1 -- 172.21.15.57:0/1289350368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21541a37c0 con 0x7f21540ff570 2023-10-08T13:49:45.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.927+0000 7f215b5cd700 1 -- 172.21.15.57:0/1289350368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2154100c90 con 0x7f21540ff570 2023-10-08T13:49:45.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.929+0000 7f214a7fc700 1 -- 172.21.15.57:0/1289350368 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f21500105a0 con 0x7f21540ff570 2023-10-08T13:49:45.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.929+0000 7f214a7fc700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f21400a14c0 0x7f21400a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:45.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.930+0000 7f2158b68700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f21400a14c0 0x7f21400a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:45.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.930+0000 7f214a7fc700 1 -- 172.21.15.57:0/1289350368 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8420+0+0 (secure 0 0 0) 0x7f2150014070 con 0x7f21540ff570 2023-10-08T13:49:45.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.930+0000 7f2158b68700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f21400a14c0 0x7f21400a3980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2144009fd0 tx=0x7f2144019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:45.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:45.933+0000 7f214a7fc700 1 -- 172.21.15.57:0/1289350368 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2150096760 con 0x7f21540ff570 2023-10-08T13:49:46.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:46.148+0000 7f215b5cd700 1 -- 172.21.15.57:0/1289350368 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f21540678a0 con 0x7f21540ff570 2023-10-08T13:49:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:45 smithi115 ceph-mon[104408]: pgmap v235: 97 pgs: 19 creating+peering, 77 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:45 smithi115 ceph-mon[104408]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T13:49:46.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:45 smithi115 ceph-mon[104408]: osdmap e83: 12 total, 12 up, 12 in 2023-10-08T13:49:46.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:45 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1666145406' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-08T13:49:46.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:45 smithi115 ceph-mon[104408]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-08T13:49:46.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:45 smithi115 ceph-mon[104408]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-08T13:49:46.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:45 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/101468673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-08T13:49:46.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:45 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/101468673' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-08T13:49:46.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:45 smithi115 ceph-mon[104408]: osdmap e84: 12 total, 12 up, 12 in 2023-10-08T13:49:46.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:45 smithi057 ceph-mon[99476]: pgmap v235: 97 pgs: 19 creating+peering, 77 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:46.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:45 smithi057 ceph-mon[99476]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T13:49:46.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:45 smithi057 ceph-mon[99476]: osdmap e83: 12 total, 12 up, 12 in 2023-10-08T13:49:46.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:45 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1666145406' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-08T13:49:46.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:45 smithi057 ceph-mon[99476]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-08T13:49:46.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:45 smithi057 ceph-mon[99476]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-08T13:49:46.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:45 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/101468673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-08T13:49:46.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:45 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/101468673' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-08T13:49:46.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:45 smithi057 ceph-mon[99476]: osdmap e84: 12 total, 12 up, 12 in 2023-10-08T13:49:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:45 smithi099 ceph-mon[103849]: pgmap v235: 97 pgs: 19 creating+peering, 77 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T13:49:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:45 smithi099 ceph-mon[103849]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T13:49:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:45 smithi099 ceph-mon[103849]: osdmap e83: 12 total, 12 up, 12 in 2023-10-08T13:49:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:45 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1666145406' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-08T13:49:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:45 smithi099 ceph-mon[103849]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-08T13:49:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:45 smithi099 ceph-mon[103849]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-08T13:49:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:45 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/101468673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-08T13:49:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:45 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/101468673' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-08T13:49:46.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:45 smithi099 ceph-mon[103849]: osdmap e84: 12 total, 12 up, 12 in 2023-10-08T13:49:47.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: pgmap v238: 97 pgs: 29 creating+peering, 39 unknown, 29 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-08T13:49:47.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] up:active 2023-10-08T13:49:47.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:49:47.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:49:47.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1289350368' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-08T13:49:47.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:47.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:47.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-08T13:49:47.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:47 smithi115 ceph-mon[104408]: Cluster is now healthy 2023-10-08T13:49:47.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: pgmap v238: 97 pgs: 29 creating+peering, 39 unknown, 29 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-08T13:49:47.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] up:active 2023-10-08T13:49:47.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1289350368' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-08T13:49:47.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:47 smithi057 ceph-mon[99476]: Cluster is now healthy 2023-10-08T13:49:47.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: pgmap v238: 97 pgs: 29 creating+peering, 39 unknown, 29 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-08T13:49:47.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] up:active 2023-10-08T13:49:47.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:49:47.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:49:47.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1289350368' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-08T13:49:47.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:49:47.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:49:47.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:49:47.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-08T13:49:47.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:47 smithi099 ceph-mon[103849]: Cluster is now healthy 2023-10-08T13:49:48.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.135+0000 7f214a7fc700 1 -- 172.21.15.57:0/1289350368 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7f2150026100 con 0x7f21540ff570 2023-10-08T13:49:48.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.139+0000 7f215b5cd700 1 -- 172.21.15.57:0/1289350368 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f21400a14c0 msgr2=0x7f21400a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:48.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.139+0000 7f215b5cd700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f21400a14c0 0x7f21400a3980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2144009fd0 tx=0x7f2144019040 comp rx=0 tx=0).stop 2023-10-08T13:49:48.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.139+0000 7f215b5cd700 1 -- 172.21.15.57:0/1289350368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21540ff570 msgr2=0x7f2154196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:48.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.139+0000 7f215b5cd700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21540ff570 0x7f2154196a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f215000e9a0 tx=0x7f215000ecb0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.139+0000 7f215b5cd700 1 -- 172.21.15.57:0/1289350368 shutdown_connections 2023-10-08T13:49:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.139+0000 7f215b5cd700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f21400a14c0 0x7f21400a3980 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.139+0000 7f215b5cd700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2154105630 0x7f215419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.139+0000 7f215b5cd700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f21540ff570 0x7f2154196a20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.139+0000 7f215b5cd700 1 --2- 172.21.15.57:0/1289350368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21540fff30 0x7f2154196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.139+0000 7f215b5cd700 1 -- 172.21.15.57:0/1289350368 >> 172.21.15.57:0/1289350368 conn(0x7f21540fb020 msgr2=0x7f21540fb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.140+0000 7f215b5cd700 1 -- 172.21.15.57:0/1289350368 shutdown_connections 2023-10-08T13:49:48.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.140+0000 7f215b5cd700 1 -- 172.21.15.57:0/1289350368 wait complete. 2023-10-08T13:49:48.145 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data_ec' created 2023-10-08T13:49:48.163 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-08T13:49:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.297+0000 7fb9cac7d700 1 -- 172.21.15.57:0/418269116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c40ff620 msgr2=0x7fb9c40ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.297+0000 7fb9cac7d700 1 --2- 172.21.15.57:0/418269116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c40ff620 0x7fb9c40ffaa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009990 tx=0x7fb9b0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.297+0000 7fb9cac7d700 1 -- 172.21.15.57:0/418269116 shutdown_connections 2023-10-08T13:49:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.297+0000 7fb9cac7d700 1 --2- 172.21.15.57:0/418269116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c40fffe0 0x7fb9c410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.297+0000 7fb9cac7d700 1 --2- 172.21.15.57:0/418269116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c41056b0 0x7fb9c4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.297+0000 7fb9cac7d700 1 --2- 172.21.15.57:0/418269116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c40ff620 0x7fb9c40ffaa0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.297+0000 7fb9cac7d700 1 -- 172.21.15.57:0/418269116 >> 172.21.15.57:0/418269116 conn(0x7fb9c4076780 msgr2=0x7fb9c4076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.297+0000 7fb9cac7d700 1 -- 172.21.15.57:0/418269116 shutdown_connections 2023-10-08T13:49:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.297+0000 7fb9cac7d700 1 -- 172.21.15.57:0/418269116 wait complete. 2023-10-08T13:49:48.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.298+0000 7fb9cac7d700 1 Processor -- start 2023-10-08T13:49:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.298+0000 7fb9cac7d700 1 -- start start 2023-10-08T13:49:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9cac7d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c40ff620 0x7fb9c4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9cac7d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c40fffe0 0x7fb9c4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9cac7d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c41056b0 0x7fb9c41a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9cac7d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4198700 con 0x7fb9c40ff620 2023-10-08T13:49:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9cac7d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c41a2f10 con 0x7fb9c40fffe0 2023-10-08T13:49:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9cac7d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c41a3050 con 0x7fb9c41056b0 2023-10-08T13:49:48.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c8a19700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c40ff620 0x7fb9c4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c8a19700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c40ff620 0x7fb9c4197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53984/0 (socket says 172.21.15.57:53984) 2023-10-08T13:49:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c8a19700 1 -- 172.21.15.57:0/3491335525 learned_addr learned my addr 172.21.15.57:0/3491335525 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c3fff700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c40fffe0 0x7fb9c4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c921a700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c41056b0 0x7fb9c41a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c8a19700 1 -- 172.21.15.57:0/3491335525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c41056b0 msgr2=0x7fb9c41a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:48.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c8a19700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c41056b0 0x7fb9c41a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c8a19700 1 -- 172.21.15.57:0/3491335525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c40fffe0 msgr2=0x7fb9c4197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c8a19700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c40fffe0 0x7fb9c4197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c8a19700 1 -- 172.21.15.57:0/3491335525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00095f0 con 0x7fb9c40ff620 2023-10-08T13:49:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c921a700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c41056b0 0x7fb9c41a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c8a19700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c40ff620 0x7fb9c4197950 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800d980 tx=0x7fb9b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c1ffb700 1 -- 172.21.15.57:0/3491335525 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800b950 con 0x7fb9c40ff620 2023-10-08T13:49:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c1ffb700 1 -- 172.21.15.57:0/3491335525 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b8007620 con 0x7fb9c40ff620 2023-10-08T13:49:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9cac7d700 1 -- 172.21.15.57:0/3491335525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c41a32e0 con 0x7fb9c40ff620 2023-10-08T13:49:48.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.299+0000 7fb9c1ffb700 1 -- 172.21.15.57:0/3491335525 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800f3c0 con 0x7fb9c40ff620 2023-10-08T13:49:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.300+0000 7fb9cac7d700 1 -- 172.21.15.57:0/3491335525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c41a3810 con 0x7fb9c40ff620 2023-10-08T13:49:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.300+0000 7fb9cac7d700 1 -- 172.21.15.57:0/3491335525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c4107c30 con 0x7fb9c40ff620 2023-10-08T13:49:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.302+0000 7fb9c1ffb700 1 -- 172.21.15.57:0/3491335525 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb9b8010640 con 0x7fb9c40ff620 2023-10-08T13:49:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.302+0000 7fb9c1ffb700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9b40a1550 0x7fb9b40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.303+0000 7fb9c3fff700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9b40a1550 0x7fb9b40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.303+0000 7fb9c1ffb700 1 -- 172.21.15.57:0/3491335525 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8913+0+0 (secure 0 0 0) 0x7fb9b80cd690 con 0x7fb9c40ff620 2023-10-08T13:49:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.303+0000 7fb9c3fff700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9b40a1550 0x7fb9b40a3a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000b5c0 tx=0x7fb9b0005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:48.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.307+0000 7fb9c1ffb700 1 -- 172.21.15.57:0/3491335525 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb9b8096a40 con 0x7fb9c40ff620 2023-10-08T13:49:48.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:48 smithi115 ceph-mon[104408]: osdmap e85: 12 total, 12 up, 12 in 2023-10-08T13:49:48.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:48 smithi057 ceph-mon[99476]: osdmap e85: 12 total, 12 up, 12 in 2023-10-08T13:49:48.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:48.510+0000 7fb9cac7d700 1 -- 172.21.15.57:0/3491335525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fb9c4067df0 con 0x7fb9c40ff620 2023-10-08T13:49:48.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:48 smithi099 ceph-mon[103849]: osdmap e85: 12 total, 12 up, 12 in 2023-10-08T13:49:49.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.137+0000 7fb9c1ffb700 1 -- 172.21.15.57:0/3491335525 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb9b80160c0 con 0x7fb9c40ff620 2023-10-08T13:49:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.141+0000 7fb9cac7d700 1 -- 172.21.15.57:0/3491335525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9b40a1550 msgr2=0x7fb9b40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.141+0000 7fb9cac7d700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9b40a1550 0x7fb9b40a3a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000b5c0 tx=0x7fb9b0005960 comp rx=0 tx=0).stop 2023-10-08T13:49:49.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.141+0000 7fb9cac7d700 1 -- 172.21.15.57:0/3491335525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c40ff620 msgr2=0x7fb9c4197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:49.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.141+0000 7fb9cac7d700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c40ff620 0x7fb9c4197950 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800d980 tx=0x7fb9b800dc90 comp rx=0 tx=0).stop 2023-10-08T13:49:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.143+0000 7fb9cac7d700 1 -- 172.21.15.57:0/3491335525 shutdown_connections 2023-10-08T13:49:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.143+0000 7fb9cac7d700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9c41056b0 0x7fb9c41a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.143+0000 7fb9cac7d700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9c40ff620 0x7fb9c4197950 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.143+0000 7fb9cac7d700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9b40a1550 0x7fb9b40a3a10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.143+0000 7fb9cac7d700 1 --2- 172.21.15.57:0/3491335525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c40fffe0 0x7fb9c4197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.143+0000 7fb9cac7d700 1 -- 172.21.15.57:0/3491335525 >> 172.21.15.57:0/3491335525 conn(0x7fb9c4076780 msgr2=0x7fb9c40fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:49.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.146+0000 7fb9cac7d700 1 -- 172.21.15.57:0/3491335525 shutdown_connections 2023-10-08T13:49:49.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.149+0000 7fb9cac7d700 1 -- 172.21.15.57:0/3491335525 wait complete. 2023-10-08T13:49:49.158 INFO:teuthology.orchestra.run.smithi057.stderr:set pool 4 allow_ec_overwrites to true 2023-10-08T13:49:49.177 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-08T13:49:49.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.310+0000 7faecb518700 1 -- 172.21.15.57:0/3809593157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faec4105630 msgr2=0x7faec4105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:49.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.310+0000 7faecb518700 1 --2- 172.21.15.57:0/3809593157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faec4105630 0x7faec4105a10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faec0009990 tx=0x7faec0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:49.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.310+0000 7faecb518700 1 -- 172.21.15.57:0/3809593157 shutdown_connections 2023-10-08T13:49:49.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.310+0000 7faecb518700 1 --2- 172.21.15.57:0/3809593157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faec4100000 0x7faec410c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:49.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.310+0000 7faecb518700 1 --2- 172.21.15.57:0/3809593157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faec4105630 0x7faec4105a10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:49.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.310+0000 7faecb518700 1 --2- 172.21.15.57:0/3809593157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faec40ff640 0x7faec40ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:49.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.310+0000 7faecb518700 1 -- 172.21.15.57:0/3809593157 >> 172.21.15.57:0/3809593157 conn(0x7faec4076790 msgr2=0x7faec4076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:49.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.310+0000 7faecb518700 1 -- 172.21.15.57:0/3809593157 shutdown_connections 2023-10-08T13:49:49.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.310+0000 7faecb518700 1 -- 172.21.15.57:0/3809593157 wait complete. 2023-10-08T13:49:49.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.310+0000 7faecb518700 1 Processor -- start 2023-10-08T13:49:49.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faecb518700 1 -- start start 2023-10-08T13:49:49.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faecb518700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faec40ff640 0x7faec4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:49.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faecb518700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faec4100000 0x7faec4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faecb518700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faec4105630 0x7faec41a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faecb518700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec4198730 con 0x7faec4105630 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faecb518700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec41a2ef0 con 0x7faec40ff640 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faecb518700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec41a3030 con 0x7faec4100000 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec9ab5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faec4105630 0x7faec41a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec9ab5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faec4105630 0x7faec41a29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54002/0 (socket says 172.21.15.57:54002) 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec9ab5700 1 -- 172.21.15.57:0/678106956 learned_addr learned my addr 172.21.15.57:0/678106956 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec8ab3700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faec4100000 0x7faec4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec92b4700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faec40ff640 0x7faec4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec9ab5700 1 -- 172.21.15.57:0/678106956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faec4100000 msgr2=0x7faec4197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:49.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec9ab5700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faec4100000 0x7faec4197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:49.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec9ab5700 1 -- 172.21.15.57:0/678106956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faec40ff640 msgr2=0x7faec4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:49.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec9ab5700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faec40ff640 0x7faec4197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:49.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec9ab5700 1 -- 172.21.15.57:0/678106956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec00095f0 con 0x7faec4105630 2023-10-08T13:49:49.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faec9ab5700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faec4105630 0x7faec41a29b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faebc007ee0 tx=0x7faebc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:49.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.311+0000 7faeba7fc700 1 -- 172.21.15.57:0/678106956 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc017070 con 0x7faec4105630 2023-10-08T13:49:49.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.312+0000 7faeba7fc700 1 -- 172.21.15.57:0/678106956 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faebc00f040 con 0x7faec4105630 2023-10-08T13:49:49.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.312+0000 7faecb518700 1 -- 172.21.15.57:0/678106956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec41a32b0 con 0x7faec4105630 2023-10-08T13:49:49.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.312+0000 7faeba7fc700 1 -- 172.21.15.57:0/678106956 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc005580 con 0x7faec4105630 2023-10-08T13:49:49.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.312+0000 7faecb518700 1 -- 172.21.15.57:0/678106956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec41a3810 con 0x7faec4105630 2023-10-08T13:49:49.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.312+0000 7faecb518700 1 -- 172.21.15.57:0/678106956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec4107c20 con 0x7faec4105630 2023-10-08T13:49:49.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.314+0000 7faeba7fc700 1 -- 172.21.15.57:0/678106956 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7faebc00db20 con 0x7faec4105630 2023-10-08T13:49:49.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.314+0000 7faeba7fc700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faeb00a1550 0x7faeb00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:49.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.314+0000 7faec92b4700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faeb00a1550 0x7faeb00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:49.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.314+0000 7faeba7fc700 1 -- 172.21.15.57:0/678106956 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8913+0+0 (secure 0 0 0) 0x7faebc0ccca0 con 0x7faec4105630 2023-10-08T13:49:49.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.315+0000 7faec92b4700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faeb00a1550 0x7faeb00a3a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faec000b5c0 tx=0x7faec0005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:49.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.318+0000 7faeba7fc700 1 -- 172.21.15.57:0/678106956 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faebc096050 con 0x7faec4105630 2023-10-08T13:49:49.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:49 smithi115 ceph-mon[104408]: pgmap v240: 97 pgs: 26 creating+peering, 6 unknown, 65 active+clean; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 708 B/s wr, 2 op/s 2023-10-08T13:49:49.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:49 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1289350368' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-08T13:49:49.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:49 smithi115 ceph-mon[104408]: osdmap e86: 12 total, 12 up, 12 in 2023-10-08T13:49:49.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:49 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3491335525' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-08T13:49:49.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:49 smithi057 ceph-mon[99476]: pgmap v240: 97 pgs: 26 creating+peering, 6 unknown, 65 active+clean; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 708 B/s wr, 2 op/s 2023-10-08T13:49:49.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:49 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1289350368' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-08T13:49:49.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:49 smithi057 ceph-mon[99476]: osdmap e86: 12 total, 12 up, 12 in 2023-10-08T13:49:49.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:49 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3491335525' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-08T13:49:49.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:49 smithi099 ceph-mon[103849]: pgmap v240: 97 pgs: 26 creating+peering, 6 unknown, 65 active+clean; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 708 B/s wr, 2 op/s 2023-10-08T13:49:49.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:49 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1289350368' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-08T13:49:49.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:49 smithi099 ceph-mon[103849]: osdmap e86: 12 total, 12 up, 12 in 2023-10-08T13:49:49.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:49 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3491335525' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-08T13:49:49.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:49.573+0000 7faecb518700 1 -- 172.21.15.57:0/678106956 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7faec4100d60 con 0x7faec4105630 2023-10-08T13:49:50.022 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 13:49:49 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b[140884]: 2023-10-08T13:49:49.743+0000 7fe27b630700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T13:49:50.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.158+0000 7faeba7fc700 1 -- 172.21.15.57:0/678106956 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7faebc099670 con 0x7faec4105630 2023-10-08T13:49:50.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 -- 172.21.15.57:0/678106956 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faeb00a1550 msgr2=0x7faeb00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faeb00a1550 0x7faeb00a3a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faec000b5c0 tx=0x7faec0005960 comp rx=0 tx=0).stop 2023-10-08T13:49:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 -- 172.21.15.57:0/678106956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faec4105630 msgr2=0x7faec41a29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faec4105630 0x7faec41a29b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faebc007ee0 tx=0x7faebc00d3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 -- 172.21.15.57:0/678106956 shutdown_connections 2023-10-08T13:49:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faec4105630 0x7faec41a29b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faeb00a1550 0x7faeb00a3a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faec40ff640 0x7faec4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 --2- 172.21.15.57:0/678106956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faec4100000 0x7faec4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 -- 172.21.15.57:0/678106956 >> 172.21.15.57:0/678106956 conn(0x7faec4076790 msgr2=0x7faec40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 -- 172.21.15.57:0/678106956 shutdown_connections 2023-10-08T13:49:50.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.162+0000 7faecb518700 1 -- 172.21.15.57:0/678106956 wait complete. 2023-10-08T13:49:50.165 INFO:teuthology.orchestra.run.smithi057.stderr:added data pool 4 to fsmap 2023-10-08T13:49:50.181 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:50.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.313+0000 7f196ed01700 1 -- 172.21.15.57:0/480837801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19680fff00 msgr2=0x7f196810c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.313+0000 7f196ed01700 1 --2- 172.21.15.57:0/480837801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19680fff00 0x7f196810c3c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f196000b0d0 tx=0x7f196000b3e0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.313+0000 7f196ed01700 1 -- 172.21.15.57:0/480837801 shutdown_connections 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.313+0000 7f196ed01700 1 --2- 172.21.15.57:0/480837801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19680fff00 0x7f196810c3c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.313+0000 7f196ed01700 1 --2- 172.21.15.57:0/480837801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1968105600 0x7f19681059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.313+0000 7f196ed01700 1 --2- 172.21.15.57:0/480837801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19680ff540 0x7f19680ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.313+0000 7f196ed01700 1 -- 172.21.15.57:0/480837801 >> 172.21.15.57:0/480837801 conn(0x7f19680fb030 msgr2=0x7f19680fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.313+0000 7f196ed01700 1 -- 172.21.15.57:0/480837801 shutdown_connections 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.313+0000 7f196ed01700 1 -- 172.21.15.57:0/480837801 wait complete. 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f196ed01700 1 Processor -- start 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f196ed01700 1 -- start start 2023-10-08T13:49:50.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f196ed01700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19680ff540 0x7f1968195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:50.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f196ed01700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19680fff00 0x7f1968195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:50.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f196ed01700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1968105600 0x7f196819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:50.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f196ed01700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196819a910 con 0x7f19680fff00 2023-10-08T13:49:50.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f196ed01700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196819aa80 con 0x7f19680ff540 2023-10-08T13:49:50.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f196ed01700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196819ac40 con 0x7f1968105600 2023-10-08T13:49:50.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f1967fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19680fff00 0x7f1968195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:50.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f1967fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19680fff00 0x7f1968195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54014/0 (socket says 172.21.15.57:54014) 2023-10-08T13:49:50.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f1967fff700 1 -- 172.21.15.57:0/1637279847 learned_addr learned my addr 172.21.15.57:0/1637279847 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:50.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f196ca9d700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19680ff540 0x7f1968195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f196d29e700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1968105600 0x7f196819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f1967fff700 1 -- 172.21.15.57:0/1637279847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1968105600 msgr2=0x7f196819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f1967fff700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1968105600 0x7f196819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f1967fff700 1 -- 172.21.15.57:0/1637279847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19680ff540 msgr2=0x7f1968195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f1967fff700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19680ff540 0x7f1968195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.314+0000 7f1967fff700 1 -- 172.21.15.57:0/1637279847 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960009d00 con 0x7f19680fff00 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.315+0000 7f1967fff700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19680fff00 0x7f1968195fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f195400d980 tx=0x7f195400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.315+0000 7f1965ffb700 1 -- 172.21.15.57:0/1637279847 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954004220 con 0x7f19680fff00 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.315+0000 7f1965ffb700 1 -- 172.21.15.57:0/1637279847 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1954007640 con 0x7f19680fff00 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.315+0000 7f196ed01700 1 -- 172.21.15.57:0/1637279847 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196819af20 con 0x7f19680fff00 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.315+0000 7f1965ffb700 1 -- 172.21.15.57:0/1637279847 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954005610 con 0x7f19680fff00 2023-10-08T13:49:50.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.315+0000 7f196ed01700 1 -- 172.21.15.57:0/1637279847 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19681a27d0 con 0x7f19680fff00 2023-10-08T13:49:50.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.316+0000 7f196ed01700 1 -- 172.21.15.57:0/1637279847 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1968107aa0 con 0x7f19680fff00 2023-10-08T13:49:50.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.317+0000 7f1965ffb700 1 -- 172.21.15.57:0/1637279847 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f1954004750 con 0x7f19680fff00 2023-10-08T13:49:50.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.318+0000 7f1965ffb700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19580a14d0 0x7f19580a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:50.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.318+0000 7f196ca9d700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19580a14d0 0x7f19580a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:50.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.318+0000 7f1965ffb700 1 -- 172.21.15.57:0/1637279847 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f19540ccef0 con 0x7f19680fff00 2023-10-08T13:49:50.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.318+0000 7f196ca9d700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19580a14d0 0x7f19580a3990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f195c009be0 tx=0x7f195c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:50.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.322+0000 7f1965ffb700 1 -- 172.21.15.57:0/1637279847 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1954096280 con 0x7f19680fff00 2023-10-08T13:49:50.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:50 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3491335525' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-08T13:49:50.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:50 smithi115 ceph-mon[104408]: osdmap e87: 12 total, 12 up, 12 in 2023-10-08T13:49:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:50 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/678106956' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-08T13:49:50.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:50 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3491335525' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-08T13:49:50.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:50 smithi057 ceph-mon[99476]: osdmap e87: 12 total, 12 up, 12 in 2023-10-08T13:49:50.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:50 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/678106956' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-08T13:49:50.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:50 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3491335525' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-08T13:49:50.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:50 smithi099 ceph-mon[103849]: osdmap e87: 12 total, 12 up, 12 in 2023-10-08T13:49:50.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:50 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/678106956' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-08T13:49:50.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.571+0000 7f196ed01700 1 -- 172.21.15.57:0/1637279847 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1968196d30 con 0x7f19680fff00 2023-10-08T13:49:50.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.571+0000 7f1965ffb700 1 -- 172.21.15.57:0/1637279847 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11249 (secure 0 0 0) 0x7f1954016020 con 0x7f19680fff00 2023-10-08T13:49:50.572 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:50.573 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14790,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24581,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24617,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24652,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:49:49.574760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24662},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-08T13:49:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.573+0000 7f196ed01700 1 -- 172.21.15.57:0/1637279847 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19580a14d0 msgr2=0x7f19580a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.573+0000 7f196ed01700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19580a14d0 0x7f19580a3990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f195c009be0 tx=0x7f195c009260 comp rx=0 tx=0).stop 2023-10-08T13:49:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.573+0000 7f196ed01700 1 -- 172.21.15.57:0/1637279847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19680fff00 msgr2=0x7f1968195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.573+0000 7f196ed01700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19680fff00 0x7f1968195fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f195400d980 tx=0x7f195400dc90 comp rx=0 tx=0).stop 2023-10-08T13:49:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.573+0000 7f196ed01700 1 -- 172.21.15.57:0/1637279847 shutdown_connections 2023-10-08T13:49:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.573+0000 7f196ed01700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1968105600 0x7f196819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.573+0000 7f196ed01700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19580a14d0 0x7f19580a3990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.573+0000 7f196ed01700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19680ff540 0x7f1968195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.573+0000 7f196ed01700 1 --2- 172.21.15.57:0/1637279847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19680fff00 0x7f1968195fc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.574+0000 7f196ed01700 1 -- 172.21.15.57:0/1637279847 >> 172.21.15.57:0/1637279847 conn(0x7f19680fb030 msgr2=0x7f19680fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.574+0000 7f196ed01700 1 -- 172.21.15.57:0/1637279847 shutdown_connections 2023-10-08T13:49:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.574+0000 7f196ed01700 1 -- 172.21.15.57:0/1637279847 wait complete. 2023-10-08T13:49:50.576 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 17 2023-10-08T13:49:50.587 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:49:50.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.711+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/4290697968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a141021e0 msgr2=0x7f6a1410a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.711+0000 7f6a1a1cd700 1 --2- 172.21.15.57:0/4290697968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a141021e0 0x7f6a1410a6c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009990 tx=0x7f6a08009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.711+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/4290697968 shutdown_connections 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.711+0000 7f6a1a1cd700 1 --2- 172.21.15.57:0/4290697968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a1410ac90 0x7f6a1410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.711+0000 7f6a1a1cd700 1 --2- 172.21.15.57:0/4290697968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a141018c0 0x7f6a14101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.711+0000 7f6a1a1cd700 1 --2- 172.21.15.57:0/4290697968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a141021e0 0x7f6a1410a6c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.711+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/4290697968 >> 172.21.15.57:0/4290697968 conn(0x7f6a140fb060 msgr2=0x7f6a140fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.711+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/4290697968 shutdown_connections 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.712+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/4290697968 wait complete. 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.712+0000 7f6a1a1cd700 1 Processor -- start 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a1a1cd700 1 -- start start 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a1a1cd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a141018c0 0x7f6a14198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a1a1cd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a141021e0 0x7f6a14199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a1a1cd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a1410ac90 0x7f6a1419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a1a1cd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1419cb30 con 0x7f6a141018c0 2023-10-08T13:49:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a1a1cd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1419cca0 con 0x7f6a1410ac90 2023-10-08T13:49:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a1a1cd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1419ce60 con 0x7f6a141021e0 2023-10-08T13:49:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a137fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a141018c0 0x7f6a14198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a137fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a141018c0 0x7f6a14198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54026/0 (socket says 172.21.15.57:54026) 2023-10-08T13:49:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a137fe700 1 -- 172.21.15.57:0/3129081805 learned_addr learned my addr 172.21.15.57:0/3129081805 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a13fff700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a1410ac90 0x7f6a1419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a12ffd700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a141021e0 0x7f6a14199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a137fe700 1 -- 172.21.15.57:0/3129081805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a141021e0 msgr2=0x7f6a14199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a137fe700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a141021e0 0x7f6a14199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a137fe700 1 -- 172.21.15.57:0/3129081805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a1410ac90 msgr2=0x7f6a1419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a137fe700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a1410ac90 0x7f6a1419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a137fe700 1 -- 172.21.15.57:0/3129081805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a080095f0 con 0x7f6a141018c0 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a12ffd700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a141021e0 0x7f6a14199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.713+0000 7f6a137fe700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a141018c0 0x7f6a14198d00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400d980 tx=0x7f6a0400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.714+0000 7f6a10ff9700 1 -- 172.21.15.57:0/3129081805 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400bb60 con 0x7f6a141018c0 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.714+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/3129081805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1419d140 con 0x7f6a141018c0 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.714+0000 7f6a10ff9700 1 -- 172.21.15.57:0/3129081805 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a04007640 con 0x7f6a141018c0 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.714+0000 7f6a10ff9700 1 -- 172.21.15.57:0/3129081805 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04004e90 con 0x7f6a141018c0 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.714+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/3129081805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a141a5aa0 con 0x7f6a141018c0 2023-10-08T13:49:50.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.715+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/3129081805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a140692b0 con 0x7f6a141018c0 2023-10-08T13:49:50.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.716+0000 7f6a10ff9700 1 -- 172.21.15.57:0/3129081805 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6a0400bcc0 con 0x7f6a141018c0 2023-10-08T13:49:50.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.716+0000 7f6a10ff9700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69fc0a1520 0x7f69fc0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:50.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.716+0000 7f6a12ffd700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69fc0a1520 0x7f69fc0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:50.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.716+0000 7f6a10ff9700 1 -- 172.21.15.57:0/3129081805 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f6a040cd1c0 con 0x7f6a141018c0 2023-10-08T13:49:50.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.717+0000 7f6a12ffd700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69fc0a1520 0x7f69fc0a39e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a080095c0 tx=0x7f6a08009510 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:50.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.720+0000 7f6a10ff9700 1 -- 172.21.15.57:0/3129081805 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6a04099fe0 con 0x7f6a141018c0 2023-10-08T13:49:50.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.923+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/3129081805 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a14066bc0 con 0x7f6a141018c0 2023-10-08T13:49:50.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.924+0000 7f6a10ff9700 1 -- 172.21.15.57:0/3129081805 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24224 (secure 0 0 0) 0x7f6a04016090 con 0x7f6a141018c0 2023-10-08T13:49:50.926 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:50.926 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":88,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:49.574582+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:49:50.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/3129081805 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69fc0a1520 msgr2=0x7f69fc0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69fc0a1520 0x7f69fc0a39e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a080095c0 tx=0x7f6a08009510 comp rx=0 tx=0).stop 2023-10-08T13:49:50.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/3129081805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a141018c0 msgr2=0x7f6a14198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:50.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a141018c0 0x7f6a14198d00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400d980 tx=0x7f6a0400dd40 comp rx=0 tx=0).stop 2023-10-08T13:49:50.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/3129081805 shutdown_connections 2023-10-08T13:49:50.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f69fc0a1520 0x7f69fc0a39e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a1410ac90 0x7f6a1419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a141018c0 0x7f6a14198d00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 --2- 172.21.15.57:0/3129081805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a141021e0 0x7f6a14199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:50.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/3129081805 >> 172.21.15.57:0/3129081805 conn(0x7f6a140fb060 msgr2=0x7f6a140ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:50.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/3129081805 shutdown_connections 2023-10-08T13:49:50.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:50.927+0000 7f6a1a1cd700 1 -- 172.21.15.57:0/3129081805 wait complete. 2023-10-08T13:49:50.942 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:49:51.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 -- 172.21.15.57:0/3260568517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5ac0fff00 msgr2=0x7fd5ac10c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 --2- 172.21.15.57:0/3260568517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5ac0fff00 0x7fd5ac10c3c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400b0d0 tx=0x7fd5a400b3e0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 -- 172.21.15.57:0/3260568517 shutdown_connections 2023-10-08T13:49:51.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 --2- 172.21.15.57:0/3260568517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5ac0fff00 0x7fd5ac10c3c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 --2- 172.21.15.57:0/3260568517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac105600 0x7fd5ac1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 --2- 172.21.15.57:0/3260568517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5ac0ff540 0x7fd5ac0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 -- 172.21.15.57:0/3260568517 >> 172.21.15.57:0/3260568517 conn(0x7fd5ac0fb030 msgr2=0x7fd5ac0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 -- 172.21.15.57:0/3260568517 shutdown_connections 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 -- 172.21.15.57:0/3260568517 wait complete. 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 Processor -- start 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 -- start start 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5ac0ff540 0x7fd5ac197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.068+0000 7fd5b452f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5ac0fff00 0x7fd5ac197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b452f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac105600 0x7fd5ac1a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b452f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac198780 con 0x7fd5ac0ff540 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b452f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1a2fd0 con 0x7fd5ac105600 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b452f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1a3110 con 0x7fd5ac0fff00 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b22cb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5ac0ff540 0x7fd5ac197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b22cb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5ac0ff540 0x7fd5ac197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54038/0 (socket says 172.21.15.57:54038) 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b22cb700 1 -- 172.21.15.57:0/1058036946 learned_addr learned my addr 172.21.15.57:0/1058036946 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b2acc700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac105600 0x7fd5ac1a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b1aca700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5ac0fff00 0x7fd5ac197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b22cb700 1 -- 172.21.15.57:0/1058036946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5ac0fff00 msgr2=0x7fd5ac197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b22cb700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5ac0fff00 0x7fd5ac197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b22cb700 1 -- 172.21.15.57:0/1058036946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac105600 msgr2=0x7fd5ac1a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b22cb700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac105600 0x7fd5ac1a2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b22cb700 1 -- 172.21.15.57:0/1058036946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a4009d00 con 0x7fd5ac0ff540 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b22cb700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5ac0ff540 0x7fd5ac197a10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800d980 tx=0x7fd5a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5a37fe700 1 -- 172.21.15.57:0/1058036946 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8004220 con 0x7fd5ac0ff540 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b452f700 1 -- 172.21.15.57:0/1058036946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac1a3370 con 0x7fd5ac0ff540 2023-10-08T13:49:51.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5a37fe700 1 -- 172.21.15.57:0/1058036946 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5a8007640 con 0x7fd5ac0ff540 2023-10-08T13:49:51.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5a37fe700 1 -- 172.21.15.57:0/1058036946 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8005610 con 0x7fd5ac0ff540 2023-10-08T13:49:51.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.069+0000 7fd5b452f700 1 -- 172.21.15.57:0/1058036946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac1a3850 con 0x7fd5ac0ff540 2023-10-08T13:49:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.070+0000 7fd5b452f700 1 -- 172.21.15.57:0/1058036946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac107aa0 con 0x7fd5ac0ff540 2023-10-08T13:49:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.071+0000 7fd5a37fe700 1 -- 172.21.15.57:0/1058036946 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd5a8004750 con 0x7fd5ac0ff540 2023-10-08T13:49:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.072+0000 7fd5a37fe700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd5980a14d0 0x7fd5980a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.072+0000 7fd5b1aca700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd5980a14d0 0x7fd5980a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.073+0000 7fd5a37fe700 1 -- 172.21.15.57:0/1058036946 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fd5a80cd2c0 con 0x7fd5ac0ff540 2023-10-08T13:49:51.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.073+0000 7fd5b1aca700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd5980a14d0 0x7fd5980a3990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac067ff0 tx=0x7fd59c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:51.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.076+0000 7fd5a37fe700 1 -- 172.21.15.57:0/1058036946 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd5a80966f0 con 0x7fd5ac0ff540 2023-10-08T13:49:51.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:51 smithi057 ceph-mon[99476]: pgmap v243: 129 pgs: 11 creating+peering, 8 unknown, 110 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2023-10-08T13:49:51.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:51 smithi057 ceph-mon[99476]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T13:49:51.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:51 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/678106956' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-08T13:49:51.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:51 smithi057 ceph-mon[99476]: osdmap e88: 12 total, 12 up, 12 in 2023-10-08T13:49:51.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:51 smithi057 ceph-mon[99476]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:49:51.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:51 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1637279847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:51.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:51 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3129081805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:51.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.282+0000 7fd5b452f700 1 -- 172.21.15.57:0/1058036946 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5ac067dd0 con 0x7fd5ac0ff540 2023-10-08T13:49:51.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.283+0000 7fd5a37fe700 1 -- 172.21.15.57:0/1058036946 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24224 (secure 0 0 0) 0x7fd5a8099d10 con 0x7fd5ac0ff540 2023-10-08T13:49:51.284 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:51.285 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":88,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:49.574582+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:49:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.285+0000 7fd5b452f700 1 -- 172.21.15.57:0/1058036946 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd5980a14d0 msgr2=0x7fd5980a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.285+0000 7fd5b452f700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd5980a14d0 0x7fd5980a3990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac067ff0 tx=0x7fd59c009260 comp rx=0 tx=0).stop 2023-10-08T13:49:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.285+0000 7fd5b452f700 1 -- 172.21.15.57:0/1058036946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5ac0ff540 msgr2=0x7fd5ac197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.285+0000 7fd5b452f700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5ac0ff540 0x7fd5ac197a10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800d980 tx=0x7fd5a800dc90 comp rx=0 tx=0).stop 2023-10-08T13:49:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.285+0000 7fd5b452f700 1 -- 172.21.15.57:0/1058036946 shutdown_connections 2023-10-08T13:49:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.285+0000 7fd5b452f700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd5980a14d0 0x7fd5980a3990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.285+0000 7fd5b452f700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac105600 0x7fd5ac1a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.285+0000 7fd5b452f700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5ac0ff540 0x7fd5ac197a10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.285+0000 7fd5b452f700 1 --2- 172.21.15.57:0/1058036946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5ac0fff00 0x7fd5ac197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.285+0000 7fd5b452f700 1 -- 172.21.15.57:0/1058036946 >> 172.21.15.57:0/1058036946 conn(0x7fd5ac0fb030 msgr2=0x7fd5ac0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.286+0000 7fd5b452f700 1 -- 172.21.15.57:0/1058036946 shutdown_connections 2023-10-08T13:49:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.286+0000 7fd5b452f700 1 -- 172.21.15.57:0/1058036946 wait complete. 2023-10-08T13:49:51.312 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:49:51.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:51 smithi115 ceph-mon[104408]: pgmap v243: 129 pgs: 11 creating+peering, 8 unknown, 110 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2023-10-08T13:49:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:51 smithi115 ceph-mon[104408]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T13:49:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:51 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/678106956' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-08T13:49:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:51 smithi115 ceph-mon[104408]: osdmap e88: 12 total, 12 up, 12 in 2023-10-08T13:49:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:51 smithi115 ceph-mon[104408]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:49:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:51 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1637279847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:51 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3129081805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:51.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.437+0000 7fea3990d700 1 -- 172.21.15.57:0/180515085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea34100640 msgr2=0x7fea34100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.437+0000 7fea3990d700 1 --2- 172.21.15.57:0/180515085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea34100640 0x7fea34100ac0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009990 tx=0x7fea1c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.437+0000 7fea3990d700 1 -- 172.21.15.57:0/180515085 shutdown_connections 2023-10-08T13:49:51.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.437+0000 7fea3990d700 1 --2- 172.21.15.57:0/180515085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea34101000 0x7fea3410d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.437+0000 7fea3990d700 1 --2- 172.21.15.57:0/180515085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea34106650 0x7fea34106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.437+0000 7fea3990d700 1 --2- 172.21.15.57:0/180515085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea34100640 0x7fea34100ac0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.437+0000 7fea3990d700 1 -- 172.21.15.57:0/180515085 >> 172.21.15.57:0/180515085 conn(0x7fea340fc170 msgr2=0x7fea340fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:51.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.437+0000 7fea3990d700 1 -- 172.21.15.57:0/180515085 shutdown_connections 2023-10-08T13:49:51.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.437+0000 7fea3990d700 1 -- 172.21.15.57:0/180515085 wait complete. 2023-10-08T13:49:51.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea3990d700 1 Processor -- start 2023-10-08T13:49:51.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea3990d700 1 -- start start 2023-10-08T13:49:51.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea3990d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea34100640 0x7fea34072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea3990d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea34101000 0x7fea34073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea3990d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea34106650 0x7fea34073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea3990d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea341a4e40 con 0x7fea34100640 2023-10-08T13:49:51.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea3990d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea341a4fb0 con 0x7fea34106650 2023-10-08T13:49:51.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea3990d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea341a5170 con 0x7fea34101000 2023-10-08T13:49:51.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea32ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea34100640 0x7fea34072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea32ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea34100640 0x7fea34072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54052/0 (socket says 172.21.15.57:54052) 2023-10-08T13:49:51.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.438+0000 7fea32ffd700 1 -- 172.21.15.57:0/3068446779 learned_addr learned my addr 172.21.15.57:0/3068446779 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:51.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea337fe700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea34106650 0x7fea34073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea327fc700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea34101000 0x7fea34073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea32ffd700 1 -- 172.21.15.57:0/3068446779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea34101000 msgr2=0x7fea34073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea32ffd700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea34101000 0x7fea34073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea32ffd700 1 -- 172.21.15.57:0/3068446779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea34106650 msgr2=0x7fea34073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea32ffd700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea34106650 0x7fea34073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea32ffd700 1 -- 172.21.15.57:0/3068446779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c0095f0 con 0x7fea34100640 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea32ffd700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea34100640 0x7fea34072af0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fea2400e9a0 tx=0x7fea2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea3890b700 1 -- 172.21.15.57:0/3068446779 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24009790 con 0x7fea34100640 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea3990d700 1 -- 172.21.15.57:0/3068446779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea341a5450 con 0x7fea34100640 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea3890b700 1 -- 172.21.15.57:0/3068446779 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2400cbd0 con 0x7fea34100640 2023-10-08T13:49:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea3890b700 1 -- 172.21.15.57:0/3068446779 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24018680 con 0x7fea34100640 2023-10-08T13:49:51.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.439+0000 7fea3990d700 1 -- 172.21.15.57:0/3068446779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea341a59b0 con 0x7fea34100640 2023-10-08T13:49:51.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.440+0000 7fea3990d700 1 -- 172.21.15.57:0/3068446779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea34108bd0 con 0x7fea34100640 2023-10-08T13:49:51.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.441+0000 7fea3890b700 1 -- 172.21.15.57:0/3068446779 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fea2400cd40 con 0x7fea34100640 2023-10-08T13:49:51.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.442+0000 7fea3890b700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fea200a14d0 0x7fea200a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.442+0000 7fea327fc700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fea200a14d0 0x7fea200a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.442+0000 7fea3890b700 1 -- 172.21.15.57:0/3068446779 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fea24014070 con 0x7fea34100640 2023-10-08T13:49:51.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.442+0000 7fea327fc700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fea200a14d0 0x7fea200a3990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fea1c006010 tx=0x7fea1c00b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:51.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.445+0000 7fea3890b700 1 -- 172.21.15.57:0/3068446779 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fea24096890 con 0x7fea34100640 2023-10-08T13:49:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:51 smithi099 ceph-mon[103849]: pgmap v243: 129 pgs: 11 creating+peering, 8 unknown, 110 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2023-10-08T13:49:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:51 smithi099 ceph-mon[103849]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T13:49:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:51 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/678106956' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-08T13:49:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:51 smithi099 ceph-mon[103849]: osdmap e88: 12 total, 12 up, 12 in 2023-10-08T13:49:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:51 smithi099 ceph-mon[103849]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:49:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:51 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1637279847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:51 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3129081805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:51.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.695+0000 7fea3990d700 1 -- 172.21.15.57:0/3068446779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea34101d80 con 0x7fea34100640 2023-10-08T13:49:51.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.696+0000 7fea3890b700 1 -- 172.21.15.57:0/3068446779 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11249 (secure 0 0 0) 0x7fea24099eb0 con 0x7fea34100640 2023-10-08T13:49:51.697 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:51.697 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14790,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24557,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24572,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24581,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24617,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24652,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:49:49.574760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24662},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-08T13:49:51.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 -- 172.21.15.57:0/3068446779 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fea200a14d0 msgr2=0x7fea200a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fea200a14d0 0x7fea200a3990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fea1c006010 tx=0x7fea1c00b260 comp rx=0 tx=0).stop 2023-10-08T13:49:51.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 -- 172.21.15.57:0/3068446779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea34100640 msgr2=0x7fea34072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea34100640 0x7fea34072af0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fea2400e9a0 tx=0x7fea2400ecb0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 -- 172.21.15.57:0/3068446779 shutdown_connections 2023-10-08T13:49:51.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fea200a14d0 0x7fea200a3990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea34106650 0x7fea34073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea34100640 0x7fea34072af0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 --2- 172.21.15.57:0/3068446779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea34101000 0x7fea34073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 -- 172.21.15.57:0/3068446779 >> 172.21.15.57:0/3068446779 conn(0x7fea340fc170 msgr2=0x7fea340fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:51.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 -- 172.21.15.57:0/3068446779 shutdown_connections 2023-10-08T13:49:51.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.698+0000 7fea3990d700 1 -- 172.21.15.57:0/3068446779 wait complete. 2023-10-08T13:49:51.701 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 17 2023-10-08T13:49:51.712 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:49:51.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 -- 172.21.15.57:0/1134707798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76240ff580 msgr2=0x7f76240ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 --2- 172.21.15.57:0/1134707798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76240ff580 0x7f76240ffa00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f76140099c0 tx=0x7f7614009cd0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 -- 172.21.15.57:0/1134707798 shutdown_connections 2023-10-08T13:49:51.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 --2- 172.21.15.57:0/1134707798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76240fff40 0x7f762410c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 --2- 172.21.15.57:0/1134707798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7624105640 0x7f7624105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 --2- 172.21.15.57:0/1134707798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76240ff580 0x7f76240ffa00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 -- 172.21.15.57:0/1134707798 >> 172.21.15.57:0/1134707798 conn(0x7f76240fb030 msgr2=0x7f76240fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 -- 172.21.15.57:0/1134707798 shutdown_connections 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 -- 172.21.15.57:0/1134707798 wait complete. 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 Processor -- start 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.838+0000 7f762bdf8700 1 -- start start 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f762bdf8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76240ff580 0x7f76241011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f762bdf8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76240fff40 0x7f7624101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f762bdf8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7624105640 0x7f7624101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f762bdf8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76241a4e00 con 0x7f76240fff40 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f762bdf8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76241a4f70 con 0x7f76240ff580 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f762bdf8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76241a5130 con 0x7f7624105640 2023-10-08T13:49:51.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629393700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76240fff40 0x7f7624101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629393700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76240fff40 0x7f7624101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54068/0 (socket says 172.21.15.57:54068) 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629393700 1 -- 172.21.15.57:0/1656514668 learned_addr learned my addr 172.21.15.57:0/1656514668 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629b94700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76240ff580 0x7f76241011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f762a395700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7624105640 0x7f7624101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629393700 1 -- 172.21.15.57:0/1656514668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7624105640 msgr2=0x7f7624101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629393700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7624105640 0x7f7624101c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629393700 1 -- 172.21.15.57:0/1656514668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76240ff580 msgr2=0x7f76241011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629393700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76240ff580 0x7f76241011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629393700 1 -- 172.21.15.57:0/1656514668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140095f0 con 0x7f76240fff40 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629b94700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76240ff580 0x7f76241011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f7629393700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76240fff40 0x7f7624101730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f761400b580 tx=0x7f7614003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:51.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f761affd700 1 -- 172.21.15.57:0/1656514668 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761401d070 con 0x7f76240fff40 2023-10-08T13:49:51.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f762bdf8700 1 -- 172.21.15.57:0/1656514668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76241a53b0 con 0x7f76240fff40 2023-10-08T13:49:51.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f761affd700 1 -- 172.21.15.57:0/1656514668 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f761400fda0 con 0x7f76240fff40 2023-10-08T13:49:51.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.839+0000 7f761affd700 1 -- 172.21.15.57:0/1656514668 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76140037c0 con 0x7f76240fff40 2023-10-08T13:49:51.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.840+0000 7f762bdf8700 1 -- 172.21.15.57:0/1656514668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76241a5830 con 0x7f76240fff40 2023-10-08T13:49:51.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.840+0000 7f762bdf8700 1 -- 172.21.15.57:0/1656514668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7624100ca0 con 0x7f76240fff40 2023-10-08T13:49:51.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.841+0000 7f761affd700 1 -- 172.21.15.57:0/1656514668 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7614022700 con 0x7f76240fff40 2023-10-08T13:49:51.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.842+0000 7f761affd700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76100a14d0 0x7f76100a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:51.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.842+0000 7f7629b94700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76100a14d0 0x7f76100a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:51.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.842+0000 7f761affd700 1 -- 172.21.15.57:0/1656514668 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f76140ce4c0 con 0x7f76240fff40 2023-10-08T13:49:51.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.842+0000 7f7629b94700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76100a14d0 0x7f76100a3990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7620005950 tx=0x7f7620005f50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:51.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:51.845+0000 7f761affd700 1 -- 172.21.15.57:0/1656514668 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f76140978e0 con 0x7f76240fff40 2023-10-08T13:49:52.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.051+0000 7f762bdf8700 1 -- 172.21.15.57:0/1656514668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f762410f4a0 con 0x7f76240fff40 2023-10-08T13:49:52.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.052+0000 7f761affd700 1 -- 172.21.15.57:0/1656514668 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24224 (secure 0 0 0) 0x7f761409af00 con 0x7f76240fff40 2023-10-08T13:49:52.053 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:52.054 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":88,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:49.574582+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:49:52.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 -- 172.21.15.57:0/1656514668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76100a14d0 msgr2=0x7f76100a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76100a14d0 0x7f76100a3990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7620005950 tx=0x7f7620005f50 comp rx=0 tx=0).stop 2023-10-08T13:49:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 -- 172.21.15.57:0/1656514668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76240fff40 msgr2=0x7f7624101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76240fff40 0x7f7624101730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f761400b580 tx=0x7f7614003710 comp rx=0 tx=0).stop 2023-10-08T13:49:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 -- 172.21.15.57:0/1656514668 shutdown_connections 2023-10-08T13:49:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7624105640 0x7f7624101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76100a14d0 0x7f76100a3990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76240ff580 0x7f76241011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 --2- 172.21.15.57:0/1656514668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f76240fff40 0x7f7624101730 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 -- 172.21.15.57:0/1656514668 >> 172.21.15.57:0/1656514668 conn(0x7f76240fb030 msgr2=0x7f76240fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 -- 172.21.15.57:0/1656514668 shutdown_connections 2023-10-08T13:49:52.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.054+0000 7f762bdf8700 1 -- 172.21.15.57:0/1656514668 wait complete. 2023-10-08T13:49:52.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T13:49:44.186896+0000', 'modified': '2023-10-08T13:49:49.574760+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24662}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24662': {'gid': 24662, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6839/687823376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 687823376}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 687823376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-08T13:49:52.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T13:49:52.069 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T13:49:52.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.195+0000 7f2a30cf7700 1 -- 172.21.15.57:0/714178160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0ff620 msgr2=0x7f2a2c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.195+0000 7f2a30cf7700 1 --2- 172.21.15.57:0/714178160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0ff620 0x7f2a2c0ffaa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009990 tx=0x7f2a20009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.195+0000 7f2a30cf7700 1 -- 172.21.15.57:0/714178160 shutdown_connections 2023-10-08T13:49:52.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.195+0000 7f2a30cf7700 1 --2- 172.21.15.57:0/714178160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a2c0fffe0 0x7f2a2c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.195+0000 7f2a30cf7700 1 --2- 172.21.15.57:0/714178160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c1056b0 0x7f2a2c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.195+0000 7f2a30cf7700 1 --2- 172.21.15.57:0/714178160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0ff620 0x7f2a2c0ffaa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.195+0000 7f2a30cf7700 1 -- 172.21.15.57:0/714178160 >> 172.21.15.57:0/714178160 conn(0x7f2a2c076780 msgr2=0x7f2a2c076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:52.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.195+0000 7f2a30cf7700 1 -- 172.21.15.57:0/714178160 shutdown_connections 2023-10-08T13:49:52.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.195+0000 7f2a30cf7700 1 -- 172.21.15.57:0/714178160 wait complete. 2023-10-08T13:49:52.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.196+0000 7f2a30cf7700 1 Processor -- start 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.196+0000 7f2a30cf7700 1 -- start start 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a30cf7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a2c0ff620 0x7f2a2c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a30cf7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0fffe0 0x7f2a2c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a30cf7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c1056b0 0x7f2a2c1a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a30cf7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c198740 con 0x7f2a2c0fffe0 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a30cf7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c1a2f50 con 0x7f2a2c0ff620 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a30cf7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c1a3090 con 0x7f2a2c1056b0 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a29d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0fffe0 0x7f2a2c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a29d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0fffe0 0x7f2a2c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54082/0 (socket says 172.21.15.57:54082) 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a29d9b700 1 -- 172.21.15.57:0/2390109545 learned_addr learned my addr 172.21.15.57:0/2390109545 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a2c0ff620 0x7f2a2c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a2ad9d700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c1056b0 0x7f2a2c1a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a29d9b700 1 -- 172.21.15.57:0/2390109545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c1056b0 msgr2=0x7f2a2c1a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c1056b0 0x7f2a2c1a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a29d9b700 1 -- 172.21.15.57:0/2390109545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a2c0ff620 msgr2=0x7f2a2c197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a2c0ff620 0x7f2a2c197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a29d9b700 1 -- 172.21.15.57:0/2390109545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a200095f0 con 0x7f2a2c0fffe0 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a2ad9d700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c1056b0 0x7f2a2c1a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.197+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0fffe0 0x7f2a2c197ed0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2a20000c00 tx=0x7f2a2000f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.198+0000 7f2a1b7fe700 1 -- 172.21.15.57:0/2390109545 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2001d070 con 0x7f2a2c0fffe0 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.198+0000 7f2a1b7fe700 1 -- 172.21.15.57:0/2390109545 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a2000fd30 con 0x7f2a2c0fffe0 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.198+0000 7f2a30cf7700 1 -- 172.21.15.57:0/2390109545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c1a32c0 con 0x7f2a2c0fffe0 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.198+0000 7f2a1b7fe700 1 -- 172.21.15.57:0/2390109545 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20017630 con 0x7f2a2c0fffe0 2023-10-08T13:49:52.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.198+0000 7f2a30cf7700 1 -- 172.21.15.57:0/2390109545 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c1a37a0 con 0x7f2a2c0fffe0 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.200+0000 7f2a1b7fe700 1 -- 172.21.15.57:0/2390109545 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f2a20017790 con 0x7f2a2c0fffe0 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.200+0000 7f2a1b7fe700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a140a14d0 0x7f2a140a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.200+0000 7f2a1b7fe700 1 -- 172.21.15.57:0/2390109545 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f2a200ce390 con 0x7f2a2c0fffe0 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.200+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a140a14d0 0x7f2a140a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.201+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a140a14d0 0x7f2a140a3990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c077ab0 tx=0x7f2a1c00a380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.201+0000 7f2a3cee0b80 1 Processor -- start 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.201+0000 7f2a3cee0b80 1 -- start start 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3be30 0x562edee3c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3c750 0x562edee470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ededa5410 con 0x562edee47600 2023-10-08T13:49:52.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eded96bd0 con 0x562edee3be30 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eded944f0 con 0x562edee3c750 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54092/0 (socket says 172.21.15.57:54092) 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2ad9d700 1 -- 172.21.15.57:0/1607070182 learned_addr learned my addr 172.21.15.57:0/1607070182 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3be30 0x562edee3c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/1607070182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3c750 0x562edee470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2ad9d700 1 -- 172.21.15.57:0/1607070182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3c750 msgr2=0x562edee470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2ad9d700 1 --2- 172.21.15.57:0/1607070182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3c750 0x562edee470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2ad9d700 1 -- 172.21.15.57:0/1607070182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3be30 msgr2=0x562edee3c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2ad9d700 1 --2- 172.21.15.57:0/1607070182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3be30 0x562edee3c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2ad9d700 1 -- 172.21.15.57:0/1607070182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562edebbad90 con 0x562edee47600 2023-10-08T13:49:52.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/1607070182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3be30 0x562edee3c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.202+0000 7f2a2ad9d700 1 --2- 172.21.15.57:0/1607070182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee499f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2a240087c0 tx=0x7f2a24008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192f3d4bf405f774 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a18ff9700 1 -- 172.21.15.57:0/1607070182 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24003c00 con 0x562edee47600 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a18ff9700 1 -- 172.21.15.57:0/1607070182 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a24010040 con 0x562edee47600 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a18ff9700 1 -- 172.21.15.57:0/1607070182 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24012740 con 0x562edee47600 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/1607070182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 msgr2=0x562edee499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/1607070182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee499f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2a240087c0 tx=0x7f2a24008ad0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/1607070182 shutdown_connections 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/1607070182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee499f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/1607070182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3be30 0x562edee3c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/1607070182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3c750 0x562edee470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/1607070182 >> 172.21.15.57:0/1607070182 conn(0x562ededb1210 msgr2=0x562edee38280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:52.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/1607070182 shutdown_connections 2023-10-08T13:49:52.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.203+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/1607070182 wait complete. 2023-10-08T13:49:52.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.204+0000 7f2a3cee0b80 1 -- start start 2023-10-08T13:49:52.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.204+0000 7f2a3cee0b80 1 Processor -- start 2023-10-08T13:49:52.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.204+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee46fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.204+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3c750 0x562edee442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.204+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee47600 0x562edee44930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.204+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eded8fde0 con 0x562edee3be30 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.204+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eded2c6c0 con 0x562edee47600 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.204+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eded2c360 con 0x562edee3c750 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.204+0000 7f2a3cee0b80 10 client.0 _ll_register_callbacks cb 0x562edeb897b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.204+0000 7f2a2a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee46fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a2a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee46fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54094/0 (socket says 172.21.15.57:54094) 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a2a59c700 1 -- 172.21.15.57:0/3421607800 learned_addr learned my addr 172.21.15.57:0/3421607800 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3c750 0x562edee442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a2ad9d700 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee47600 0x562edee44930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a2a59c700 1 -- 172.21.15.57:0/3421607800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3c750 msgr2=0x562edee442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3c750 0x562edee442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a2a59c700 1 -- 172.21.15.57:0/3421607800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee47600 msgr2=0x562edee44930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee47600 0x562edee44930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a2a59c700 1 -- 172.21.15.57:0/3421607800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c00f9c0 con 0x562edee3be30 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a2ad9d700 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee47600 0x562edee44930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee46fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c01de30 tx=0x7f2a1c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5515fcd6d8f9743 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a11ffb700 1 -- 172.21.15.57:0/3421607800 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c010ac0 con 0x562edee3be30 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a11ffb700 1 -- 172.21.15.57:0/3421607800 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f2a1c010c20 con 0x562edee3be30 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/3421607800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562eded57e60 con 0x562edee3be30 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a11ffb700 1 -- 172.21.15.57:0/3421607800 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a1c010e20 con 0x562edee3be30 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a3cee0b80 10 client.14904 fetch_fsmap learned FSMap version 17 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.205+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/3421607800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7f2a240083f0 con 0x562edee3be30 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.206+0000 7f2a11ffb700 1 -- 172.21.15.57:0/3421607800 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562eded57e60 con 0x562edee3be30 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.206+0000 7f2a11ffb700 1 -- 172.21.15.57:0/3421607800 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2a1c0255b0 con 0x562edee3be30 2023-10-08T13:49:52.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.206+0000 7f2a3cee0b80 10 client.14904 fetch_fsmap finished waiting for FSMap version 17 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.206+0000 7f2a3cee0b80 10 client.14904 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.206+0000 7f2a3cee0b80 20 client.14904 populate_metadata read hostname 'smithi057' 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.206+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562edee57200 0x562edee55cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.206+0000 7f2a3cee0b80 4 client.14904 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.206+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/3421607800 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562eded88dc0 con 0x562edee57200 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.206+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562edee57200 0x562edee55cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.207+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562edee57200 0x562edee55cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4fe45138edc4cb1 server_cookie=b1d8d4d3dd98c4e6 in_seq=0 out_seq=0 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.207+0000 7f2a11ffb700 10 client.14904 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.207+0000 7f2a11ffb700 1 -- 172.21.15.57:0/3421607800 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562eded88dc0 con 0x562edee57200 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.207+0000 7f2a11ffb700 10 client.14904 handle_command_reply: tid=0 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.207+0000 7f2a3cee0b80 1 client.14904 shutdown 2023-10-08T13:49:52.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.207+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/3421607800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 msgr2=0x562edee46fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.207+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee46fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c01de30 tx=0x7f2a1c0047c0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.208+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/3421607800 shutdown_connections 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.208+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562edee57200 0x562edee55cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.208+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee47600 0x562edee44930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.208+0000 7f2a11ffb700 0 client.14904 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.208+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee46fc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.208+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/3421607800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3c750 0x562edee442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.208+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/3421607800 >> 172.21.15.57:0/3421607800 conn(0x562ededb1210 msgr2=0x562edee37e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.208+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/3421607800 shutdown_connections 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.208+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/3421607800 wait complete. 2023-10-08T13:49:52.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.208+0000 7f2a3cee0b80 20 client.14904 trim_cache size 0 max 16384 2023-10-08T13:49:52.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.225+0000 7f2a3cee0b80 1 Processor -- start 2023-10-08T13:49:52.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.225+0000 7f2a3cee0b80 1 -- start start 2023-10-08T13:49:52.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee3ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 0x562edee53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee47600 0x562edee5fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562edec108d0 con 0x562edee3be30 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eded364d0 con 0x562edee3c750 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562edec1cde0 con 0x562edee47600 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a2a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee3ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a2a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee3ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54102/0 (socket says 172.21.15.57:54102) 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a2a59c700 1 -- 172.21.15.57:0/116723598 learned_addr learned my addr 172.21.15.57:0/116723598 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/116723598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 0x562edee53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a2a59c700 1 -- 172.21.15.57:0/116723598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee47600 msgr2=0x562edee5fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/116723598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee47600 0x562edee5fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a2a59c700 1 -- 172.21.15.57:0/116723598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 msgr2=0x562edee53170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:49:52.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/116723598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 0x562edee53170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a2a59c700 1 -- 172.21.15.57:0/116723598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562edeba3c10 con 0x562edee3be30 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/116723598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 0x562edee53170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/116723598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee3ed60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c005b60 tx=0x7f2a1c00f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7c9d7faaa1289f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a197fa700 1 -- 172.21.15.57:0/116723598 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c0253f0 con 0x562edee3be30 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a197fa700 1 -- 172.21.15.57:0/116723598 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a1c014070 con 0x562edee3be30 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.226+0000 7f2a197fa700 1 -- 172.21.15.57:0/116723598 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c035690 con 0x562edee3be30 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/116723598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 msgr2=0x562edee3ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/116723598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee3ed60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c005b60 tx=0x7f2a1c00f880 comp rx=0 tx=0).stop 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/116723598 shutdown_connections 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/116723598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee47600 0x562edee5fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/116723598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee3be30 0x562edee3ed60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/116723598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 0x562edee53170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/116723598 >> 172.21.15.57:0/116723598 conn(0x562ededb1210 msgr2=0x562edee42b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/116723598 shutdown_connections 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/116723598 wait complete. 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 -- start start 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.227+0000 7f2a3cee0b80 1 Processor -- start 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3be30 0x562edee5e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 0x562edee5e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a3cee0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee59b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562edee5a100 con 0x562edee47600 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562edee5a270 con 0x562edee3c750 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a3cee0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562edee5a430 con 0x562edee3be30 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a3cee0b80 10 client.0 _ll_register_callbacks cb 0x562edeb897b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:49:52.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a2ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee59b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a2ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee59b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54112/0 (socket says 172.21.15.57:54112) 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a2ad9d700 1 -- 172.21.15.57:0/683974221 learned_addr learned my addr 172.21.15.57:0/683974221 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 0x562edee5e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a2ad9d700 1 -- 172.21.15.57:0/683974221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3be30 msgr2=0x562edee5e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a2ad9d700 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3be30 0x562edee5e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a2ad9d700 1 -- 172.21.15.57:0/683974221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 msgr2=0x562edee5e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a2ad9d700 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 0x562edee5e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a2ad9d700 1 -- 172.21.15.57:0/683974221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a240083f0 con 0x562edee47600 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a29d9b700 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 0x562edee5e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a2ad9d700 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee59b30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2a24007fd0 tx=0x7f2a24012ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa69b767fb6d0f0f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:49:52.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a11ffb700 1 -- 172.21.15.57:0/683974221 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24007770 con 0x562edee47600 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a11ffb700 1 -- 172.21.15.57:0/683974221 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f2a24012cd0 con 0x562edee47600 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/683974221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a24010040 con 0x562edee47600 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a11ffb700 1 -- 172.21.15.57:0/683974221 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a24012e90 con 0x562edee47600 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a3cee0b80 10 client.14910 fetch_fsmap learned FSMap version 17 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.228+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/683974221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x562edebe1200 con 0x562edee47600 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a11ffb700 1 -- 172.21.15.57:0/683974221 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a24010040 con 0x562edee47600 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a11ffb700 1 -- 172.21.15.57:0/683974221 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2a2401fbb0 con 0x562edee47600 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a3cee0b80 10 client.14910 fetch_fsmap finished waiting for FSMap version 17 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a3cee0b80 10 client.14910 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a3cee0b80 20 client.14910 populate_metadata read hostname 'smithi057' 2023-10-08T13:49:52.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562edeeaf7b0 0x562edeeabbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a3cee0b80 4 client.14910 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/683974221 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562eded88ae0 con 0x562edeeaf7b0 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562edeeaf7b0 0x562edeeabbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a11ffb700 10 client.14910 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.229+0000 7f2a2a59c700 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562edeeaf7b0 0x562edeeabbd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b838c1b84d6d06 server_cookie=5c975ee139503742 in_seq=0 out_seq=0 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a11ffb700 1 -- 172.21.15.57:0/683974221 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x562eded88ae0 con 0x562edeeaf7b0 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a11ffb700 10 client.14910 handle_command_reply: tid=0 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 client.14910 shutdown 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/683974221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 msgr2=0x562edee59b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee59b30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2a24007fd0 tx=0x7f2a24012ab0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/683974221 shutdown_connections 2023-10-08T13:49:52.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562edeeaf7b0 0x562edeeabbd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562edee47600 0x562edee59b30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a11ffb700 0 client.14910 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:49:52.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562edee3be30 0x562edee5e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 --2- 172.21.15.57:0/683974221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562edee3c750 0x562edee5e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/683974221 >> 172.21.15.57:0/683974221 conn(0x562ededb1210 msgr2=0x562edee427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:52.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/683974221 shutdown_connections 2023-10-08T13:49:52.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 1 -- 172.21.15.57:0/683974221 wait complete. 2023-10-08T13:49:52.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a3cee0b80 20 client.14910 trim_cache size 0 max 16384 2023-10-08T13:49:52.238 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24662, 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 7.1161542339999997, 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 17, 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 88, 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 87, 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 13.045549775 2023-10-08T13:49:52.239 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:49:52.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a30cf7700 1 -- 172.21.15.57:0/2390109545 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a140a14d0 msgr2=0x7f2a140a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a30cf7700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a140a14d0 0x7f2a140a3990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c077ab0 tx=0x7f2a1c00a380 comp rx=0 tx=0).stop 2023-10-08T13:49:52.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a30cf7700 1 -- 172.21.15.57:0/2390109545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0fffe0 msgr2=0x7f2a2c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:52.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a30cf7700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0fffe0 0x7f2a2c197ed0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2a20000c00 tx=0x7f2a2000f7d0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a30cf7700 1 -- 172.21.15.57:0/2390109545 shutdown_connections 2023-10-08T13:49:52.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a30cf7700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c1056b0 0x7f2a2c1a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a30cf7700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a140a14d0 0x7f2a140a3990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a30cf7700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a2c0ff620 0x7f2a2c197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a30cf7700 1 --2- 172.21.15.57:0/2390109545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a2c0fffe0 0x7f2a2c197ed0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:52.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.230+0000 7f2a30cf7700 1 -- 172.21.15.57:0/2390109545 >> 172.21.15.57:0/2390109545 conn(0x7f2a2c076780 msgr2=0x7f2a2c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:52.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.231+0000 7f2a30cf7700 1 -- 172.21.15.57:0/2390109545 shutdown_connections 2023-10-08T13:49:52.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:52.231+0000 7f2a30cf7700 1 -- 172.21.15.57:0/2390109545 wait complete. 2023-10-08T13:49:52.244 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T13:49:52.244 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-08T13:49:52.244 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-08T13:49:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:52 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1058036946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:52 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3068446779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:52.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:52 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1656514668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:52.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:52 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1058036946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:52.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:52 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3068446779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:52.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:52 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1656514668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:52.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:52 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1058036946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:52.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:52 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3068446779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:49:52.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:52 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1656514668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:52.803 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:49:53.263 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:53 smithi057 ceph-mon[99476]: pgmap v245: 129 pgs: 11 creating+peering, 1 unknown, 117 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2023-10-08T13:49:53.263 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:53 smithi057 ceph-mon[99476]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T13:49:53.263 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:53 smithi057 ceph-mon[99476]: Cluster is now healthy 2023-10-08T13:49:53.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:53 smithi115 ceph-mon[104408]: pgmap v245: 129 pgs: 11 creating+peering, 1 unknown, 117 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2023-10-08T13:49:53.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:53 smithi115 ceph-mon[104408]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T13:49:53.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:53 smithi115 ceph-mon[104408]: Cluster is now healthy 2023-10-08T13:49:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:53 smithi099 ceph-mon[103849]: pgmap v245: 129 pgs: 11 creating+peering, 1 unknown, 117 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2023-10-08T13:49:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:53 smithi099 ceph-mon[103849]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T13:49:53.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:53 smithi099 ceph-mon[103849]: Cluster is now healthy 2023-10-08T13:49:53.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.868+0000 7fcd1e295700 1 -- 172.21.15.57:0/3367166730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd181000a0 msgr2=0x7fcd1810c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:53.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.868+0000 7fcd1e295700 1 --2- 172.21.15.57:0/3367166730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd181000a0 0x7fcd1810c560 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000b0a0 tx=0x7fcd1000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.868+0000 7fcd1e295700 1 -- 172.21.15.57:0/3367166730 shutdown_connections 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.868+0000 7fcd1e295700 1 --2- 172.21.15.57:0/3367166730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd181000a0 0x7fcd1810c560 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.868+0000 7fcd1e295700 1 --2- 172.21.15.57:0/3367166730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd18105690 0x7fcd18105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.868+0000 7fcd1e295700 1 --2- 172.21.15.57:0/3367166730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd180ff6e0 0x7fcd180ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.868+0000 7fcd1e295700 1 -- 172.21.15.57:0/3367166730 >> 172.21.15.57:0/3367166730 conn(0x7fcd180766c0 msgr2=0x7fcd18076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.868+0000 7fcd1e295700 1 -- 172.21.15.57:0/3367166730 shutdown_connections 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.868+0000 7fcd1e295700 1 -- 172.21.15.57:0/3367166730 wait complete. 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1e295700 1 Processor -- start 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1e295700 1 -- start start 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1e295700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd180ff6e0 0x7fcd18101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:53.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1e295700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd181000a0 0x7fcd181017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1e295700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd18105690 0x7fcd18101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1e295700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181a5120 con 0x7fcd18105690 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1e295700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181a5290 con 0x7fcd180ff6e0 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1e295700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181a5450 con 0x7fcd181000a0 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1c832700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd18105690 0x7fcd18101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1c832700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd18105690 0x7fcd18101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54124/0 (socket says 172.21.15.57:54124) 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd1c832700 1 -- 172.21.15.57:0/1276438323 learned_addr learned my addr 172.21.15.57:0/1276438323 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd177fe700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd181000a0 0x7fcd181017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.869+0000 7fcd17fff700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd180ff6e0 0x7fcd18101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd1c832700 1 -- 172.21.15.57:0/1276438323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd181000a0 msgr2=0x7fcd181017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd1c832700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd181000a0 0x7fcd181017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd1c832700 1 -- 172.21.15.57:0/1276438323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd180ff6e0 msgr2=0x7fcd18101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:53.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd1c832700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd180ff6e0 0x7fcd18101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:53.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd1c832700 1 -- 172.21.15.57:0/1276438323 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd10009d00 con 0x7fcd18105690 2023-10-08T13:49:53.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd1c832700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd18105690 0x7fcd18101ce0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000b070 tx=0x7fcd10007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:53.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd157fa700 1 -- 172.21.15.57:0/1276438323 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000e040 con 0x7fcd18105690 2023-10-08T13:49:53.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd1e295700 1 -- 172.21.15.57:0/1276438323 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd181a56d0 con 0x7fcd18105690 2023-10-08T13:49:53.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd157fa700 1 -- 172.21.15.57:0/1276438323 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd10009440 con 0x7fcd18105690 2023-10-08T13:49:53.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd157fa700 1 -- 172.21.15.57:0/1276438323 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1001b450 con 0x7fcd18105690 2023-10-08T13:49:53.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.870+0000 7fcd1e295700 1 -- 172.21.15.57:0/1276438323 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd181a5b90 con 0x7fcd18105690 2023-10-08T13:49:53.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.871+0000 7fcd1e295700 1 -- 172.21.15.57:0/1276438323 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd18108cc0 con 0x7fcd18105690 2023-10-08T13:49:53.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.872+0000 7fcd157fa700 1 -- 172.21.15.57:0/1276438323 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fcd10004cf0 con 0x7fcd18105690 2023-10-08T13:49:53.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.873+0000 7fcd157fa700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcd040a14d0 0x7fcd040a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:53.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.873+0000 7fcd17fff700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcd040a14d0 0x7fcd040a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:53.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.873+0000 7fcd157fa700 1 -- 172.21.15.57:0/1276438323 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fcd100cec90 con 0x7fcd18105690 2023-10-08T13:49:53.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.873+0000 7fcd17fff700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcd040a14d0 0x7fcd040a3990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000dad0 tx=0x7fcd0000d4b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:53.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:53.877+0000 7fcd157fa700 1 -- 172.21.15.57:0/1276438323 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcd100980b0 con 0x7fcd18105690 2023-10-08T13:49:54.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.123+0000 7fcd1e295700 1 -- 172.21.15.57:0/1276438323 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcd18102740 con 0x7fcd18105690 2023-10-08T13:49:54.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.123+0000 7fcd157fa700 1 -- 172.21.15.57:0/1276438323 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fcd10011390 con 0x7fcd18105690 2023-10-08T13:49:54.124 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T13:49:54.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 -- 172.21.15.57:0/1276438323 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcd040a14d0 msgr2=0x7fcd040a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcd040a14d0 0x7fcd040a3990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000dad0 tx=0x7fcd0000d4b0 comp rx=0 tx=0).stop 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 -- 172.21.15.57:0/1276438323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd18105690 msgr2=0x7fcd18101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd18105690 0x7fcd18101ce0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000b070 tx=0x7fcd10007870 comp rx=0 tx=0).stop 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 -- 172.21.15.57:0/1276438323 shutdown_connections 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd18105690 0x7fcd18101ce0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcd040a14d0 0x7fcd040a3990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd180ff6e0 0x7fcd18101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 --2- 172.21.15.57:0/1276438323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd181000a0 0x7fcd181017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 -- 172.21.15.57:0/1276438323 >> 172.21.15.57:0/1276438323 conn(0x7fcd180766c0 msgr2=0x7fcd18069260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 -- 172.21.15.57:0/1276438323 shutdown_connections 2023-10-08T13:49:54.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:54.125+0000 7fcd1e295700 1 -- 172.21.15.57:0/1276438323 wait complete. 2023-10-08T13:49:54.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:54 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1276438323' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T13:49:54.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:54 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1276438323' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T13:49:54.566 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T13:49:54.566 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T13:49:54.566 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-10-08T13:49:54.566 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-08T13:49:54.566 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-08T13:49:54.566 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T13:49:54.566 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T13:49:54.566 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T13:49:54.566 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-08T13:49:54.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:54.567 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-10-08T13:49:54.586 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T13:49:54.586 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T13:49:54.586 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T13:49:54.586 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-08T13:49:54.586 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-10-08T13:49:54.586 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-08T13:49:54.586 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T13:49:54.586 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-10-08T13:49:54.587 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T13:49:54.587 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-10-08T13:49:54.587 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T13:49:54.587 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T13:49:54.587 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-08T13:49:54.587 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41907sec preferred_lft 41907sec 2023-10-08T13:49:54.587 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-08T13:49:54.587 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-08T13:49:54.587 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T13:49:54.587 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T13:49:54.588 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T13:49:54.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:54.588 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-08T13:49:54.588 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-08T13:49:54.588 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add name ceph-brx type bridge 2023-10-08T13:49:54.588 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr flush dev ceph-brx 2023-10-08T13:49:54.588 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set ceph-brx up 2023-10-08T13:49:54.588 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T13:49:54.589 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-08T13:49:54.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:54 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1276438323' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T13:49:54.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:54.759 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T13:49:54.789 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-10-08T13:49:54.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:54.790 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-10-08T13:49:54.874 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-10-08T13:49:54.875 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T13:49:54.875 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-08T13:49:54.875 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-08T13:49:54.875 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T13:49:54.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:54.876 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-08T13:49:54.876 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-08T13:49:54.876 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-08T13:49:54.876 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-08T13:49:54.876 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-08T13:49:54.876 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-08T13:49:55.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:55.124 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-08T13:49:55.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:55.141 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2023-10-08T13:49:55.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:55.199 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-08T13:49:55.200 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-08T13:49:55.200 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T13:49:55.200 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-08T13:49:55.200 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-08T13:49:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:55 smithi057 ceph-mon[99476]: pgmap v246: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2023-10-08T13:49:55.293 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-08T13:49:55.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:55.294 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-08T13:49:55.294 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-08T13:49:55.294 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-08T13:49:55.294 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T13:49:55.294 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-08T13:49:55.294 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-08T13:49:55.294 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-08T13:49:55.294 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-08T13:49:55.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:55.497 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-08T13:49:55.497 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-08T13:49:55.497 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-10-08T13:49:55.497 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T13:49:55.497 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-08T13:49:55.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:55 smithi099 ceph-mon[103849]: pgmap v246: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2023-10-08T13:49:55.596 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-08T13:49:55.596 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T13:49:55.597 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-08T13:49:55.614 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-08T13:49:55.614 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T13:49:55.614 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-08T13:49:55.671 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-08T13:49:55.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:55 smithi115 ceph-mon[104408]: pgmap v246: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2023-10-08T13:49:55.740 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2023-10-08T13:49:55.797 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/systemd 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2023-10-08T13:49:55.798 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/cpuset 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/pids 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/devices 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/hugetlb 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/memory 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/blkio 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/perf_event 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/rdma 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/freezer 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2023-10-08T13:49:55.799 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2023-10-08T13:49:55.800 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/8d21e02541d23be47e93e5b6edd1a62b90df520b8640b4a5970704c3c5f93d25/merged 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e81c9eb28775e4b2a9c2900ade28fb0fe3edc3be6a611d9cee0350c8c0a34332/merged 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f5c61220aad320541da1e44047ca4be361f0c87acd7dc83c83c59ebe40bd1293/merged 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/a3c341dc7ea0d7e0126249450d39311f778d372a13671f341994ade570b010c4/merged 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/0443ae2a350ba0f0e3a7062c7d6c43087d9714ad1d023d2edd4f9fb9fedd7e8a/merged 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/a199bb960f20d37a9fafc283dd284267a31fc1f8929a7e7c3ce14217de86fc46/merged 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/25632f389c7f43016ca942979a2378e3ba779cc4075d3cd392d99ff30376649e/merged 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/23416b84fdeb905a93b7089631a0e61bbd88847cd99d2ba4d3c04a898f4263e6/merged 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/2e01f65ad907ed13b484411791536a52cd121d70f5010364cab9cac508503e15/merged 2023-10-08T13:49:55.801 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/61af0144ab97eb053693f4295bd7c98e9dd86bc5ded21451becf6734b0371b58/merged 2023-10-08T13:49:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2023-10-08T13:49:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T13:49:55.802 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T13:49:55.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:55.802 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-08T13:49:55.855 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T13:49:55.855 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-08T13:49:55.897 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-08T13:49:55.924 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2023-10-08T13:49:55.983 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2023-10-08T13:49:55.983 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2023-10-08T13:49:55.983 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2023-10-08T13:49:55.983 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2023-10-08T13:49:55.983 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2023-10-08T13:49:55.983 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2023-10-08T13:49:55.983 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2023-10-08T13:49:55.983 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2023-10-08T13:49:55.983 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/systemd 2023-10-08T13:49:55.983 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/cpuset 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/pids 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/devices 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/hugetlb 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/memory 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/blkio 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/perf_event 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/rdma 2023-10-08T13:49:55.984 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup/freezer 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2023-10-08T13:49:55.985 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/8d21e02541d23be47e93e5b6edd1a62b90df520b8640b4a5970704c3c5f93d25/merged 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e81c9eb28775e4b2a9c2900ade28fb0fe3edc3be6a611d9cee0350c8c0a34332/merged 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f5c61220aad320541da1e44047ca4be361f0c87acd7dc83c83c59ebe40bd1293/merged 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/a3c341dc7ea0d7e0126249450d39311f778d372a13671f341994ade570b010c4/merged 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/0443ae2a350ba0f0e3a7062c7d6c43087d9714ad1d023d2edd4f9fb9fedd7e8a/merged 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/a199bb960f20d37a9fafc283dd284267a31fc1f8929a7e7c3ce14217de86fc46/merged 2023-10-08T13:49:55.986 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/25632f389c7f43016ca942979a2378e3ba779cc4075d3cd392d99ff30376649e/merged 2023-10-08T13:49:55.987 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/23416b84fdeb905a93b7089631a0e61bbd88847cd99d2ba4d3c04a898f4263e6/merged 2023-10-08T13:49:55.987 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/2e01f65ad907ed13b484411791536a52cd121d70f5010364cab9cac508503e15/merged 2023-10-08T13:49:55.987 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/61af0144ab97eb053693f4295bd7c98e9dd86bc5ded21451becf6734b0371b58/merged 2023-10-08T13:49:55.987 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2023-10-08T13:49:55.987 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T13:49:55.987 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T13:49:55.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:55.988 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-08T13:49:56.005 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:2023-10-08T13:49:56.004+0000 7f18abf3b5c0 -1 init, newargv = 0x555a38f8c020 newargc=15 2023-10-08T13:49:56.005 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:ceph-fuse[160198]: starting ceph client 2023-10-08T13:49:56.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:ceph-fuse[160198]: starting fuse 2023-10-08T13:49:56.023 INFO:teuthology.orchestra.run.smithi057.stdout:124 2023-10-08T13:49:56.023 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> sudo stdin-killer -- python3 -c ' 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-08T13:49:56.024 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T13:49:56.025 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-08T13:49:56.026 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.admin")) 2023-10-08T13:49:56.026 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-08T13:49:56.150 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.admin.160198.asok 2023-10-08T13:49:56.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-08T13:49:56.159 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.160198.asok 2023-10-08T13:49:56.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:56.160 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.160198.asok status 2023-10-08T13:49:56.321 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:49:56.321 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-10-08T13:49:56.321 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "7b3143bae3b6870a749bfb9810ed6e7d0e89d7de", 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "admin", 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "160198", 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 14922, 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-10-08T13:49:56.322 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 14922 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3925654388 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-08T13:49:56.323 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3925654388 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.14922 192.168.0.1:0/3925654388", 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/3925654388", 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 17, 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 88, 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-10-08T13:49:56.324 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:49:56.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:56.331 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-10-08T13:49:56.331 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-08T13:49:56.349 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-10-08T13:49:56.349 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-08T13:49:56.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:56.349 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-08T13:49:56.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:49:56.417 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-08T13:49:56.517 INFO:teuthology.orchestra.run.smithi057.stdout:# file: . 2023-10-08T13:49:56.518 INFO:teuthology.orchestra.run.smithi057.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-08T13:49:56.518 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:56.518 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:49:56.518 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T13:49:56.536 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:49:56.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 -- 172.21.15.57:0/1557167543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a94106660 msgr2=0x7f5a94106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:56.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 --2- 172.21.15.57:0/1557167543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a94106660 0x7f5a94106a40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009990 tx=0x7f5a90009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 -- 172.21.15.57:0/1557167543 shutdown_connections 2023-10-08T13:49:56.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 --2- 172.21.15.57:0/1557167543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a94101010 0x7f5a9410d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 --2- 172.21.15.57:0/1557167543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a94106660 0x7f5a94106a40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 --2- 172.21.15.57:0/1557167543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a94100650 0x7f5a94100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 -- 172.21.15.57:0/1557167543 >> 172.21.15.57:0/1557167543 conn(0x7f5a940fc180 msgr2=0x7f5a940fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:56.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 -- 172.21.15.57:0/1557167543 shutdown_connections 2023-10-08T13:49:56.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 -- 172.21.15.57:0/1557167543 wait complete. 2023-10-08T13:49:56.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 Processor -- start 2023-10-08T13:49:56.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.702+0000 7f5a9c39c700 1 -- start start 2023-10-08T13:49:56.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a9c39c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a94100650 0x7f5a94072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:56.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a9c39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a94101010 0x7f5a94073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:56.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a9c39c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a94106660 0x7f5a94073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:56.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a9c39c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a941a4e50 con 0x7f5a94101010 2023-10-08T13:49:56.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a9c39c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a941a4fc0 con 0x7f5a94106660 2023-10-08T13:49:56.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a9c39c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a941a5180 con 0x7f5a94100650 2023-10-08T13:49:56.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a99937700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a94101010 0x7f5a94073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:56.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a99937700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a94101010 0x7f5a94073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54150/0 (socket says 172.21.15.57:54150) 2023-10-08T13:49:56.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a99937700 1 -- 172.21.15.57:0/3760084972 learned_addr learned my addr 172.21.15.57:0/3760084972 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a9a939700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a94106660 0x7f5a94073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a9a138700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a94100650 0x7f5a94072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a99937700 1 -- 172.21.15.57:0/3760084972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a94100650 msgr2=0x7f5a94072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a99937700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a94100650 0x7f5a94072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a99937700 1 -- 172.21.15.57:0/3760084972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a94106660 msgr2=0x7f5a94073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a99937700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a94106660 0x7f5a94073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a99937700 1 -- 172.21.15.57:0/3760084972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a900095f0 con 0x7f5a94101010 2023-10-08T13:49:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a99937700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a94101010 0x7f5a94073030 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400e9a0 tx=0x7f5a8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a8b7fe700 1 -- 172.21.15.57:0/3760084972 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84009790 con 0x7f5a94101010 2023-10-08T13:49:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a9c39c700 1 -- 172.21.15.57:0/3760084972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a941a5460 con 0x7f5a94101010 2023-10-08T13:49:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a8b7fe700 1 -- 172.21.15.57:0/3760084972 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a84010430 con 0x7f5a94101010 2023-10-08T13:49:56.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.703+0000 7f5a8b7fe700 1 -- 172.21.15.57:0/3760084972 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84018680 con 0x7f5a94101010 2023-10-08T13:49:56.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.704+0000 7f5a9c39c700 1 -- 172.21.15.57:0/3760084972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a941a59c0 con 0x7f5a94101010 2023-10-08T13:49:56.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.704+0000 7f5a9c39c700 1 -- 172.21.15.57:0/3760084972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a94108c50 con 0x7f5a94101010 2023-10-08T13:49:56.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.706+0000 7f5a8b7fe700 1 -- 172.21.15.57:0/3760084972 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f5a840105a0 con 0x7f5a94101010 2023-10-08T13:49:56.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.706+0000 7f5a8b7fe700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5a800a14d0 0x7f5a800a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:56.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.706+0000 7f5a9a138700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5a800a14d0 0x7f5a800a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:56.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.706+0000 7f5a8b7fe700 1 -- 172.21.15.57:0/3760084972 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f5a84014070 con 0x7f5a94101010 2023-10-08T13:49:56.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.707+0000 7f5a9a138700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5a800a14d0 0x7f5a800a3990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000b2e0 tx=0x7f5a90005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:56.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.710+0000 7f5a8b7fe700 1 -- 172.21.15.57:0/3760084972 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5a840967c0 con 0x7f5a94101010 2023-10-08T13:49:56.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.916+0000 7f5a9c39c700 1 -- 172.21.15.57:0/3760084972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a94073fd0 con 0x7f5a94101010 2023-10-08T13:49:56.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.917+0000 7f5a8b7fe700 1 -- 172.21.15.57:0/3760084972 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24224 (secure 0 0 0) 0x7f5a84026020 con 0x7f5a94101010 2023-10-08T13:49:56.919 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:56.919 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":88,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:49.574582+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.919+0000 7f5a9c39c700 1 -- 172.21.15.57:0/3760084972 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5a800a14d0 msgr2=0x7f5a800a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.919+0000 7f5a9c39c700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5a800a14d0 0x7f5a800a3990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000b2e0 tx=0x7f5a90005960 comp rx=0 tx=0).stop 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.919+0000 7f5a9c39c700 1 -- 172.21.15.57:0/3760084972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a94101010 msgr2=0x7f5a94073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.919+0000 7f5a9c39c700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a94101010 0x7f5a94073030 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400e9a0 tx=0x7f5a8400ecb0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.919+0000 7f5a9c39c700 1 -- 172.21.15.57:0/3760084972 shutdown_connections 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.919+0000 7f5a9c39c700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5a800a14d0 0x7f5a800a3990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.920+0000 7f5a9c39c700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a94106660 0x7f5a94073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.920+0000 7f5a9c39c700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a94100650 0x7f5a94072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.920+0000 7f5a9c39c700 1 --2- 172.21.15.57:0/3760084972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5a94101010 0x7f5a94073030 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.920+0000 7f5a9c39c700 1 -- 172.21.15.57:0/3760084972 >> 172.21.15.57:0/3760084972 conn(0x7f5a940fc180 msgr2=0x7f5a940fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:56.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.920+0000 7f5a9c39c700 1 -- 172.21.15.57:0/3760084972 shutdown_connections 2023-10-08T13:49:56.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:56.920+0000 7f5a9c39c700 1 -- 172.21.15.57:0/3760084972 wait complete. 2023-10-08T13:49:56.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T13:49:56.933 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:49:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.061+0000 7f7450f80700 1 -- 172.21.15.57:0/563422163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f744c105630 msgr2=0x7f744c105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.061+0000 7f7450f80700 1 --2- 172.21.15.57:0/563422163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f744c105630 0x7f744c105a10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f743c009990 tx=0x7f743c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.061+0000 7f7450f80700 1 -- 172.21.15.57:0/563422163 shutdown_connections 2023-10-08T13:49:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.061+0000 7f7450f80700 1 --2- 172.21.15.57:0/563422163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c100000 0x7f744c10c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.061+0000 7f7450f80700 1 --2- 172.21.15.57:0/563422163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f744c105630 0x7f744c105a10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.061+0000 7f7450f80700 1 --2- 172.21.15.57:0/563422163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c0ff640 0x7f744c0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.061+0000 7f7450f80700 1 -- 172.21.15.57:0/563422163 >> 172.21.15.57:0/563422163 conn(0x7f744c076790 msgr2=0x7f744c076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:57.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.061+0000 7f7450f80700 1 -- 172.21.15.57:0/563422163 shutdown_connections 2023-10-08T13:49:57.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.061+0000 7f7450f80700 1 -- 172.21.15.57:0/563422163 wait complete. 2023-10-08T13:49:57.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.062+0000 7f7450f80700 1 Processor -- start 2023-10-08T13:49:57.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.062+0000 7f7450f80700 1 -- start start 2023-10-08T13:49:57.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.062+0000 7f7450f80700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c0ff640 0x7f744c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.062+0000 7f7450f80700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c100000 0x7f744c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f7450f80700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f744c105630 0x7f744c1a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f7450f80700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c198790 con 0x7f744c105630 2023-10-08T13:49:57.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f7450f80700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c1a2fc0 con 0x7f744c100000 2023-10-08T13:49:57.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f7450f80700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c1a3100 con 0x7f744c0ff640 2023-10-08T13:49:57.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f744ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f744c105630 0x7f744c1a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f744ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f744c105630 0x7f744c1a2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54166/0 (socket says 172.21.15.57:54166) 2023-10-08T13:49:57.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f744ad9d700 1 -- 172.21.15.57:0/536510579 learned_addr learned my addr 172.21.15.57:0/536510579 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:57.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f744a59c700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c0ff640 0x7f744c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f7449d9b700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c100000 0x7f744c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f744ad9d700 1 -- 172.21.15.57:0/536510579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c0ff640 msgr2=0x7f744c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f744ad9d700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c0ff640 0x7f744c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f744ad9d700 1 -- 172.21.15.57:0/536510579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c100000 msgr2=0x7f744c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f744ad9d700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c100000 0x7f744c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f744ad9d700 1 -- 172.21.15.57:0/536510579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c0095f0 con 0x7f744c105630 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f744ad9d700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f744c105630 0x7f744c1a2a80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f744400ef60 tx=0x7f744400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f74437fe700 1 -- 172.21.15.57:0/536510579 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444018070 con 0x7f744c105630 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f74437fe700 1 -- 172.21.15.57:0/536510579 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f744400f040 con 0x7f744c105630 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f74437fe700 1 -- 172.21.15.57:0/536510579 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744400ccc0 con 0x7f744c105630 2023-10-08T13:49:57.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f7450f80700 1 -- 172.21.15.57:0/536510579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744c1a3300 con 0x7f744c105630 2023-10-08T13:49:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.063+0000 7f7450f80700 1 -- 172.21.15.57:0/536510579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c1a37c0 con 0x7f744c105630 2023-10-08T13:49:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.064+0000 7f7450f80700 1 -- 172.21.15.57:0/536510579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744c107c20 con 0x7f744c105630 2023-10-08T13:49:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.065+0000 7f74437fe700 1 -- 172.21.15.57:0/536510579 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7444003a00 con 0x7f744c105630 2023-10-08T13:49:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.066+0000 7f74437fe700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74380a14d0 0x7f74380a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.066+0000 7f74437fe700 1 -- 172.21.15.57:0/536510579 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f74440cd430 con 0x7f744c105630 2023-10-08T13:49:57.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.066+0000 7f744a59c700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74380a14d0 0x7f74380a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.067+0000 7f744a59c700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74380a14d0 0x7f74380a3990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b2e0 tx=0x7f743c005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:57.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.070+0000 7f74437fe700 1 -- 172.21.15.57:0/536510579 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7444096850 con 0x7f744c105630 2023-10-08T13:49:57.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.279+0000 7f7450f80700 1 -- 172.21.15.57:0/536510579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f744c069230 con 0x7f744c105630 2023-10-08T13:49:57.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.279+0000 7f74437fe700 1 -- 172.21.15.57:0/536510579 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7444099e70 con 0x7f744c105630 2023-10-08T13:49:57.280 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:49:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 -- 172.21.15.57:0/536510579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74380a14d0 msgr2=0x7f74380a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74380a14d0 0x7f74380a3990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b2e0 tx=0x7f743c005fd0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 -- 172.21.15.57:0/536510579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f744c105630 msgr2=0x7f744c1a2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f744c105630 0x7f744c1a2a80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f744400ef60 tx=0x7f744400ef90 comp rx=0 tx=0).stop 2023-10-08T13:49:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 -- 172.21.15.57:0/536510579 shutdown_connections 2023-10-08T13:49:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f744c105630 0x7f744c1a2a80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f744c0ff640 0x7f744c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74380a14d0 0x7f74380a3990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 --2- 172.21.15.57:0/536510579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c100000 0x7f744c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 -- 172.21.15.57:0/536510579 >> 172.21.15.57:0/536510579 conn(0x7f744c076790 msgr2=0x7f744c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 -- 172.21.15.57:0/536510579 shutdown_connections 2023-10-08T13:49:57.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.281+0000 7f7450f80700 1 -- 172.21.15.57:0/536510579 wait complete. 2023-10-08T13:49:57.294 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T13:49:57.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.422+0000 7f7bf24b1700 1 -- 172.21.15.57:0/2225158531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0ff620 msgr2=0x7f7bec0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.422+0000 7f7bf24b1700 1 --2- 172.21.15.57:0/2225158531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0ff620 0x7f7bec0ffaa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009990 tx=0x7f7bd4009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.422+0000 7f7bf24b1700 1 -- 172.21.15.57:0/2225158531 shutdown_connections 2023-10-08T13:49:57.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.422+0000 7f7bf24b1700 1 --2- 172.21.15.57:0/2225158531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bec0fffe0 0x7f7bec10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.422+0000 7f7bf24b1700 1 --2- 172.21.15.57:0/2225158531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec1056b0 0x7f7bec105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.422+0000 7f7bf24b1700 1 --2- 172.21.15.57:0/2225158531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0ff620 0x7f7bec0ffaa0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.422+0000 7f7bf24b1700 1 -- 172.21.15.57:0/2225158531 >> 172.21.15.57:0/2225158531 conn(0x7f7bec076780 msgr2=0x7f7bec076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:57.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.422+0000 7f7bf24b1700 1 -- 172.21.15.57:0/2225158531 shutdown_connections 2023-10-08T13:49:57.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.422+0000 7f7bf24b1700 1 -- 172.21.15.57:0/2225158531 wait complete. 2023-10-08T13:49:57.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7bf24b1700 1 Processor -- start 2023-10-08T13:49:57.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7bf24b1700 1 -- start start 2023-10-08T13:49:57.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7bf24b1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bec0ff620 0x7f7bec197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7bf24b1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0fffe0 0x7f7bec197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7bf24b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec1056b0 0x7f7bec198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7bf24b1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec1a2cc0 con 0x7f7bec0fffe0 2023-10-08T13:49:57.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7bf24b1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec1a2e30 con 0x7f7bec0ff620 2023-10-08T13:49:57.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7bf24b1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec1a2ff0 con 0x7f7bec1056b0 2023-10-08T13:49:57.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7beb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0fffe0 0x7f7bec197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7beb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0fffe0 0x7f7bec197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54196/0 (socket says 172.21.15.57:54196) 2023-10-08T13:49:57.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7beb7fe700 1 -- 172.21.15.57:0/1040382529 learned_addr learned my addr 172.21.15.57:0/1040382529 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:57.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7bebfff700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bec0ff620 0x7f7bec197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.423+0000 7f7bf0a4e700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec1056b0 0x7f7bec198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7beb7fe700 1 -- 172.21.15.57:0/1040382529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec1056b0 msgr2=0x7f7bec198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7beb7fe700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec1056b0 0x7f7bec198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7beb7fe700 1 -- 172.21.15.57:0/1040382529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bec0ff620 msgr2=0x7f7bec197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7beb7fe700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bec0ff620 0x7f7bec197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7beb7fe700 1 -- 172.21.15.57:0/1040382529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd40095f0 con 0x7f7bec0fffe0 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7bf0a4e700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec1056b0 0x7f7bec198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7beb7fe700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0fffe0 0x7f7bec197f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4005910 tx=0x7f7bd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7be97fa700 1 -- 172.21.15.57:0/1040382529 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd401d070 con 0x7f7bec0fffe0 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7be97fa700 1 -- 172.21.15.57:0/1040382529 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd4022470 con 0x7f7bec0fffe0 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7bf24b1700 1 -- 172.21.15.57:0/1040382529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec1a3270 con 0x7f7bec0fffe0 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7be97fa700 1 -- 172.21.15.57:0/1040382529 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4017650 con 0x7f7bec0fffe0 2023-10-08T13:49:57.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.424+0000 7f7bf24b1700 1 -- 172.21.15.57:0/1040382529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec1a37a0 con 0x7f7bec0fffe0 2023-10-08T13:49:57.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.425+0000 7f7bf24b1700 1 -- 172.21.15.57:0/1040382529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bec107c30 con 0x7f7bec0fffe0 2023-10-08T13:49:57.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.426+0000 7f7be97fa700 1 -- 172.21.15.57:0/1040382529 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7bd40225e0 con 0x7f7bec0fffe0 2023-10-08T13:49:57.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.427+0000 7f7be97fa700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bd80a1550 0x7f7bd80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.427+0000 7f7be97fa700 1 -- 172.21.15.57:0/1040382529 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f7bd40ce790 con 0x7f7bec0fffe0 2023-10-08T13:49:57.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.427+0000 7f7bebfff700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bd80a1550 0x7f7bd80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.427+0000 7f7bebfff700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bd80a1550 0x7f7bd80a3a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7bec077ab0 tx=0x7f7bdc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:57.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.430+0000 7f7be97fa700 1 -- 172.21.15.57:0/1040382529 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7bd4097b50 con 0x7f7bec0fffe0 2023-10-08T13:49:57.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:57 smithi057 ceph-mon[99476]: pgmap v247: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2023-10-08T13:49:57.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:57 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3760084972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:57.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:57 smithi099 ceph-mon[103849]: pgmap v247: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2023-10-08T13:49:57.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:57 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3760084972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:57.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.634+0000 7f7bf24b1700 1 -- 172.21.15.57:0/1040382529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7bec067df0 con 0x7f7bec0fffe0 2023-10-08T13:49:57.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.634+0000 7f7be97fa700 1 -- 172.21.15.57:0/1040382529 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7bd409b170 con 0x7f7bec0fffe0 2023-10-08T13:49:57.636 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:49:57.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 -- 172.21.15.57:0/1040382529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bd80a1550 msgr2=0x7f7bd80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bd80a1550 0x7f7bd80a3a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7bec077ab0 tx=0x7f7bdc009260 comp rx=0 tx=0).stop 2023-10-08T13:49:57.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 -- 172.21.15.57:0/1040382529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0fffe0 msgr2=0x7f7bec197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0fffe0 0x7f7bec197f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4005910 tx=0x7f7bd4004730 comp rx=0 tx=0).stop 2023-10-08T13:49:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 -- 172.21.15.57:0/1040382529 shutdown_connections 2023-10-08T13:49:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec1056b0 0x7f7bec198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bd80a1550 0x7f7bd80a3a10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bec0ff620 0x7f7bec197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 --2- 172.21.15.57:0/1040382529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0fffe0 0x7f7bec197f40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 -- 172.21.15.57:0/1040382529 >> 172.21.15.57:0/1040382529 conn(0x7f7bec076780 msgr2=0x7f7bec0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:57.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 -- 172.21.15.57:0/1040382529 shutdown_connections 2023-10-08T13:49:57.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.637+0000 7f7bf24b1700 1 -- 172.21.15.57:0/1040382529 wait complete. 2023-10-08T13:49:57.650 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T13:49:57.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:57 smithi115 ceph-mon[104408]: pgmap v247: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2023-10-08T13:49:57.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:57 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3760084972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:57.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.779+0000 7f0a1b131700 1 -- 172.21.15.57:0/932085694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a14106750 msgr2=0x7f0a14106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.779+0000 7f0a1b131700 1 --2- 172.21.15.57:0/932085694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a14106750 0x7f0a14106b30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009990 tx=0x7f0a08009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.779+0000 7f0a1b131700 1 -- 172.21.15.57:0/932085694 shutdown_connections 2023-10-08T13:49:57.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.779+0000 7f0a1b131700 1 --2- 172.21.15.57:0/932085694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a14101100 0x7f0a1410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.779+0000 7f0a1b131700 1 --2- 172.21.15.57:0/932085694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a14106750 0x7f0a14106b30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.779+0000 7f0a1b131700 1 --2- 172.21.15.57:0/932085694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a14100740 0x7f0a14100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.779+0000 7f0a1b131700 1 -- 172.21.15.57:0/932085694 >> 172.21.15.57:0/932085694 conn(0x7f0a140fc230 msgr2=0x7f0a140fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.779+0000 7f0a1b131700 1 -- 172.21.15.57:0/932085694 shutdown_connections 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.779+0000 7f0a1b131700 1 -- 172.21.15.57:0/932085694 wait complete. 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a1b131700 1 Processor -- start 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a1b131700 1 -- start start 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a1b131700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a14100740 0x7f0a1419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a1b131700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a14101100 0x7f0a1419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a1b131700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a14106750 0x7f0a14196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a1b131700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14196fe0 con 0x7f0a14106750 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a1b131700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14197150 con 0x7f0a14101100 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a1b131700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14197310 con 0x7f0a14100740 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a196ce700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a14106750 0x7f0a14196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a196ce700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a14106750 0x7f0a14196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54214/0 (socket says 172.21.15.57:54214) 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a196ce700 1 -- 172.21.15.57:0/4071869817 learned_addr learned my addr 172.21.15.57:0/4071869817 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a13fff700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a14101100 0x7f0a1419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a18ecd700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a14100740 0x7f0a1419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a196ce700 1 -- 172.21.15.57:0/4071869817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a14100740 msgr2=0x7f0a1419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a196ce700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a14100740 0x7f0a1419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a196ce700 1 -- 172.21.15.57:0/4071869817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a14101100 msgr2=0x7f0a1419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a196ce700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a14101100 0x7f0a1419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.780+0000 7f0a196ce700 1 -- 172.21.15.57:0/4071869817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080095f0 con 0x7f0a14106750 2023-10-08T13:49:57.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.781+0000 7f0a196ce700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a14106750 0x7f0a14196980 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00ef30 tx=0x7f0a0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:57.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.781+0000 7f0a11ffb700 1 -- 172.21.15.57:0/4071869817 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c018070 con 0x7f0a14106750 2023-10-08T13:49:57.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.781+0000 7f0a11ffb700 1 -- 172.21.15.57:0/4071869817 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a0c00f040 con 0x7f0a14106750 2023-10-08T13:49:57.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.781+0000 7f0a1b131700 1 -- 172.21.15.57:0/4071869817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a14197590 con 0x7f0a14106750 2023-10-08T13:49:57.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.781+0000 7f0a11ffb700 1 -- 172.21.15.57:0/4071869817 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c00cdb0 con 0x7f0a14106750 2023-10-08T13:49:57.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.781+0000 7f0a1b131700 1 -- 172.21.15.57:0/4071869817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a141a3840 con 0x7f0a14106750 2023-10-08T13:49:57.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.782+0000 7f0a1b131700 1 -- 172.21.15.57:0/4071869817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a14108d40 con 0x7f0a14106750 2023-10-08T13:49:57.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.783+0000 7f0a11ffb700 1 -- 172.21.15.57:0/4071869817 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f0a0c003a00 con 0x7f0a14106750 2023-10-08T13:49:57.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.783+0000 7f0a11ffb700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a040a1550 0x7f0a040a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:57.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.784+0000 7f0a18ecd700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a040a1550 0x7f0a040a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:57.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.784+0000 7f0a11ffb700 1 -- 172.21.15.57:0/4071869817 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f0a0c0ccb80 con 0x7f0a14106750 2023-10-08T13:49:57.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.784+0000 7f0a18ecd700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a040a1550 0x7f0a040a3a10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0a08006010 tx=0x7f0a0800b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:57.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.788+0000 7f0a11ffb700 1 -- 172.21.15.57:0/4071869817 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0a0c095f40 con 0x7f0a14106750 2023-10-08T13:49:57.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.988+0000 7f0a1b131700 1 -- 172.21.15.57:0/4071869817 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a14197e40 con 0x7f0a14106750 2023-10-08T13:49:57.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.989+0000 7f0a11ffb700 1 -- 172.21.15.57:0/4071869817 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a0c099560 con 0x7f0a14106750 2023-10-08T13:49:57.990 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T13:49:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 -- 172.21.15.57:0/4071869817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a040a1550 msgr2=0x7f0a040a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a040a1550 0x7f0a040a3a10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0a08006010 tx=0x7f0a0800b260 comp rx=0 tx=0).stop 2023-10-08T13:49:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 -- 172.21.15.57:0/4071869817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a14106750 msgr2=0x7f0a14196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a14106750 0x7f0a14196980 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00ef30 tx=0x7f0a0c00c3a0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 -- 172.21.15.57:0/4071869817 shutdown_connections 2023-10-08T13:49:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a14106750 0x7f0a14196980 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a14100740 0x7f0a1419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0a040a1550 0x7f0a040a3a10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 --2- 172.21.15.57:0/4071869817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a14101100 0x7f0a1419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 -- 172.21.15.57:0/4071869817 >> 172.21.15.57:0/4071869817 conn(0x7f0a140fc230 msgr2=0x7f0a140fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 -- 172.21.15.57:0/4071869817 shutdown_connections 2023-10-08T13:49:57.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:57.991+0000 7f0a1b131700 1 -- 172.21.15.57:0/4071869817 wait complete. 2023-10-08T13:49:58.004 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T13:49:58.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.131+0000 7f52be102700 1 -- 172.21.15.57:0/4221610311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b8106750 msgr2=0x7f52b8106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.131+0000 7f52be102700 1 --2- 172.21.15.57:0/4221610311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b8106750 0x7f52b8106b30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009990 tx=0x7f52a8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.131+0000 7f52be102700 1 -- 172.21.15.57:0/4221610311 shutdown_connections 2023-10-08T13:49:58.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.131+0000 7f52be102700 1 --2- 172.21.15.57:0/4221610311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8101100 0x7f52b810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.131+0000 7f52be102700 1 --2- 172.21.15.57:0/4221610311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b8106750 0x7f52b8106b30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.131+0000 7f52be102700 1 --2- 172.21.15.57:0/4221610311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52b8100740 0x7f52b8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.131+0000 7f52be102700 1 -- 172.21.15.57:0/4221610311 >> 172.21.15.57:0/4221610311 conn(0x7f52b80fc230 msgr2=0x7f52b80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:58.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.131+0000 7f52be102700 1 -- 172.21.15.57:0/4221610311 shutdown_connections 2023-10-08T13:49:58.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.131+0000 7f52be102700 1 -- 172.21.15.57:0/4221610311 wait complete. 2023-10-08T13:49:58.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52be102700 1 Processor -- start 2023-10-08T13:49:58.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52be102700 1 -- start start 2023-10-08T13:49:58.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52be102700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52b8100740 0x7f52b8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52be102700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b8101100 0x7f52b8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52be102700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8106750 0x7f52b8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52be102700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b81a4f50 con 0x7f52b8101100 2023-10-08T13:49:58.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52be102700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b81a50c0 con 0x7f52b8100740 2023-10-08T13:49:58.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52be102700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b81a5280 con 0x7f52b8106750 2023-10-08T13:49:58.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52b6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b8101100 0x7f52b8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52b6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b8101100 0x7f52b8073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54226/0 (socket says 172.21.15.57:54226) 2023-10-08T13:49:58.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.132+0000 7f52b6ffd700 1 -- 172.21.15.57:0/4108932680 learned_addr learned my addr 172.21.15.57:0/4108932680 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:58.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b7fff700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8106750 0x7f52b8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b77fe700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52b8100740 0x7f52b8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b6ffd700 1 -- 172.21.15.57:0/4108932680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8106750 msgr2=0x7f52b8073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b6ffd700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8106750 0x7f52b8073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b6ffd700 1 -- 172.21.15.57:0/4108932680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52b8100740 msgr2=0x7f52b8072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b6ffd700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52b8100740 0x7f52b8072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b6ffd700 1 -- 172.21.15.57:0/4108932680 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a80095f0 con 0x7f52b8101100 2023-10-08T13:49:58.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b6ffd700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b8101100 0x7f52b8073030 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00d980 tx=0x7f52ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:58.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b4ff9700 1 -- 172.21.15.57:0/4108932680 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac004220 con 0x7f52b8101100 2023-10-08T13:49:58.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52be102700 1 -- 172.21.15.57:0/4108932680 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b81a5560 con 0x7f52b8101100 2023-10-08T13:49:58.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b4ff9700 1 -- 172.21.15.57:0/4108932680 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52ac007640 con 0x7f52b8101100 2023-10-08T13:49:58.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52b4ff9700 1 -- 172.21.15.57:0/4108932680 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac005690 con 0x7f52b8101100 2023-10-08T13:49:58.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.133+0000 7f52be102700 1 -- 172.21.15.57:0/4108932680 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b81a5a70 con 0x7f52b8101100 2023-10-08T13:49:58.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.134+0000 7f52be102700 1 -- 172.21.15.57:0/4108932680 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b8108d40 con 0x7f52b8101100 2023-10-08T13:49:58.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.135+0000 7f52b4ff9700 1 -- 172.21.15.57:0/4108932680 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f52ac004750 con 0x7f52b8101100 2023-10-08T13:49:58.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.136+0000 7f52b4ff9700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a00a14d0 0x7f52a00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.136+0000 7f52b4ff9700 1 -- 172.21.15.57:0/4108932680 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f52ac0cd2d0 con 0x7f52b8101100 2023-10-08T13:49:58.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.136+0000 7f52b77fe700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a00a14d0 0x7f52a00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.136+0000 7f52b77fe700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a00a14d0 0x7f52a00a3990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52a800b2e0 tx=0x7f52a8005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:58.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.139+0000 7f52b4ff9700 1 -- 172.21.15.57:0/4108932680 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f52ac096660 con 0x7f52b8101100 2023-10-08T13:49:58.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.342+0000 7f52be102700 1 -- 172.21.15.57:0/4108932680 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f52b8073fd0 con 0x7f52b8101100 2023-10-08T13:49:58.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.342+0000 7f52b4ff9700 1 -- 172.21.15.57:0/4108932680 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f52ac0160a0 con 0x7f52b8101100 2023-10-08T13:49:58.344 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:49:58.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.344+0000 7f52be102700 1 -- 172.21.15.57:0/4108932680 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a00a14d0 msgr2=0x7f52a00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.344+0000 7f52be102700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a00a14d0 0x7f52a00a3990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52a800b2e0 tx=0x7f52a8005fd0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.344+0000 7f52be102700 1 -- 172.21.15.57:0/4108932680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b8101100 msgr2=0x7f52b8073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.344+0000 7f52be102700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b8101100 0x7f52b8073030 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00d980 tx=0x7f52ac00dc90 comp rx=0 tx=0).stop 2023-10-08T13:49:58.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.344+0000 7f52be102700 1 -- 172.21.15.57:0/4108932680 shutdown_connections 2023-10-08T13:49:58.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.344+0000 7f52be102700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52b8106750 0x7f52b8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.344+0000 7f52be102700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a00a14d0 0x7f52a00a3990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.344+0000 7f52be102700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52b8100740 0x7f52b8072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.344+0000 7f52be102700 1 --2- 172.21.15.57:0/4108932680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b8101100 0x7f52b8073030 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.344+0000 7f52be102700 1 -- 172.21.15.57:0/4108932680 >> 172.21.15.57:0/4108932680 conn(0x7f52b80fc230 msgr2=0x7f52b80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.345+0000 7f52be102700 1 -- 172.21.15.57:0/4108932680 shutdown_connections 2023-10-08T13:49:58.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.345+0000 7f52be102700 1 -- 172.21.15.57:0/4108932680 wait complete. 2023-10-08T13:49:58.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:58 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/536510579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:58.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:58 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1040382529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:49:58.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:58 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4071869817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:49:58.358 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T13:49:58.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.484+0000 7f27d0242700 1 -- 172.21.15.57:0/2283474494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27c80fffe0 msgr2=0x7f27c810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.484+0000 7f27d0242700 1 --2- 172.21.15.57:0/2283474494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27c80fffe0 0x7f27c810c4e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f27c000b0a0 tx=0x7f27c000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.484+0000 7f27d0242700 1 -- 172.21.15.57:0/2283474494 shutdown_connections 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.484+0000 7f27d0242700 1 --2- 172.21.15.57:0/2283474494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27c80fffe0 0x7f27c810c4e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.484+0000 7f27d0242700 1 --2- 172.21.15.57:0/2283474494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27c81056b0 0x7f27c8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.484+0000 7f27d0242700 1 --2- 172.21.15.57:0/2283474494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c80ff620 0x7f27c80ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.484+0000 7f27d0242700 1 -- 172.21.15.57:0/2283474494 >> 172.21.15.57:0/2283474494 conn(0x7f27c8076780 msgr2=0x7f27c8076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.484+0000 7f27d0242700 1 -- 172.21.15.57:0/2283474494 shutdown_connections 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.484+0000 7f27d0242700 1 -- 172.21.15.57:0/2283474494 wait complete. 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27d0242700 1 Processor -- start 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27d0242700 1 -- start start 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27d0242700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27c80ff620 0x7f27c8197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27d0242700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c80fffe0 0x7f27c8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27d0242700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27c81056b0 0x7f27c81a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27d0242700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81986d0 con 0x7f27c81056b0 2023-10-08T13:49:58.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27d0242700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81a2f00 con 0x7f27c80fffe0 2023-10-08T13:49:58.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27d0242700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81a3040 con 0x7f27c80ff620 2023-10-08T13:49:58.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27ce7df700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27c81056b0 0x7f27c81a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27ce7df700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27c81056b0 0x7f27c81a29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54240/0 (socket says 172.21.15.57:54240) 2023-10-08T13:49:58.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27ce7df700 1 -- 172.21.15.57:0/540736009 learned_addr learned my addr 172.21.15.57:0/540736009 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:58.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27cd7dd700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c80fffe0 0x7f27c8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27cdfde700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27c80ff620 0x7f27c8197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27ce7df700 1 -- 172.21.15.57:0/540736009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27c80ff620 msgr2=0x7f27c8197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27ce7df700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27c80ff620 0x7f27c8197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27ce7df700 1 -- 172.21.15.57:0/540736009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c80fffe0 msgr2=0x7f27c8197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27ce7df700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c80fffe0 0x7f27c8197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27ce7df700 1 -- 172.21.15.57:0/540736009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c0009d00 con 0x7f27c81056b0 2023-10-08T13:49:58.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.485+0000 7f27ce7df700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27c81056b0 0x7f27c81a29c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f27c000bfa0 tx=0x7f27c0008ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:58.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.486+0000 7f27beffd700 1 -- 172.21.15.57:0/540736009 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000e040 con 0x7f27c81056b0 2023-10-08T13:49:58.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.486+0000 7f27d0242700 1 -- 172.21.15.57:0/540736009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c81a3240 con 0x7f27c81056b0 2023-10-08T13:49:58.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.486+0000 7f27beffd700 1 -- 172.21.15.57:0/540736009 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c0003c00 con 0x7f27c81056b0 2023-10-08T13:49:58.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.486+0000 7f27beffd700 1 -- 172.21.15.57:0/540736009 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00052a0 con 0x7f27c81056b0 2023-10-08T13:49:58.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.486+0000 7f27d0242700 1 -- 172.21.15.57:0/540736009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c81a36c0 con 0x7f27c81056b0 2023-10-08T13:49:58.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.487+0000 7f27d0242700 1 -- 172.21.15.57:0/540736009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c8107c30 con 0x7f27c81056b0 2023-10-08T13:49:58.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.488+0000 7f27beffd700 1 -- 172.21.15.57:0/540736009 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f27c001d470 con 0x7f27c81056b0 2023-10-08T13:49:58.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.488+0000 7f27beffd700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27b40a14d0 0x7f27b40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.489+0000 7f27cdfde700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27b40a14d0 0x7f27b40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.489+0000 7f27beffd700 1 -- 172.21.15.57:0/540736009 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f27c00cebc0 con 0x7f27c81056b0 2023-10-08T13:49:58.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.489+0000 7f27cdfde700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27b40a14d0 0x7f27b40a3990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005950 tx=0x7f27c40058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:58.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.492+0000 7f27beffd700 1 -- 172.21.15.57:0/540736009 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f27c0097f80 con 0x7f27c81056b0 2023-10-08T13:49:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:58 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/536510579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:58 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1040382529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:49:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:58 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4071869817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:49:58.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:58 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/536510579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:49:58.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:58 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1040382529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:49:58.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:58 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4071869817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:49:58.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.709+0000 7f27d0242700 1 -- 172.21.15.57:0/540736009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f27c8100d50 con 0x7f27c81056b0 2023-10-08T13:49:58.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.709+0000 7f27beffd700 1 -- 172.21.15.57:0/540736009 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v88) v1 ==== 81+0+882 (secure 0 0 0) 0x7f27c0024350 con 0x7f27c81056b0 2023-10-08T13:49:58.710 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/466175178 2023-10-09T13:44:14.025065+0000 2023-10-08T13:49:58.710 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1455537828 2023-10-09T13:44:14.025065+0000 2023-10-08T13:49:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/2534706720 2023-10-09T13:44:14.025065+0000 2023-10-08T13:49:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/2534706720 2023-10-09T13:44:14.025065+0000 2023-10-08T13:49:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1379042066 2023-10-09T13:44:14.025065+0000 2023-10-08T13:49:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/19108071 2023-10-09T13:43:06.093214+0000 2023-10-08T13:49:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3114287016 2023-10-09T13:43:06.093214+0000 2023-10-08T13:49:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3318653521 2023-10-09T13:43:06.093214+0000 2023-10-08T13:49:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2210095932 2023-10-09T13:42:39.884205+0000 2023-10-08T13:49:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/955690770 2023-10-09T13:42:39.884205+0000 2023-10-08T13:49:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/140726743 2023-10-09T13:42:39.884205+0000 2023-10-08T13:49:58.711 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/2406655290 2023-10-09T13:43:06.093214+0000 2023-10-08T13:49:58.712 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2229924806 2023-10-09T13:42:39.884205+0000 2023-10-08T13:49:58.712 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/140726743 2023-10-09T13:42:39.884205+0000 2023-10-08T13:49:58.712 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/2406655290 2023-10-09T13:43:06.093214+0000 2023-10-08T13:49:58.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 -- 172.21.15.57:0/540736009 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27b40a14d0 msgr2=0x7f27b40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27b40a14d0 0x7f27b40a3990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005950 tx=0x7f27c40058e0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 -- 172.21.15.57:0/540736009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27c81056b0 msgr2=0x7f27c81a29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27c81056b0 0x7f27c81a29c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f27c000bfa0 tx=0x7f27c0008ff0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 -- 172.21.15.57:0/540736009 shutdown_connections 2023-10-08T13:49:58.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27c81056b0 0x7f27c81a29c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27c80ff620 0x7f27c8197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27b40a14d0 0x7f27b40a3990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 --2- 172.21.15.57:0/540736009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27c80fffe0 0x7f27c8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 -- 172.21.15.57:0/540736009 >> 172.21.15.57:0/540736009 conn(0x7f27c8076780 msgr2=0x7f27c80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:58.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 -- 172.21.15.57:0/540736009 shutdown_connections 2023-10-08T13:49:58.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.711+0000 7f27d0242700 1 -- 172.21.15.57:0/540736009 wait complete. 2023-10-08T13:49:58.714 INFO:teuthology.orchestra.run.smithi057.stderr:listed 15 entries 2023-10-08T13:49:58.724 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:49:58.724 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T13:49:58.743 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:49:58.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.909+0000 7eff519ab700 1 -- 172.21.15.57:0/3354819842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff4c102100 msgr2=0x7eff4c10a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.909+0000 7eff519ab700 1 --2- 172.21.15.57:0/3354819842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff4c102100 0x7eff4c10a5e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7eff40009990 tx=0x7eff40009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.909+0000 7eff519ab700 1 -- 172.21.15.57:0/3354819842 shutdown_connections 2023-10-08T13:49:58.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.909+0000 7eff519ab700 1 --2- 172.21.15.57:0/3354819842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff4c10ac00 0x7eff4c10d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.909+0000 7eff519ab700 1 --2- 172.21.15.57:0/3354819842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff4c1017e0 0x7eff4c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.909+0000 7eff519ab700 1 --2- 172.21.15.57:0/3354819842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff4c102100 0x7eff4c10a5e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.909+0000 7eff519ab700 1 -- 172.21.15.57:0/3354819842 >> 172.21.15.57:0/3354819842 conn(0x7eff4c0fb040 msgr2=0x7eff4c0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:58.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.909+0000 7eff519ab700 1 -- 172.21.15.57:0/3354819842 shutdown_connections 2023-10-08T13:49:58.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.909+0000 7eff519ab700 1 -- 172.21.15.57:0/3354819842 wait complete. 2023-10-08T13:49:58.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.909+0000 7eff519ab700 1 Processor -- start 2023-10-08T13:49:58.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff519ab700 1 -- start start 2023-10-08T13:49:58.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff519ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff4c1017e0 0x7eff4c0ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff519ab700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff4c102100 0x7eff4c0ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff519ab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff4c10ac00 0x7eff4c0ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff519ab700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c1a4ed0 con 0x7eff4c10ac00 2023-10-08T13:49:58.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff519ab700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c1a5040 con 0x7eff4c102100 2023-10-08T13:49:58.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff519ab700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c1a5200 con 0x7eff4c1017e0 2023-10-08T13:49:58.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff4c10ac00 0x7eff4c0ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff4c10ac00 0x7eff4c0ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54254/0 (socket says 172.21.15.57:54254) 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4b7fe700 1 -- 172.21.15.57:0/810525665 learned_addr learned my addr 172.21.15.57:0/810525665 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4affd700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff4c1017e0 0x7eff4c0ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4a7fc700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff4c102100 0x7eff4c0ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4b7fe700 1 -- 172.21.15.57:0/810525665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff4c1017e0 msgr2=0x7eff4c0ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4b7fe700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff4c1017e0 0x7eff4c0ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4b7fe700 1 -- 172.21.15.57:0/810525665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff4c102100 msgr2=0x7eff4c0ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4b7fe700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff4c102100 0x7eff4c0ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4b7fe700 1 -- 172.21.15.57:0/810525665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400095f0 con 0x7eff4c10ac00 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.910+0000 7eff4b7fe700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff4c10ac00 0x7eff4c0ffeb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7eff44007ee0 tx=0x7eff4400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.911+0000 7eff509a9700 1 -- 172.21.15.57:0/810525665 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff44017070 con 0x7eff4c10ac00 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.911+0000 7eff519ab700 1 -- 172.21.15.57:0/810525665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4c1a54e0 con 0x7eff4c10ac00 2023-10-08T13:49:58.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.911+0000 7eff509a9700 1 -- 172.21.15.57:0/810525665 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff4400f040 con 0x7eff4c10ac00 2023-10-08T13:49:58.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.911+0000 7eff509a9700 1 -- 172.21.15.57:0/810525665 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff44005580 con 0x7eff4c10ac00 2023-10-08T13:49:58.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.911+0000 7eff519ab700 1 -- 172.21.15.57:0/810525665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4c1a59c0 con 0x7eff4c10ac00 2023-10-08T13:49:58.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.912+0000 7eff519ab700 1 -- 172.21.15.57:0/810525665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4c105cd0 con 0x7eff4c10ac00 2023-10-08T13:49:58.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.913+0000 7eff509a9700 1 -- 172.21.15.57:0/810525665 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7eff4400db20 con 0x7eff4c10ac00 2023-10-08T13:49:58.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.913+0000 7eff509a9700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff340a14d0 0x7eff340a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:58.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.913+0000 7eff4affd700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff340a14d0 0x7eff340a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:58.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.913+0000 7eff509a9700 1 -- 172.21.15.57:0/810525665 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7eff440ccd70 con 0x7eff4c10ac00 2023-10-08T13:49:58.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.914+0000 7eff4affd700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff340a14d0 0x7eff340a3990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009c60 tx=0x7eff3c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:58.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:58.917+0000 7eff509a9700 1 -- 172.21.15.57:0/810525665 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7eff44096100 con 0x7eff4c10ac00 2023-10-08T13:49:59.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.123+0000 7eff519ab700 1 -- 172.21.15.57:0/810525665 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff4c1009b0 con 0x7eff4c10ac00 2023-10-08T13:49:59.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.125+0000 7eff509a9700 1 -- 172.21.15.57:0/810525665 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24224 (secure 0 0 0) 0x7eff44099720 con 0x7eff4c10ac00 2023-10-08T13:49:59.127 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:49:59.127 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":88,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:49.574582+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:49:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 -- 172.21.15.57:0/810525665 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff340a14d0 msgr2=0x7eff340a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff340a14d0 0x7eff340a3990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009c60 tx=0x7eff3c009260 comp rx=0 tx=0).stop 2023-10-08T13:49:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 -- 172.21.15.57:0/810525665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff4c10ac00 msgr2=0x7eff4c0ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff4c10ac00 0x7eff4c0ffeb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7eff44007ee0 tx=0x7eff4400d3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 -- 172.21.15.57:0/810525665 shutdown_connections 2023-10-08T13:49:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7eff4c10ac00 0x7eff4c0ffeb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff4c1017e0 0x7eff4c0ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7eff340a14d0 0x7eff340a3990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 --2- 172.21.15.57:0/810525665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff4c102100 0x7eff4c0ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 -- 172.21.15.57:0/810525665 >> 172.21.15.57:0/810525665 conn(0x7eff4c0fb040 msgr2=0x7eff4c0fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 -- 172.21.15.57:0/810525665 shutdown_connections 2023-10-08T13:49:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.127+0000 7eff519ab700 1 -- 172.21.15.57:0/810525665 wait complete. 2023-10-08T13:49:59.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T13:49:59.141 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:49:59.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.269+0000 7f522f2d0700 1 -- 172.21.15.57:0/3597590573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52281056b0 msgr2=0x7f5228105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.269+0000 7f522f2d0700 1 --2- 172.21.15.57:0/3597590573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52281056b0 0x7f5228105a90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5224009990 tx=0x7f5224009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.269+0000 7f522f2d0700 1 -- 172.21.15.57:0/3597590573 shutdown_connections 2023-10-08T13:49:59.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.269+0000 7f522f2d0700 1 --2- 172.21.15.57:0/3597590573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52280fffe0 0x7f522810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.269+0000 7f522f2d0700 1 --2- 172.21.15.57:0/3597590573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52281056b0 0x7f5228105a90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.269+0000 7f522f2d0700 1 --2- 172.21.15.57:0/3597590573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52280ff620 0x7f52280ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.269+0000 7f522f2d0700 1 -- 172.21.15.57:0/3597590573 >> 172.21.15.57:0/3597590573 conn(0x7f5228076780 msgr2=0x7f5228076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.269+0000 7f522f2d0700 1 -- 172.21.15.57:0/3597590573 shutdown_connections 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.269+0000 7f522f2d0700 1 -- 172.21.15.57:0/3597590573 wait complete. 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522f2d0700 1 Processor -- start 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522f2d0700 1 -- start start 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522f2d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52280ff620 0x7f5228195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522f2d0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52280fffe0 0x7f5228195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522f2d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52281056b0 0x7f5228196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522f2d0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52281a4f10 con 0x7f52280ff620 2023-10-08T13:49:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522f2d0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52281a5080 con 0x7f52281056b0 2023-10-08T13:49:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522f2d0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52281a5240 con 0x7f52280fffe0 2023-10-08T13:49:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522d06c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52280ff620 0x7f5228195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522d06c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52280ff620 0x7f5228195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54264/0 (socket says 172.21.15.57:54264) 2023-10-08T13:49:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522d06c700 1 -- 172.21.15.57:0/2035372227 learned_addr learned my addr 172.21.15.57:0/2035372227 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522c86b700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52280fffe0 0x7f5228195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522d06c700 1 -- 172.21.15.57:0/2035372227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52280fffe0 msgr2=0x7f5228195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522d06c700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52280fffe0 0x7f5228195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522d06c700 1 -- 172.21.15.57:0/2035372227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52281056b0 msgr2=0x7f5228196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522d06c700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52281056b0 0x7f5228196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522d06c700 1 -- 172.21.15.57:0/2035372227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240095f0 con 0x7f52280ff620 2023-10-08T13:49:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.270+0000 7f522c86b700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52280fffe0 0x7f5228195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:49:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.271+0000 7f522d06c700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52280ff620 0x7f5228195800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5224003730 tx=0x7f522400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.271+0000 7f521a7fc700 1 -- 172.21.15.57:0/2035372227 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522401d070 con 0x7f52280ff620 2023-10-08T13:49:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.271+0000 7f521a7fc700 1 -- 172.21.15.57:0/2035372227 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52240054f0 con 0x7f52280ff620 2023-10-08T13:49:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.271+0000 7f521a7fc700 1 -- 172.21.15.57:0/2035372227 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224004bd0 con 0x7f52280ff620 2023-10-08T13:49:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.271+0000 7f522f2d0700 1 -- 172.21.15.57:0/2035372227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52281a54c0 con 0x7f52280ff620 2023-10-08T13:49:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.271+0000 7f522f2d0700 1 -- 172.21.15.57:0/2035372227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52281a5950 con 0x7f52280ff620 2023-10-08T13:49:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.272+0000 7f522f2d0700 1 -- 172.21.15.57:0/2035372227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5228107c30 con 0x7f52280ff620 2023-10-08T13:49:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.273+0000 7f521a7fc700 1 -- 172.21.15.57:0/2035372227 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f5224004d30 con 0x7f52280ff620 2023-10-08T13:49:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.273+0000 7f521a7fc700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52140a14d0 0x7f52140a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.273+0000 7f522c86b700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52140a14d0 0x7f52140a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.273+0000 7f521a7fc700 1 -- 172.21.15.57:0/2035372227 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f52240cdef0 con 0x7f52280ff620 2023-10-08T13:49:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.274+0000 7f522c86b700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52140a14d0 0x7f52140a3990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f521c005fd0 tx=0x7f521c005ee0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:59.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.278+0000 7f521a7fc700 1 -- 172.21.15.57:0/2035372227 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5224097310 con 0x7f52280ff620 2023-10-08T13:49:59.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.478+0000 7f522f2d0700 1 -- 172.21.15.57:0/2035372227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5228067d80 con 0x7f52280ff620 2023-10-08T13:49:59.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.478+0000 7f521a7fc700 1 -- 172.21.15.57:0/2035372227 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f522409fe60 con 0x7f52280ff620 2023-10-08T13:49:59.480 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:49:59.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.480+0000 7f522f2d0700 1 -- 172.21.15.57:0/2035372227 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52140a14d0 msgr2=0x7f52140a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.480+0000 7f522f2d0700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52140a14d0 0x7f52140a3990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f521c005fd0 tx=0x7f521c005ee0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.480+0000 7f522f2d0700 1 -- 172.21.15.57:0/2035372227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52280ff620 msgr2=0x7f5228195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.480+0000 7f522f2d0700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52280ff620 0x7f5228195800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5224003730 tx=0x7f522400f740 comp rx=0 tx=0).stop 2023-10-08T13:49:59.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.480+0000 7f522f2d0700 1 -- 172.21.15.57:0/2035372227 shutdown_connections 2023-10-08T13:49:59.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.480+0000 7f522f2d0700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52140a14d0 0x7f52140a3990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.480+0000 7f522f2d0700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52281056b0 0x7f5228196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.481+0000 7f522f2d0700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52280ff620 0x7f5228195800 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.481+0000 7f522f2d0700 1 --2- 172.21.15.57:0/2035372227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52280fffe0 0x7f5228195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.481+0000 7f522f2d0700 1 -- 172.21.15.57:0/2035372227 >> 172.21.15.57:0/2035372227 conn(0x7f5228076780 msgr2=0x7f52280fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:59.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.481+0000 7f522f2d0700 1 -- 172.21.15.57:0/2035372227 shutdown_connections 2023-10-08T13:49:59.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.481+0000 7f522f2d0700 1 -- 172.21.15.57:0/2035372227 wait complete. 2023-10-08T13:49:59.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:59 smithi057 ceph-mon[99476]: pgmap v248: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2023-10-08T13:49:59.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:59 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4108932680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:49:59.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:59 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/540736009' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T13:49:59.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:49:59 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/810525665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:59.494 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T13:49:59.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:59 smithi099 ceph-mon[103849]: pgmap v248: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2023-10-08T13:49:59.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:59 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4108932680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:49:59.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:59 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/540736009' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T13:49:59.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:49:59 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/810525665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:59.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 -- 172.21.15.57:0/1702955030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd450106730 msgr2=0x7fd450106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 --2- 172.21.15.57:0/1702955030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd450106730 0x7fd450106b10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd440009990 tx=0x7fd440009ca0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 -- 172.21.15.57:0/1702955030 shutdown_connections 2023-10-08T13:49:59.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 --2- 172.21.15.57:0/1702955030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4501010e0 0x7fd45010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 --2- 172.21.15.57:0/1702955030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd450106730 0x7fd450106b10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 --2- 172.21.15.57:0/1702955030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd450100720 0x7fd450100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 -- 172.21.15.57:0/1702955030 >> 172.21.15.57:0/1702955030 conn(0x7fd4500fc230 msgr2=0x7fd4500fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 -- 172.21.15.57:0/1702955030 shutdown_connections 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 -- 172.21.15.57:0/1702955030 wait complete. 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 Processor -- start 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.622+0000 7fd4580d7700 1 -- start start 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd4580d7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd450100720 0x7fd45019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd4580d7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4501010e0 0x7fd45019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd4580d7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd450106730 0x7fd4501968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd4580d7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501970c0 con 0x7fd450100720 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd4580d7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450197230 con 0x7fd4501010e0 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd4580d7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501973f0 con 0x7fd450106730 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd455e73700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd450100720 0x7fd45019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd455e73700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd450100720 0x7fd45019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54278/0 (socket says 172.21.15.57:54278) 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd455e73700 1 -- 172.21.15.57:0/3994290622 learned_addr learned my addr 172.21.15.57:0/3994290622 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd455672700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4501010e0 0x7fd45019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd456674700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd450106730 0x7fd4501968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd455e73700 1 -- 172.21.15.57:0/3994290622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd450106730 msgr2=0x7fd4501968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd455e73700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd450106730 0x7fd4501968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd455e73700 1 -- 172.21.15.57:0/3994290622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4501010e0 msgr2=0x7fd45019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd455e73700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4501010e0 0x7fd45019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd455e73700 1 -- 172.21.15.57:0/3994290622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4400095f0 con 0x7fd450100720 2023-10-08T13:49:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd455e73700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd450100720 0x7fd45019c850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd4400098e0 tx=0x7fd44000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd446ffd700 1 -- 172.21.15.57:0/3994290622 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44001d070 con 0x7fd450100720 2023-10-08T13:49:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd4580d7700 1 -- 172.21.15.57:0/3994290622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd450197670 con 0x7fd450100720 2023-10-08T13:49:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd446ffd700 1 -- 172.21.15.57:0/3994290622 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd440022470 con 0x7fd450100720 2023-10-08T13:49:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.623+0000 7fd446ffd700 1 -- 172.21.15.57:0/3994290622 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440017650 con 0x7fd450100720 2023-10-08T13:49:59.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.624+0000 7fd4580d7700 1 -- 172.21.15.57:0/3994290622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4501a37d0 con 0x7fd450100720 2023-10-08T13:49:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.625+0000 7fd4580d7700 1 -- 172.21.15.57:0/3994290622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd450108cb0 con 0x7fd450100720 2023-10-08T13:49:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.625+0000 7fd446ffd700 1 -- 172.21.15.57:0/3994290622 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd4400225e0 con 0x7fd450100720 2023-10-08T13:49:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.626+0000 7fd446ffd700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd43c0a14d0 0x7fd43c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.626+0000 7fd455672700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd43c0a14d0 0x7fd43c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.626+0000 7fd446ffd700 1 -- 172.21.15.57:0/3994290622 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fd4400ce060 con 0x7fd450100720 2023-10-08T13:49:59.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.627+0000 7fd455672700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd43c0a14d0 0x7fd43c0a3990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd450198060 tx=0x7fd44c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:59.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.631+0000 7fd446ffd700 1 -- 172.21.15.57:0/3994290622 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd440097480 con 0x7fd450100720 2023-10-08T13:49:59.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:59 smithi115 ceph-mon[104408]: pgmap v248: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2023-10-08T13:49:59.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:59 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4108932680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:49:59.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:59 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/540736009' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T13:49:59.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:49:59 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/810525665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:49:59.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.837+0000 7fd4580d7700 1 -- 172.21.15.57:0/3994290622 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd450197dd0 con 0x7fd450100720 2023-10-08T13:49:59.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.837+0000 7fd446ffd700 1 -- 172.21.15.57:0/3994290622 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd44009aaa0 con 0x7fd450100720 2023-10-08T13:49:59.839 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:49:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 -- 172.21.15.57:0/3994290622 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd43c0a14d0 msgr2=0x7fd43c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd43c0a14d0 0x7fd43c0a3990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd450198060 tx=0x7fd44c00b410 comp rx=0 tx=0).stop 2023-10-08T13:49:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 -- 172.21.15.57:0/3994290622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd450100720 msgr2=0x7fd45019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd450100720 0x7fd45019c850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd4400098e0 tx=0x7fd44000f670 comp rx=0 tx=0).stop 2023-10-08T13:49:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 -- 172.21.15.57:0/3994290622 shutdown_connections 2023-10-08T13:49:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd450106730 0x7fd4501968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd450100720 0x7fd45019c850 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd43c0a14d0 0x7fd43c0a3990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 --2- 172.21.15.57:0/3994290622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4501010e0 0x7fd45019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 -- 172.21.15.57:0/3994290622 >> 172.21.15.57:0/3994290622 conn(0x7fd4500fc230 msgr2=0x7fd4500fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:59.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 -- 172.21.15.57:0/3994290622 shutdown_connections 2023-10-08T13:49:59.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.839+0000 7fd4580d7700 1 -- 172.21.15.57:0/3994290622 wait complete. 2023-10-08T13:49:59.852 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T13:49:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.980+0000 7f74b1d61700 1 -- 172.21.15.57:0/1854737780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac101100 msgr2=0x7f74ac10d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.980+0000 7f74b1d61700 1 --2- 172.21.15.57:0/1854737780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac101100 0x7f74ac10d610 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f74a400b0a0 tx=0x7f74a400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.980+0000 7f74b1d61700 1 -- 172.21.15.57:0/1854737780 shutdown_connections 2023-10-08T13:49:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.980+0000 7f74b1d61700 1 --2- 172.21.15.57:0/1854737780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac101100 0x7f74ac10d610 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.980+0000 7f74b1d61700 1 --2- 172.21.15.57:0/1854737780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ac106750 0x7f74ac106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.980+0000 7f74b1d61700 1 --2- 172.21.15.57:0/1854737780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac100740 0x7f74ac100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.980+0000 7f74b1d61700 1 -- 172.21.15.57:0/1854737780 >> 172.21.15.57:0/1854737780 conn(0x7f74ac0fc230 msgr2=0x7f74ac0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:49:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.980+0000 7f74b1d61700 1 -- 172.21.15.57:0/1854737780 shutdown_connections 2023-10-08T13:49:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.980+0000 7f74b1d61700 1 -- 172.21.15.57:0/1854737780 wait complete. 2023-10-08T13:49:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.980+0000 7f74b1d61700 1 Processor -- start 2023-10-08T13:49:59.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74b1d61700 1 -- start start 2023-10-08T13:49:59.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74b1d61700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ac100740 0x7f74ac19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74b1d61700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac101100 0x7f74ac19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74b1d61700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac106750 0x7f74ac1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74b1d61700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac196f30 con 0x7f74ac106750 2023-10-08T13:49:59.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74b1d61700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac1970a0 con 0x7f74ac100740 2023-10-08T13:49:59.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74b1d61700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac197260 con 0x7f74ac101100 2023-10-08T13:49:59.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74abfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac106750 0x7f74ac1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74abfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac106750 0x7f74ac1968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54298/0 (socket says 172.21.15.57:54298) 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74abfff700 1 -- 172.21.15.57:0/3339258475 learned_addr learned my addr 172.21.15.57:0/3339258475 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74ab7fe700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ac100740 0x7f74ac19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74aaffd700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac101100 0x7f74ac19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74abfff700 1 -- 172.21.15.57:0/3339258475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac101100 msgr2=0x7f74ac19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74abfff700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac101100 0x7f74ac19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74abfff700 1 -- 172.21.15.57:0/3339258475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ac100740 msgr2=0x7f74ac19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74abfff700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ac100740 0x7f74ac19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74abfff700 1 -- 172.21.15.57:0/3339258475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a4009d00 con 0x7f74ac106750 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.981+0000 7f74ab7fe700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ac100740 0x7f74ac19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:49:59.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.982+0000 7f74abfff700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac106750 0x7f74ac1968d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f74a4007b60 tx=0x7f74a4005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:59.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.982+0000 7f74a8ff9700 1 -- 172.21.15.57:0/3339258475 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a400e040 con 0x7f74ac106750 2023-10-08T13:49:59.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.982+0000 7f74a8ff9700 1 -- 172.21.15.57:0/3339258475 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74a401d480 con 0x7f74ac106750 2023-10-08T13:49:59.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.982+0000 7f74b1d61700 1 -- 172.21.15.57:0/3339258475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac1974e0 con 0x7f74ac106750 2023-10-08T13:49:59.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.982+0000 7f74a8ff9700 1 -- 172.21.15.57:0/3339258475 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a401b6b0 con 0x7f74ac106750 2023-10-08T13:49:59.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.982+0000 7f74b1d61700 1 -- 172.21.15.57:0/3339258475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac1a3770 con 0x7f74ac106750 2023-10-08T13:49:59.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.983+0000 7f74b1d61700 1 -- 172.21.15.57:0/3339258475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac108d60 con 0x7f74ac106750 2023-10-08T13:49:59.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.984+0000 7f74a8ff9700 1 -- 172.21.15.57:0/3339258475 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f74a401d5f0 con 0x7f74ac106750 2023-10-08T13:49:59.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.984+0000 7f74a8ff9700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74940a14d0 0x7f74940a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:49:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.984+0000 7f74a8ff9700 1 -- 172.21.15.57:0/3339258475 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f74a40cef70 con 0x7f74ac106750 2023-10-08T13:49:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.985+0000 7f74ab7fe700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74940a14d0 0x7f74940a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:49:59.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.985+0000 7f74ab7fe700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74940a14d0 0x7f74940a3990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f749c005fd0 tx=0x7f749c005dc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:49:59.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:49:59.989+0000 7f74a8ff9700 1 -- 172.21.15.57:0/3339258475 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f74a4098330 con 0x7f74ac106750 2023-10-08T13:50:00.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.192+0000 7f74b1d61700 1 -- 172.21.15.57:0/3339258475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74ac101e80 con 0x7f74ac106750 2023-10-08T13:50:00.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.192+0000 7f74a8ff9700 1 -- 172.21.15.57:0/3339258475 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f74ac101e80 con 0x7f74ac106750 2023-10-08T13:50:00.193 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T13:50:00.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 -- 172.21.15.57:0/3339258475 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74940a14d0 msgr2=0x7f74940a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74940a14d0 0x7f74940a3990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f749c005fd0 tx=0x7f749c005dc0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 -- 172.21.15.57:0/3339258475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac106750 msgr2=0x7f74ac1968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac106750 0x7f74ac1968d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f74a4007b60 tx=0x7f74a4005b10 comp rx=0 tx=0).stop 2023-10-08T13:50:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 -- 172.21.15.57:0/3339258475 shutdown_connections 2023-10-08T13:50:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74ac106750 0x7f74ac1968d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f74940a14d0 0x7f74940a3990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ac100740 0x7f74ac19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 --2- 172.21.15.57:0/3339258475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac101100 0x7f74ac19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 -- 172.21.15.57:0/3339258475 >> 172.21.15.57:0/3339258475 conn(0x7f74ac0fc230 msgr2=0x7f74ac0fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:00.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 -- 172.21.15.57:0/3339258475 shutdown_connections 2023-10-08T13:50:00.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.194+0000 7f74b1d61700 1 -- 172.21.15.57:0/3339258475 wait complete. 2023-10-08T13:50:00.207 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T13:50:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.339+0000 7f19f590c700 1 -- 172.21.15.57:0/2738690348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f0100000 msgr2=0x7f19f010c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.339+0000 7f19f590c700 1 --2- 172.21.15.57:0/2738690348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f0100000 0x7f19f010c4a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f19e800b0a0 tx=0x7f19e800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.339+0000 7f19f590c700 1 -- 172.21.15.57:0/2738690348 shutdown_connections 2023-10-08T13:50:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.339+0000 7f19f590c700 1 --2- 172.21.15.57:0/2738690348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f0100000 0x7f19f010c4a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.339+0000 7f19f590c700 1 --2- 172.21.15.57:0/2738690348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19f01056e0 0x7f19f0105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.339+0000 7f19f590c700 1 --2- 172.21.15.57:0/2738690348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f00ff640 0x7f19f00ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.339+0000 7f19f590c700 1 -- 172.21.15.57:0/2738690348 >> 172.21.15.57:0/2738690348 conn(0x7f19f0076760 msgr2=0x7f19f0076b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:00.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.339+0000 7f19f590c700 1 -- 172.21.15.57:0/2738690348 shutdown_connections 2023-10-08T13:50:00.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.339+0000 7f19f590c700 1 -- 172.21.15.57:0/2738690348 wait complete. 2023-10-08T13:50:00.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.339+0000 7f19f590c700 1 Processor -- start 2023-10-08T13:50:00.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19f590c700 1 -- start start 2023-10-08T13:50:00.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19f590c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19f00ff640 0x7f19f0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:00.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19f590c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f0100000 0x7f19f0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:00.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19f590c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f01056e0 0x7f19f01a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:00.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19f590c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f01988e0 con 0x7f19f01056e0 2023-10-08T13:50:00.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19f590c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f01a3110 con 0x7f19f0100000 2023-10-08T13:50:00.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19f590c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f01a3250 con 0x7f19f00ff640 2023-10-08T13:50:00.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19ef7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f01056e0 0x7f19f01a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19ef7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f01056e0 0x7f19f01a2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54312/0 (socket says 172.21.15.57:54312) 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19ef7fe700 1 -- 172.21.15.57:0/1129902192 learned_addr learned my addr 172.21.15.57:0/1129902192 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19eeffd700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19f00ff640 0x7f19f0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19ee7fc700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f0100000 0x7f19f0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19ef7fe700 1 -- 172.21.15.57:0/1129902192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19f00ff640 msgr2=0x7f19f0197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19ef7fe700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19f00ff640 0x7f19f0197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19ef7fe700 1 -- 172.21.15.57:0/1129902192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f0100000 msgr2=0x7f19f0198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19ef7fe700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f0100000 0x7f19f0198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19ef7fe700 1 -- 172.21.15.57:0/1129902192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e8009d00 con 0x7f19f01056e0 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19eeffd700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19f00ff640 0x7f19f0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19ef7fe700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f01056e0 0x7f19f01a2bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f19e8007930 tx=0x7f19e8007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:00.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.340+0000 7f19f490a700 1 -- 172.21.15.57:0/1129902192 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e800e040 con 0x7f19f01056e0 2023-10-08T13:50:00.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.341+0000 7f19f490a700 1 -- 172.21.15.57:0/1129902192 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e8004c50 con 0x7f19f01056e0 2023-10-08T13:50:00.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.341+0000 7f19f490a700 1 -- 172.21.15.57:0/1129902192 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e80052a0 con 0x7f19f01056e0 2023-10-08T13:50:00.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.341+0000 7f19f590c700 1 -- 172.21.15.57:0/1129902192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f01a3450 con 0x7f19f01056e0 2023-10-08T13:50:00.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.341+0000 7f19f590c700 1 -- 172.21.15.57:0/1129902192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f01a3910 con 0x7f19f01056e0 2023-10-08T13:50:00.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.342+0000 7f19f590c700 1 -- 172.21.15.57:0/1129902192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f0107b80 con 0x7f19f01056e0 2023-10-08T13:50:00.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.343+0000 7f19f490a700 1 -- 172.21.15.57:0/1129902192 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f19e801d470 con 0x7f19f01056e0 2023-10-08T13:50:00.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.343+0000 7f19f490a700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19d80a14d0 0x7f19d80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:00.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.343+0000 7f19f490a700 1 -- 172.21.15.57:0/1129902192 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f19e80ce520 con 0x7f19f01056e0 2023-10-08T13:50:00.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.343+0000 7f19eeffd700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19d80a14d0 0x7f19d80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:00.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.344+0000 7f19eeffd700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19d80a14d0 0x7f19d80a3990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f19e400bb90 tx=0x7f19e4005d80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:00.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.348+0000 7f19f490a700 1 -- 172.21.15.57:0/1129902192 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f19e8097940 con 0x7f19f01056e0 2023-10-08T13:50:00.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:00 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2035372227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:50:00.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:00 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3994290622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:50:00.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:00 smithi057 ceph-mon[99476]: overall HEALTH_OK 2023-10-08T13:50:00.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:00 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3339258475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:50:00.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:00 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2035372227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:50:00.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:00 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3994290622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:50:00.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:00 smithi099 ceph-mon[103849]: overall HEALTH_OK 2023-10-08T13:50:00.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:00 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3339258475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:50:00.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.547+0000 7f19f590c700 1 -- 172.21.15.57:0/1129902192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f19f0069340 con 0x7f19f01056e0 2023-10-08T13:50:00.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.548+0000 7f19f490a700 1 -- 172.21.15.57:0/1129902192 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f19e809af60 con 0x7f19f01056e0 2023-10-08T13:50:00.549 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:50:00.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 -- 172.21.15.57:0/1129902192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19d80a14d0 msgr2=0x7f19d80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19d80a14d0 0x7f19d80a3990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f19e400bb90 tx=0x7f19e4005d80 comp rx=0 tx=0).stop 2023-10-08T13:50:00.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 -- 172.21.15.57:0/1129902192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f01056e0 msgr2=0x7f19f01a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f01056e0 0x7f19f01a2bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f19e8007930 tx=0x7f19e8007a10 comp rx=0 tx=0).stop 2023-10-08T13:50:00.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 -- 172.21.15.57:0/1129902192 shutdown_connections 2023-10-08T13:50:00.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f01056e0 0x7f19f01a2bd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19f00ff640 0x7f19f0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f19d80a14d0 0x7f19d80a3990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 --2- 172.21.15.57:0/1129902192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f0100000 0x7f19f0198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 -- 172.21.15.57:0/1129902192 >> 172.21.15.57:0/1129902192 conn(0x7f19f0076760 msgr2=0x7f19f00fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:00.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 -- 172.21.15.57:0/1129902192 shutdown_connections 2023-10-08T13:50:00.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.550+0000 7f19f590c700 1 -- 172.21.15.57:0/1129902192 wait complete. 2023-10-08T13:50:00.564 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T13:50:00.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:00 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2035372227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:50:00.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:00 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3994290622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:50:00.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:00 smithi115 ceph-mon[104408]: overall HEALTH_OK 2023-10-08T13:50:00.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:00 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3339258475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:50:00.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.691+0000 7fc32b71f700 1 -- 172.21.15.57:0/4133167060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc324106750 msgr2=0x7fc324106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.691+0000 7fc32b71f700 1 --2- 172.21.15.57:0/4133167060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc324106750 0x7fc324106b30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc314009990 tx=0x7fc314009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.691+0000 7fc32b71f700 1 -- 172.21.15.57:0/4133167060 shutdown_connections 2023-10-08T13:50:00.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.691+0000 7fc32b71f700 1 --2- 172.21.15.57:0/4133167060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc324101100 0x7fc32410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.691+0000 7fc32b71f700 1 --2- 172.21.15.57:0/4133167060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc324106750 0x7fc324106b30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.691+0000 7fc32b71f700 1 --2- 172.21.15.57:0/4133167060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc324100740 0x7fc324100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.691+0000 7fc32b71f700 1 -- 172.21.15.57:0/4133167060 >> 172.21.15.57:0/4133167060 conn(0x7fc3240fc230 msgr2=0x7fc3240fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:00.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.691+0000 7fc32b71f700 1 -- 172.21.15.57:0/4133167060 shutdown_connections 2023-10-08T13:50:00.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.691+0000 7fc32b71f700 1 -- 172.21.15.57:0/4133167060 wait complete. 2023-10-08T13:50:00.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc32b71f700 1 Processor -- start 2023-10-08T13:50:00.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc32b71f700 1 -- start start 2023-10-08T13:50:00.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc32b71f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc324100740 0x7fc32419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:00.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc32b71f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc324101100 0x7fc32419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:00.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc32b71f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc324106750 0x7fc3241969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:00.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc32b71f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc324197170 con 0x7fc324100740 2023-10-08T13:50:00.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc32b71f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3241972e0 con 0x7fc324101100 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc32b71f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3241974a0 con 0x7fc324106750 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc3294bb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc324100740 0x7fc32419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc3294bb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc324100740 0x7fc32419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54322/0 (socket says 172.21.15.57:54322) 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc3294bb700 1 -- 172.21.15.57:0/4273351517 learned_addr learned my addr 172.21.15.57:0/4273351517 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc329cbc700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc324106750 0x7fc3241969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc328cba700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc324101100 0x7fc32419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc3294bb700 1 -- 172.21.15.57:0/4273351517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc324106750 msgr2=0x7fc3241969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc3294bb700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc324106750 0x7fc3241969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc3294bb700 1 -- 172.21.15.57:0/4273351517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc324101100 msgr2=0x7fc32419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc3294bb700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc324101100 0x7fc32419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc3294bb700 1 -- 172.21.15.57:0/4273351517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3140095f0 con 0x7fc324100740 2023-10-08T13:50:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc328cba700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc324101100 0x7fc32419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:00.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.692+0000 7fc3294bb700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc324100740 0x7fc32419c900 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc3140098e0 tx=0x7fc31400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:00.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.693+0000 7fc31a7fc700 1 -- 172.21.15.57:0/4273351517 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31401d070 con 0x7fc324100740 2023-10-08T13:50:00.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.693+0000 7fc31a7fc700 1 -- 172.21.15.57:0/4273351517 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc314022470 con 0x7fc324100740 2023-10-08T13:50:00.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.693+0000 7fc32b71f700 1 -- 172.21.15.57:0/4273351517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc324197720 con 0x7fc324100740 2023-10-08T13:50:00.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.693+0000 7fc31a7fc700 1 -- 172.21.15.57:0/4273351517 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314017650 con 0x7fc324100740 2023-10-08T13:50:00.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.693+0000 7fc32b71f700 1 -- 172.21.15.57:0/4273351517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3241a3880 con 0x7fc324100740 2023-10-08T13:50:00.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.694+0000 7fc32b71f700 1 -- 172.21.15.57:0/4273351517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc324108d40 con 0x7fc324100740 2023-10-08T13:50:00.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.695+0000 7fc31a7fc700 1 -- 172.21.15.57:0/4273351517 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc3140225e0 con 0x7fc324100740 2023-10-08T13:50:00.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.696+0000 7fc31a7fc700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc3100a1550 0x7fc3100a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:00.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.696+0000 7fc328cba700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc3100a1550 0x7fc3100a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:00.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.696+0000 7fc31a7fc700 1 -- 172.21.15.57:0/4273351517 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fc3140ce380 con 0x7fc324100740 2023-10-08T13:50:00.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.696+0000 7fc328cba700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc3100a1550 0x7fc3100a3a10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc320005fd0 tx=0x7fc320005e30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:00.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.699+0000 7fc31a7fc700 1 -- 172.21.15.57:0/4273351517 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc3140977a0 con 0x7fc324100740 2023-10-08T13:50:00.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.920+0000 7fc32b71f700 1 -- 172.21.15.57:0/4273351517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc324197e80 con 0x7fc324100740 2023-10-08T13:50:00.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.921+0000 7fc31a7fc700 1 -- 172.21.15.57:0/4273351517 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v88) v1 ==== 81+0+882 (secure 0 0 0) 0x7fc314027080 con 0x7fc324100740 2023-10-08T13:50:00.922 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/466175178 2023-10-09T13:44:14.025065+0000 2023-10-08T13:50:00.922 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1455537828 2023-10-09T13:44:14.025065+0000 2023-10-08T13:50:00.922 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/2534706720 2023-10-09T13:44:14.025065+0000 2023-10-08T13:50:00.922 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/2534706720 2023-10-09T13:44:14.025065+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1379042066 2023-10-09T13:44:14.025065+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/19108071 2023-10-09T13:43:06.093214+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3114287016 2023-10-09T13:43:06.093214+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3318653521 2023-10-09T13:43:06.093214+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2210095932 2023-10-09T13:42:39.884205+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/955690770 2023-10-09T13:42:39.884205+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/140726743 2023-10-09T13:42:39.884205+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/2406655290 2023-10-09T13:43:06.093214+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2229924806 2023-10-09T13:42:39.884205+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/140726743 2023-10-09T13:42:39.884205+0000 2023-10-08T13:50:00.923 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/2406655290 2023-10-09T13:43:06.093214+0000 2023-10-08T13:50:00.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 -- 172.21.15.57:0/4273351517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc3100a1550 msgr2=0x7fc3100a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc3100a1550 0x7fc3100a3a10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc320005fd0 tx=0x7fc320005e30 comp rx=0 tx=0).stop 2023-10-08T13:50:00.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 -- 172.21.15.57:0/4273351517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc324100740 msgr2=0x7fc32419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:00.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc324100740 0x7fc32419c900 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc3140098e0 tx=0x7fc31400f670 comp rx=0 tx=0).stop 2023-10-08T13:50:00.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 -- 172.21.15.57:0/4273351517 shutdown_connections 2023-10-08T13:50:00.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc324106750 0x7fc3241969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc324100740 0x7fc32419c900 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc3100a1550 0x7fc3100a3a10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 --2- 172.21.15.57:0/4273351517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc324101100 0x7fc32419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:00.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 -- 172.21.15.57:0/4273351517 >> 172.21.15.57:0/4273351517 conn(0x7fc3240fc230 msgr2=0x7fc3240fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:00.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 -- 172.21.15.57:0/4273351517 shutdown_connections 2023-10-08T13:50:00.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:00.923+0000 7fc32b71f700 1 -- 172.21.15.57:0/4273351517 wait complete. 2023-10-08T13:50:00.926 INFO:teuthology.orchestra.run.smithi057.stderr:listed 15 entries 2023-10-08T13:50:00.936 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-10-08T13:50:00.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:50:00.936 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-08T13:50:00.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:ceph-fuse[160198]: fuse finished with error 0 and tester_r 0 2023-10-08T13:50:00.971 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T13:50:00.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T13:50:00.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:50:00.995 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-08T13:50:01.061 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:50:01.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.233+0000 7f4e41a8a700 1 -- 172.21.15.57:0/4053456650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3c0fff00 msgr2=0x7f4e3c10c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.233+0000 7f4e41a8a700 1 --2- 172.21.15.57:0/4053456650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3c0fff00 0x7f4e3c10c3c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4e340099c0 tx=0x7f4e34009cd0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.234+0000 7f4e41a8a700 1 -- 172.21.15.57:0/4053456650 shutdown_connections 2023-10-08T13:50:01.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.234+0000 7f4e41a8a700 1 --2- 172.21.15.57:0/4053456650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3c0fff00 0x7f4e3c10c3c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.234+0000 7f4e41a8a700 1 --2- 172.21.15.57:0/4053456650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c105600 0x7f4e3c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.234+0000 7f4e41a8a700 1 --2- 172.21.15.57:0/4053456650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e3c0ff540 0x7f4e3c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.234+0000 7f4e41a8a700 1 -- 172.21.15.57:0/4053456650 >> 172.21.15.57:0/4053456650 conn(0x7f4e3c0fb030 msgr2=0x7f4e3c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:01.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.234+0000 7f4e41a8a700 1 -- 172.21.15.57:0/4053456650 shutdown_connections 2023-10-08T13:50:01.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.234+0000 7f4e41a8a700 1 -- 172.21.15.57:0/4053456650 wait complete. 2023-10-08T13:50:01.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.234+0000 7f4e41a8a700 1 Processor -- start 2023-10-08T13:50:01.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e41a8a700 1 -- start start 2023-10-08T13:50:01.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e41a8a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c0ff540 0x7f4e3c19b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e41a8a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3c0fff00 0x7f4e3c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e41a8a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e3c105600 0x7f4e3c195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e41a8a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c195dc0 con 0x7f4e3c0fff00 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e41a8a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c195f30 con 0x7f4e3c0ff540 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e41a8a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c1960f0 con 0x7f4e3c105600 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e3affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3c0fff00 0x7f4e3c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e3affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3c0fff00 0x7f4e3c19bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54340/0 (socket says 172.21.15.57:54340) 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e3affd700 1 -- 172.21.15.57:0/1584011384 learned_addr learned my addr 172.21.15.57:0/1584011384 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e3bfff700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e3c105600 0x7f4e3c195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e3b7fe700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c0ff540 0x7f4e3c19b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e3affd700 1 -- 172.21.15.57:0/1584011384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e3c105600 msgr2=0x7f4e3c195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.235+0000 7f4e3affd700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e3c105600 0x7f4e3c195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.236+0000 7f4e3affd700 1 -- 172.21.15.57:0/1584011384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c0ff540 msgr2=0x7f4e3c19b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.236+0000 7f4e3affd700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c0ff540 0x7f4e3c19b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.236+0000 7f4e3affd700 1 -- 172.21.15.57:0/1584011384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e340095f0 con 0x7f4e3c0fff00 2023-10-08T13:50:01.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.236+0000 7f4e3b7fe700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c0ff540 0x7f4e3c19b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:01.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.236+0000 7f4e3affd700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3c0fff00 0x7f4e3c19bb20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400e9a0 tx=0x7f4e2400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:01.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.236+0000 7f4e38ff9700 1 -- 172.21.15.57:0/1584011384 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24009790 con 0x7f4e3c0fff00 2023-10-08T13:50:01.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.236+0000 7f4e41a8a700 1 -- 172.21.15.57:0/1584011384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c1963d0 con 0x7f4e3c0fff00 2023-10-08T13:50:01.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.236+0000 7f4e38ff9700 1 -- 172.21.15.57:0/1584011384 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e2400cbd0 con 0x7f4e3c0fff00 2023-10-08T13:50:01.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.236+0000 7f4e38ff9700 1 -- 172.21.15.57:0/1584011384 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24018680 con 0x7f4e3c0fff00 2023-10-08T13:50:01.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.236+0000 7f4e41a8a700 1 -- 172.21.15.57:0/1584011384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c1a25d0 con 0x7f4e3c0fff00 2023-10-08T13:50:01.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.237+0000 7f4e41a8a700 1 -- 172.21.15.57:0/1584011384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3c107aa0 con 0x7f4e3c0fff00 2023-10-08T13:50:01.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.238+0000 7f4e38ff9700 1 -- 172.21.15.57:0/1584011384 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f4e2400cd40 con 0x7f4e3c0fff00 2023-10-08T13:50:01.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.239+0000 7f4e38ff9700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4e280a14d0 0x7f4e280a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.239+0000 7f4e3b7fe700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4e280a14d0 0x7f4e280a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.239+0000 7f4e38ff9700 1 -- 172.21.15.57:0/1584011384 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f4e24014070 con 0x7f4e3c0fff00 2023-10-08T13:50:01.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.239+0000 7f4e3b7fe700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4e280a14d0 0x7f4e280a3990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009fd0 tx=0x7f4e2c00b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:01.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.242+0000 7f4e38ff9700 1 -- 172.21.15.57:0/1584011384 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4e24096be0 con 0x7f4e3c0fff00 2023-10-08T13:50:01.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.445+0000 7f4e41a8a700 1 -- 172.21.15.57:0/1584011384 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e3c100c60 con 0x7f4e3c0fff00 2023-10-08T13:50:01.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.447+0000 7f4e38ff9700 1 -- 172.21.15.57:0/1584011384 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24224 (secure 0 0 0) 0x7f4e2409a200 con 0x7f4e3c0fff00 2023-10-08T13:50:01.448 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:01.448 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":88,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:49.574582+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:50:01.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 -- 172.21.15.57:0/1584011384 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4e280a14d0 msgr2=0x7f4e280a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4e280a14d0 0x7f4e280a3990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009fd0 tx=0x7f4e2c00b040 comp rx=0 tx=0).stop 2023-10-08T13:50:01.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 -- 172.21.15.57:0/1584011384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3c0fff00 msgr2=0x7f4e3c19bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3c0fff00 0x7f4e3c19bb20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400e9a0 tx=0x7f4e2400ed60 comp rx=0 tx=0).stop 2023-10-08T13:50:01.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 -- 172.21.15.57:0/1584011384 shutdown_connections 2023-10-08T13:50:01.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e3c105600 0x7f4e3c195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4e280a14d0 0x7f4e280a3990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e3c0ff540 0x7f4e3c19b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 --2- 172.21.15.57:0/1584011384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3c0fff00 0x7f4e3c19bb20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 -- 172.21.15.57:0/1584011384 >> 172.21.15.57:0/1584011384 conn(0x7f4e3c0fb030 msgr2=0x7f4e3c0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:01.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 -- 172.21.15.57:0/1584011384 shutdown_connections 2023-10-08T13:50:01.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.449+0000 7f4e41a8a700 1 -- 172.21.15.57:0/1584011384 wait complete. 2023-10-08T13:50:01.463 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:50:01.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:01 smithi057 ceph-mon[99476]: pgmap v249: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2023-10-08T13:50:01.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:01 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1129902192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:50:01.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:01 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4273351517' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T13:50:01.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:01 smithi099 ceph-mon[103849]: pgmap v249: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2023-10-08T13:50:01.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:01 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1129902192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:50:01.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:01 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4273351517' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T13:50:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 -- 172.21.15.57:0/2236810980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9980ff8f0 msgr2=0x7fd99810bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 --2- 172.21.15.57:0/2236810980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9980ff8f0 0x7fd99810bba0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd99000b0d0 tx=0x7fd99000b3e0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 -- 172.21.15.57:0/2236810980 shutdown_connections 2023-10-08T13:50:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 --2- 172.21.15.57:0/2236810980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9980ff8f0 0x7fd99810bba0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 --2- 172.21.15.57:0/2236810980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd998103ca0 0x7fd998104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 --2- 172.21.15.57:0/2236810980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9980fef30 0x7fd9980ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 -- 172.21.15.57:0/2236810980 >> 172.21.15.57:0/2236810980 conn(0x7fd998074820 msgr2=0x7fd998074c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:01.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 -- 172.21.15.57:0/2236810980 shutdown_connections 2023-10-08T13:50:01.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 -- 172.21.15.57:0/2236810980 wait complete. 2023-10-08T13:50:01.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 Processor -- start 2023-10-08T13:50:01.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.588+0000 7fd99ccc8700 1 -- start start 2023-10-08T13:50:01.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd99ccc8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9980fef30 0x7fd998194810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd99ccc8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9980ff8f0 0x7fd998194d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd99ccc8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd998103ca0 0x7fd998195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd99ccc8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9981a4f70 con 0x7fd998103ca0 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd99ccc8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9981a50e0 con 0x7fd9980fef30 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd99ccc8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9981a52a0 con 0x7fd9980ff8f0 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd996d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd998103ca0 0x7fd998195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd996d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd998103ca0 0x7fd998195290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54356/0 (socket says 172.21.15.57:54356) 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd996d9d700 1 -- 172.21.15.57:0/1928887408 learned_addr learned my addr 172.21.15.57:0/1928887408 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd99659c700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9980fef30 0x7fd998194810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd996d9d700 1 -- 172.21.15.57:0/1928887408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9980ff8f0 msgr2=0x7fd998194d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd995d9b700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9980ff8f0 0x7fd998194d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd996d9d700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9980ff8f0 0x7fd998194d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd996d9d700 1 -- 172.21.15.57:0/1928887408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9980fef30 msgr2=0x7fd998194810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd996d9d700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9980fef30 0x7fd998194810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd996d9d700 1 -- 172.21.15.57:0/1928887408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd990009d00 con 0x7fd998103ca0 2023-10-08T13:50:01.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd996d9d700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd998103ca0 0x7fd998195290 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd9900094c0 tx=0x7fd9900094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:01.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd9877fe700 1 -- 172.21.15.57:0/1928887408 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99000e040 con 0x7fd998103ca0 2023-10-08T13:50:01.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.589+0000 7fd99ccc8700 1 -- 172.21.15.57:0/1928887408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9981a5520 con 0x7fd998103ca0 2023-10-08T13:50:01.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.590+0000 7fd9877fe700 1 -- 172.21.15.57:0/1928887408 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd99001d940 con 0x7fd998103ca0 2023-10-08T13:50:01.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.590+0000 7fd9877fe700 1 -- 172.21.15.57:0/1928887408 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99001b9e0 con 0x7fd998103ca0 2023-10-08T13:50:01.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.590+0000 7fd99ccc8700 1 -- 172.21.15.57:0/1928887408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9981a59a0 con 0x7fd998103ca0 2023-10-08T13:50:01.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.590+0000 7fd99ccc8700 1 -- 172.21.15.57:0/1928887408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd998107220 con 0x7fd998103ca0 2023-10-08T13:50:01.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.592+0000 7fd9877fe700 1 -- 172.21.15.57:0/1928887408 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd99001d480 con 0x7fd998103ca0 2023-10-08T13:50:01.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.592+0000 7fd9877fe700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9800a14d0 0x7fd9800a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.592+0000 7fd9877fe700 1 -- 172.21.15.57:0/1928887408 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fd9900cf090 con 0x7fd998103ca0 2023-10-08T13:50:01.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.592+0000 7fd99659c700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9800a14d0 0x7fd9800a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.593+0000 7fd99659c700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9800a14d0 0x7fd9800a3990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd988005950 tx=0x7fd98800a300 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:01.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.596+0000 7fd9877fe700 1 -- 172.21.15.57:0/1928887408 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd9900984b0 con 0x7fd998103ca0 2023-10-08T13:50:01.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:01 smithi115 ceph-mon[104408]: pgmap v249: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2023-10-08T13:50:01.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:01 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1129902192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:50:01.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:01 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4273351517' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T13:50:01.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.802+0000 7fd99ccc8700 1 -- 172.21.15.57:0/1928887408 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd998195d40 con 0x7fd998103ca0 2023-10-08T13:50:01.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.804+0000 7fd9877fe700 1 -- 172.21.15.57:0/1928887408 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24224 (secure 0 0 0) 0x7fd99009bad0 con 0x7fd998103ca0 2023-10-08T13:50:01.805 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:01.806 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":88,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:49.574582+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:50:01.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 -- 172.21.15.57:0/1928887408 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9800a14d0 msgr2=0x7fd9800a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9800a14d0 0x7fd9800a3990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd988005950 tx=0x7fd98800a300 comp rx=0 tx=0).stop 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 -- 172.21.15.57:0/1928887408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd998103ca0 msgr2=0x7fd998195290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd998103ca0 0x7fd998195290 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd9900094c0 tx=0x7fd9900094f0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 -- 172.21.15.57:0/1928887408 shutdown_connections 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd998103ca0 0x7fd998195290 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9800a14d0 0x7fd9800a3990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9980fef30 0x7fd998194810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 --2- 172.21.15.57:0/1928887408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9980ff8f0 0x7fd998194d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 -- 172.21.15.57:0/1928887408 >> 172.21.15.57:0/1928887408 conn(0x7fd998074820 msgr2=0x7fd9980fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 -- 172.21.15.57:0/1928887408 shutdown_connections 2023-10-08T13:50:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.806+0000 7fd99ccc8700 1 -- 172.21.15.57:0/1928887408 wait complete. 2023-10-08T13:50:01.820 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T13:50:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.987+0000 7f9e795dc700 1 -- 172.21.15.57:0/2630000714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e74105b30 msgr2=0x7f9e7410d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.987+0000 7f9e795dc700 1 --2- 172.21.15.57:0/2630000714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e74105b30 0x7f9e7410d6d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00b0a0 tx=0x7f9e6c00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.987+0000 7f9e795dc700 1 -- 172.21.15.57:0/2630000714 shutdown_connections 2023-10-08T13:50:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.987+0000 7f9e795dc700 1 --2- 172.21.15.57:0/2630000714 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e74105b30 0x7f9e7410d6d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.987+0000 7f9e795dc700 1 --2- 172.21.15.57:0/2630000714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e740ff320 0x7f9e740ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.987+0000 7f9e795dc700 1 --2- 172.21.15.57:0/2630000714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e740ffcd0 0x7f9e74105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.987+0000 7f9e795dc700 1 -- 172.21.15.57:0/2630000714 >> 172.21.15.57:0/2630000714 conn(0x7f9e740749a0 msgr2=0x7f9e74074db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.987+0000 7f9e795dc700 1 -- 172.21.15.57:0/2630000714 shutdown_connections 2023-10-08T13:50:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.987+0000 7f9e795dc700 1 -- 172.21.15.57:0/2630000714 wait complete. 2023-10-08T13:50:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.988+0000 7f9e795dc700 1 Processor -- start 2023-10-08T13:50:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.988+0000 7f9e795dc700 1 -- start start 2023-10-08T13:50:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.988+0000 7f9e795dc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e740ff320 0x7f9e74100290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.988+0000 7f9e795dc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e74105b30 0x7f9e741007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.988+0000 7f9e795dc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e74100d10 0x7f9e741a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.988+0000 7f9e795dc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e741a5520 con 0x7f9e74100d10 2023-10-08T13:50:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.988+0000 7f9e795dc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e741a5690 con 0x7f9e74105b30 2023-10-08T13:50:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.988+0000 7f9e795dc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e741a5850 con 0x7f9e740ff320 2023-10-08T13:50:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.988+0000 7f9e737fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e74100d10 0x7f9e741a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e737fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e74100d10 0x7f9e741a4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54370/0 (socket says 172.21.15.57:54370) 2023-10-08T13:50:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e737fe700 1 -- 172.21.15.57:0/1306360419 learned_addr learned my addr 172.21.15.57:0/1306360419 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e727fc700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e74105b30 0x7f9e741007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e72ffd700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e740ff320 0x7f9e74100290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e737fe700 1 -- 172.21.15.57:0/1306360419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e740ff320 msgr2=0x7f9e74100290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e737fe700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e740ff320 0x7f9e74100290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e737fe700 1 -- 172.21.15.57:0/1306360419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e74105b30 msgr2=0x7f9e741007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e737fe700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e74105b30 0x7f9e741007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e737fe700 1 -- 172.21.15.57:0/1306360419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c009d00 con 0x7f9e74100d10 2023-10-08T13:50:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e737fe700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e74100d10 0x7f9e741a4f50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c006130 tx=0x7f9e6c007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e5bfff700 1 -- 172.21.15.57:0/1306360419 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c00e040 con 0x7f9e74100d10 2023-10-08T13:50:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e795dc700 1 -- 172.21.15.57:0/1306360419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e741a5ad0 con 0x7f9e74100d10 2023-10-08T13:50:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e5bfff700 1 -- 172.21.15.57:0/1306360419 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e6c009480 con 0x7f9e74100d10 2023-10-08T13:50:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e5bfff700 1 -- 172.21.15.57:0/1306360419 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c01b6b0 con 0x7f9e74100d10 2023-10-08T13:50:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.989+0000 7f9e795dc700 1 -- 172.21.15.57:0/1306360419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e741a5f50 con 0x7f9e74100d10 2023-10-08T13:50:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.990+0000 7f9e795dc700 1 -- 172.21.15.57:0/1306360419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e74108d20 con 0x7f9e74100d10 2023-10-08T13:50:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.991+0000 7f9e5bfff700 1 -- 172.21.15.57:0/1306360419 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f9e6c01d480 con 0x7f9e74100d10 2023-10-08T13:50:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.992+0000 7f9e5bfff700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9e5c0a14d0 0x7f9e5c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.992+0000 7f9e5bfff700 1 -- 172.21.15.57:0/1306360419 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f9e6c0cebf0 con 0x7f9e74100d10 2023-10-08T13:50:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.992+0000 7f9e72ffd700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9e5c0a14d0 0x7f9e5c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.992+0000 7f9e72ffd700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9e5c0a14d0 0x7f9e5c0a3990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9e68007be0 tx=0x7f9e680073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:01.995+0000 7f9e5bfff700 1 -- 172.21.15.57:0/1306360419 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9e6c098010 con 0x7f9e74100d10 2023-10-08T13:50:02.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:02.247+0000 7f9e795dc700 1 -- 172.21.15.57:0/1306360419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9e74068800 con 0x7f9e74100d10 2023-10-08T13:50:02.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:02 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1584011384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:02.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:02 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1928887408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:02.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:02 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1584011384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:02.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:02 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1928887408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:02.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:02 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1584011384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:02.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:02 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1928887408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:03.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.248+0000 7f9e5bfff700 1 -- 172.21.15.57:0/1306360419 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9e6c09b630 con 0x7f9e74100d10 2023-10-08T13:50:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.250+0000 7f9e795dc700 1 -- 172.21.15.57:0/1306360419 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9e5c0a14d0 msgr2=0x7f9e5c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9e5c0a14d0 0x7f9e5c0a3990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9e68007be0 tx=0x7f9e680073d0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 -- 172.21.15.57:0/1306360419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e74100d10 msgr2=0x7f9e741a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e74100d10 0x7f9e741a4f50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c006130 tx=0x7f9e6c007cf0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 -- 172.21.15.57:0/1306360419 shutdown_connections 2023-10-08T13:50:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e74100d10 0x7f9e741a4f50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e740ff320 0x7f9e74100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9e5c0a14d0 0x7f9e5c0a3990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 --2- 172.21.15.57:0/1306360419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e74105b30 0x7f9e741007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 -- 172.21.15.57:0/1306360419 >> 172.21.15.57:0/1306360419 conn(0x7f9e740749a0 msgr2=0x7f9e740fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:03.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 -- 172.21.15.57:0/1306360419 shutdown_connections 2023-10-08T13:50:03.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.251+0000 7f9e795dc700 1 -- 172.21.15.57:0/1306360419 wait complete. 2023-10-08T13:50:03.267 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-08T13:50:03.267 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-10-08T13:50:03.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.402+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1683492153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bbc1056b0 msgr2=0x7f7bbc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:03.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.402+0000 7f7bc2c7a700 1 --2- 172.21.15.57:0/1683492153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bbc1056b0 0x7f7bbc105a90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009990 tx=0x7f7bb0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.402+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1683492153 shutdown_connections 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.402+0000 7f7bc2c7a700 1 --2- 172.21.15.57:0/1683492153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bbc0fffe0 0x7f7bbc10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.402+0000 7f7bc2c7a700 1 --2- 172.21.15.57:0/1683492153 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bbc1056b0 0x7f7bbc105a90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.402+0000 7f7bc2c7a700 1 --2- 172.21.15.57:0/1683492153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bbc0ff620 0x7f7bbc0ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.402+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1683492153 >> 172.21.15.57:0/1683492153 conn(0x7f7bbc076780 msgr2=0x7f7bbc076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.402+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1683492153 shutdown_connections 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.402+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1683492153 wait complete. 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc2c7a700 1 Processor -- start 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc2c7a700 1 -- start start 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc2c7a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bbc0ff620 0x7f7bbc19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:03.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc2c7a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bbc0fffe0 0x7f7bbc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc2c7a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bbc1056b0 0x7f7bbc1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc2c7a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc196f50 con 0x7f7bbc1056b0 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc2c7a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc1970c0 con 0x7f7bbc0ff620 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc2c7a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc197280 con 0x7f7bbc0fffe0 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc1217700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bbc1056b0 0x7f7bbc1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc1217700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bbc1056b0 0x7f7bbc1968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54384/0 (socket says 172.21.15.57:54384) 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc1217700 1 -- 172.21.15.57:0/1218409969 learned_addr learned my addr 172.21.15.57:0/1218409969 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc0a16700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bbc0ff620 0x7f7bbc19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bbbfff700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bbc0fffe0 0x7f7bbc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc1217700 1 -- 172.21.15.57:0/1218409969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bbc0fffe0 msgr2=0x7f7bbc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc1217700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bbc0fffe0 0x7f7bbc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc1217700 1 -- 172.21.15.57:0/1218409969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bbc0ff620 msgr2=0x7f7bbc19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc1217700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bbc0ff620 0x7f7bbc19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc1217700 1 -- 172.21.15.57:0/1218409969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb00095f0 con 0x7f7bbc1056b0 2023-10-08T13:50:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.403+0000 7f7bc1217700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bbc1056b0 0x7f7bbc1968f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4007ee0 tx=0x7f7bb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.404+0000 7f7bb9ffb700 1 -- 172.21.15.57:0/1218409969 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4017070 con 0x7f7bbc1056b0 2023-10-08T13:50:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.404+0000 7f7bb9ffb700 1 -- 172.21.15.57:0/1218409969 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bb400f040 con 0x7f7bbc1056b0 2023-10-08T13:50:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.404+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1218409969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc197560 con 0x7f7bbc1056b0 2023-10-08T13:50:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.404+0000 7f7bb9ffb700 1 -- 172.21.15.57:0/1218409969 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4005740 con 0x7f7bbc1056b0 2023-10-08T13:50:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.404+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1218409969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc1a37b0 con 0x7f7bbc1056b0 2023-10-08T13:50:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.405+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1218409969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bbc100d50 con 0x7f7bbc1056b0 2023-10-08T13:50:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.406+0000 7f7bb9ffb700 1 -- 172.21.15.57:0/1218409969 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7bb400dc60 con 0x7f7bbc1056b0 2023-10-08T13:50:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.407+0000 7f7bb9ffb700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bac0a1550 0x7f7bac0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.407+0000 7f7bc0a16700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bac0a1550 0x7f7bac0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.407+0000 7f7bb9ffb700 1 -- 172.21.15.57:0/1218409969 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f7bb40cc7d0 con 0x7f7bbc1056b0 2023-10-08T13:50:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.407+0000 7f7bc0a16700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bac0a1550 0x7f7bac0a3a10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0006010 tx=0x7f7bb0005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:03.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.411+0000 7f7bb9ffb700 1 -- 172.21.15.57:0/1218409969 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7bb4095b60 con 0x7f7bbc1056b0 2023-10-08T13:50:03.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:03 smithi099 ceph-mon[103849]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 257 B/s wr, 0 op/s 2023-10-08T13:50:03.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:03 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1306360419' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-08T13:50:03.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:03.662+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1218409969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f7bbc10f500 con 0x7f7bbc1056b0 2023-10-08T13:50:03.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:03 smithi115 ceph-mon[104408]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 257 B/s wr, 0 op/s 2023-10-08T13:50:03.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:03 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1306360419' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-08T13:50:03.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:03 smithi057 ceph-mon[99476]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 257 B/s wr, 0 op/s 2023-10-08T13:50:03.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:03 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1306360419' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-08T13:50:04.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.259+0000 7f7bb9ffb700 1 -- 172.21.15.57:0/1218409969 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7bb4099180 con 0x7f7bbc1056b0 2023-10-08T13:50:04.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1218409969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bac0a1550 msgr2=0x7f7bac0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:04.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bac0a1550 0x7f7bac0a3a10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0006010 tx=0x7f7bb0005960 comp rx=0 tx=0).stop 2023-10-08T13:50:04.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1218409969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bbc1056b0 msgr2=0x7f7bbc1968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:04.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bbc1056b0 0x7f7bbc1968f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4007ee0 tx=0x7f7bb400d3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1218409969 shutdown_connections 2023-10-08T13:50:04.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bbc1056b0 0x7f7bbc1968f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7bac0a1550 0x7f7bac0a3a10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bbc0ff620 0x7f7bbc19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 --2- 172.21.15.57:0/1218409969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bbc0fffe0 0x7f7bbc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1218409969 >> 172.21.15.57:0/1218409969 conn(0x7f7bbc076780 msgr2=0x7f7bbc067b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:04.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1218409969 shutdown_connections 2023-10-08T13:50:04.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.263+0000 7f7bc2c7a700 1 -- 172.21.15.57:0/1218409969 wait complete. 2023-10-08T13:50:04.286 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-08T13:50:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.421+0000 7f8b116d4700 1 -- 172.21.15.57:0/1609681588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c105650 msgr2=0x7f8b0c105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.421+0000 7f8b116d4700 1 --2- 172.21.15.57:0/1609681588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c105650 0x7f8b0c105a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009990 tx=0x7f8afc009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.422+0000 7f8b116d4700 1 -- 172.21.15.57:0/1609681588 shutdown_connections 2023-10-08T13:50:04.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.422+0000 7f8b116d4700 1 --2- 172.21.15.57:0/1609681588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b0c100000 0x7f8b0c10c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.422+0000 7f8b116d4700 1 --2- 172.21.15.57:0/1609681588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c105650 0x7f8b0c105a30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.422+0000 7f8b116d4700 1 --2- 172.21.15.57:0/1609681588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b0c0ff640 0x7f8b0c0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.422+0000 7f8b116d4700 1 -- 172.21.15.57:0/1609681588 >> 172.21.15.57:0/1609681588 conn(0x7f8b0c076850 msgr2=0x7f8b0c076c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.422+0000 7f8b116d4700 1 -- 172.21.15.57:0/1609681588 shutdown_connections 2023-10-08T13:50:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.422+0000 7f8b116d4700 1 -- 172.21.15.57:0/1609681588 wait complete. 2023-10-08T13:50:04.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.422+0000 7f8b116d4700 1 Processor -- start 2023-10-08T13:50:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b116d4700 1 -- start start 2023-10-08T13:50:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b116d4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b0c0ff640 0x7f8b0c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b116d4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100000 0x7f8b0c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:04.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b116d4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b0c105650 0x7f8b0c1a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b116d4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c198770 con 0x7f8b0c100000 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b116d4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c1a2f80 con 0x7f8b0c105650 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b116d4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c1a30c0 con 0x7f8b0c0ff640 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b0a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100000 0x7f8b0c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b0a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100000 0x7f8b0c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54396/0 (socket says 172.21.15.57:54396) 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b0a7fc700 1 -- 172.21.15.57:0/942567254 learned_addr learned my addr 172.21.15.57:0/942567254 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b0affd700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b0c0ff640 0x7f8b0c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b0b7fe700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b0c105650 0x7f8b0c1a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b0a7fc700 1 -- 172.21.15.57:0/942567254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b0c0ff640 msgr2=0x7f8b0c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b0a7fc700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b0c0ff640 0x7f8b0c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b0a7fc700 1 -- 172.21.15.57:0/942567254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b0c105650 msgr2=0x7f8b0c1a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b0a7fc700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b0c105650 0x7f8b0c1a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.423+0000 7f8b0a7fc700 1 -- 172.21.15.57:0/942567254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc0095f0 con 0x7f8b0c100000 2023-10-08T13:50:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.424+0000 7f8b0a7fc700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100000 0x7f8b0c197f00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000d950 tx=0x7f8b0000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.424+0000 7f8af3fff700 1 -- 172.21.15.57:0/942567254 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000f890 con 0x7f8b0c100000 2023-10-08T13:50:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.424+0000 7f8b116d4700 1 -- 172.21.15.57:0/942567254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c1a3350 con 0x7f8b0c100000 2023-10-08T13:50:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.424+0000 7f8af3fff700 1 -- 172.21.15.57:0/942567254 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b00010460 con 0x7f8b0c100000 2023-10-08T13:50:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.424+0000 7f8af3fff700 1 -- 172.21.15.57:0/942567254 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000e610 con 0x7f8b0c100000 2023-10-08T13:50:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.424+0000 7f8b116d4700 1 -- 172.21.15.57:0/942567254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c1a38b0 con 0x7f8b0c100000 2023-10-08T13:50:04.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.425+0000 7f8b116d4700 1 -- 172.21.15.57:0/942567254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c107c40 con 0x7f8b0c100000 2023-10-08T13:50:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.426+0000 7f8af3fff700 1 -- 172.21.15.57:0/942567254 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f8b000108d0 con 0x7f8b0c100000 2023-10-08T13:50:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.427+0000 7f8af3fff700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8af40a14d0 0x7f8af40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.427+0000 7f8b0affd700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8af40a14d0 0x7f8af40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.427+0000 7f8af3fff700 1 -- 172.21.15.57:0/942567254 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f8b000ccdc0 con 0x7f8b0c100000 2023-10-08T13:50:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.427+0000 7f8b0affd700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8af40a14d0 0x7f8af40a3990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00b5c0 tx=0x7f8afc005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.431+0000 7f8af3fff700 1 -- 172.21.15.57:0/942567254 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8b00096150 con 0x7f8b0c100000 2023-10-08T13:50:04.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:04 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1306360419' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-08T13:50:04.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:04 smithi099 ceph-mon[103849]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:50:04.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:04 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1218409969' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-08T13:50:04.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:04.680+0000 7f8b116d4700 1 -- 172.21.15.57:0/942567254 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f8b0c066cd0 con 0x7f8b0c100000 2023-10-08T13:50:04.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:04 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1306360419' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-08T13:50:04.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:04 smithi115 ceph-mon[104408]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:50:04.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:04 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1218409969' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-08T13:50:04.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:04 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1306360419' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-08T13:50:04.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:04 smithi057 ceph-mon[99476]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:50:04.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:04 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1218409969' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-08T13:50:05.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.279+0000 7f8af3fff700 1 -- 172.21.15.57:0/942567254 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f8b000160c0 con 0x7f8b0c100000 2023-10-08T13:50:05.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.281+0000 7f8b116d4700 1 -- 172.21.15.57:0/942567254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8af40a14d0 msgr2=0x7f8af40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:05.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.281+0000 7f8b116d4700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8af40a14d0 0x7f8af40a3990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00b5c0 tx=0x7f8afc005960 comp rx=0 tx=0).stop 2023-10-08T13:50:05.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.281+0000 7f8b116d4700 1 -- 172.21.15.57:0/942567254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100000 msgr2=0x7f8b0c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:05.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.281+0000 7f8b116d4700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100000 0x7f8b0c197f00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000d950 tx=0x7f8b0000dc60 comp rx=0 tx=0).stop 2023-10-08T13:50:05.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.282+0000 7f8b116d4700 1 -- 172.21.15.57:0/942567254 shutdown_connections 2023-10-08T13:50:05.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.282+0000 7f8b116d4700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8af40a14d0 0x7f8af40a3990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:05.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.282+0000 7f8b116d4700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b0c105650 0x7f8b0c1a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.282+0000 7f8b116d4700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b0c0ff640 0x7f8b0c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.282+0000 7f8b116d4700 1 --2- 172.21.15.57:0/942567254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b0c100000 0x7f8b0c197f00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.282+0000 7f8b116d4700 1 -- 172.21.15.57:0/942567254 >> 172.21.15.57:0/942567254 conn(0x7f8b0c076850 msgr2=0x7f8b0c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.282+0000 7f8b116d4700 1 -- 172.21.15.57:0/942567254 shutdown_connections 2023-10-08T13:50:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.282+0000 7f8b116d4700 1 -- 172.21.15.57:0/942567254 wait complete. 2023-10-08T13:50:05.300 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-08T13:50:05.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 -- 172.21.15.57:0/3556939832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8834105650 msgr2=0x7f8834105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:05.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 --2- 172.21.15.57:0/3556939832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8834105650 0x7f8834105a30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f881c009990 tx=0x7f881c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:05.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 -- 172.21.15.57:0/3556939832 shutdown_connections 2023-10-08T13:50:05.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 --2- 172.21.15.57:0/3556939832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8834100000 0x7f883410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:05.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 --2- 172.21.15.57:0/3556939832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8834105650 0x7f8834105a30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:05.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 --2- 172.21.15.57:0/3556939832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88340ff640 0x7f88340ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:05.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 -- 172.21.15.57:0/3556939832 >> 172.21.15.57:0/3556939832 conn(0x7f88340767c0 msgr2=0x7f8834076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:05.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 -- 172.21.15.57:0/3556939832 shutdown_connections 2023-10-08T13:50:05.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 -- 172.21.15.57:0/3556939832 wait complete. 2023-10-08T13:50:05.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 Processor -- start 2023-10-08T13:50:05.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.437+0000 7f883a440700 1 -- start start 2023-10-08T13:50:05.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:05 smithi115 ceph-mon[104408]: pgmap v251: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T13:50:05.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:05 smithi115 ceph-mon[104408]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-08T13:50:05.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:05 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1218409969' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-08T13:50:05.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:05 smithi115 ceph-mon[104408]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-08T13:50:05.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:05 smithi115 ceph-mon[104408]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:50:05.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:05 smithi115 ceph-mon[104408]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-08T13:50:05.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:05 smithi115 ceph-mon[104408]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-08T13:50:05.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:05 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/942567254' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-08T13:50:05.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f883a440700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88340ff640 0x7f883419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:05.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f883a440700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8834100000 0x7f883419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:05.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f883a440700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8834105650 0x7f8834195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:05.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f883a440700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8834195ec0 con 0x7f8834100000 2023-10-08T13:50:05.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f883a440700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8834196030 con 0x7f88340ff640 2023-10-08T13:50:05.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f883a440700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88341961f0 con 0x7f8834105650 2023-10-08T13:50:05.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88337fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8834100000 0x7f883419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:05.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88337fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8834100000 0x7f883419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54410/0 (socket says 172.21.15.57:54410) 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88337fe700 1 -- 172.21.15.57:0/3119493863 learned_addr learned my addr 172.21.15.57:0/3119493863 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88389dd700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8834105650 0x7f8834195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f8833fff700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88340ff640 0x7f883419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88337fe700 1 -- 172.21.15.57:0/3119493863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8834105650 msgr2=0x7f8834195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88337fe700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8834105650 0x7f8834195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88337fe700 1 -- 172.21.15.57:0/3119493863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88340ff640 msgr2=0x7f883419b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88337fe700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88340ff640 0x7f883419b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88337fe700 1 -- 172.21.15.57:0/3119493863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c0095f0 con 0x7f8834100000 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88389dd700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8834105650 0x7f8834195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f8833fff700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88340ff640 0x7f883419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.438+0000 7f88337fe700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8834100000 0x7f883419bc60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8824007b50 tx=0x7f8824007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:05.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.439+0000 7f88317fa700 1 -- 172.21.15.57:0/3119493863 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824011070 con 0x7f8834100000 2023-10-08T13:50:05.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.439+0000 7f88317fa700 1 -- 172.21.15.57:0/3119493863 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f882400c2e0 con 0x7f8834100000 2023-10-08T13:50:05.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.439+0000 7f883a440700 1 -- 172.21.15.57:0/3119493863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88341964d0 con 0x7f8834100000 2023-10-08T13:50:05.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.439+0000 7f88317fa700 1 -- 172.21.15.57:0/3119493863 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882400e5c0 con 0x7f8834100000 2023-10-08T13:50:05.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.439+0000 7f883a440700 1 -- 172.21.15.57:0/3119493863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8834068bb0 con 0x7f8834100000 2023-10-08T13:50:05.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.440+0000 7f883a440700 1 -- 172.21.15.57:0/3119493863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8834100d80 con 0x7f8834100000 2023-10-08T13:50:05.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.441+0000 7f88317fa700 1 -- 172.21.15.57:0/3119493863 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f8824004750 con 0x7f8834100000 2023-10-08T13:50:05.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.442+0000 7f88317fa700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88200a14d0 0x7f88200a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:05.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.442+0000 7f8833fff700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88200a14d0 0x7f88200a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:05.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.442+0000 7f88317fa700 1 -- 172.21.15.57:0/3119493863 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f88240ce390 con 0x7f8834100000 2023-10-08T13:50:05.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.442+0000 7f8833fff700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88200a14d0 0x7f88200a3990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f881c00b5c0 tx=0x7f881c005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:05.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.446+0000 7f88317fa700 1 -- 172.21.15.57:0/3119493863 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f88240977b0 con 0x7f8834100000 2023-10-08T13:50:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:05 smithi099 ceph-mon[103849]: pgmap v251: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T13:50:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:05 smithi099 ceph-mon[103849]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-08T13:50:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:05 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1218409969' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-08T13:50:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:05 smithi099 ceph-mon[103849]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-08T13:50:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:05 smithi099 ceph-mon[103849]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:50:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:05 smithi099 ceph-mon[103849]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-08T13:50:05.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:05 smithi099 ceph-mon[103849]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-08T13:50:05.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:05 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/942567254' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-08T13:50:05.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:05.699+0000 7f883a440700 1 -- 172.21.15.57:0/3119493863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f8834066ca0 con 0x7f8834100000 2023-10-08T13:50:05.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:05 smithi057 ceph-mon[99476]: pgmap v251: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T13:50:05.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:05 smithi057 ceph-mon[99476]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-08T13:50:05.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:05 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1218409969' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-08T13:50:05.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:05 smithi057 ceph-mon[99476]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-08T13:50:05.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:05 smithi057 ceph-mon[99476]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T13:50:05.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:05 smithi057 ceph-mon[99476]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-08T13:50:05.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:05 smithi057 ceph-mon[99476]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-08T13:50:05.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:05 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/942567254' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-08T13:50:06.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.296+0000 7f88317fa700 1 -- 172.21.15.57:0/3119493863 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f882409add0 con 0x7f8834100000 2023-10-08T13:50:06.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 -- 172.21.15.57:0/3119493863 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88200a14d0 msgr2=0x7f88200a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:06.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88200a14d0 0x7f88200a3990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f881c00b5c0 tx=0x7f881c005fb0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 -- 172.21.15.57:0/3119493863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8834100000 msgr2=0x7f883419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:06.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8834100000 0x7f883419bc60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8824007b50 tx=0x7f8824007f10 comp rx=0 tx=0).stop 2023-10-08T13:50:06.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 -- 172.21.15.57:0/3119493863 shutdown_connections 2023-10-08T13:50:06.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8834105650 0x7f8834195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88200a14d0 0x7f88200a3990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88340ff640 0x7f883419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 --2- 172.21.15.57:0/3119493863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8834100000 0x7f883419bc60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 -- 172.21.15.57:0/3119493863 >> 172.21.15.57:0/3119493863 conn(0x7f88340767c0 msgr2=0x7f88340fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:06.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 -- 172.21.15.57:0/3119493863 shutdown_connections 2023-10-08T13:50:06.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.299+0000 7f883a440700 1 -- 172.21.15.57:0/3119493863 wait complete. 2023-10-08T13:50:06.318 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-10-08T13:50:06.318 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-10-08T13:50:06.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.455+0000 7fe386aa8700 1 -- 172.21.15.57:0/2513896598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe380067ce0 msgr2=0x7fe3800680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:06.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.455+0000 7fe386aa8700 1 --2- 172.21.15.57:0/2513896598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe380067ce0 0x7fe3800680c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe3740099c0 tx=0x7fe374009cd0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.455+0000 7fe386aa8700 1 -- 172.21.15.57:0/2513896598 shutdown_connections 2023-10-08T13:50:06.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.455+0000 7fe386aa8700 1 --2- 172.21.15.57:0/2513896598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe380105b80 0x7fe38010d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.455+0000 7fe386aa8700 1 --2- 172.21.15.57:0/2513896598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe380067ce0 0x7fe3800680c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.455+0000 7fe386aa8700 1 --2- 172.21.15.57:0/2513896598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3801051c0 0x7fe380105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.455+0000 7fe386aa8700 1 -- 172.21.15.57:0/2513896598 >> 172.21.15.57:0/2513896598 conn(0x7fe380076780 msgr2=0x7fe380076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.455+0000 7fe386aa8700 1 -- 172.21.15.57:0/2513896598 shutdown_connections 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.455+0000 7fe386aa8700 1 -- 172.21.15.57:0/2513896598 wait complete. 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.455+0000 7fe386aa8700 1 Processor -- start 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe386aa8700 1 -- start start 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe386aa8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe380067ce0 0x7fe38019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe386aa8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3801051c0 0x7fe38019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe386aa8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe380105b80 0x7fe3801969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe386aa8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380197050 con 0x7fe3801051c0 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe386aa8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3801971c0 con 0x7fe380067ce0 2023-10-08T13:50:06.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe386aa8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380197380 con 0x7fe380105b80 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe37ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3801051c0 0x7fe38019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe37ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3801051c0 0x7fe38019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54424/0 (socket says 172.21.15.57:54424) 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe37ffff700 1 -- 172.21.15.57:0/3355890074 learned_addr learned my addr 172.21.15.57:0/3355890074 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe384844700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe380067ce0 0x7fe38019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe385045700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe380105b80 0x7fe3801969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe37ffff700 1 -- 172.21.15.57:0/3355890074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe380105b80 msgr2=0x7fe3801969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe37ffff700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe380105b80 0x7fe3801969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe37ffff700 1 -- 172.21.15.57:0/3355890074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe380067ce0 msgr2=0x7fe38019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe37ffff700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe380067ce0 0x7fe38019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe37ffff700 1 -- 172.21.15.57:0/3355890074 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740095f0 con 0x7fe3801051c0 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.456+0000 7fe37ffff700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3801051c0 0x7fe38019ce00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00c9b0 tx=0x7fe36c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:06.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.457+0000 7fe37dffb700 1 -- 172.21.15.57:0/3355890074 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c007b00 con 0x7fe3801051c0 2023-10-08T13:50:06.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.457+0000 7fe37dffb700 1 -- 172.21.15.57:0/3355890074 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36c007c60 con 0x7fe3801051c0 2023-10-08T13:50:06.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.457+0000 7fe386aa8700 1 -- 172.21.15.57:0/3355890074 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe380197660 con 0x7fe3801051c0 2023-10-08T13:50:06.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.457+0000 7fe37dffb700 1 -- 172.21.15.57:0/3355890074 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c018660 con 0x7fe3801051c0 2023-10-08T13:50:06.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.457+0000 7fe386aa8700 1 -- 172.21.15.57:0/3355890074 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3801a38b0 con 0x7fe3801051c0 2023-10-08T13:50:06.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.458+0000 7fe386aa8700 1 -- 172.21.15.57:0/3355890074 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe380108c70 con 0x7fe3801051c0 2023-10-08T13:50:06.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.459+0000 7fe37dffb700 1 -- 172.21.15.57:0/3355890074 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe36c018880 con 0x7fe3801051c0 2023-10-08T13:50:06.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.459+0000 7fe37dffb700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3700a1550 0x7fe3700a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:06.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.459+0000 7fe37dffb700 1 -- 172.21.15.57:0/3355890074 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fe36c0cd020 con 0x7fe3801051c0 2023-10-08T13:50:06.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.459+0000 7fe384844700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3700a1550 0x7fe3700a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:06.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.460+0000 7fe384844700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3700a1550 0x7fe3700a3a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe374009fd0 tx=0x7fe374005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:06.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.464+0000 7fe37dffb700 1 -- 172.21.15.57:0/3355890074 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe36c0963b0 con 0x7fe3801051c0 2023-10-08T13:50:06.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:06 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/942567254' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-08T13:50:06.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:06 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] up:active 2023-10-08T13:50:06.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:06 smithi115 ceph-mon[104408]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-08T13:50:06.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:06 smithi115 ceph-mon[104408]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-08T13:50:06.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:06 smithi115 ceph-mon[104408]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-10-08T13:50:06.690 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:06 smithi115 ceph-mon[104408]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-10-08T13:50:06.690 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:06 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3119493863' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-08T13:50:06.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:06.703+0000 7fe386aa8700 1 -- 172.21.15.57:0/3355890074 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3800675a0 con 0x7fe3700a1550 2023-10-08T13:50:06.710 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/942567254' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-08T13:50:06.710 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] up:active 2023-10-08T13:50:06.710 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-mon[99476]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-08T13:50:06.710 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-mon[99476]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-08T13:50:06.710 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-mon[99476]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-10-08T13:50:06.710 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-mon[99476]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-10-08T13:50:06.710 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3119493863' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-08T13:50:06.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:06 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/942567254' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-08T13:50:06.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:06 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] up:active 2023-10-08T13:50:06.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:06 smithi099 ceph-mon[103849]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-08T13:50:06.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:06 smithi099 ceph-mon[103849]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-08T13:50:06.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:06 smithi099 ceph-mon[103849]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-10-08T13:50:06.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:06 smithi099 ceph-mon[103849]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-10-08T13:50:06.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:06 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3119493863' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-08T13:50:06.987 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:50:06.708+0000 7fe49fb89700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T13:50:06.988 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:50:06.708+0000 7fe49fb89700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T13:50:06.988 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:50:06.708+0000 7fe49fb89700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T13:50:06.988 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:50:06.708+0000 7fe49fb89700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T13:50:06.988 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:50:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:50:06.708+0000 7fe49fb89700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T13:50:07.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.350+0000 7fe37dffb700 1 -- 172.21.15.57:0/3355890074 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe3800675a0 con 0x7fe3700a1550 2023-10-08T13:50:07.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 -- 172.21.15.57:0/3355890074 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3700a1550 msgr2=0x7fe3700a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:07.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3700a1550 0x7fe3700a3a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe374009fd0 tx=0x7fe374005fb0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 -- 172.21.15.57:0/3355890074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3801051c0 msgr2=0x7fe38019ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:07.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3801051c0 0x7fe38019ce00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00c9b0 tx=0x7fe36c00cd70 comp rx=0 tx=0).stop 2023-10-08T13:50:07.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 -- 172.21.15.57:0/3355890074 shutdown_connections 2023-10-08T13:50:07.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe380105b80 0x7fe3801969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3700a1550 0x7fe3700a3a10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe380067ce0 0x7fe38019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 --2- 172.21.15.57:0/3355890074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3801051c0 0x7fe38019ce00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 -- 172.21.15.57:0/3355890074 >> 172.21.15.57:0/3355890074 conn(0x7fe380076780 msgr2=0x7fe3800fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:07.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 -- 172.21.15.57:0/3355890074 shutdown_connections 2023-10-08T13:50:07.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.353+0000 7fe386aa8700 1 -- 172.21.15.57:0/3355890074 wait complete. 2023-10-08T13:50:07.368 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-10-08T13:50:07.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 -- 172.21.15.57:0/1609494085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88dc0fffe0 msgr2=0x7f88dc10c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:07.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 --2- 172.21.15.57:0/1609494085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88dc0fffe0 0x7f88dc10c4e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88d400b0a0 tx=0x7f88d400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 -- 172.21.15.57:0/1609494085 shutdown_connections 2023-10-08T13:50:07.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 --2- 172.21.15.57:0/1609494085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88dc0fffe0 0x7f88dc10c4e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 --2- 172.21.15.57:0/1609494085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88dc1056b0 0x7f88dc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 --2- 172.21.15.57:0/1609494085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc0ff620 0x7f88dc0ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 -- 172.21.15.57:0/1609494085 >> 172.21.15.57:0/1609494085 conn(0x7f88dc076780 msgr2=0x7f88dc076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 -- 172.21.15.57:0/1609494085 shutdown_connections 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 -- 172.21.15.57:0/1609494085 wait complete. 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 Processor -- start 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.505+0000 7f88e42ee700 1 -- start start 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e42ee700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc0ff620 0x7f88dc19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e42ee700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88dc0fffe0 0x7f88dc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e42ee700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88dc1056b0 0x7f88dc196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e42ee700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc196f80 con 0x7f88dc0fffe0 2023-10-08T13:50:07.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e42ee700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc1970f0 con 0x7f88dc0ff620 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e42ee700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88dc1972b0 con 0x7f88dc1056b0 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e1889700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88dc0fffe0 0x7f88dc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e1889700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88dc0fffe0 0x7f88dc19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54456/0 (socket says 172.21.15.57:54456) 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e1889700 1 -- 172.21.15.57:0/2338503798 learned_addr learned my addr 172.21.15.57:0/2338503798 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e208a700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc0ff620 0x7f88dc19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e288b700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88dc1056b0 0x7f88dc196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e1889700 1 -- 172.21.15.57:0/2338503798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88dc1056b0 msgr2=0x7f88dc196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e1889700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88dc1056b0 0x7f88dc196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e1889700 1 -- 172.21.15.57:0/2338503798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc0ff620 msgr2=0x7f88dc19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e1889700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc0ff620 0x7f88dc19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e1889700 1 -- 172.21.15.57:0/2338503798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d4009d00 con 0x7f88dc0fffe0 2023-10-08T13:50:07.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e1889700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88dc0fffe0 0x7f88dc19cd30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f88d800d980 tx=0x7f88d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:07.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88d37fe700 1 -- 172.21.15.57:0/2338503798 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8004220 con 0x7f88dc0fffe0 2023-10-08T13:50:07.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88d37fe700 1 -- 172.21.15.57:0/2338503798 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88d8007640 con 0x7f88dc0fffe0 2023-10-08T13:50:07.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.506+0000 7f88e42ee700 1 -- 172.21.15.57:0/2338503798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88dc197590 con 0x7f88dc0fffe0 2023-10-08T13:50:07.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.507+0000 7f88d37fe700 1 -- 172.21.15.57:0/2338503798 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8005610 con 0x7f88dc0fffe0 2023-10-08T13:50:07.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.507+0000 7f88e42ee700 1 -- 172.21.15.57:0/2338503798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88dc1a37e0 con 0x7f88dc0fffe0 2023-10-08T13:50:07.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.507+0000 7f88e42ee700 1 -- 172.21.15.57:0/2338503798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88dc100d50 con 0x7f88dc0fffe0 2023-10-08T13:50:07.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.509+0000 7f88d37fe700 1 -- 172.21.15.57:0/2338503798 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f88d8004750 con 0x7f88dc0fffe0 2023-10-08T13:50:07.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.509+0000 7f88d37fe700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88c80a1550 0x7f88c80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:07.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.509+0000 7f88e208a700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88c80a1550 0x7f88c80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:07.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.509+0000 7f88d37fe700 1 -- 172.21.15.57:0/2338503798 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f88d80cd3a0 con 0x7f88dc0fffe0 2023-10-08T13:50:07.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.510+0000 7f88e208a700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88c80a1550 0x7f88c80a3a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f88dc077ab0 tx=0x7f88cc007400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:07.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.513+0000 7f88d37fe700 1 -- 172.21.15.57:0/2338503798 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f88d8096730 con 0x7f88dc0fffe0 2023-10-08T13:50:07.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:07 smithi115 ceph-mon[104408]: pgmap v252: 129 pgs: 129 active+clean; 582 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2023-10-08T13:50:07.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:07 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3119493863' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-08T13:50:07.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:07 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] up:active 2023-10-08T13:50:07.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:07 smithi115 ceph-mon[104408]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-08T13:50:07.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:07 smithi115 ceph-mon[104408]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-10-08T13:50:07.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:07 smithi115 ceph-mon[104408]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2023-10-08T13:50:07.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:07 smithi115 ceph-mon[104408]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-10-08T13:50:07.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:07 smithi115 ceph-mon[104408]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:07.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:07 smithi057 ceph-mon[99476]: pgmap v252: 129 pgs: 129 active+clean; 582 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2023-10-08T13:50:07.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:07 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3119493863' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-08T13:50:07.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:07 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] up:active 2023-10-08T13:50:07.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:07 smithi057 ceph-mon[99476]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-08T13:50:07.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:07 smithi057 ceph-mon[99476]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-10-08T13:50:07.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:07 smithi057 ceph-mon[99476]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2023-10-08T13:50:07.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:07 smithi057 ceph-mon[99476]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-10-08T13:50:07.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:07 smithi057 ceph-mon[99476]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:07.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.755+0000 7f88e42ee700 1 -- 172.21.15.57:0/2338503798 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f88dc10f4f0 con 0x7f88c80a1550 2023-10-08T13:50:07.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.759+0000 7f88d37fe700 1 -- 172.21.15.57:0/2338503798 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f88dc10f4f0 con 0x7f88c80a1550 2023-10-08T13:50:07.759 INFO:teuthology.orchestra.run.smithi057.stdout:/volumes/_nogroup/sv_0/d13b4afb-cd16-49d2-a6b3-78c0f9641493 2023-10-08T13:50:07.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.761+0000 7f88e42ee700 1 -- 172.21.15.57:0/2338503798 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88c80a1550 msgr2=0x7f88c80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:07.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.761+0000 7f88e42ee700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88c80a1550 0x7f88c80a3a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f88dc077ab0 tx=0x7f88cc007400 comp rx=0 tx=0).stop 2023-10-08T13:50:07.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.761+0000 7f88e42ee700 1 -- 172.21.15.57:0/2338503798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88dc0fffe0 msgr2=0x7f88dc19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:07.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.761+0000 7f88e42ee700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88dc0fffe0 0x7f88dc19cd30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f88d800d980 tx=0x7f88d800dc90 comp rx=0 tx=0).stop 2023-10-08T13:50:07.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.761+0000 7f88e42ee700 1 -- 172.21.15.57:0/2338503798 shutdown_connections 2023-10-08T13:50:07.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.761+0000 7f88e42ee700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88dc1056b0 0x7f88dc196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.761+0000 7f88e42ee700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f88c80a1550 0x7f88c80a3a10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.761+0000 7f88e42ee700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88dc0ff620 0x7f88dc19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.761+0000 7f88e42ee700 1 --2- 172.21.15.57:0/2338503798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88dc0fffe0 0x7f88dc19cd30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.761+0000 7f88e42ee700 1 -- 172.21.15.57:0/2338503798 >> 172.21.15.57:0/2338503798 conn(0x7f88dc076780 msgr2=0x7f88dc067b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:07.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.762+0000 7f88e42ee700 1 -- 172.21.15.57:0/2338503798 shutdown_connections 2023-10-08T13:50:07.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.762+0000 7f88e42ee700 1 -- 172.21.15.57:0/2338503798 wait complete. 2023-10-08T13:50:07.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:07 smithi099 ceph-mon[103849]: pgmap v252: 129 pgs: 129 active+clean; 582 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2023-10-08T13:50:07.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:07 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3119493863' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-08T13:50:07.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:07 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] up:active 2023-10-08T13:50:07.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:07 smithi099 ceph-mon[103849]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-08T13:50:07.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:07 smithi099 ceph-mon[103849]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-10-08T13:50:07.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:07 smithi099 ceph-mon[103849]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2023-10-08T13:50:07.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:07 smithi099 ceph-mon[103849]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-10-08T13:50:07.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:07 smithi099 ceph-mon[103849]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:07.774 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-10-08T13:50:07.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.905+0000 7f098d6fe700 1 -- 172.21.15.57:0/3091667818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09880ff640 msgr2=0x7f09880ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.905+0000 7f098d6fe700 1 --2- 172.21.15.57:0/3091667818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09880ff640 0x7f09880ffac0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0978009990 tx=0x7f0978009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.905+0000 7f098d6fe700 1 -- 172.21.15.57:0/3091667818 shutdown_connections 2023-10-08T13:50:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.905+0000 7f098d6fe700 1 --2- 172.21.15.57:0/3091667818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988100000 0x7f098810c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.905+0000 7f098d6fe700 1 --2- 172.21.15.57:0/3091667818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0988105630 0x7f0988105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.905+0000 7f098d6fe700 1 --2- 172.21.15.57:0/3091667818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f09880ff640 0x7f09880ffac0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.905+0000 7f098d6fe700 1 -- 172.21.15.57:0/3091667818 >> 172.21.15.57:0/3091667818 conn(0x7f0988076790 msgr2=0x7f0988076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.905+0000 7f098d6fe700 1 -- 172.21.15.57:0/3091667818 shutdown_connections 2023-10-08T13:50:07.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.905+0000 7f098d6fe700 1 -- 172.21.15.57:0/3091667818 wait complete. 2023-10-08T13:50:07.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.906+0000 7f098d6fe700 1 Processor -- start 2023-10-08T13:50:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.906+0000 7f098d6fe700 1 -- start start 2023-10-08T13:50:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.906+0000 7f098d6fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09880ff640 0x7f098819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.906+0000 7f098d6fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0988100000 0x7f098819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.906+0000 7f098d6fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0988105630 0x7f0988195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:07.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.906+0000 7f098d6fe700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0988195e90 con 0x7f0988105630 2023-10-08T13:50:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.906+0000 7f098d6fe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0988196000 con 0x7f09880ff640 2023-10-08T13:50:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.906+0000 7f098d6fe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09881961c0 con 0x7f0988100000 2023-10-08T13:50:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.906+0000 7f09877fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0988105630 0x7f0988195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f09877fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0988105630 0x7f0988195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54492/0 (socket says 172.21.15.57:54492) 2023-10-08T13:50:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f09877fe700 1 -- 172.21.15.57:0/871503667 learned_addr learned my addr 172.21.15.57:0/871503667 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f09867fc700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0988100000 0x7f098819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f0986ffd700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09880ff640 0x7f098819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f09877fe700 1 -- 172.21.15.57:0/871503667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0988100000 msgr2=0x7f098819bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:07.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f09877fe700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0988100000 0x7f098819bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f09877fe700 1 -- 172.21.15.57:0/871503667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09880ff640 msgr2=0x7f098819b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f09877fe700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09880ff640 0x7f098819b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f09877fe700 1 -- 172.21.15.57:0/871503667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09780095f0 con 0x7f0988105630 2023-10-08T13:50:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f09877fe700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0988105630 0x7f0988195820 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0980007d70 tx=0x7f098000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f096ffff700 1 -- 172.21.15.57:0/871503667 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098000ac90 con 0x7f0988105630 2023-10-08T13:50:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f096ffff700 1 -- 172.21.15.57:0/871503667 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f098000adf0 con 0x7f0988105630 2023-10-08T13:50:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f098d6fe700 1 -- 172.21.15.57:0/871503667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09881964a0 con 0x7f0988105630 2023-10-08T13:50:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f096ffff700 1 -- 172.21.15.57:0/871503667 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09800115e0 con 0x7f0988105630 2023-10-08T13:50:07.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.907+0000 7f098d6fe700 1 -- 172.21.15.57:0/871503667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09881a26e0 con 0x7f0988105630 2023-10-08T13:50:07.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.908+0000 7f098d6fe700 1 -- 172.21.15.57:0/871503667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0988107c40 con 0x7f0988105630 2023-10-08T13:50:07.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.909+0000 7f096ffff700 1 -- 172.21.15.57:0/871503667 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f0980012420 con 0x7f0988105630 2023-10-08T13:50:07.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.910+0000 7f096ffff700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f09700a14d0 0x7f09700a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:07.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.910+0000 7f0986ffd700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f09700a14d0 0x7f09700a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:07.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.910+0000 7f096ffff700 1 -- 172.21.15.57:0/871503667 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f09800cd210 con 0x7f0988105630 2023-10-08T13:50:07.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.910+0000 7f0986ffd700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f09700a14d0 0x7f09700a3990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f097c00abd0 tx=0x7f097c00a380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:07.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:07.914+0000 7f096ffff700 1 -- 172.21.15.57:0/871503667 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f09800965a0 con 0x7f0988105630 2023-10-08T13:50:08.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.154+0000 7f098d6fe700 1 -- 172.21.15.57:0/871503667 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f0988196ce0 con 0x7f09700a14d0 2023-10-08T13:50:08.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.181+0000 7f096ffff700 1 -- 172.21.15.57:0/871503667 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0988196ce0 con 0x7f09700a14d0 2023-10-08T13:50:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 -- 172.21.15.57:0/871503667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f09700a14d0 msgr2=0x7f09700a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f09700a14d0 0x7f09700a3990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f097c00abd0 tx=0x7f097c00a380 comp rx=0 tx=0).stop 2023-10-08T13:50:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 -- 172.21.15.57:0/871503667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0988105630 msgr2=0x7f0988195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0988105630 0x7f0988195820 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0980007d70 tx=0x7f098000a340 comp rx=0 tx=0).stop 2023-10-08T13:50:08.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 -- 172.21.15.57:0/871503667 shutdown_connections 2023-10-08T13:50:08.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0988105630 0x7f0988195820 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f09700a14d0 0x7f09700a3990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09880ff640 0x7f098819b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 --2- 172.21.15.57:0/871503667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0988100000 0x7f098819bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 -- 172.21.15.57:0/871503667 >> 172.21.15.57:0/871503667 conn(0x7f0988076790 msgr2=0x7f09880fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:08.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 -- 172.21.15.57:0/871503667 shutdown_connections 2023-10-08T13:50:08.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.183+0000 7f098d6fe700 1 -- 172.21.15.57:0/871503667 wait complete. 2023-10-08T13:50:08.195 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-10-08T13:50:08.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.324+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/2863064670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f480ff620 msgr2=0x7f2f480ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.324+0000 7f2f4f6ae700 1 --2- 172.21.15.57:0/2863064670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f480ff620 0x7f2f480ffaa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009990 tx=0x7f2f38009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.324+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/2863064670 shutdown_connections 2023-10-08T13:50:08.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.324+0000 7f2f4f6ae700 1 --2- 172.21.15.57:0/2863064670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f480fffe0 0x7f2f4810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.324+0000 7f2f4f6ae700 1 --2- 172.21.15.57:0/2863064670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f481056b0 0x7f2f48105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.324+0000 7f2f4f6ae700 1 --2- 172.21.15.57:0/2863064670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f480ff620 0x7f2f480ffaa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.324+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/2863064670 >> 172.21.15.57:0/2863064670 conn(0x7f2f48076780 msgr2=0x7f2f48076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.324+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/2863064670 shutdown_connections 2023-10-08T13:50:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.324+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/2863064670 wait complete. 2023-10-08T13:50:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.325+0000 7f2f4f6ae700 1 Processor -- start 2023-10-08T13:50:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.325+0000 7f2f4f6ae700 1 -- start start 2023-10-08T13:50:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.325+0000 7f2f4f6ae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f480ff620 0x7f2f4819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.325+0000 7f2f4f6ae700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f480fffe0 0x7f2f4819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:08.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.325+0000 7f2f4f6ae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f481056b0 0x7f2f48195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:08.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.325+0000 7f2f4f6ae700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48195df0 con 0x7f2f480fffe0 2023-10-08T13:50:08.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.325+0000 7f2f4f6ae700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48195f60 con 0x7f2f481056b0 2023-10-08T13:50:08.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.325+0000 7f2f4f6ae700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48196120 con 0x7f2f480ff620 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4cc49700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f480fffe0 0x7f2f4819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4cc49700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f480fffe0 0x7f2f4819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54506/0 (socket says 172.21.15.57:54506) 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4d44a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f480ff620 0x7f2f4819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4cc49700 1 -- 172.21.15.57:0/1851328413 learned_addr learned my addr 172.21.15.57:0/1851328413 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4dc4b700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f481056b0 0x7f2f48195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4cc49700 1 -- 172.21.15.57:0/1851328413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f480ff620 msgr2=0x7f2f4819b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4cc49700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f480ff620 0x7f2f4819b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4cc49700 1 -- 172.21.15.57:0/1851328413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f481056b0 msgr2=0x7f2f48195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4cc49700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f481056b0 0x7f2f48195790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4cc49700 1 -- 172.21.15.57:0/1851328413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f380095f0 con 0x7f2f480fffe0 2023-10-08T13:50:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4cc49700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f480fffe0 0x7f2f4819bb50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800f6a0 tx=0x7f2f3800f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:08.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f3e7fc700 1 -- 172.21.15.57:0/1851328413 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3801d070 con 0x7f2f480fffe0 2023-10-08T13:50:08.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f3e7fc700 1 -- 172.21.15.57:0/1851328413 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f3800fbf0 con 0x7f2f480fffe0 2023-10-08T13:50:08.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f3e7fc700 1 -- 172.21.15.57:0/1851328413 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38017650 con 0x7f2f480fffe0 2023-10-08T13:50:08.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/1851328413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f48196400 con 0x7f2f480fffe0 2023-10-08T13:50:08.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.326+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/1851328413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f481a2600 con 0x7f2f480fffe0 2023-10-08T13:50:08.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.327+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/1851328413 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f48107c30 con 0x7f2f480fffe0 2023-10-08T13:50:08.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.328+0000 7f2f3e7fc700 1 -- 172.21.15.57:0/1851328413 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f2f38022470 con 0x7f2f480fffe0 2023-10-08T13:50:08.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.329+0000 7f2f3e7fc700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2f340a1550 0x7f2f340a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:08.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.329+0000 7f2f3e7fc700 1 -- 172.21.15.57:0/1851328413 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f2f380ce3b0 con 0x7f2f480fffe0 2023-10-08T13:50:08.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.329+0000 7f2f4d44a700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2f340a1550 0x7f2f340a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:08.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.330+0000 7f2f4d44a700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2f340a1550 0x7f2f340a3a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2f48077ab0 tx=0x7f2f4400a3b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:08.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.333+0000 7f2f3e7fc700 1 -- 172.21.15.57:0/1851328413 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2f380978f0 con 0x7f2f480fffe0 2023-10-08T13:50:08.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:08 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] up:active 2023-10-08T13:50:08.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:08 smithi057 ceph-mon[99476]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-08T13:50:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:08 smithi057 ceph-mon[99476]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-08T13:50:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:08 smithi057 ceph-mon[99476]: Cluster is now healthy 2023-10-08T13:50:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:08 smithi057 ceph-mon[99476]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2023-10-08T13:50:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:08 smithi057 ceph-mon[99476]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-10-08T13:50:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:08 smithi057 ceph-mon[99476]: daemon mds.h is now active in filesystem cephfs as rank 4 2023-10-08T13:50:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:08 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] up:active 2023-10-08T13:50:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:08 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:50:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:08 smithi057 ceph-mon[99476]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:08.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.569+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/1851328413 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f48196c00 con 0x7f2f340a1550 2023-10-08T13:50:08.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.573+0000 7f2f3e7fc700 1 -- 172.21.15.57:0/1851328413 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f2f48196c00 con 0x7f2f340a1550 2023-10-08T13:50:08.574 INFO:teuthology.orchestra.run.smithi057.stdout:/volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725 2023-10-08T13:50:08.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/1851328413 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2f340a1550 msgr2=0x7f2f340a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2f340a1550 0x7f2f340a3a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2f48077ab0 tx=0x7f2f4400a3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/1851328413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f480fffe0 msgr2=0x7f2f4819bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f480fffe0 0x7f2f4819bb50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800f6a0 tx=0x7f2f3800f6d0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/1851328413 shutdown_connections 2023-10-08T13:50:08.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2f340a1550 0x7f2f340a3a10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f481056b0 0x7f2f48195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f480ff620 0x7f2f4819b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 --2- 172.21.15.57:0/1851328413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f480fffe0 0x7f2f4819bb50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/1851328413 >> 172.21.15.57:0/1851328413 conn(0x7f2f48076780 msgr2=0x7f2f480fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:08.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/1851328413 shutdown_connections 2023-10-08T13:50:08.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.576+0000 7f2f4f6ae700 1 -- 172.21.15.57:0/1851328413 wait complete. 2023-10-08T13:50:08.588 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:50:08.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:08 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] up:active 2023-10-08T13:50:08.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:08 smithi115 ceph-mon[104408]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-08T13:50:08.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:08 smithi115 ceph-mon[104408]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-08T13:50:08.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:08 smithi115 ceph-mon[104408]: Cluster is now healthy 2023-10-08T13:50:08.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:08 smithi115 ceph-mon[104408]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2023-10-08T13:50:08.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:08 smithi115 ceph-mon[104408]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-10-08T13:50:08.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:08 smithi115 ceph-mon[104408]: daemon mds.h is now active in filesystem cephfs as rank 4 2023-10-08T13:50:08.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:08 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] up:active 2023-10-08T13:50:08.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:08 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:50:08.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:08 smithi115 ceph-mon[104408]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:08.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.714+0000 7f3abd355700 1 -- 172.21.15.57:0/2726869730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ab8101850 msgr2=0x7f3ab8101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.714+0000 7f3abd355700 1 --2- 172.21.15.57:0/2726869730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ab8101850 0x7f3ab8101c30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009990 tx=0x7f3aa8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.714+0000 7f3abd355700 1 -- 172.21.15.57:0/2726869730 shutdown_connections 2023-10-08T13:50:08.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.714+0000 7f3abd355700 1 --2- 172.21.15.57:0/2726869730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab810ac20 0x7f3ab810d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.714+0000 7f3abd355700 1 --2- 172.21.15.57:0/2726869730 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ab8101850 0x7f3ab8101c30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.714+0000 7f3abd355700 1 --2- 172.21.15.57:0/2726869730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab8102170 0x7f3ab810a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.714+0000 7f3abd355700 1 -- 172.21.15.57:0/2726869730 >> 172.21.15.57:0/2726869730 conn(0x7f3ab80fb030 msgr2=0x7f3ab80fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.714+0000 7f3abd355700 1 -- 172.21.15.57:0/2726869730 shutdown_connections 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.714+0000 7f3abd355700 1 -- 172.21.15.57:0/2726869730 wait complete. 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3abd355700 1 Processor -- start 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3abd355700 1 -- start start 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3abd355700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab8101850 0x7f3ab80ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3abd355700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ab8102170 0x7f3ab80ff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3abd355700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab810ac20 0x7f3ab80fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:08.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3abd355700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab81a4ef0 con 0x7f3ab8102170 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3abd355700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab81a5060 con 0x7f3ab810ac20 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3abd355700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab81a5220 con 0x7f3ab8101850 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3ab67fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ab8102170 0x7f3ab80ff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3ab67fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ab8102170 0x7f3ab80ff9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54520/0 (socket says 172.21.15.57:54520) 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3ab67fc700 1 -- 172.21.15.57:0/782714939 learned_addr learned my addr 172.21.15.57:0/782714939 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3ab77fe700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab810ac20 0x7f3ab80fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.715+0000 7f3ab6ffd700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab8101850 0x7f3ab80ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3ab67fc700 1 -- 172.21.15.57:0/782714939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab8101850 msgr2=0x7f3ab80ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3ab67fc700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab8101850 0x7f3ab80ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3ab67fc700 1 -- 172.21.15.57:0/782714939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab810ac20 msgr2=0x7f3ab80fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3ab67fc700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab810ac20 0x7f3ab80fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3ab67fc700 1 -- 172.21.15.57:0/782714939 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80095f0 con 0x7f3ab8102170 2023-10-08T13:50:08.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3ab67fc700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ab8102170 0x7f3ab80ff9e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00e9a0 tx=0x7f3aac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:08.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3a9ffff700 1 -- 172.21.15.57:0/782714939 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac009790 con 0x7f3ab8102170 2023-10-08T13:50:08.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3abd355700 1 -- 172.21.15.57:0/782714939 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab81a5500 con 0x7f3ab8102170 2023-10-08T13:50:08.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3a9ffff700 1 -- 172.21.15.57:0/782714939 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aac00cbd0 con 0x7f3ab8102170 2023-10-08T13:50:08.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3a9ffff700 1 -- 172.21.15.57:0/782714939 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac018680 con 0x7f3ab8102170 2023-10-08T13:50:08.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.716+0000 7f3abd355700 1 -- 172.21.15.57:0/782714939 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab81a5a10 con 0x7f3ab8102170 2023-10-08T13:50:08.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.717+0000 7f3abd355700 1 -- 172.21.15.57:0/782714939 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab8105da0 con 0x7f3ab8102170 2023-10-08T13:50:08.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.718+0000 7f3a9ffff700 1 -- 172.21.15.57:0/782714939 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f3aac00cd40 con 0x7f3ab8102170 2023-10-08T13:50:08.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.719+0000 7f3a9ffff700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa00a14d0 0x7f3aa00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:08.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.719+0000 7f3a9ffff700 1 -- 172.21.15.57:0/782714939 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f3aac014070 con 0x7f3ab8102170 2023-10-08T13:50:08.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.719+0000 7f3ab6ffd700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa00a14d0 0x7f3aa00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:08.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.719+0000 7f3ab6ffd700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa00a14d0 0x7f3aa00a3990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8006010 tx=0x7f3aa800b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:08.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.722+0000 7f3a9ffff700 1 -- 172.21.15.57:0/782714939 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3aac096910 con 0x7f3ab8102170 2023-10-08T13:50:08.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.756+0000 7f3a9ffff700 1 -- 172.21.15.57:0/782714939 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f3aac096ce0 con 0x7f3ab8102170 2023-10-08T13:50:08.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:08 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] up:active 2023-10-08T13:50:08.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:08 smithi099 ceph-mon[103849]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-08T13:50:08.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:08 smithi099 ceph-mon[103849]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-08T13:50:08.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:08 smithi099 ceph-mon[103849]: Cluster is now healthy 2023-10-08T13:50:08.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:08 smithi099 ceph-mon[103849]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2023-10-08T13:50:08.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:08 smithi099 ceph-mon[103849]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-10-08T13:50:08.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:08 smithi099 ceph-mon[103849]: daemon mds.h is now active in filesystem cephfs as rank 4 2023-10-08T13:50:08.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:08 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] up:active 2023-10-08T13:50:08.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:08 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:50:08.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:08 smithi099 ceph-mon[103849]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:08.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.983+0000 7f3abd355700 1 -- 172.21.15.57:0/782714939 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ab8100930 con 0x7f3ab8102170 2023-10-08T13:50:08.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.984+0000 7f3a9ffff700 1 -- 172.21.15.57:0/782714939 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f3aac0c0d20 con 0x7f3ab8102170 2023-10-08T13:50:08.984 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:08.984 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:50:07.719573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:50:08.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 -- 172.21.15.57:0/782714939 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa00a14d0 msgr2=0x7f3aa00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa00a14d0 0x7f3aa00a3990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8006010 tx=0x7f3aa800b260 comp rx=0 tx=0).stop 2023-10-08T13:50:08.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 -- 172.21.15.57:0/782714939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ab8102170 msgr2=0x7f3ab80ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:08.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ab8102170 0x7f3ab80ff9e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00e9a0 tx=0x7f3aac00ecb0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 -- 172.21.15.57:0/782714939 shutdown_connections 2023-10-08T13:50:08.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa00a14d0 0x7f3aa00a3990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab810ac20 0x7f3ab80fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab8101850 0x7f3ab80ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 --2- 172.21.15.57:0/782714939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3ab8102170 0x7f3ab80ff9e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:08.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 -- 172.21.15.57:0/782714939 >> 172.21.15.57:0/782714939 conn(0x7f3ab80fb030 msgr2=0x7f3ab80fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:08.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 -- 172.21.15.57:0/782714939 shutdown_connections 2023-10-08T13:50:08.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:08.986+0000 7f3abd355700 1 -- 172.21.15.57:0/782714939 wait complete. 2023-10-08T13:50:08.987 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:50:08.999 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:50:09.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.124+0000 7f1c61354700 1 -- 172.21.15.57:0/2803369961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c5c0ff530 msgr2=0x7f1c5c0ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.125+0000 7f1c61354700 1 --2- 172.21.15.57:0/2803369961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c5c0ff530 0x7f1c5c0ff9b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1c500099c0 tx=0x7f1c50009cd0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.125+0000 7f1c61354700 1 -- 172.21.15.57:0/2803369961 shutdown_connections 2023-10-08T13:50:09.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.125+0000 7f1c61354700 1 --2- 172.21.15.57:0/2803369961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c5c0ffef0 0x7f1c5c10c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.125+0000 7f1c61354700 1 --2- 172.21.15.57:0/2803369961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c5c1055f0 0x7f1c5c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.125+0000 7f1c61354700 1 --2- 172.21.15.57:0/2803369961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c5c0ff530 0x7f1c5c0ff9b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.125+0000 7f1c61354700 1 -- 172.21.15.57:0/2803369961 >> 172.21.15.57:0/2803369961 conn(0x7f1c5c0fb040 msgr2=0x7f1c5c0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:09.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.125+0000 7f1c61354700 1 -- 172.21.15.57:0/2803369961 shutdown_connections 2023-10-08T13:50:09.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.125+0000 7f1c61354700 1 -- 172.21.15.57:0/2803369961 wait complete. 2023-10-08T13:50:09.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.125+0000 7f1c61354700 1 Processor -- start 2023-10-08T13:50:09.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.125+0000 7f1c61354700 1 -- start start 2023-10-08T13:50:09.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c61354700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c5c0ff530 0x7f1c5c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c61354700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c5c0ffef0 0x7f1c5c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c61354700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c5c1055f0 0x7f1c5c1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c61354700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c196f20 con 0x7f1c5c1055f0 2023-10-08T13:50:09.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c61354700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c197090 con 0x7f1c5c0ffef0 2023-10-08T13:50:09.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c61354700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c197250 con 0x7f1c5c0ff530 2023-10-08T13:50:09.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c5c1055f0 0x7f1c5c1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c5c1055f0 0x7f1c5c1968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54536/0 (socket says 172.21.15.57:54536) 2023-10-08T13:50:09.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5b7fe700 1 -- 172.21.15.57:0/171048795 learned_addr learned my addr 172.21.15.57:0/171048795 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:09.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5a7fc700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c5c0ffef0 0x7f1c5c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5affd700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c5c0ff530 0x7f1c5c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5b7fe700 1 -- 172.21.15.57:0/171048795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c5c0ff530 msgr2=0x7f1c5c19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5b7fe700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c5c0ff530 0x7f1c5c19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5b7fe700 1 -- 172.21.15.57:0/171048795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c5c0ffef0 msgr2=0x7f1c5c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5b7fe700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c5c0ffef0 0x7f1c5c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5b7fe700 1 -- 172.21.15.57:0/171048795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c500095f0 con 0x7f1c5c1055f0 2023-10-08T13:50:09.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c5b7fe700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c5c1055f0 0x7f1c5c1968c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400ada0 tx=0x7f1c5400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:09.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.126+0000 7f1c43fff700 1 -- 172.21.15.57:0/171048795 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400f070 con 0x7f1c5c1055f0 2023-10-08T13:50:09.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.127+0000 7f1c43fff700 1 -- 172.21.15.57:0/171048795 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c54007b30 con 0x7f1c5c1055f0 2023-10-08T13:50:09.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.127+0000 7f1c61354700 1 -- 172.21.15.57:0/171048795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c197530 con 0x7f1c5c1055f0 2023-10-08T13:50:09.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.127+0000 7f1c43fff700 1 -- 172.21.15.57:0/171048795 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54005280 con 0x7f1c5c1055f0 2023-10-08T13:50:09.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.127+0000 7f1c61354700 1 -- 172.21.15.57:0/171048795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c1a3780 con 0x7f1c5c1055f0 2023-10-08T13:50:09.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.127+0000 7f1c61354700 1 -- 172.21.15.57:0/171048795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c100c50 con 0x7f1c5c1055f0 2023-10-08T13:50:09.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.129+0000 7f1c43fff700 1 -- 172.21.15.57:0/171048795 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1c5400a090 con 0x7f1c5c1055f0 2023-10-08T13:50:09.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.130+0000 7f1c43fff700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c440a1560 0x7f1c440a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.130+0000 7f1c5affd700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c440a1560 0x7f1c440a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.130+0000 7f1c43fff700 1 -- 172.21.15.57:0/171048795 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f1c540ccd20 con 0x7f1c5c1055f0 2023-10-08T13:50:09.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.131+0000 7f1c5affd700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c440a1560 0x7f1c440a3a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c005950 tx=0x7f1c4c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:09.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.134+0000 7f1c43fff700 1 -- 172.21.15.57:0/171048795 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1c54099a10 con 0x7f1c5c1055f0 2023-10-08T13:50:09.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.340+0000 7f1c61354700 1 -- 172.21.15.57:0/171048795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c5c0678a0 con 0x7f1c5c1055f0 2023-10-08T13:50:09.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.341+0000 7f1c43fff700 1 -- 172.21.15.57:0/171048795 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24224 (secure 0 0 0) 0x7f1c540995a0 con 0x7f1c5c1055f0 2023-10-08T13:50:09.341 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:09.342 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":88,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:49:49.574582+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:50:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.343+0000 7f1c61354700 1 -- 172.21.15.57:0/171048795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c440a1560 msgr2=0x7f1c440a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.343+0000 7f1c61354700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c440a1560 0x7f1c440a3a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c005950 tx=0x7f1c4c00b410 comp rx=0 tx=0).stop 2023-10-08T13:50:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.343+0000 7f1c61354700 1 -- 172.21.15.57:0/171048795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c5c1055f0 msgr2=0x7f1c5c1968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.343+0000 7f1c61354700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c5c1055f0 0x7f1c5c1968c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400ada0 tx=0x7f1c5400c330 comp rx=0 tx=0).stop 2023-10-08T13:50:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.344+0000 7f1c61354700 1 -- 172.21.15.57:0/171048795 shutdown_connections 2023-10-08T13:50:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.344+0000 7f1c61354700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1c5c1055f0 0x7f1c5c1968c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.344+0000 7f1c61354700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c5c0ff530 0x7f1c5c19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.344+0000 7f1c61354700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1c440a1560 0x7f1c440a3a20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.344+0000 7f1c61354700 1 --2- 172.21.15.57:0/171048795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c5c0ffef0 0x7f1c5c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.344+0000 7f1c61354700 1 -- 172.21.15.57:0/171048795 >> 172.21.15.57:0/171048795 conn(0x7f1c5c0fb040 msgr2=0x7f1c5c068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.344+0000 7f1c61354700 1 -- 172.21.15.57:0/171048795 shutdown_connections 2023-10-08T13:50:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.344+0000 7f1c61354700 1 -- 172.21.15.57:0/171048795 wait complete. 2023-10-08T13:50:09.356 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T13:50:09.357 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T13:50:09.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.482+0000 7fc73d07f700 1 -- 172.21.15.57:0/3051822038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc738105600 msgr2=0x7fc7381059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.482+0000 7fc73d07f700 1 --2- 172.21.15.57:0/3051822038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc738105600 0x7fc7381059e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc7280099c0 tx=0x7fc728009cd0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.482+0000 7fc73d07f700 1 -- 172.21.15.57:0/3051822038 shutdown_connections 2023-10-08T13:50:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.482+0000 7fc73d07f700 1 --2- 172.21.15.57:0/3051822038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7380fff00 0x7fc73810c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.482+0000 7fc73d07f700 1 --2- 172.21.15.57:0/3051822038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc738105600 0x7fc7381059e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.482+0000 7fc73d07f700 1 --2- 172.21.15.57:0/3051822038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7380ff540 0x7fc7380ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.482+0000 7fc73d07f700 1 -- 172.21.15.57:0/3051822038 >> 172.21.15.57:0/3051822038 conn(0x7fc7380fb030 msgr2=0x7fc7380fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.482+0000 7fc73d07f700 1 -- 172.21.15.57:0/3051822038 shutdown_connections 2023-10-08T13:50:09.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.482+0000 7fc73d07f700 1 -- 172.21.15.57:0/3051822038 wait complete. 2023-10-08T13:50:09.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.483+0000 7fc73d07f700 1 Processor -- start 2023-10-08T13:50:09.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.483+0000 7fc73d07f700 1 -- start start 2023-10-08T13:50:09.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.483+0000 7fc73d07f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7380ff540 0x7fc73819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.483+0000 7fc73d07f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7380fff00 0x7fc73819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.483+0000 7fc73d07f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc738105600 0x7fc738198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.483+0000 7fc73d07f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7381990e0 con 0x7fc738105600 2023-10-08T13:50:09.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.483+0000 7fc73d07f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738199250 con 0x7fc7380ff540 2023-10-08T13:50:09.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.483+0000 7fc73d07f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738199410 con 0x7fc7380fff00 2023-10-08T13:50:09.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.483+0000 7fc73759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc738105600 0x7fc738198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc738105600 0x7fc738198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54550/0 (socket says 172.21.15.57:54550) 2023-10-08T13:50:09.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73759e700 1 -- 172.21.15.57:0/2084805271 learned_addr learned my addr 172.21.15.57:0/2084805271 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:09.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73659c700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7380fff00 0x7fc73819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc736d9d700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7380ff540 0x7fc73819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73759e700 1 -- 172.21.15.57:0/2084805271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7380fff00 msgr2=0x7fc73819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73759e700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7380fff00 0x7fc73819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73759e700 1 -- 172.21.15.57:0/2084805271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7380ff540 msgr2=0x7fc73819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73759e700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7380ff540 0x7fc73819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73759e700 1 -- 172.21.15.57:0/2084805271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7280095f0 con 0x7fc738105600 2023-10-08T13:50:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73759e700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc738105600 0x7fc738198a80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc730007ee0 tx=0x7fc73000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc71ffff700 1 -- 172.21.15.57:0/2084805271 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730017070 con 0x7fc738105600 2023-10-08T13:50:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc71ffff700 1 -- 172.21.15.57:0/2084805271 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc73000f040 con 0x7fc738105600 2023-10-08T13:50:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73d07f700 1 -- 172.21.15.57:0/2084805271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7381996f0 con 0x7fc738105600 2023-10-08T13:50:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc71ffff700 1 -- 172.21.15.57:0/2084805271 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730005580 con 0x7fc738105600 2023-10-08T13:50:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.484+0000 7fc73d07f700 1 -- 172.21.15.57:0/2084805271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7381a5940 con 0x7fc738105600 2023-10-08T13:50:09.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.485+0000 7fc73d07f700 1 -- 172.21.15.57:0/2084805271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc738100c60 con 0x7fc738105600 2023-10-08T13:50:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.486+0000 7fc71ffff700 1 -- 172.21.15.57:0/2084805271 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fc73000db20 con 0x7fc738105600 2023-10-08T13:50:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.487+0000 7fc71ffff700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7200a1560 0x7fc7200a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.487+0000 7fc736d9d700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7200a1560 0x7fc7200a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.487+0000 7fc71ffff700 1 -- 172.21.15.57:0/2084805271 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7fc7300ccf20 con 0x7fc738105600 2023-10-08T13:50:09.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.487+0000 7fc736d9d700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7200a1560 0x7fc7200a3a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc728009990 tx=0x7fc728003680 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:09.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.490+0000 7fc71ffff700 1 -- 172.21.15.57:0/2084805271 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc730096150 con 0x7fc738105600 2023-10-08T13:50:09.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.688+0000 7fc73d07f700 1 -- 172.21.15.57:0/2084805271 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7381105b0 con 0x7fc7200a1560 2023-10-08T13:50:09.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.704+0000 7fc71ffff700 1 -- 172.21.15.57:0/2084805271 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418865 (secure 0 0 0) 0x7fc7381105b0 con 0x7fc7200a1560 2023-10-08T13:50:09.709 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:09.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 -- 172.21.15.57:0/2084805271 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7200a1560 msgr2=0x7fc7200a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7200a1560 0x7fc7200a3a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc728009990 tx=0x7fc728003680 comp rx=0 tx=0).stop 2023-10-08T13:50:09.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 -- 172.21.15.57:0/2084805271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc738105600 msgr2=0x7fc738198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc738105600 0x7fc738198a80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc730007ee0 tx=0x7fc73000d3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 -- 172.21.15.57:0/2084805271 shutdown_connections 2023-10-08T13:50:09.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc738105600 0x7fc738198a80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc7200a1560 0x7fc7200a3a20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7380ff540 0x7fc73819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 --2- 172.21.15.57:0/2084805271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7380fff00 0x7fc73819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 -- 172.21.15.57:0/2084805271 >> 172.21.15.57:0/2084805271 conn(0x7fc7380fb030 msgr2=0x7fc7380fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 -- 172.21.15.57:0/2084805271 shutdown_connections 2023-10-08T13:50:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.708+0000 7fc73d07f700 1 -- 172.21.15.57:0/2084805271 wait complete. 2023-10-08T13:50:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-08T13:50:09.721 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2023-10-08T13:50:08.104169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597283,"num_objects":48,"num_object_clones":0,"num_object_copies":145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":138,"num_read_kb":120,"num_write":284,"num_write_kb":2966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337648,"kb_used_data":13424,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604176,"statfs":{"total":1151940427776,"available":1151594676224,"internally_reserved":0,"allocated":13746176,"data_stored":7642653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4505,"num_objects":24,"num_object_clones":0,"num_object_copies":73,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":58,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003611"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.154854+0000","last_change":"2023-10-08T13:49:44.786130+0000","last_active":"2023-10-08T13:49:49.154854+0000","last_peered":"2023-10-08T13:49:49.154854+0000","last_clean":"2023-10-08T13:49:49.154854+0000","last_became_active":"2023-10-08T13:49:44.785792+0000","last_became_peered":"2023-10-08T13:49:44.785792+0000","last_unstale":"2023-10-08T13:49:49.154854+0000","last_undegraded":"2023-10-08T13:49:49.154854+0000","last_fullsized":"2023-10-08T13:49:49.154854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:17.046800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149324+0000","last_change":"2023-10-08T13:49:44.777601+0000","last_active":"2023-10-08T13:49:49.149324+0000","last_peered":"2023-10-08T13:49:49.149324+0000","last_clean":"2023-10-08T13:49:49.149324+0000","last_became_active":"2023-10-08T13:49:44.777287+0000","last_became_peered":"2023-10-08T13:49:44.777287+0000","last_unstale":"2023-10-08T13:49:49.149324+0000","last_undegraded":"2023-10-08T13:49:49.149324+0000","last_fullsized":"2023-10-08T13:49:49.149324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:09:17.317629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148154+0000","last_change":"2023-10-08T13:49:44.787035+0000","last_active":"2023-10-08T13:49:49.148154+0000","last_peered":"2023-10-08T13:49:49.148154+0000","last_clean":"2023-10-08T13:49:49.148154+0000","last_became_active":"2023-10-08T13:49:44.785834+0000","last_became_peered":"2023-10-08T13:49:44.785834+0000","last_unstale":"2023-10-08T13:49:49.148154+0000","last_undegraded":"2023-10-08T13:49:49.148154+0000","last_fullsized":"2023-10-08T13:49:49.148154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:11.611404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143282+0000","last_change":"2023-10-08T13:49:44.781937+0000","last_active":"2023-10-08T13:49:49.143282+0000","last_peered":"2023-10-08T13:49:49.143282+0000","last_clean":"2023-10-08T13:49:49.143282+0000","last_became_active":"2023-10-08T13:49:44.780712+0000","last_became_peered":"2023-10-08T13:49:44.780712+0000","last_unstale":"2023-10-08T13:49:49.143282+0000","last_undegraded":"2023-10-08T13:49:49.143282+0000","last_fullsized":"2023-10-08T13:49:49.143282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:02.514007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146636+0000","last_change":"2023-10-08T13:49:44.788661+0000","last_active":"2023-10-08T13:49:49.146636+0000","last_peered":"2023-10-08T13:49:49.146636+0000","last_clean":"2023-10-08T13:49:49.146636+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:49:49.146636+0000","last_undegraded":"2023-10-08T13:49:49.146636+0000","last_fullsized":"2023-10-08T13:49:49.146636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:35.770287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.544588+0000","last_change":"2023-10-08T13:49:44.778519+0000","last_active":"2023-10-08T13:49:49.544588+0000","last_peered":"2023-10-08T13:49:49.544588+0000","last_clean":"2023-10-08T13:49:49.544588+0000","last_became_active":"2023-10-08T13:49:44.778193+0000","last_became_peered":"2023-10-08T13:49:44.778193+0000","last_unstale":"2023-10-08T13:49:49.544588+0000","last_undegraded":"2023-10-08T13:49:49.544588+0000","last_fullsized":"2023-10-08T13:49:49.544588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:12.916019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147737+0000","last_change":"2023-10-08T13:49:44.791759+0000","last_active":"2023-10-08T13:49:49.147737+0000","last_peered":"2023-10-08T13:49:49.147737+0000","last_clean":"2023-10-08T13:49:49.147737+0000","last_became_active":"2023-10-08T13:49:44.791494+0000","last_became_peered":"2023-10-08T13:49:44.791494+0000","last_unstale":"2023-10-08T13:49:49.147737+0000","last_undegraded":"2023-10-08T13:49:49.147737+0000","last_fullsized":"2023-10-08T13:49:49.147737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:55.859609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149225+0000","last_change":"2023-10-08T13:49:44.786831+0000","last_active":"2023-10-08T13:49:49.149225+0000","last_peered":"2023-10-08T13:49:49.149225+0000","last_clean":"2023-10-08T13:49:49.149225+0000","last_became_active":"2023-10-08T13:49:44.786516+0000","last_became_peered":"2023-10-08T13:49:44.786516+0000","last_unstale":"2023-10-08T13:49:49.149225+0000","last_undegraded":"2023-10-08T13:49:49.149225+0000","last_fullsized":"2023-10-08T13:49:49.149225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:07.616821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149102+0000","last_change":"2023-10-08T13:49:44.789827+0000","last_active":"2023-10-08T13:49:49.149102+0000","last_peered":"2023-10-08T13:49:49.149102+0000","last_clean":"2023-10-08T13:49:49.149102+0000","last_became_active":"2023-10-08T13:49:44.786635+0000","last_became_peered":"2023-10-08T13:49:44.786635+0000","last_unstale":"2023-10-08T13:49:49.149102+0000","last_undegraded":"2023-10-08T13:49:49.149102+0000","last_fullsized":"2023-10-08T13:49:49.149102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:35.492437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.545458+0000","last_change":"2023-10-08T13:49:44.784240+0000","last_active":"2023-10-08T13:49:49.545458+0000","last_peered":"2023-10-08T13:49:49.545458+0000","last_clean":"2023-10-08T13:49:49.545458+0000","last_became_active":"2023-10-08T13:49:44.783475+0000","last_became_peered":"2023-10-08T13:49:44.783475+0000","last_unstale":"2023-10-08T13:49:49.545458+0000","last_undegraded":"2023-10-08T13:49:49.545458+0000","last_fullsized":"2023-10-08T13:49:49.545458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:39.730679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148185+0000","last_change":"2023-10-08T13:49:44.790901+0000","last_active":"2023-10-08T13:49:49.148185+0000","last_peered":"2023-10-08T13:49:49.148185+0000","last_clean":"2023-10-08T13:49:49.148185+0000","last_became_active":"2023-10-08T13:49:44.790611+0000","last_became_peered":"2023-10-08T13:49:44.790611+0000","last_unstale":"2023-10-08T13:49:49.148185+0000","last_undegraded":"2023-10-08T13:49:49.148185+0000","last_fullsized":"2023-10-08T13:49:49.148185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:19:21.764105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143349+0000","last_change":"2023-10-08T13:49:44.775226+0000","last_active":"2023-10-08T13:49:49.143349+0000","last_peered":"2023-10-08T13:49:49.143349+0000","last_clean":"2023-10-08T13:49:49.143349+0000","last_became_active":"2023-10-08T13:49:44.774756+0000","last_became_peered":"2023-10-08T13:49:44.774756+0000","last_unstale":"2023-10-08T13:49:49.143349+0000","last_undegraded":"2023-10-08T13:49:49.143349+0000","last_fullsized":"2023-10-08T13:49:49.143349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:57:14.749066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.141527+0000","last_change":"2023-10-08T13:49:44.781610+0000","last_active":"2023-10-08T13:49:49.141527+0000","last_peered":"2023-10-08T13:49:49.141527+0000","last_clean":"2023-10-08T13:49:49.141527+0000","last_became_active":"2023-10-08T13:49:44.781281+0000","last_became_peered":"2023-10-08T13:49:44.781281+0000","last_unstale":"2023-10-08T13:49:49.141527+0000","last_undegraded":"2023-10-08T13:49:49.141527+0000","last_fullsized":"2023-10-08T13:49:49.141527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:50.823909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147896+0000","last_change":"2023-10-08T13:49:44.775836+0000","last_active":"2023-10-08T13:49:49.147896+0000","last_peered":"2023-10-08T13:49:49.147896+0000","last_clean":"2023-10-08T13:49:49.147896+0000","last_became_active":"2023-10-08T13:49:44.775206+0000","last_became_peered":"2023-10-08T13:49:44.775206+0000","last_unstale":"2023-10-08T13:49:49.147896+0000","last_undegraded":"2023-10-08T13:49:49.147896+0000","last_fullsized":"2023-10-08T13:49:49.147896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:51.566800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148540+0000","last_change":"2023-10-08T13:49:44.778495+0000","last_active":"2023-10-08T13:49:49.148540+0000","last_peered":"2023-10-08T13:49:49.148540+0000","last_clean":"2023-10-08T13:49:49.148540+0000","last_became_active":"2023-10-08T13:49:44.778196+0000","last_became_peered":"2023-10-08T13:49:44.778196+0000","last_unstale":"2023-10-08T13:49:49.148540+0000","last_undegraded":"2023-10-08T13:49:49.148540+0000","last_fullsized":"2023-10-08T13:49:49.148540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:57:38.531853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148003+0000","last_change":"2023-10-08T13:49:44.792960+0000","last_active":"2023-10-08T13:49:49.148003+0000","last_peered":"2023-10-08T13:49:49.148003+0000","last_clean":"2023-10-08T13:49:49.148003+0000","last_became_active":"2023-10-08T13:49:44.792674+0000","last_became_peered":"2023-10-08T13:49:44.792674+0000","last_unstale":"2023-10-08T13:49:49.148003+0000","last_undegraded":"2023-10-08T13:49:49.148003+0000","last_fullsized":"2023-10-08T13:49:49.148003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:13.934954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.154810+0000","last_change":"2023-10-08T13:49:44.785971+0000","last_active":"2023-10-08T13:49:49.154810+0000","last_peered":"2023-10-08T13:49:49.154810+0000","last_clean":"2023-10-08T13:49:49.154810+0000","last_became_active":"2023-10-08T13:49:44.785680+0000","last_became_peered":"2023-10-08T13:49:44.785680+0000","last_unstale":"2023-10-08T13:49:49.154810+0000","last_undegraded":"2023-10-08T13:49:49.154810+0000","last_fullsized":"2023-10-08T13:49:49.154810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:35.447519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.161467+0000","last_change":"2023-10-08T13:49:44.787553+0000","last_active":"2023-10-08T13:49:49.161467+0000","last_peered":"2023-10-08T13:49:49.161467+0000","last_clean":"2023-10-08T13:49:49.161467+0000","last_became_active":"2023-10-08T13:49:44.787288+0000","last_became_peered":"2023-10-08T13:49:44.787288+0000","last_unstale":"2023-10-08T13:49:49.161467+0000","last_undegraded":"2023-10-08T13:49:49.161467+0000","last_fullsized":"2023-10-08T13:49:49.161467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:48.386120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143084+0000","last_change":"2023-10-08T13:49:44.789252+0000","last_active":"2023-10-08T13:49:49.143084+0000","last_peered":"2023-10-08T13:49:49.143084+0000","last_clean":"2023-10-08T13:49:49.143084+0000","last_became_active":"2023-10-08T13:49:44.788947+0000","last_became_peered":"2023-10-08T13:49:44.788947+0000","last_unstale":"2023-10-08T13:49:49.143084+0000","last_undegraded":"2023-10-08T13:49:49.143084+0000","last_fullsized":"2023-10-08T13:49:49.143084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:30.266913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.176271+0000","last_change":"2023-10-08T13:49:49.163619+0000","last_active":"2023-10-08T13:49:50.176271+0000","last_peered":"2023-10-08T13:49:50.176271+0000","last_clean":"2023-10-08T13:49:50.176271+0000","last_became_active":"2023-10-08T13:49:49.163020+0000","last_became_peered":"2023-10-08T13:49:49.163020+0000","last_unstale":"2023-10-08T13:49:50.176271+0000","last_undegraded":"2023-10-08T13:49:50.176271+0000","last_fullsized":"2023-10-08T13:49:50.176271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:56.092404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.162203+0000","last_change":"2023-10-08T13:49:43.782863+0000","last_active":"2023-10-08T13:49:49.162203+0000","last_peered":"2023-10-08T13:49:49.162203+0000","last_clean":"2023-10-08T13:49:49.162203+0000","last_became_active":"2023-10-08T13:49:43.782477+0000","last_became_peered":"2023-10-08T13:49:43.782477+0000","last_unstale":"2023-10-08T13:49:49.162203+0000","last_undegraded":"2023-10-08T13:49:49.162203+0000","last_fullsized":"2023-10-08T13:49:49.162203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:10.134907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149259+0000","last_change":"2023-10-08T13:49:44.788452+0000","last_active":"2023-10-08T13:49:49.149259+0000","last_peered":"2023-10-08T13:49:49.149259+0000","last_clean":"2023-10-08T13:49:49.149259+0000","last_became_active":"2023-10-08T13:49:44.788122+0000","last_became_peered":"2023-10-08T13:49:44.788122+0000","last_unstale":"2023-10-08T13:49:49.149259+0000","last_undegraded":"2023-10-08T13:49:49.149259+0000","last_fullsized":"2023-10-08T13:49:49.149259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:25:45.766329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164248+0000","last_change":"2023-10-08T13:49:49.164248+0000","last_active":"2023-10-08T13:49:50.164248+0000","last_peered":"2023-10-08T13:49:50.164248+0000","last_clean":"2023-10-08T13:49:50.164248+0000","last_became_active":"2023-10-08T13:49:49.163624+0000","last_became_peered":"2023-10-08T13:49:49.163624+0000","last_unstale":"2023-10-08T13:49:50.164248+0000","last_undegraded":"2023-10-08T13:49:50.164248+0000","last_fullsized":"2023-10-08T13:49:50.164248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:04.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175015+0000","last_change":"2023-10-08T13:49:43.788026+0000","last_active":"2023-10-08T13:49:50.175015+0000","last_peered":"2023-10-08T13:49:50.175015+0000","last_clean":"2023-10-08T13:49:50.175015+0000","last_became_active":"2023-10-08T13:49:43.787639+0000","last_became_peered":"2023-10-08T13:49:43.787639+0000","last_unstale":"2023-10-08T13:49:50.175015+0000","last_undegraded":"2023-10-08T13:49:50.175015+0000","last_fullsized":"2023-10-08T13:49:50.175015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:25:48.367765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148792+0000","last_change":"2023-10-08T13:49:44.786322+0000","last_active":"2023-10-08T13:49:49.148792+0000","last_peered":"2023-10-08T13:49:49.148792+0000","last_clean":"2023-10-08T13:49:49.148792+0000","last_became_active":"2023-10-08T13:49:44.785953+0000","last_became_peered":"2023-10-08T13:49:44.785953+0000","last_unstale":"2023-10-08T13:49:49.148792+0000","last_undegraded":"2023-10-08T13:49:49.148792+0000","last_fullsized":"2023-10-08T13:49:49.148792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:39.866785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164557+0000","last_change":"2023-10-08T13:49:49.164501+0000","last_active":"2023-10-08T13:49:50.164557+0000","last_peered":"2023-10-08T13:49:50.164557+0000","last_clean":"2023-10-08T13:49:50.164557+0000","last_became_active":"2023-10-08T13:49:49.163994+0000","last_became_peered":"2023-10-08T13:49:49.163994+0000","last_unstale":"2023-10-08T13:49:50.164557+0000","last_undegraded":"2023-10-08T13:49:50.164557+0000","last_fullsized":"2023-10-08T13:49:50.164557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:38.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147836+0000","last_change":"2023-10-08T13:49:43.776239+0000","last_active":"2023-10-08T13:49:49.147836+0000","last_peered":"2023-10-08T13:49:49.147836+0000","last_clean":"2023-10-08T13:49:49.147836+0000","last_became_active":"2023-10-08T13:49:43.775969+0000","last_became_peered":"2023-10-08T13:49:43.775969+0000","last_unstale":"2023-10-08T13:49:49.147836+0000","last_undegraded":"2023-10-08T13:49:49.147836+0000","last_fullsized":"2023-10-08T13:49:49.147836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:24.011655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148812+0000","last_change":"2023-10-08T13:49:44.794912+0000","last_active":"2023-10-08T13:49:49.148812+0000","last_peered":"2023-10-08T13:49:49.148812+0000","last_clean":"2023-10-08T13:49:49.148812+0000","last_became_active":"2023-10-08T13:49:44.794590+0000","last_became_peered":"2023-10-08T13:49:44.794590+0000","last_unstale":"2023-10-08T13:49:49.148812+0000","last_undegraded":"2023-10-08T13:49:49.148812+0000","last_fullsized":"2023-10-08T13:49:49.148812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:51.631386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164345+0000","last_change":"2023-10-08T13:49:49.152149+0000","last_active":"2023-10-08T13:49:50.164345+0000","last_peered":"2023-10-08T13:49:50.164345+0000","last_clean":"2023-10-08T13:49:50.164345+0000","last_became_active":"2023-10-08T13:49:49.151764+0000","last_became_peered":"2023-10-08T13:49:49.151764+0000","last_unstale":"2023-10-08T13:49:50.164345+0000","last_undegraded":"2023-10-08T13:49:50.164345+0000","last_fullsized":"2023-10-08T13:49:50.164345+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:25:08.232556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147334+0000","last_change":"2023-10-08T13:49:43.774505+0000","last_active":"2023-10-08T13:49:49.147334+0000","last_peered":"2023-10-08T13:49:49.147334+0000","last_clean":"2023-10-08T13:49:49.147334+0000","last_became_active":"2023-10-08T13:49:43.774301+0000","last_became_peered":"2023-10-08T13:49:43.774301+0000","last_unstale":"2023-10-08T13:49:49.147334+0000","last_undegraded":"2023-10-08T13:49:49.147334+0000","last_fullsized":"2023-10-08T13:49:49.147334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:59:31.708311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148676+0000","last_change":"2023-10-08T13:49:44.783915+0000","last_active":"2023-10-08T13:49:49.148676+0000","last_peered":"2023-10-08T13:49:49.148676+0000","last_clean":"2023-10-08T13:49:49.148676+0000","last_became_active":"2023-10-08T13:49:44.783628+0000","last_became_peered":"2023-10-08T13:49:44.783628+0000","last_unstale":"2023-10-08T13:49:49.148676+0000","last_undegraded":"2023-10-08T13:49:49.148676+0000","last_fullsized":"2023-10-08T13:49:49.148676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:43.759681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175588+0000","last_change":"2023-10-08T13:49:49.156241+0000","last_active":"2023-10-08T13:49:50.175588+0000","last_peered":"2023-10-08T13:49:50.175588+0000","last_clean":"2023-10-08T13:49:50.175588+0000","last_became_active":"2023-10-08T13:49:49.155652+0000","last_became_peered":"2023-10-08T13:49:49.155652+0000","last_unstale":"2023-10-08T13:49:50.175588+0000","last_undegraded":"2023-10-08T13:49:50.175588+0000","last_fullsized":"2023-10-08T13:49:50.175588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:04.283647+0000","last_change":"2023-10-08T13:49:43.781263+0000","last_active":"2023-10-08T13:50:04.283647+0000","last_peered":"2023-10-08T13:50:04.283647+0000","last_clean":"2023-10-08T13:50:04.283647+0000","last_became_active":"2023-10-08T13:49:43.780981+0000","last_became_peered":"2023-10-08T13:49:43.780981+0000","last_unstale":"2023-10-08T13:50:04.283647+0000","last_undegraded":"2023-10-08T13:50:04.283647+0000","last_fullsized":"2023-10-08T13:50:04.283647+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:22:18.130688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146867+0000","last_change":"2023-10-08T13:49:44.780935+0000","last_active":"2023-10-08T13:49:49.146867+0000","last_peered":"2023-10-08T13:49:49.146867+0000","last_clean":"2023-10-08T13:49:49.146867+0000","last_became_active":"2023-10-08T13:49:44.780613+0000","last_became_peered":"2023-10-08T13:49:44.780613+0000","last_unstale":"2023-10-08T13:49:49.146867+0000","last_undegraded":"2023-10-08T13:49:49.146867+0000","last_fullsized":"2023-10-08T13:49:49.146867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:57.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175911+0000","last_change":"2023-10-08T13:49:49.163652+0000","last_active":"2023-10-08T13:49:50.175911+0000","last_peered":"2023-10-08T13:49:50.175911+0000","last_clean":"2023-10-08T13:49:50.175911+0000","last_became_active":"2023-10-08T13:49:49.163140+0000","last_became_peered":"2023-10-08T13:49:49.163140+0000","last_unstale":"2023-10-08T13:49:50.175911+0000","last_undegraded":"2023-10-08T13:49:50.175911+0000","last_fullsized":"2023-10-08T13:49:50.175911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:48.733675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:06.314146+0000","last_change":"2023-10-08T13:49:43.775799+0000","last_active":"2023-10-08T13:50:06.314146+0000","last_peered":"2023-10-08T13:50:06.314146+0000","last_clean":"2023-10-08T13:50:06.314146+0000","last_became_active":"2023-10-08T13:49:43.775470+0000","last_became_peered":"2023-10-08T13:49:43.775470+0000","last_unstale":"2023-10-08T13:50:06.314146+0000","last_undegraded":"2023-10-08T13:50:06.314146+0000","last_fullsized":"2023-10-08T13:50:06.314146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:50.404554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149414+0000","last_change":"2023-10-08T13:49:44.789392+0000","last_active":"2023-10-08T13:49:49.149414+0000","last_peered":"2023-10-08T13:49:49.149414+0000","last_clean":"2023-10-08T13:49:49.149414+0000","last_became_active":"2023-10-08T13:49:44.789079+0000","last_became_peered":"2023-10-08T13:49:44.789079+0000","last_unstale":"2023-10-08T13:49:49.149414+0000","last_undegraded":"2023-10-08T13:49:49.149414+0000","last_fullsized":"2023-10-08T13:49:49.149414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:57:56.239854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175082+0000","last_change":"2023-10-08T13:49:49.166702+0000","last_active":"2023-10-08T13:49:50.175082+0000","last_peered":"2023-10-08T13:49:50.175082+0000","last_clean":"2023-10-08T13:49:50.175082+0000","last_became_active":"2023-10-08T13:49:49.166256+0000","last_became_peered":"2023-10-08T13:49:49.166256+0000","last_unstale":"2023-10-08T13:49:50.175082+0000","last_undegraded":"2023-10-08T13:49:50.175082+0000","last_fullsized":"2023-10-08T13:49:50.175082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:18:57.565040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:05.297390+0000","last_change":"2023-10-08T13:49:43.785085+0000","last_active":"2023-10-08T13:50:05.297390+0000","last_peered":"2023-10-08T13:50:05.297390+0000","last_clean":"2023-10-08T13:50:05.297390+0000","last_became_active":"2023-10-08T13:49:43.784746+0000","last_became_peered":"2023-10-08T13:49:43.784746+0000","last_unstale":"2023-10-08T13:50:05.297390+0000","last_undegraded":"2023-10-08T13:50:05.297390+0000","last_fullsized":"2023-10-08T13:50:05.297390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:13.471834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149099+0000","last_change":"2023-10-08T13:49:44.785670+0000","last_active":"2023-10-08T13:49:49.149099+0000","last_peered":"2023-10-08T13:49:49.149099+0000","last_clean":"2023-10-08T13:49:49.149099+0000","last_became_active":"2023-10-08T13:49:44.785221+0000","last_became_peered":"2023-10-08T13:49:44.785221+0000","last_unstale":"2023-10-08T13:49:49.149099+0000","last_undegraded":"2023-10-08T13:49:49.149099+0000","last_fullsized":"2023-10-08T13:49:49.149099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:25.415168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.389166+0000","last_change":"2023-10-08T13:49:49.156294+0000","last_active":"2023-10-08T13:49:50.389166+0000","last_peered":"2023-10-08T13:49:50.389166+0000","last_clean":"2023-10-08T13:49:50.389166+0000","last_became_active":"2023-10-08T13:49:49.155821+0000","last_became_peered":"2023-10-08T13:49:49.155821+0000","last_unstale":"2023-10-08T13:49:50.389166+0000","last_undegraded":"2023-10-08T13:49:50.389166+0000","last_fullsized":"2023-10-08T13:49:50.389166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:59:43.831634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174567+0000","last_change":"2023-10-08T13:49:43.780052+0000","last_active":"2023-10-08T13:49:50.174567+0000","last_peered":"2023-10-08T13:49:50.174567+0000","last_clean":"2023-10-08T13:49:50.174567+0000","last_became_active":"2023-10-08T13:49:43.779731+0000","last_became_peered":"2023-10-08T13:49:43.779731+0000","last_unstale":"2023-10-08T13:49:50.174567+0000","last_undegraded":"2023-10-08T13:49:50.174567+0000","last_fullsized":"2023-10-08T13:49:50.174567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:38.800783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.159663+0000","last_change":"2023-10-08T13:49:44.787880+0000","last_active":"2023-10-08T13:49:49.159663+0000","last_peered":"2023-10-08T13:49:49.159663+0000","last_clean":"2023-10-08T13:49:49.159663+0000","last_became_active":"2023-10-08T13:49:44.787591+0000","last_became_peered":"2023-10-08T13:49:44.787591+0000","last_unstale":"2023-10-08T13:49:49.159663+0000","last_undegraded":"2023-10-08T13:49:49.159663+0000","last_fullsized":"2023-10-08T13:49:49.159663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:53.078903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175396+0000","last_change":"2023-10-08T13:49:49.155004+0000","last_active":"2023-10-08T13:49:50.175396+0000","last_peered":"2023-10-08T13:49:50.175396+0000","last_clean":"2023-10-08T13:49:50.175396+0000","last_became_active":"2023-10-08T13:49:49.154503+0000","last_became_peered":"2023-10-08T13:49:49.154503+0000","last_unstale":"2023-10-08T13:49:50.175396+0000","last_undegraded":"2023-10-08T13:49:50.175396+0000","last_fullsized":"2023-10-08T13:49:50.175396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:21.565816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:04.283502+0000","last_change":"2023-10-08T13:49:43.775819+0000","last_active":"2023-10-08T13:50:04.283502+0000","last_peered":"2023-10-08T13:50:04.283502+0000","last_clean":"2023-10-08T13:50:04.283502+0000","last_became_active":"2023-10-08T13:49:43.775578+0000","last_became_peered":"2023-10-08T13:49:43.775578+0000","last_unstale":"2023-10-08T13:50:04.283502+0000","last_undegraded":"2023-10-08T13:50:04.283502+0000","last_fullsized":"2023-10-08T13:50:04.283502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:31.518191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148667+0000","last_change":"2023-10-08T13:49:44.785869+0000","last_active":"2023-10-08T13:49:49.148667+0000","last_peered":"2023-10-08T13:49:49.148667+0000","last_clean":"2023-10-08T13:49:49.148667+0000","last_became_active":"2023-10-08T13:49:44.785566+0000","last_became_peered":"2023-10-08T13:49:44.785566+0000","last_unstale":"2023-10-08T13:49:49.148667+0000","last_undegraded":"2023-10-08T13:49:49.148667+0000","last_fullsized":"2023-10-08T13:49:49.148667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:29.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.162217+0000","last_change":"2023-10-08T13:49:49.162075+0000","last_active":"2023-10-08T13:49:50.162217+0000","last_peered":"2023-10-08T13:49:50.162217+0000","last_clean":"2023-10-08T13:49:50.162217+0000","last_became_active":"2023-10-08T13:49:49.161485+0000","last_became_peered":"2023-10-08T13:49:49.161485+0000","last_unstale":"2023-10-08T13:49:50.162217+0000","last_undegraded":"2023-10-08T13:49:50.162217+0000","last_fullsized":"2023-10-08T13:49:50.162217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:16.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:07.323980+0000","last_change":"2023-10-08T13:49:43.776555+0000","last_active":"2023-10-08T13:50:07.323980+0000","last_peered":"2023-10-08T13:50:07.323980+0000","last_clean":"2023-10-08T13:50:07.323980+0000","last_became_active":"2023-10-08T13:49:43.776259+0000","last_became_peered":"2023-10-08T13:49:43.776259+0000","last_unstale":"2023-10-08T13:50:07.323980+0000","last_undegraded":"2023-10-08T13:50:07.323980+0000","last_fullsized":"2023-10-08T13:50:07.323980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:34.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148899+0000","last_change":"2023-10-08T13:49:44.779253+0000","last_active":"2023-10-08T13:49:49.148899+0000","last_peered":"2023-10-08T13:49:49.148899+0000","last_clean":"2023-10-08T13:49:49.148899+0000","last_became_active":"2023-10-08T13:49:44.778902+0000","last_became_peered":"2023-10-08T13:49:44.778902+0000","last_unstale":"2023-10-08T13:49:49.148899+0000","last_undegraded":"2023-10-08T13:49:49.148899+0000","last_fullsized":"2023-10-08T13:49:49.148899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:31:01.091424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164641+0000","last_change":"2023-10-08T13:49:49.164859+0000","last_active":"2023-10-08T13:49:50.164641+0000","last_peered":"2023-10-08T13:49:50.164641+0000","last_clean":"2023-10-08T13:49:50.164641+0000","last_became_active":"2023-10-08T13:49:49.164221+0000","last_became_peered":"2023-10-08T13:49:49.164221+0000","last_unstale":"2023-10-08T13:49:50.164641+0000","last_undegraded":"2023-10-08T13:49:50.164641+0000","last_fullsized":"2023-10-08T13:49:50.164641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:22.484934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175938+0000","last_change":"2023-10-08T13:49:43.788136+0000","last_active":"2023-10-08T13:49:50.175938+0000","last_peered":"2023-10-08T13:49:50.175938+0000","last_clean":"2023-10-08T13:49:50.175938+0000","last_became_active":"2023-10-08T13:49:43.787910+0000","last_became_peered":"2023-10-08T13:49:43.787910+0000","last_unstale":"2023-10-08T13:49:50.175938+0000","last_undegraded":"2023-10-08T13:49:50.175938+0000","last_fullsized":"2023-10-08T13:49:50.175938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:00:32.897972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148905+0000","last_change":"2023-10-08T13:49:44.786833+0000","last_active":"2023-10-08T13:49:49.148905+0000","last_peered":"2023-10-08T13:49:49.148905+0000","last_clean":"2023-10-08T13:49:49.148905+0000","last_became_active":"2023-10-08T13:49:44.786523+0000","last_became_peered":"2023-10-08T13:49:44.786523+0000","last_unstale":"2023-10-08T13:49:49.148905+0000","last_undegraded":"2023-10-08T13:49:49.148905+0000","last_fullsized":"2023-10-08T13:49:49.148905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:36:22.156252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.162461+0000","last_change":"2023-10-08T13:49:49.160653+0000","last_active":"2023-10-08T13:49:50.162461+0000","last_peered":"2023-10-08T13:49:50.162461+0000","last_clean":"2023-10-08T13:49:50.162461+0000","last_became_active":"2023-10-08T13:49:49.160253+0000","last_became_peered":"2023-10-08T13:49:49.160253+0000","last_unstale":"2023-10-08T13:49:50.162461+0000","last_undegraded":"2023-10-08T13:49:50.162461+0000","last_fullsized":"2023-10-08T13:49:50.162461+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:07.838617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'12","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:04.282958+0000","last_change":"2023-10-08T13:49:43.777736+0000","last_active":"2023-10-08T13:50:04.282958+0000","last_peered":"2023-10-08T13:50:04.282958+0000","last_clean":"2023-10-08T13:50:04.282958+0000","last_became_active":"2023-10-08T13:49:43.777431+0000","last_became_peered":"2023-10-08T13:49:43.777431+0000","last_unstale":"2023-10-08T13:50:04.282958+0000","last_undegraded":"2023-10-08T13:50:04.282958+0000","last_fullsized":"2023-10-08T13:50:04.282958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:12.907196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147573+0000","last_change":"2023-10-08T13:49:44.789122+0000","last_active":"2023-10-08T13:49:49.147573+0000","last_peered":"2023-10-08T13:49:49.147573+0000","last_clean":"2023-10-08T13:49:49.147573+0000","last_became_active":"2023-10-08T13:49:44.788831+0000","last_became_peered":"2023-10-08T13:49:44.788831+0000","last_unstale":"2023-10-08T13:49:49.147573+0000","last_undegraded":"2023-10-08T13:49:49.147573+0000","last_fullsized":"2023-10-08T13:49:49.147573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:45.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.176310+0000","last_change":"2023-10-08T13:49:49.157319+0000","last_active":"2023-10-08T13:49:50.176310+0000","last_peered":"2023-10-08T13:49:50.176310+0000","last_clean":"2023-10-08T13:49:50.176310+0000","last_became_active":"2023-10-08T13:49:49.156897+0000","last_became_peered":"2023-10-08T13:49:49.156897+0000","last_unstale":"2023-10-08T13:49:50.176310+0000","last_undegraded":"2023-10-08T13:49:50.176310+0000","last_fullsized":"2023-10-08T13:49:50.176310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:25.460707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:04.283662+0000","last_change":"2023-10-08T13:49:43.776699+0000","last_active":"2023-10-08T13:50:04.283662+0000","last_peered":"2023-10-08T13:50:04.283662+0000","last_clean":"2023-10-08T13:50:04.283662+0000","last_became_active":"2023-10-08T13:49:43.776293+0000","last_became_peered":"2023-10-08T13:49:43.776293+0000","last_unstale":"2023-10-08T13:50:04.283662+0000","last_undegraded":"2023-10-08T13:50:04.283662+0000","last_fullsized":"2023-10-08T13:50:04.283662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:19.056804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.141415+0000","last_change":"2023-10-08T13:49:44.789915+0000","last_active":"2023-10-08T13:49:49.141415+0000","last_peered":"2023-10-08T13:49:49.141415+0000","last_clean":"2023-10-08T13:49:49.141415+0000","last_became_active":"2023-10-08T13:49:44.789617+0000","last_became_peered":"2023-10-08T13:49:44.789617+0000","last_unstale":"2023-10-08T13:49:49.141415+0000","last_undegraded":"2023-10-08T13:49:49.141415+0000","last_fullsized":"2023-10-08T13:49:49.141415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:06.286155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175275+0000","last_change":"2023-10-08T13:49:49.165946+0000","last_active":"2023-10-08T13:49:50.175275+0000","last_peered":"2023-10-08T13:49:50.175275+0000","last_clean":"2023-10-08T13:49:50.175275+0000","last_became_active":"2023-10-08T13:49:49.165581+0000","last_became_peered":"2023-10-08T13:49:49.165581+0000","last_unstale":"2023-10-08T13:49:50.175275+0000","last_undegraded":"2023-10-08T13:49:50.175275+0000","last_fullsized":"2023-10-08T13:49:50.175275+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:13.772198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:04.283236+0000","last_change":"2023-10-08T13:49:43.779161+0000","last_active":"2023-10-08T13:50:04.283236+0000","last_peered":"2023-10-08T13:50:04.283236+0000","last_clean":"2023-10-08T13:50:04.283236+0000","last_became_active":"2023-10-08T13:49:43.778862+0000","last_became_peered":"2023-10-08T13:49:43.778862+0000","last_unstale":"2023-10-08T13:50:04.283236+0000","last_undegraded":"2023-10-08T13:50:04.283236+0000","last_fullsized":"2023-10-08T13:50:04.283236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:57:44.075322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147263+0000","last_change":"2023-10-08T13:49:44.782072+0000","last_active":"2023-10-08T13:49:49.147263+0000","last_peered":"2023-10-08T13:49:49.147263+0000","last_clean":"2023-10-08T13:49:49.147263+0000","last_became_active":"2023-10-08T13:49:44.781756+0000","last_became_peered":"2023-10-08T13:49:44.781756+0000","last_unstale":"2023-10-08T13:49:49.147263+0000","last_undegraded":"2023-10-08T13:49:49.147263+0000","last_fullsized":"2023-10-08T13:49:49.147263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:32.877818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.162940+0000","last_change":"2023-10-08T13:49:49.167457+0000","last_active":"2023-10-08T13:49:50.162940+0000","last_peered":"2023-10-08T13:49:50.162940+0000","last_clean":"2023-10-08T13:49:50.162940+0000","last_became_active":"2023-10-08T13:49:49.166861+0000","last_became_peered":"2023-10-08T13:49:49.166861+0000","last_unstale":"2023-10-08T13:49:50.162940+0000","last_undegraded":"2023-10-08T13:49:50.162940+0000","last_fullsized":"2023-10-08T13:49:50.162940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:55.805271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142732+0000","last_change":"2023-10-08T13:49:43.782822+0000","last_active":"2023-10-08T13:49:49.142732+0000","last_peered":"2023-10-08T13:49:49.142732+0000","last_clean":"2023-10-08T13:49:49.142732+0000","last_became_active":"2023-10-08T13:49:43.782524+0000","last_became_peered":"2023-10-08T13:49:43.782524+0000","last_unstale":"2023-10-08T13:49:49.142732+0000","last_undegraded":"2023-10-08T13:49:49.142732+0000","last_fullsized":"2023-10-08T13:49:49.142732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:11.847851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147586+0000","last_change":"2023-10-08T13:49:44.794296+0000","last_active":"2023-10-08T13:49:49.147586+0000","last_peered":"2023-10-08T13:49:49.147586+0000","last_clean":"2023-10-08T13:49:49.147586+0000","last_became_active":"2023-10-08T13:49:44.793987+0000","last_became_peered":"2023-10-08T13:49:44.793987+0000","last_unstale":"2023-10-08T13:49:49.147586+0000","last_undegraded":"2023-10-08T13:49:49.147586+0000","last_fullsized":"2023-10-08T13:49:49.147586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:41.562569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174829+0000","last_change":"2023-10-08T13:49:49.165016+0000","last_active":"2023-10-08T13:49:50.174829+0000","last_peered":"2023-10-08T13:49:50.174829+0000","last_clean":"2023-10-08T13:49:50.174829+0000","last_became_active":"2023-10-08T13:49:49.164531+0000","last_became_peered":"2023-10-08T13:49:49.164531+0000","last_unstale":"2023-10-08T13:49:50.174829+0000","last_undegraded":"2023-10-08T13:49:50.174829+0000","last_fullsized":"2023-10-08T13:49:50.174829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:45.014512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146511+0000","last_change":"2023-10-08T13:49:43.786929+0000","last_active":"2023-10-08T13:49:49.146511+0000","last_peered":"2023-10-08T13:49:49.146511+0000","last_clean":"2023-10-08T13:49:49.146511+0000","last_became_active":"2023-10-08T13:49:43.786641+0000","last_became_peered":"2023-10-08T13:49:43.786641+0000","last_unstale":"2023-10-08T13:49:49.146511+0000","last_undegraded":"2023-10-08T13:49:49.146511+0000","last_fullsized":"2023-10-08T13:49:49.146511+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:30.054067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148947+0000","last_change":"2023-10-08T13:49:44.786718+0000","last_active":"2023-10-08T13:49:49.148947+0000","last_peered":"2023-10-08T13:49:49.148947+0000","last_clean":"2023-10-08T13:49:49.148947+0000","last_became_active":"2023-10-08T13:49:44.786413+0000","last_became_peered":"2023-10-08T13:49:44.786413+0000","last_unstale":"2023-10-08T13:49:49.148947+0000","last_undegraded":"2023-10-08T13:49:49.148947+0000","last_fullsized":"2023-10-08T13:49:49.148947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.302426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174625+0000","last_change":"2023-10-08T13:49:49.152226+0000","last_active":"2023-10-08T13:49:50.174625+0000","last_peered":"2023-10-08T13:49:50.174625+0000","last_clean":"2023-10-08T13:49:50.174625+0000","last_became_active":"2023-10-08T13:49:49.151772+0000","last_became_peered":"2023-10-08T13:49:49.151772+0000","last_unstale":"2023-10-08T13:49:50.174625+0000","last_undegraded":"2023-10-08T13:49:50.174625+0000","last_fullsized":"2023-10-08T13:49:50.174625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:53.856496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146977+0000","last_change":"2023-10-08T13:49:43.777648+0000","last_active":"2023-10-08T13:49:49.146977+0000","last_peered":"2023-10-08T13:49:49.146977+0000","last_clean":"2023-10-08T13:49:49.146977+0000","last_became_active":"2023-10-08T13:49:43.777361+0000","last_became_peered":"2023-10-08T13:49:43.777361+0000","last_unstale":"2023-10-08T13:49:49.146977+0000","last_undegraded":"2023-10-08T13:49:49.146977+0000","last_fullsized":"2023-10-08T13:49:49.146977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:53:19.939752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.155523+0000","last_change":"2023-10-08T13:49:44.782805+0000","last_active":"2023-10-08T13:49:49.155523+0000","last_peered":"2023-10-08T13:49:49.155523+0000","last_clean":"2023-10-08T13:49:49.155523+0000","last_became_active":"2023-10-08T13:49:44.782485+0000","last_became_peered":"2023-10-08T13:49:44.782485+0000","last_unstale":"2023-10-08T13:49:49.155523+0000","last_undegraded":"2023-10-08T13:49:49.155523+0000","last_fullsized":"2023-10-08T13:49:49.155523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:19.805734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.167364+0000","last_change":"2023-10-08T13:49:49.546226+0000","last_active":"2023-10-08T13:49:50.167364+0000","last_peered":"2023-10-08T13:49:50.167364+0000","last_clean":"2023-10-08T13:49:50.167364+0000","last_became_active":"2023-10-08T13:49:49.545626+0000","last_became_peered":"2023-10-08T13:49:49.545626+0000","last_unstale":"2023-10-08T13:49:50.167364+0000","last_undegraded":"2023-10-08T13:49:50.167364+0000","last_fullsized":"2023-10-08T13:49:50.167364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:37.184944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.160970+0000","last_change":"2023-10-08T13:49:43.784705+0000","last_active":"2023-10-08T13:49:49.160970+0000","last_peered":"2023-10-08T13:49:49.160970+0000","last_clean":"2023-10-08T13:49:49.160970+0000","last_became_active":"2023-10-08T13:49:43.784391+0000","last_became_peered":"2023-10-08T13:49:43.784391+0000","last_unstale":"2023-10-08T13:49:49.160970+0000","last_undegraded":"2023-10-08T13:49:49.160970+0000","last_fullsized":"2023-10-08T13:49:49.160970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:45.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.545021+0000","last_change":"2023-10-08T13:49:44.789845+0000","last_active":"2023-10-08T13:49:49.545021+0000","last_peered":"2023-10-08T13:49:49.545021+0000","last_clean":"2023-10-08T13:49:49.545021+0000","last_became_active":"2023-10-08T13:49:44.789580+0000","last_became_peered":"2023-10-08T13:49:44.789580+0000","last_unstale":"2023-10-08T13:49:49.545021+0000","last_undegraded":"2023-10-08T13:49:49.545021+0000","last_fullsized":"2023-10-08T13:49:49.545021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:58.098757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175738+0000","last_change":"2023-10-08T13:49:49.153277+0000","last_active":"2023-10-08T13:49:50.175738+0000","last_peered":"2023-10-08T13:49:50.175738+0000","last_clean":"2023-10-08T13:49:50.175738+0000","last_became_active":"2023-10-08T13:49:49.152575+0000","last_became_peered":"2023-10-08T13:49:49.152575+0000","last_unstale":"2023-10-08T13:49:50.175738+0000","last_undegraded":"2023-10-08T13:49:50.175738+0000","last_fullsized":"2023-10-08T13:49:50.175738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:35.677823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:07.735809+0000","last_change":"2023-10-08T13:49:43.778442+0000","last_active":"2023-10-08T13:50:07.735809+0000","last_peered":"2023-10-08T13:50:07.735809+0000","last_clean":"2023-10-08T13:50:07.735809+0000","last_became_active":"2023-10-08T13:49:43.774799+0000","last_became_peered":"2023-10-08T13:49:43.774799+0000","last_unstale":"2023-10-08T13:50:07.735809+0000","last_undegraded":"2023-10-08T13:50:07.735809+0000","last_fullsized":"2023-10-08T13:50:07.735809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:15.464736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.544687+0000","last_change":"2023-10-08T13:49:44.780097+0000","last_active":"2023-10-08T13:49:49.544687+0000","last_peered":"2023-10-08T13:49:49.544687+0000","last_clean":"2023-10-08T13:49:49.544687+0000","last_became_active":"2023-10-08T13:49:44.779774+0000","last_became_peered":"2023-10-08T13:49:44.779774+0000","last_unstale":"2023-10-08T13:49:49.544687+0000","last_undegraded":"2023-10-08T13:49:49.544687+0000","last_fullsized":"2023-10-08T13:49:49.544687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:19.132295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175603+0000","last_change":"2023-10-08T13:49:49.547129+0000","last_active":"2023-10-08T13:49:50.175603+0000","last_peered":"2023-10-08T13:49:50.175603+0000","last_clean":"2023-10-08T13:49:50.175603+0000","last_became_active":"2023-10-08T13:49:49.546694+0000","last_became_peered":"2023-10-08T13:49:49.546694+0000","last_unstale":"2023-10-08T13:49:50.175603+0000","last_undegraded":"2023-10-08T13:49:50.175603+0000","last_fullsized":"2023-10-08T13:49:50.175603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:45.524650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149911+0000","last_change":"2023-10-08T13:49:44.779875+0000","last_active":"2023-10-08T13:49:49.149911+0000","last_peered":"2023-10-08T13:49:49.149911+0000","last_clean":"2023-10-08T13:49:49.149911+0000","last_became_active":"2023-10-08T13:49:44.779570+0000","last_became_peered":"2023-10-08T13:49:44.779570+0000","last_unstale":"2023-10-08T13:49:49.149911+0000","last_undegraded":"2023-10-08T13:49:49.149911+0000","last_fullsized":"2023-10-08T13:49:49.149911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:02.981998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:05.298544+0000","last_change":"2023-10-08T13:49:43.787574+0000","last_active":"2023-10-08T13:50:05.298544+0000","last_peered":"2023-10-08T13:50:05.298544+0000","last_clean":"2023-10-08T13:50:05.298544+0000","last_became_active":"2023-10-08T13:49:43.787261+0000","last_became_peered":"2023-10-08T13:49:43.787261+0000","last_unstale":"2023-10-08T13:50:05.298544+0000","last_undegraded":"2023-10-08T13:50:05.298544+0000","last_fullsized":"2023-10-08T13:50:05.298544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:28.676135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:52.935506+0000","last_change":"2023-10-08T13:49:49.163856+0000","last_active":"2023-10-08T13:49:52.935506+0000","last_peered":"2023-10-08T13:49:52.935506+0000","last_clean":"2023-10-08T13:49:52.935506+0000","last_became_active":"2023-10-08T13:49:49.163456+0000","last_became_peered":"2023-10-08T13:49:49.163456+0000","last_unstale":"2023-10-08T13:49:52.935506+0000","last_undegraded":"2023-10-08T13:49:52.935506+0000","last_fullsized":"2023-10-08T13:49:52.935506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:31.654309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149129+0000","last_change":"2023-10-08T13:49:44.792935+0000","last_active":"2023-10-08T13:49:49.149129+0000","last_peered":"2023-10-08T13:49:49.149129+0000","last_clean":"2023-10-08T13:49:49.149129+0000","last_became_active":"2023-10-08T13:49:44.792667+0000","last_became_peered":"2023-10-08T13:49:44.792667+0000","last_unstale":"2023-10-08T13:49:49.149129+0000","last_undegraded":"2023-10-08T13:49:49.149129+0000","last_fullsized":"2023-10-08T13:49:49.149129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:40.964452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142640+0000","last_change":"2023-10-08T13:49:43.781424+0000","last_active":"2023-10-08T13:49:49.142640+0000","last_peered":"2023-10-08T13:49:49.142640+0000","last_clean":"2023-10-08T13:49:49.142640+0000","last_became_active":"2023-10-08T13:49:43.780930+0000","last_became_peered":"2023-10-08T13:49:43.780930+0000","last_unstale":"2023-10-08T13:49:49.142640+0000","last_undegraded":"2023-10-08T13:49:49.142640+0000","last_fullsized":"2023-10-08T13:49:49.142640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:56:50.397289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":323,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146439+0000","last_change":"2023-10-08T13:48:45.192219+0000","last_active":"2023-10-08T13:49:49.146439+0000","last_peered":"2023-10-08T13:49:49.146439+0000","last_clean":"2023-10-08T13:49:49.146439+0000","last_became_active":"2023-10-08T13:48:45.191813+0000","last_became_peered":"2023-10-08T13:48:45.191813+0000","last_unstale":"2023-10-08T13:49:49.146439+0000","last_undegraded":"2023-10-08T13:49:49.146439+0000","last_fullsized":"2023-10-08T13:49:49.146439+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_clean_scrub_stamp":"2023-10-08T13:45:51.040695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.303937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175811+0000","last_change":"2023-10-08T13:49:49.546181+0000","last_active":"2023-10-08T13:49:50.175811+0000","last_peered":"2023-10-08T13:49:50.175811+0000","last_clean":"2023-10-08T13:49:50.175811+0000","last_became_active":"2023-10-08T13:49:49.545726+0000","last_became_peered":"2023-10-08T13:49:49.545726+0000","last_unstale":"2023-10-08T13:49:50.175811+0000","last_undegraded":"2023-10-08T13:49:50.175811+0000","last_fullsized":"2023-10-08T13:49:50.175811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:45.370909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.141204+0000","last_change":"2023-10-08T13:49:44.785544+0000","last_active":"2023-10-08T13:49:49.141204+0000","last_peered":"2023-10-08T13:49:49.141204+0000","last_clean":"2023-10-08T13:49:49.141204+0000","last_became_active":"2023-10-08T13:49:44.785007+0000","last_became_peered":"2023-10-08T13:49:44.785007+0000","last_unstale":"2023-10-08T13:49:49.141204+0000","last_undegraded":"2023-10-08T13:49:49.141204+0000","last_fullsized":"2023-10-08T13:49:49.141204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:16:55.058693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:05.296736+0000","last_change":"2023-10-08T13:49:43.779207+0000","last_active":"2023-10-08T13:50:05.296736+0000","last_peered":"2023-10-08T13:50:05.296736+0000","last_clean":"2023-10-08T13:50:05.296736+0000","last_became_active":"2023-10-08T13:49:43.778846+0000","last_became_peered":"2023-10-08T13:49:43.778846+0000","last_unstale":"2023-10-08T13:50:05.296736+0000","last_undegraded":"2023-10-08T13:50:05.296736+0000","last_fullsized":"2023-10-08T13:50:05.296736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:45.868187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175105+0000","last_change":"2023-10-08T13:49:49.162156+0000","last_active":"2023-10-08T13:49:50.175105+0000","last_peered":"2023-10-08T13:49:50.175105+0000","last_clean":"2023-10-08T13:49:50.175105+0000","last_became_active":"2023-10-08T13:49:49.161720+0000","last_became_peered":"2023-10-08T13:49:49.161720+0000","last_unstale":"2023-10-08T13:49:50.175105+0000","last_undegraded":"2023-10-08T13:49:50.175105+0000","last_fullsized":"2023-10-08T13:49:50.175105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:13.499915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147669+0000","last_change":"2023-10-08T13:49:43.782322+0000","last_active":"2023-10-08T13:49:49.147669+0000","last_peered":"2023-10-08T13:49:49.147669+0000","last_clean":"2023-10-08T13:49:49.147669+0000","last_became_active":"2023-10-08T13:49:43.781938+0000","last_became_peered":"2023-10-08T13:49:43.781938+0000","last_unstale":"2023-10-08T13:49:49.147669+0000","last_undegraded":"2023-10-08T13:49:49.147669+0000","last_fullsized":"2023-10-08T13:49:49.147669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:57.627507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142286+0000","last_change":"2023-10-08T13:49:44.778340+0000","last_active":"2023-10-08T13:49:49.142286+0000","last_peered":"2023-10-08T13:49:49.142286+0000","last_clean":"2023-10-08T13:49:49.142286+0000","last_became_active":"2023-10-08T13:49:44.778000+0000","last_became_peered":"2023-10-08T13:49:44.778000+0000","last_unstale":"2023-10-08T13:49:49.142286+0000","last_undegraded":"2023-10-08T13:49:49.142286+0000","last_fullsized":"2023-10-08T13:49:49.142286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:23.350449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174377+0000","last_change":"2023-10-08T13:49:49.545915+0000","last_active":"2023-10-08T13:49:50.174377+0000","last_peered":"2023-10-08T13:49:50.174377+0000","last_clean":"2023-10-08T13:49:50.174377+0000","last_became_active":"2023-10-08T13:49:49.545396+0000","last_became_peered":"2023-10-08T13:49:49.545396+0000","last_unstale":"2023-10-08T13:49:50.174377+0000","last_undegraded":"2023-10-08T13:49:50.174377+0000","last_fullsized":"2023-10-08T13:49:50.174377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:07:03.850707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.188889+0000","last_change":"2023-10-08T13:49:43.776479+0000","last_active":"2023-10-08T13:49:49.188889+0000","last_peered":"2023-10-08T13:49:49.188889+0000","last_clean":"2023-10-08T13:49:49.188889+0000","last_became_active":"2023-10-08T13:49:43.776130+0000","last_became_peered":"2023-10-08T13:49:43.776130+0000","last_unstale":"2023-10-08T13:49:49.188889+0000","last_undegraded":"2023-10-08T13:49:49.188889+0000","last_fullsized":"2023-10-08T13:49:49.188889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.760660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149144+0000","last_change":"2023-10-08T13:49:44.793864+0000","last_active":"2023-10-08T13:49:49.149144+0000","last_peered":"2023-10-08T13:49:49.149144+0000","last_clean":"2023-10-08T13:49:49.149144+0000","last_became_active":"2023-10-08T13:49:44.793469+0000","last_became_peered":"2023-10-08T13:49:44.793469+0000","last_unstale":"2023-10-08T13:49:49.149144+0000","last_undegraded":"2023-10-08T13:49:49.149144+0000","last_fullsized":"2023-10-08T13:49:49.149144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:28.669711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.176066+0000","last_change":"2023-10-08T13:49:49.150696+0000","last_active":"2023-10-08T13:49:50.176066+0000","last_peered":"2023-10-08T13:49:50.176066+0000","last_clean":"2023-10-08T13:49:50.176066+0000","last_became_active":"2023-10-08T13:49:49.150188+0000","last_became_peered":"2023-10-08T13:49:49.150188+0000","last_unstale":"2023-10-08T13:49:50.176066+0000","last_undegraded":"2023-10-08T13:49:50.176066+0000","last_fullsized":"2023-10-08T13:49:50.176066+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:51:35.775240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142893+0000","last_change":"2023-10-08T13:49:43.786245+0000","last_active":"2023-10-08T13:49:49.142893+0000","last_peered":"2023-10-08T13:49:49.142893+0000","last_clean":"2023-10-08T13:49:49.142893+0000","last_became_active":"2023-10-08T13:49:43.785986+0000","last_became_peered":"2023-10-08T13:49:43.785986+0000","last_unstale":"2023-10-08T13:49:49.142893+0000","last_undegraded":"2023-10-08T13:49:49.142893+0000","last_fullsized":"2023-10-08T13:49:49.142893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:53:59.643447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149418+0000","last_change":"2023-10-08T13:49:44.790494+0000","last_active":"2023-10-08T13:49:49.149418+0000","last_peered":"2023-10-08T13:49:49.149418+0000","last_clean":"2023-10-08T13:49:49.149418+0000","last_became_active":"2023-10-08T13:49:44.790091+0000","last_became_peered":"2023-10-08T13:49:44.790091+0000","last_unstale":"2023-10-08T13:49:49.149418+0000","last_undegraded":"2023-10-08T13:49:49.149418+0000","last_fullsized":"2023-10-08T13:49:49.149418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:49.309597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164913+0000","last_change":"2023-10-08T13:49:49.547271+0000","last_active":"2023-10-08T13:49:50.164913+0000","last_peered":"2023-10-08T13:49:50.164913+0000","last_clean":"2023-10-08T13:49:50.164913+0000","last_became_active":"2023-10-08T13:49:49.546603+0000","last_became_peered":"2023-10-08T13:49:49.546603+0000","last_unstale":"2023-10-08T13:49:50.164913+0000","last_undegraded":"2023-10-08T13:49:50.164913+0000","last_fullsized":"2023-10-08T13:49:50.164913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:29.908321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148090+0000","last_change":"2023-10-08T13:49:43.788638+0000","last_active":"2023-10-08T13:49:49.148090+0000","last_peered":"2023-10-08T13:49:49.148090+0000","last_clean":"2023-10-08T13:49:49.148090+0000","last_became_active":"2023-10-08T13:49:43.788303+0000","last_became_peered":"2023-10-08T13:49:43.788303+0000","last_unstale":"2023-10-08T13:49:49.148090+0000","last_undegraded":"2023-10-08T13:49:49.148090+0000","last_fullsized":"2023-10-08T13:49:49.148090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:52.637775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146530+0000","last_change":"2023-10-08T13:49:44.777546+0000","last_active":"2023-10-08T13:49:49.146530+0000","last_peered":"2023-10-08T13:49:49.146530+0000","last_clean":"2023-10-08T13:49:49.146530+0000","last_became_active":"2023-10-08T13:49:44.777261+0000","last_became_peered":"2023-10-08T13:49:44.777261+0000","last_unstale":"2023-10-08T13:49:49.146530+0000","last_undegraded":"2023-10-08T13:49:49.146530+0000","last_fullsized":"2023-10-08T13:49:49.146530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:26:36.368188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.163099+0000","last_change":"2023-10-08T13:49:49.163071+0000","last_active":"2023-10-08T13:49:50.163099+0000","last_peered":"2023-10-08T13:49:50.163099+0000","last_clean":"2023-10-08T13:49:50.163099+0000","last_became_active":"2023-10-08T13:49:49.162667+0000","last_became_peered":"2023-10-08T13:49:49.162667+0000","last_unstale":"2023-10-08T13:49:50.163099+0000","last_undegraded":"2023-10-08T13:49:50.163099+0000","last_fullsized":"2023-10-08T13:49:50.163099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:54:42.645837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:06.313233+0000","last_change":"2023-10-08T13:49:43.776693+0000","last_active":"2023-10-08T13:50:06.313233+0000","last_peered":"2023-10-08T13:50:06.313233+0000","last_clean":"2023-10-08T13:50:06.313233+0000","last_became_active":"2023-10-08T13:49:43.776367+0000","last_became_peered":"2023-10-08T13:49:43.776367+0000","last_unstale":"2023-10-08T13:50:06.313233+0000","last_undegraded":"2023-10-08T13:50:06.313233+0000","last_fullsized":"2023-10-08T13:50:06.313233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:17.415011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.152599+0000","last_change":"2023-10-08T13:49:44.779472+0000","last_active":"2023-10-08T13:49:49.152599+0000","last_peered":"2023-10-08T13:49:49.152599+0000","last_clean":"2023-10-08T13:49:49.152599+0000","last_became_active":"2023-10-08T13:49:44.779172+0000","last_became_peered":"2023-10-08T13:49:44.779172+0000","last_unstale":"2023-10-08T13:49:49.152599+0000","last_undegraded":"2023-10-08T13:49:49.152599+0000","last_fullsized":"2023-10-08T13:49:49.152599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:17.314243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164634+0000","last_change":"2023-10-08T13:49:49.164535+0000","last_active":"2023-10-08T13:49:50.164634+0000","last_peered":"2023-10-08T13:49:50.164634+0000","last_clean":"2023-10-08T13:49:50.164634+0000","last_became_active":"2023-10-08T13:49:49.163467+0000","last_became_peered":"2023-10-08T13:49:49.163467+0000","last_unstale":"2023-10-08T13:49:50.164634+0000","last_undegraded":"2023-10-08T13:49:50.164634+0000","last_fullsized":"2023-10-08T13:49:50.164634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:55.058656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142401+0000","last_change":"2023-10-08T13:49:43.775311+0000","last_active":"2023-10-08T13:49:49.142401+0000","last_peered":"2023-10-08T13:49:49.142401+0000","last_clean":"2023-10-08T13:49:49.142401+0000","last_became_active":"2023-10-08T13:49:43.775005+0000","last_became_peered":"2023-10-08T13:49:43.775005+0000","last_unstale":"2023-10-08T13:49:49.142401+0000","last_undegraded":"2023-10-08T13:49:49.142401+0000","last_fullsized":"2023-10-08T13:49:49.142401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:33.713395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149584+0000","last_change":"2023-10-08T13:49:44.792051+0000","last_active":"2023-10-08T13:49:49.149584+0000","last_peered":"2023-10-08T13:49:49.149584+0000","last_clean":"2023-10-08T13:49:49.149584+0000","last_became_active":"2023-10-08T13:49:44.790045+0000","last_became_peered":"2023-10-08T13:49:44.790045+0000","last_unstale":"2023-10-08T13:49:49.149584+0000","last_undegraded":"2023-10-08T13:49:49.149584+0000","last_fullsized":"2023-10-08T13:49:49.149584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:28.699799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164975+0000","last_change":"2023-10-08T13:49:49.546219+0000","last_active":"2023-10-08T13:49:50.164975+0000","last_peered":"2023-10-08T13:49:50.164975+0000","last_clean":"2023-10-08T13:49:50.164975+0000","last_became_active":"2023-10-08T13:49:49.545783+0000","last_became_peered":"2023-10-08T13:49:49.545783+0000","last_unstale":"2023-10-08T13:49:50.164975+0000","last_undegraded":"2023-10-08T13:49:50.164975+0000","last_fullsized":"2023-10-08T13:49:50.164975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:00.497908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146753+0000","last_change":"2023-10-08T13:49:43.776252+0000","last_active":"2023-10-08T13:49:49.146753+0000","last_peered":"2023-10-08T13:49:49.146753+0000","last_clean":"2023-10-08T13:49:49.146753+0000","last_became_active":"2023-10-08T13:49:43.775935+0000","last_became_peered":"2023-10-08T13:49:43.775935+0000","last_unstale":"2023-10-08T13:49:49.146753+0000","last_undegraded":"2023-10-08T13:49:49.146753+0000","last_fullsized":"2023-10-08T13:49:49.146753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:58.949365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146922+0000","last_change":"2023-10-08T13:49:44.788628+0000","last_active":"2023-10-08T13:49:49.146922+0000","last_peered":"2023-10-08T13:49:49.146922+0000","last_clean":"2023-10-08T13:49:49.146922+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:49:49.146922+0000","last_undegraded":"2023-10-08T13:49:49.146922+0000","last_fullsized":"2023-10-08T13:49:49.146922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:46:42.311296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174221+0000","last_change":"2023-10-08T13:49:49.153641+0000","last_active":"2023-10-08T13:49:50.174221+0000","last_peered":"2023-10-08T13:49:50.174221+0000","last_clean":"2023-10-08T13:49:50.174221+0000","last_became_active":"2023-10-08T13:49:49.153172+0000","last_became_peered":"2023-10-08T13:49:49.153172+0000","last_unstale":"2023-10-08T13:49:50.174221+0000","last_undegraded":"2023-10-08T13:49:50.174221+0000","last_fullsized":"2023-10-08T13:49:50.174221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:40:51.726343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143719+0000","last_change":"2023-10-08T13:49:43.782683+0000","last_active":"2023-10-08T13:49:49.143719+0000","last_peered":"2023-10-08T13:49:49.143719+0000","last_clean":"2023-10-08T13:49:49.143719+0000","last_became_active":"2023-10-08T13:49:43.782269+0000","last_became_peered":"2023-10-08T13:49:43.782269+0000","last_unstale":"2023-10-08T13:49:49.143719+0000","last_undegraded":"2023-10-08T13:49:49.143719+0000","last_fullsized":"2023-10-08T13:49:49.143719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:50.850581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148276+0000","last_change":"2023-10-08T13:49:44.787894+0000","last_active":"2023-10-08T13:49:49.148276+0000","last_peered":"2023-10-08T13:49:49.148276+0000","last_clean":"2023-10-08T13:49:49.148276+0000","last_became_active":"2023-10-08T13:49:44.787599+0000","last_became_peered":"2023-10-08T13:49:44.787599+0000","last_unstale":"2023-10-08T13:49:49.148276+0000","last_undegraded":"2023-10-08T13:49:49.148276+0000","last_fullsized":"2023-10-08T13:49:49.148276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:17:15.131051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.389031+0000","last_change":"2023-10-08T13:49:49.164256+0000","last_active":"2023-10-08T13:49:50.389031+0000","last_peered":"2023-10-08T13:49:50.389031+0000","last_clean":"2023-10-08T13:49:50.389031+0000","last_became_active":"2023-10-08T13:49:49.163793+0000","last_became_peered":"2023-10-08T13:49:49.163793+0000","last_unstale":"2023-10-08T13:49:50.389031+0000","last_undegraded":"2023-10-08T13:49:50.389031+0000","last_fullsized":"2023-10-08T13:49:50.389031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:19.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149616+0000","last_change":"2023-10-08T13:49:43.778311+0000","last_active":"2023-10-08T13:49:49.149616+0000","last_peered":"2023-10-08T13:49:49.149616+0000","last_clean":"2023-10-08T13:49:49.149616+0000","last_became_active":"2023-10-08T13:49:43.777984+0000","last_became_peered":"2023-10-08T13:49:43.777984+0000","last_unstale":"2023-10-08T13:49:49.149616+0000","last_undegraded":"2023-10-08T13:49:49.149616+0000","last_fullsized":"2023-10-08T13:49:49.149616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:56.890729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147428+0000","last_change":"2023-10-08T13:49:44.793424+0000","last_active":"2023-10-08T13:49:49.147428+0000","last_peered":"2023-10-08T13:49:49.147428+0000","last_clean":"2023-10-08T13:49:49.147428+0000","last_became_active":"2023-10-08T13:49:44.792825+0000","last_became_peered":"2023-10-08T13:49:44.792825+0000","last_unstale":"2023-10-08T13:49:49.147428+0000","last_undegraded":"2023-10-08T13:49:49.147428+0000","last_fullsized":"2023-10-08T13:49:49.147428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:39.463676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174266+0000","last_change":"2023-10-08T13:49:49.151915+0000","last_active":"2023-10-08T13:49:50.174266+0000","last_peered":"2023-10-08T13:49:50.174266+0000","last_clean":"2023-10-08T13:49:50.174266+0000","last_became_active":"2023-10-08T13:49:49.151434+0000","last_became_peered":"2023-10-08T13:49:49.151434+0000","last_unstale":"2023-10-08T13:49:50.174266+0000","last_undegraded":"2023-10-08T13:49:50.174266+0000","last_fullsized":"2023-10-08T13:49:50.174266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:54:07.631498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142601+0000","last_change":"2023-10-08T13:49:43.772363+0000","last_active":"2023-10-08T13:49:49.142601+0000","last_peered":"2023-10-08T13:49:49.142601+0000","last_clean":"2023-10-08T13:49:49.142601+0000","last_became_active":"2023-10-08T13:49:43.772072+0000","last_became_peered":"2023-10-08T13:49:43.772072+0000","last_unstale":"2023-10-08T13:49:49.142601+0000","last_undegraded":"2023-10-08T13:49:49.142601+0000","last_fullsized":"2023-10-08T13:49:49.142601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:07:30.877217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148135+0000","last_change":"2023-10-08T13:49:44.790949+0000","last_active":"2023-10-08T13:49:49.148135+0000","last_peered":"2023-10-08T13:49:49.148135+0000","last_clean":"2023-10-08T13:49:49.148135+0000","last_became_active":"2023-10-08T13:49:44.790645+0000","last_became_peered":"2023-10-08T13:49:44.790645+0000","last_unstale":"2023-10-08T13:49:49.148135+0000","last_undegraded":"2023-10-08T13:49:49.148135+0000","last_fullsized":"2023-10-08T13:49:49.148135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:48.890730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.152709+0000","last_change":"2023-10-08T13:49:44.781165+0000","last_active":"2023-10-08T13:49:49.152709+0000","last_peered":"2023-10-08T13:49:49.152709+0000","last_clean":"2023-10-08T13:49:49.152709+0000","last_became_active":"2023-10-08T13:49:44.780828+0000","last_became_peered":"2023-10-08T13:49:44.780828+0000","last_unstale":"2023-10-08T13:49:49.152709+0000","last_undegraded":"2023-10-08T13:49:49.152709+0000","last_fullsized":"2023-10-08T13:49:49.152709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:04.259482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143422+0000","last_change":"2023-10-08T13:49:44.781271+0000","last_active":"2023-10-08T13:49:49.143422+0000","last_peered":"2023-10-08T13:49:49.143422+0000","last_clean":"2023-10-08T13:49:49.143422+0000","last_became_active":"2023-10-08T13:49:44.780912+0000","last_became_peered":"2023-10-08T13:49:44.780912+0000","last_unstale":"2023-10-08T13:49:49.143422+0000","last_undegraded":"2023-10-08T13:49:49.143422+0000","last_fullsized":"2023-10-08T13:49:49.143422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:17.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142288+0000","last_change":"2023-10-08T13:49:44.777613+0000","last_active":"2023-10-08T13:49:49.142288+0000","last_peered":"2023-10-08T13:49:49.142288+0000","last_clean":"2023-10-08T13:49:49.142288+0000","last_became_active":"2023-10-08T13:49:44.777304+0000","last_became_peered":"2023-10-08T13:49:44.777304+0000","last_unstale":"2023-10-08T13:49:49.142288+0000","last_undegraded":"2023-10-08T13:49:49.142288+0000","last_fullsized":"2023-10-08T13:49:49.142288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:45.482935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.150763+0000","last_change":"2023-10-08T13:49:44.782685+0000","last_active":"2023-10-08T13:49:49.150763+0000","last_peered":"2023-10-08T13:49:49.150763+0000","last_clean":"2023-10-08T13:49:49.150763+0000","last_became_active":"2023-10-08T13:49:44.781816+0000","last_became_peered":"2023-10-08T13:49:44.781816+0000","last_unstale":"2023-10-08T13:49:49.150763+0000","last_undegraded":"2023-10-08T13:49:49.150763+0000","last_fullsized":"2023-10-08T13:49:49.150763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:53.781237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142383+0000","last_change":"2023-10-08T13:49:44.783973+0000","last_active":"2023-10-08T13:49:49.142383+0000","last_peered":"2023-10-08T13:49:49.142383+0000","last_clean":"2023-10-08T13:49:49.142383+0000","last_became_active":"2023-10-08T13:49:44.783648+0000","last_became_peered":"2023-10-08T13:49:44.783648+0000","last_unstale":"2023-10-08T13:49:49.142383+0000","last_undegraded":"2023-10-08T13:49:49.142383+0000","last_fullsized":"2023-10-08T13:49:49.142383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:01.253610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.153009+0000","last_change":"2023-10-08T13:49:44.781848+0000","last_active":"2023-10-08T13:49:49.153009+0000","last_peered":"2023-10-08T13:49:49.153009+0000","last_clean":"2023-10-08T13:49:49.153009+0000","last_became_active":"2023-10-08T13:49:44.781542+0000","last_became_peered":"2023-10-08T13:49:44.781542+0000","last_unstale":"2023-10-08T13:49:49.153009+0000","last_undegraded":"2023-10-08T13:49:49.153009+0000","last_fullsized":"2023-10-08T13:49:49.153009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:26:38.726298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143093+0000","last_change":"2023-10-08T13:49:44.786541+0000","last_active":"2023-10-08T13:49:49.143093+0000","last_peered":"2023-10-08T13:49:49.143093+0000","last_clean":"2023-10-08T13:49:49.143093+0000","last_became_active":"2023-10-08T13:49:44.786214+0000","last_became_peered":"2023-10-08T13:49:44.786214+0000","last_unstale":"2023-10-08T13:49:49.143093+0000","last_undegraded":"2023-10-08T13:49:49.143093+0000","last_fullsized":"2023-10-08T13:49:49.143093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:30:18.342774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149227+0000","last_change":"2023-10-08T13:49:44.788445+0000","last_active":"2023-10-08T13:49:49.149227+0000","last_peered":"2023-10-08T13:49:49.149227+0000","last_clean":"2023-10-08T13:49:49.149227+0000","last_became_active":"2023-10-08T13:49:44.788075+0000","last_became_peered":"2023-10-08T13:49:44.788075+0000","last_unstale":"2023-10-08T13:49:49.149227+0000","last_undegraded":"2023-10-08T13:49:49.149227+0000","last_fullsized":"2023-10-08T13:49:49.149227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:08:20.701231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149588+0000","last_change":"2023-10-08T13:49:44.783755+0000","last_active":"2023-10-08T13:49:49.149588+0000","last_peered":"2023-10-08T13:49:49.149588+0000","last_clean":"2023-10-08T13:49:49.149588+0000","last_became_active":"2023-10-08T13:49:44.783467+0000","last_became_peered":"2023-10-08T13:49:44.783467+0000","last_unstale":"2023-10-08T13:49:49.149588+0000","last_undegraded":"2023-10-08T13:49:49.149588+0000","last_fullsized":"2023-10-08T13:49:49.149588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:39.838940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149966+0000","last_change":"2023-10-08T13:49:44.779342+0000","last_active":"2023-10-08T13:49:49.149966+0000","last_peered":"2023-10-08T13:49:49.149966+0000","last_clean":"2023-10-08T13:49:49.149966+0000","last_became_active":"2023-10-08T13:49:44.778518+0000","last_became_peered":"2023-10-08T13:49:44.778518+0000","last_unstale":"2023-10-08T13:49:49.149966+0000","last_undegraded":"2023-10-08T13:49:49.149966+0000","last_fullsized":"2023-10-08T13:49:49.149966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:33.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148573+0000","last_change":"2023-10-08T13:49:44.792955+0000","last_active":"2023-10-08T13:49:49.148573+0000","last_peered":"2023-10-08T13:49:49.148573+0000","last_clean":"2023-10-08T13:49:49.148573+0000","last_became_active":"2023-10-08T13:49:44.792637+0000","last_became_peered":"2023-10-08T13:49:44.792637+0000","last_unstale":"2023-10-08T13:49:49.148573+0000","last_undegraded":"2023-10-08T13:49:49.148573+0000","last_fullsized":"2023-10-08T13:49:49.148573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:40.635913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.141628+0000","last_change":"2023-10-08T13:49:44.786817+0000","last_active":"2023-10-08T13:49:49.141628+0000","last_peered":"2023-10-08T13:49:49.141628+0000","last_clean":"2023-10-08T13:49:49.141628+0000","last_became_active":"2023-10-08T13:49:44.786439+0000","last_became_peered":"2023-10-08T13:49:44.786439+0000","last_unstale":"2023-10-08T13:49:49.141628+0000","last_undegraded":"2023-10-08T13:49:49.141628+0000","last_fullsized":"2023-10-08T13:49:49.141628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:45.308600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.152693+0000","last_change":"2023-10-08T13:49:44.782776+0000","last_active":"2023-10-08T13:49:49.152693+0000","last_peered":"2023-10-08T13:49:49.152693+0000","last_clean":"2023-10-08T13:49:49.152693+0000","last_became_active":"2023-10-08T13:49:44.782460+0000","last_became_peered":"2023-10-08T13:49:44.782460+0000","last_unstale":"2023-10-08T13:49:49.152693+0000","last_undegraded":"2023-10-08T13:49:49.152693+0000","last_fullsized":"2023-10-08T13:49:49.152693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:15:12.580688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6792,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":102,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":22866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":486499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":10,"up_from":71,"seq":304942678036,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1083117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.41999999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68300000000000005}]},{"osd":3,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65500000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48499999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78300000000000003}]},{"osd":6,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.70699999999999996}]},{"osd":7,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55300000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.72199999999999998}]},{"osd":9,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75800000000000001}]},{"osd":11,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77000000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841559,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":487989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.35499999999999998}]},{"osd":1,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.442}]},{"osd":3,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.375},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64300000000000002}]},{"osd":5,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.42899999999999999}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.9570000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.345}]},{"osd":7,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.66600000000000004}]},{"osd":8,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.498}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.65500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":8,"up_from":58,"seq":249108103195,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":987136,"data_stored":487809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.57399999999999995,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":5.5860000000000003,"5min":5.5860000000000003,"15min":5.5860000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.55700000000000005,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.80300000000000005}]},{"osd":1,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.628,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.77700000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.51300000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56799999999999995,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84199999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70499999999999996}]},{"osd":4,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.876}]},{"osd":5,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.59799999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91500000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.69399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.89200000000000002}]},{"osd":7,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":0.96999999999999997}]},{"osd":9,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.747},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.92800000000000005}]},{"osd":10,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.99099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364833,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1085201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.439}]},{"osd":1,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.27000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.438,"15min":0.42799999999999999},"min":{"1min":0.28299999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45600000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.497},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57299999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.40699999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.627,"15min":0.62},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58599999999999997}]},{"osd":6,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67400000000000004,"15min":0.66900000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.69999999999999996}]},{"osd":8,"last update":"Sun Oct 8 13:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67000000000000004}]},{"osd":9,"last update":"Sun Oct 8 13:49:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.65800000000000003}]},{"osd":10,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.626}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528357,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":487205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.27300000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.41599999999999998,"15min":0.40699999999999997},"min":{"1min":0.29399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.61199999999999999}]},{"osd":1,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.38200000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.433,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.752}]},{"osd":2,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89000000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.91200000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.50900000000000001,"15min":0.50600000000000001},"min":{"1min":0.29699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.877,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.35399999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.91500000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69499999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58599999999999997,"15min":0.57199999999999995},"min":{"1min":0.435,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.627,"15min":0.61899999999999999},"min":{"1min":0.47999999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.84399999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0549999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.71999999999999997}]},{"osd":7,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73799999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:50:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1080000000000001,"5min":1.147,"15min":1.147},"last":1.0069999999999999}]},{"osd":9,"last update":"Sun Oct 8 13:49:20 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.89600000000000002}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":1.0469999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":491047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.503},"min":{"1min":0.34599999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.51600000000000001},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52400000000000002,"15min":0.50700000000000001},"min":{"1min":0.441,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80300000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65600000000000003}]},{"osd":3,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.873,"5min":3.7959999999999998,"15min":3.7959999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.39000000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94099999999999995,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.60799999999999998}]},{"osd":4,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.316,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.378,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.58599999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94599999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77100000000000002}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92400000000000004,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.75}]},{"osd":7,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.35499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.93500000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89500000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.71899999999999997}]},{"osd":8,"last update":"Sun Oct 8 13:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84299999999999997}]},{"osd":9,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.82799999999999996}]},{"osd":10,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705727,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":489271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 13:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.54500000000000004,"15min":0.53000000000000003},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56299999999999994,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85299999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.625}]},{"osd":3,"last update":"Sun Oct 8 13:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.47099999999999997,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 13:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57999999999999996,"15min":0.56000000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.43099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.46700000000000003}]},{"osd":5,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85999999999999999,"5min":1.012,"15min":1.012},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.60099999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.069,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.5}]},{"osd":7,"last update":"Sun Oct 8 13:49:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.50700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.80800000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76600000000000001,"15min":0.77800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1419999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.57499999999999996}]},{"osd":9,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.74199999999999999}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.78600000000000003}]},{"osd":11,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.64700000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378677,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":491081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.55200000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.56799999999999995}]},{"osd":1,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.501},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80600000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78800000000000003}]},{"osd":3,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82099999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.69799999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73699999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.61399999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44700000000000001,"15min":0.42899999999999999},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.38200000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.71899999999999997}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.45700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.44900000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.80500000000000005}]},{"osd":7,"last update":"Sun Oct 8 13:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.93200000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.68500000000000005}]},{"osd":9,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.96599999999999997}]},{"osd":10,"last update":"Sun Oct 8 13:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149746,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":487489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.33600000000000002},"min":{"1min":0.252,"5min":0.22,"15min":0.22},"max":{"1min":0.54900000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.441,"15min":0.40899999999999997},"min":{"1min":0.316,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77900000000000003,"5min":0.87,"15min":0.87},"last":0.372}]},{"osd":1,"last update":"Sun Oct 8 13:49:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.52100000000000002},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.41499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83299999999999996,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.57199999999999995}]},{"osd":2,"last update":"Sun Oct 8 13:49:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.42199999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.621},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59299999999999997,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97399999999999998,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.44600000000000001}]},{"osd":4,"last update":"Sun Oct 8 13:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59999999999999998,"15min":0.56200000000000006},"min":{"1min":0.47299999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91300000000000003,"5min":2.077,"15min":2.077},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59099999999999997,"15min":0.55100000000000005},"min":{"1min":0.49299999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81100000000000005,"5min":2.129,"15min":2.129},"last":0.52300000000000002}]},{"osd":5,"last update":"Sun Oct 8 13:49:59 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.60599999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96999999999999997,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Oct 8 13:49:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63900000000000001,"15min":0.626},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.50600000000000001}]},{"osd":7,"last update":"Sun Oct 8 13:49:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.55300000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99399999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.67200000000000004}]},{"osd":8,"last update":"Sun Oct 8 13:48:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.46800000000000003}]},{"osd":9,"last update":"Sun Oct 8 13:49:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63100000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.68600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920813,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":487227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.51800000000000002,"15min":0.47999999999999998},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.371,"15min":0.35199999999999998},"min":{"1min":0.32400000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.38100000000000001,"15min":0.35699999999999998},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.437}]},{"osd":2,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.504,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65800000000000003}]},{"osd":3,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.67700000000000005}]},{"osd":5,"last update":"Sun Oct 8 13:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.48099999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.621},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Oct 8 13:49:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53100000000000003}]},{"osd":7,"last update":"Sun Oct 8 13:49:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.41599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.51400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.73499999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.755},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69299999999999995}]},{"osd":9,"last update":"Sun Oct 8 13:49:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70599999999999996}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757289,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1078718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48599999999999999,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.49299999999999999,"15min":0.46600000000000003},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.871,"15min":0.871},"last":0.71299999999999997}]},{"osd":1,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.55000000000000004,"15min":0.53000000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84999999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61499999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.38600000000000001,"15min":0.36899999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58699999999999997,"5min":0.629,"15min":0.629},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.40300000000000002,"15min":0.38300000000000001},"min":{"1min":0.29599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.45200000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.313,"5min":0.309,"15min":0.309},"max":{"1min":0.71499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.40699999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.77400000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.67500000000000004}]},{"osd":4,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.42999999999999999}]},{"osd":6,"last update":"Sun Oct 8 13:49:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.502,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84799999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65700000000000003,"15min":0.65500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77900000000000003}]},{"osd":8,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.57699999999999996}]},{"osd":9,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.749},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.86699999999999999}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.83999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T13:50:09.731 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T13:50:09.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.887+0000 7f2555840700 1 -- 172.21.15.57:0/491345720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25501066d0 msgr2=0x7f2550106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.887+0000 7f2555840700 1 --2- 172.21.15.57:0/491345720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25501066d0 0x7f2550106ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2544009990 tx=0x7f2544009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.887+0000 7f2555840700 1 -- 172.21.15.57:0/491345720 shutdown_connections 2023-10-08T13:50:09.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.887+0000 7f2555840700 1 --2- 172.21.15.57:0/491345720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25500ff570 0x7f255010d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.887+0000 7f2555840700 1 --2- 172.21.15.57:0/491345720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25501066d0 0x7f2550106ab0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.887+0000 7f2555840700 1 --2- 172.21.15.57:0/491345720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25500febb0 0x7f25500ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.887+0000 7f2555840700 1 -- 172.21.15.57:0/491345720 >> 172.21.15.57:0/491345720 conn(0x7f2550075a60 msgr2=0x7f25500fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:09.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.887+0000 7f2555840700 1 -- 172.21.15.57:0/491345720 shutdown_connections 2023-10-08T13:50:09.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.887+0000 7f2555840700 1 -- 172.21.15.57:0/491345720 wait complete. 2023-10-08T13:50:09.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f2555840700 1 Processor -- start 2023-10-08T13:50:09.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f2555840700 1 -- start start 2023-10-08T13:50:09.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f2555840700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500febb0 0x7f255019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f2555840700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25500ff570 0x7f255019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f2555840700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25501066d0 0x7f2550198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f2555840700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2550199120 con 0x7f25500febb0 2023-10-08T13:50:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f2555840700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2550199290 con 0x7f25500ff570 2023-10-08T13:50:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f2555840700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2550199450 con 0x7f25501066d0 2023-10-08T13:50:09.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f254effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500febb0 0x7f255019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f254effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500febb0 0x7f255019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54560/0 (socket says 172.21.15.57:54560) 2023-10-08T13:50:09.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.888+0000 7f254effd700 1 -- 172.21.15.57:0/674621813 learned_addr learned my addr 172.21.15.57:0/674621813 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:09.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f254f7fe700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25501066d0 0x7f2550198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f254e7fc700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25500ff570 0x7f255019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f254effd700 1 -- 172.21.15.57:0/674621813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25501066d0 msgr2=0x7f2550198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f254effd700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25501066d0 0x7f2550198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f254effd700 1 -- 172.21.15.57:0/674621813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25500ff570 msgr2=0x7f255019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f254effd700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25500ff570 0x7f255019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f254effd700 1 -- 172.21.15.57:0/674621813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25440095f0 con 0x7f25500febb0 2023-10-08T13:50:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f254effd700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500febb0 0x7f255019e990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2544005770 tx=0x7f25440057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f255483e700 1 -- 172.21.15.57:0/674621813 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254401d070 con 0x7f25500febb0 2023-10-08T13:50:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f255483e700 1 -- 172.21.15.57:0/674621813 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2544021410 con 0x7f25500febb0 2023-10-08T13:50:09.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f2555840700 1 -- 172.21.15.57:0/674621813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2550199730 con 0x7f25500febb0 2023-10-08T13:50:09.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f255483e700 1 -- 172.21.15.57:0/674621813 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544017650 con 0x7f25500febb0 2023-10-08T13:50:09.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.889+0000 7f2555840700 1 -- 172.21.15.57:0/674621813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501a5980 con 0x7f25500febb0 2023-10-08T13:50:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.890+0000 7f2555840700 1 -- 172.21.15.57:0/674621813 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255010d080 con 0x7f25500febb0 2023-10-08T13:50:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.891+0000 7f255483e700 1 -- 172.21.15.57:0/674621813 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f25440217f0 con 0x7f25500febb0 2023-10-08T13:50:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.892+0000 7f255483e700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25380a15e0 0x7f25380a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.892+0000 7f255483e700 1 -- 172.21.15.57:0/674621813 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f25440ce3e0 con 0x7f25500febb0 2023-10-08T13:50:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.892+0000 7f254e7fc700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25380a15e0 0x7f25380a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.892+0000 7f254e7fc700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25380a15e0 0x7f25380a3aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2550199c80 tx=0x7f254000a380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:09.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:09.896+0000 7f255483e700 1 -- 172.21.15.57:0/674621813 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f25440976a0 con 0x7f25500febb0 2023-10-08T13:50:09.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:09 smithi057 ceph-mon[99476]: pgmap v253: 129 pgs: 129 active+clean; 583 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.2 KiB/s wr, 4 op/s 2023-10-08T13:50:09.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:09 smithi057 ceph-mon[99476]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:09.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:09 smithi057 ceph-mon[99476]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:09.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:09 smithi057 ceph-mon[99476]: mgrmap e21: x(active, since 5m), standbys: z, y 2023-10-08T13:50:09.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:09 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/782714939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:09.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:09 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/171048795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:10.022 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 13:50:09 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b[140884]: 2023-10-08T13:50:09.744+0000 7fe27b630700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-08T13:50:10.022 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 13:50:09 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b[140884]: 2023-10-08T13:50:09.744+0000 7fe27b630700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-10-08T13:50:10.022 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 13:50:09 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b[140884]: 2023-10-08T13:50:09.744+0000 7fe27b630700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-10-08T13:50:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:09 smithi099 ceph-mon[103849]: pgmap v253: 129 pgs: 129 active+clean; 583 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.2 KiB/s wr, 4 op/s 2023-10-08T13:50:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:09 smithi099 ceph-mon[103849]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:09 smithi099 ceph-mon[103849]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:09 smithi099 ceph-mon[103849]: mgrmap e21: x(active, since 5m), standbys: z, y 2023-10-08T13:50:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:09 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/782714939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:10.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:09 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/171048795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:10.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.092+0000 7f2555840700 1 -- 172.21.15.57:0/674621813 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2550067910 con 0x7f25380a15e0 2023-10-08T13:50:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.107+0000 7f255483e700 1 -- 172.21.15.57:0/674621813 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418865 (secure 0 0 0) 0x7f2550067910 con 0x7f25380a15e0 2023-10-08T13:50:10.112 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.110+0000 7f2555840700 1 -- 172.21.15.57:0/674621813 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25380a15e0 msgr2=0x7f25380a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.110+0000 7f2555840700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25380a15e0 0x7f25380a3aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2550199c80 tx=0x7f254000a380 comp rx=0 tx=0).stop 2023-10-08T13:50:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.110+0000 7f2555840700 1 -- 172.21.15.57:0/674621813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500febb0 msgr2=0x7f255019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.110+0000 7f2555840700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500febb0 0x7f255019e990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2544005770 tx=0x7f25440057a0 comp rx=0 tx=0).stop 2023-10-08T13:50:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.111+0000 7f2555840700 1 -- 172.21.15.57:0/674621813 shutdown_connections 2023-10-08T13:50:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.111+0000 7f2555840700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25501066d0 0x7f2550198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.111+0000 7f2555840700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500febb0 0x7f255019e990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.111+0000 7f2555840700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25380a15e0 0x7f25380a3aa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.111+0000 7f2555840700 1 --2- 172.21.15.57:0/674621813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25500ff570 0x7f255019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.111+0000 7f2555840700 1 -- 172.21.15.57:0/674621813 >> 172.21.15.57:0/674621813 conn(0x7f2550075a60 msgr2=0x7f2550103780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.111+0000 7f2555840700 1 -- 172.21.15.57:0/674621813 shutdown_connections 2023-10-08T13:50:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:10.111+0000 7f2555840700 1 -- 172.21.15.57:0/674621813 wait complete. 2023-10-08T13:50:10.117 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-08T13:50:10.123 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2023-10-08T13:50:08.104169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597283,"num_objects":48,"num_object_clones":0,"num_object_copies":145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":138,"num_read_kb":120,"num_write":284,"num_write_kb":2966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337648,"kb_used_data":13424,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604176,"statfs":{"total":1151940427776,"available":1151594676224,"internally_reserved":0,"allocated":13746176,"data_stored":7642653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4505,"num_objects":24,"num_object_clones":0,"num_object_copies":73,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":58,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003611"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.154854+0000","last_change":"2023-10-08T13:49:44.786130+0000","last_active":"2023-10-08T13:49:49.154854+0000","last_peered":"2023-10-08T13:49:49.154854+0000","last_clean":"2023-10-08T13:49:49.154854+0000","last_became_active":"2023-10-08T13:49:44.785792+0000","last_became_peered":"2023-10-08T13:49:44.785792+0000","last_unstale":"2023-10-08T13:49:49.154854+0000","last_undegraded":"2023-10-08T13:49:49.154854+0000","last_fullsized":"2023-10-08T13:49:49.154854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:17.046800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149324+0000","last_change":"2023-10-08T13:49:44.777601+0000","last_active":"2023-10-08T13:49:49.149324+0000","last_peered":"2023-10-08T13:49:49.149324+0000","last_clean":"2023-10-08T13:49:49.149324+0000","last_became_active":"2023-10-08T13:49:44.777287+0000","last_became_peered":"2023-10-08T13:49:44.777287+0000","last_unstale":"2023-10-08T13:49:49.149324+0000","last_undegraded":"2023-10-08T13:49:49.149324+0000","last_fullsized":"2023-10-08T13:49:49.149324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:09:17.317629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148154+0000","last_change":"2023-10-08T13:49:44.787035+0000","last_active":"2023-10-08T13:49:49.148154+0000","last_peered":"2023-10-08T13:49:49.148154+0000","last_clean":"2023-10-08T13:49:49.148154+0000","last_became_active":"2023-10-08T13:49:44.785834+0000","last_became_peered":"2023-10-08T13:49:44.785834+0000","last_unstale":"2023-10-08T13:49:49.148154+0000","last_undegraded":"2023-10-08T13:49:49.148154+0000","last_fullsized":"2023-10-08T13:49:49.148154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:11.611404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143282+0000","last_change":"2023-10-08T13:49:44.781937+0000","last_active":"2023-10-08T13:49:49.143282+0000","last_peered":"2023-10-08T13:49:49.143282+0000","last_clean":"2023-10-08T13:49:49.143282+0000","last_became_active":"2023-10-08T13:49:44.780712+0000","last_became_peered":"2023-10-08T13:49:44.780712+0000","last_unstale":"2023-10-08T13:49:49.143282+0000","last_undegraded":"2023-10-08T13:49:49.143282+0000","last_fullsized":"2023-10-08T13:49:49.143282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:02.514007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146636+0000","last_change":"2023-10-08T13:49:44.788661+0000","last_active":"2023-10-08T13:49:49.146636+0000","last_peered":"2023-10-08T13:49:49.146636+0000","last_clean":"2023-10-08T13:49:49.146636+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:49:49.146636+0000","last_undegraded":"2023-10-08T13:49:49.146636+0000","last_fullsized":"2023-10-08T13:49:49.146636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:35.770287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.544588+0000","last_change":"2023-10-08T13:49:44.778519+0000","last_active":"2023-10-08T13:49:49.544588+0000","last_peered":"2023-10-08T13:49:49.544588+0000","last_clean":"2023-10-08T13:49:49.544588+0000","last_became_active":"2023-10-08T13:49:44.778193+0000","last_became_peered":"2023-10-08T13:49:44.778193+0000","last_unstale":"2023-10-08T13:49:49.544588+0000","last_undegraded":"2023-10-08T13:49:49.544588+0000","last_fullsized":"2023-10-08T13:49:49.544588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:12.916019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147737+0000","last_change":"2023-10-08T13:49:44.791759+0000","last_active":"2023-10-08T13:49:49.147737+0000","last_peered":"2023-10-08T13:49:49.147737+0000","last_clean":"2023-10-08T13:49:49.147737+0000","last_became_active":"2023-10-08T13:49:44.791494+0000","last_became_peered":"2023-10-08T13:49:44.791494+0000","last_unstale":"2023-10-08T13:49:49.147737+0000","last_undegraded":"2023-10-08T13:49:49.147737+0000","last_fullsized":"2023-10-08T13:49:49.147737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:55.859609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149225+0000","last_change":"2023-10-08T13:49:44.786831+0000","last_active":"2023-10-08T13:49:49.149225+0000","last_peered":"2023-10-08T13:49:49.149225+0000","last_clean":"2023-10-08T13:49:49.149225+0000","last_became_active":"2023-10-08T13:49:44.786516+0000","last_became_peered":"2023-10-08T13:49:44.786516+0000","last_unstale":"2023-10-08T13:49:49.149225+0000","last_undegraded":"2023-10-08T13:49:49.149225+0000","last_fullsized":"2023-10-08T13:49:49.149225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:07.616821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149102+0000","last_change":"2023-10-08T13:49:44.789827+0000","last_active":"2023-10-08T13:49:49.149102+0000","last_peered":"2023-10-08T13:49:49.149102+0000","last_clean":"2023-10-08T13:49:49.149102+0000","last_became_active":"2023-10-08T13:49:44.786635+0000","last_became_peered":"2023-10-08T13:49:44.786635+0000","last_unstale":"2023-10-08T13:49:49.149102+0000","last_undegraded":"2023-10-08T13:49:49.149102+0000","last_fullsized":"2023-10-08T13:49:49.149102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:35.492437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.545458+0000","last_change":"2023-10-08T13:49:44.784240+0000","last_active":"2023-10-08T13:49:49.545458+0000","last_peered":"2023-10-08T13:49:49.545458+0000","last_clean":"2023-10-08T13:49:49.545458+0000","last_became_active":"2023-10-08T13:49:44.783475+0000","last_became_peered":"2023-10-08T13:49:44.783475+0000","last_unstale":"2023-10-08T13:49:49.545458+0000","last_undegraded":"2023-10-08T13:49:49.545458+0000","last_fullsized":"2023-10-08T13:49:49.545458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:39.730679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148185+0000","last_change":"2023-10-08T13:49:44.790901+0000","last_active":"2023-10-08T13:49:49.148185+0000","last_peered":"2023-10-08T13:49:49.148185+0000","last_clean":"2023-10-08T13:49:49.148185+0000","last_became_active":"2023-10-08T13:49:44.790611+0000","last_became_peered":"2023-10-08T13:49:44.790611+0000","last_unstale":"2023-10-08T13:49:49.148185+0000","last_undegraded":"2023-10-08T13:49:49.148185+0000","last_fullsized":"2023-10-08T13:49:49.148185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:19:21.764105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143349+0000","last_change":"2023-10-08T13:49:44.775226+0000","last_active":"2023-10-08T13:49:49.143349+0000","last_peered":"2023-10-08T13:49:49.143349+0000","last_clean":"2023-10-08T13:49:49.143349+0000","last_became_active":"2023-10-08T13:49:44.774756+0000","last_became_peered":"2023-10-08T13:49:44.774756+0000","last_unstale":"2023-10-08T13:49:49.143349+0000","last_undegraded":"2023-10-08T13:49:49.143349+0000","last_fullsized":"2023-10-08T13:49:49.143349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:57:14.749066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.141527+0000","last_change":"2023-10-08T13:49:44.781610+0000","last_active":"2023-10-08T13:49:49.141527+0000","last_peered":"2023-10-08T13:49:49.141527+0000","last_clean":"2023-10-08T13:49:49.141527+0000","last_became_active":"2023-10-08T13:49:44.781281+0000","last_became_peered":"2023-10-08T13:49:44.781281+0000","last_unstale":"2023-10-08T13:49:49.141527+0000","last_undegraded":"2023-10-08T13:49:49.141527+0000","last_fullsized":"2023-10-08T13:49:49.141527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:50.823909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147896+0000","last_change":"2023-10-08T13:49:44.775836+0000","last_active":"2023-10-08T13:49:49.147896+0000","last_peered":"2023-10-08T13:49:49.147896+0000","last_clean":"2023-10-08T13:49:49.147896+0000","last_became_active":"2023-10-08T13:49:44.775206+0000","last_became_peered":"2023-10-08T13:49:44.775206+0000","last_unstale":"2023-10-08T13:49:49.147896+0000","last_undegraded":"2023-10-08T13:49:49.147896+0000","last_fullsized":"2023-10-08T13:49:49.147896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:51.566800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148540+0000","last_change":"2023-10-08T13:49:44.778495+0000","last_active":"2023-10-08T13:49:49.148540+0000","last_peered":"2023-10-08T13:49:49.148540+0000","last_clean":"2023-10-08T13:49:49.148540+0000","last_became_active":"2023-10-08T13:49:44.778196+0000","last_became_peered":"2023-10-08T13:49:44.778196+0000","last_unstale":"2023-10-08T13:49:49.148540+0000","last_undegraded":"2023-10-08T13:49:49.148540+0000","last_fullsized":"2023-10-08T13:49:49.148540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:57:38.531853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148003+0000","last_change":"2023-10-08T13:49:44.792960+0000","last_active":"2023-10-08T13:49:49.148003+0000","last_peered":"2023-10-08T13:49:49.148003+0000","last_clean":"2023-10-08T13:49:49.148003+0000","last_became_active":"2023-10-08T13:49:44.792674+0000","last_became_peered":"2023-10-08T13:49:44.792674+0000","last_unstale":"2023-10-08T13:49:49.148003+0000","last_undegraded":"2023-10-08T13:49:49.148003+0000","last_fullsized":"2023-10-08T13:49:49.148003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:13.934954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.154810+0000","last_change":"2023-10-08T13:49:44.785971+0000","last_active":"2023-10-08T13:49:49.154810+0000","last_peered":"2023-10-08T13:49:49.154810+0000","last_clean":"2023-10-08T13:49:49.154810+0000","last_became_active":"2023-10-08T13:49:44.785680+0000","last_became_peered":"2023-10-08T13:49:44.785680+0000","last_unstale":"2023-10-08T13:49:49.154810+0000","last_undegraded":"2023-10-08T13:49:49.154810+0000","last_fullsized":"2023-10-08T13:49:49.154810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:35.447519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.161467+0000","last_change":"2023-10-08T13:49:44.787553+0000","last_active":"2023-10-08T13:49:49.161467+0000","last_peered":"2023-10-08T13:49:49.161467+0000","last_clean":"2023-10-08T13:49:49.161467+0000","last_became_active":"2023-10-08T13:49:44.787288+0000","last_became_peered":"2023-10-08T13:49:44.787288+0000","last_unstale":"2023-10-08T13:49:49.161467+0000","last_undegraded":"2023-10-08T13:49:49.161467+0000","last_fullsized":"2023-10-08T13:49:49.161467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:48.386120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143084+0000","last_change":"2023-10-08T13:49:44.789252+0000","last_active":"2023-10-08T13:49:49.143084+0000","last_peered":"2023-10-08T13:49:49.143084+0000","last_clean":"2023-10-08T13:49:49.143084+0000","last_became_active":"2023-10-08T13:49:44.788947+0000","last_became_peered":"2023-10-08T13:49:44.788947+0000","last_unstale":"2023-10-08T13:49:49.143084+0000","last_undegraded":"2023-10-08T13:49:49.143084+0000","last_fullsized":"2023-10-08T13:49:49.143084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:30.266913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.176271+0000","last_change":"2023-10-08T13:49:49.163619+0000","last_active":"2023-10-08T13:49:50.176271+0000","last_peered":"2023-10-08T13:49:50.176271+0000","last_clean":"2023-10-08T13:49:50.176271+0000","last_became_active":"2023-10-08T13:49:49.163020+0000","last_became_peered":"2023-10-08T13:49:49.163020+0000","last_unstale":"2023-10-08T13:49:50.176271+0000","last_undegraded":"2023-10-08T13:49:50.176271+0000","last_fullsized":"2023-10-08T13:49:50.176271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:56.092404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.162203+0000","last_change":"2023-10-08T13:49:43.782863+0000","last_active":"2023-10-08T13:49:49.162203+0000","last_peered":"2023-10-08T13:49:49.162203+0000","last_clean":"2023-10-08T13:49:49.162203+0000","last_became_active":"2023-10-08T13:49:43.782477+0000","last_became_peered":"2023-10-08T13:49:43.782477+0000","last_unstale":"2023-10-08T13:49:49.162203+0000","last_undegraded":"2023-10-08T13:49:49.162203+0000","last_fullsized":"2023-10-08T13:49:49.162203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:10.134907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149259+0000","last_change":"2023-10-08T13:49:44.788452+0000","last_active":"2023-10-08T13:49:49.149259+0000","last_peered":"2023-10-08T13:49:49.149259+0000","last_clean":"2023-10-08T13:49:49.149259+0000","last_became_active":"2023-10-08T13:49:44.788122+0000","last_became_peered":"2023-10-08T13:49:44.788122+0000","last_unstale":"2023-10-08T13:49:49.149259+0000","last_undegraded":"2023-10-08T13:49:49.149259+0000","last_fullsized":"2023-10-08T13:49:49.149259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:25:45.766329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164248+0000","last_change":"2023-10-08T13:49:49.164248+0000","last_active":"2023-10-08T13:49:50.164248+0000","last_peered":"2023-10-08T13:49:50.164248+0000","last_clean":"2023-10-08T13:49:50.164248+0000","last_became_active":"2023-10-08T13:49:49.163624+0000","last_became_peered":"2023-10-08T13:49:49.163624+0000","last_unstale":"2023-10-08T13:49:50.164248+0000","last_undegraded":"2023-10-08T13:49:50.164248+0000","last_fullsized":"2023-10-08T13:49:50.164248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:04.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175015+0000","last_change":"2023-10-08T13:49:43.788026+0000","last_active":"2023-10-08T13:49:50.175015+0000","last_peered":"2023-10-08T13:49:50.175015+0000","last_clean":"2023-10-08T13:49:50.175015+0000","last_became_active":"2023-10-08T13:49:43.787639+0000","last_became_peered":"2023-10-08T13:49:43.787639+0000","last_unstale":"2023-10-08T13:49:50.175015+0000","last_undegraded":"2023-10-08T13:49:50.175015+0000","last_fullsized":"2023-10-08T13:49:50.175015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:25:48.367765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148792+0000","last_change":"2023-10-08T13:49:44.786322+0000","last_active":"2023-10-08T13:49:49.148792+0000","last_peered":"2023-10-08T13:49:49.148792+0000","last_clean":"2023-10-08T13:49:49.148792+0000","last_became_active":"2023-10-08T13:49:44.785953+0000","last_became_peered":"2023-10-08T13:49:44.785953+0000","last_unstale":"2023-10-08T13:49:49.148792+0000","last_undegraded":"2023-10-08T13:49:49.148792+0000","last_fullsized":"2023-10-08T13:49:49.148792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:39.866785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164557+0000","last_change":"2023-10-08T13:49:49.164501+0000","last_active":"2023-10-08T13:49:50.164557+0000","last_peered":"2023-10-08T13:49:50.164557+0000","last_clean":"2023-10-08T13:49:50.164557+0000","last_became_active":"2023-10-08T13:49:49.163994+0000","last_became_peered":"2023-10-08T13:49:49.163994+0000","last_unstale":"2023-10-08T13:49:50.164557+0000","last_undegraded":"2023-10-08T13:49:50.164557+0000","last_fullsized":"2023-10-08T13:49:50.164557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:38.826822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147836+0000","last_change":"2023-10-08T13:49:43.776239+0000","last_active":"2023-10-08T13:49:49.147836+0000","last_peered":"2023-10-08T13:49:49.147836+0000","last_clean":"2023-10-08T13:49:49.147836+0000","last_became_active":"2023-10-08T13:49:43.775969+0000","last_became_peered":"2023-10-08T13:49:43.775969+0000","last_unstale":"2023-10-08T13:49:49.147836+0000","last_undegraded":"2023-10-08T13:49:49.147836+0000","last_fullsized":"2023-10-08T13:49:49.147836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:24.011655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148812+0000","last_change":"2023-10-08T13:49:44.794912+0000","last_active":"2023-10-08T13:49:49.148812+0000","last_peered":"2023-10-08T13:49:49.148812+0000","last_clean":"2023-10-08T13:49:49.148812+0000","last_became_active":"2023-10-08T13:49:44.794590+0000","last_became_peered":"2023-10-08T13:49:44.794590+0000","last_unstale":"2023-10-08T13:49:49.148812+0000","last_undegraded":"2023-10-08T13:49:49.148812+0000","last_fullsized":"2023-10-08T13:49:49.148812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:51.631386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164345+0000","last_change":"2023-10-08T13:49:49.152149+0000","last_active":"2023-10-08T13:49:50.164345+0000","last_peered":"2023-10-08T13:49:50.164345+0000","last_clean":"2023-10-08T13:49:50.164345+0000","last_became_active":"2023-10-08T13:49:49.151764+0000","last_became_peered":"2023-10-08T13:49:49.151764+0000","last_unstale":"2023-10-08T13:49:50.164345+0000","last_undegraded":"2023-10-08T13:49:50.164345+0000","last_fullsized":"2023-10-08T13:49:50.164345+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:25:08.232556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147334+0000","last_change":"2023-10-08T13:49:43.774505+0000","last_active":"2023-10-08T13:49:49.147334+0000","last_peered":"2023-10-08T13:49:49.147334+0000","last_clean":"2023-10-08T13:49:49.147334+0000","last_became_active":"2023-10-08T13:49:43.774301+0000","last_became_peered":"2023-10-08T13:49:43.774301+0000","last_unstale":"2023-10-08T13:49:49.147334+0000","last_undegraded":"2023-10-08T13:49:49.147334+0000","last_fullsized":"2023-10-08T13:49:49.147334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:59:31.708311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148676+0000","last_change":"2023-10-08T13:49:44.783915+0000","last_active":"2023-10-08T13:49:49.148676+0000","last_peered":"2023-10-08T13:49:49.148676+0000","last_clean":"2023-10-08T13:49:49.148676+0000","last_became_active":"2023-10-08T13:49:44.783628+0000","last_became_peered":"2023-10-08T13:49:44.783628+0000","last_unstale":"2023-10-08T13:49:49.148676+0000","last_undegraded":"2023-10-08T13:49:49.148676+0000","last_fullsized":"2023-10-08T13:49:49.148676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:43.759681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175588+0000","last_change":"2023-10-08T13:49:49.156241+0000","last_active":"2023-10-08T13:49:50.175588+0000","last_peered":"2023-10-08T13:49:50.175588+0000","last_clean":"2023-10-08T13:49:50.175588+0000","last_became_active":"2023-10-08T13:49:49.155652+0000","last_became_peered":"2023-10-08T13:49:49.155652+0000","last_unstale":"2023-10-08T13:49:50.175588+0000","last_undegraded":"2023-10-08T13:49:50.175588+0000","last_fullsized":"2023-10-08T13:49:50.175588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:04.283647+0000","last_change":"2023-10-08T13:49:43.781263+0000","last_active":"2023-10-08T13:50:04.283647+0000","last_peered":"2023-10-08T13:50:04.283647+0000","last_clean":"2023-10-08T13:50:04.283647+0000","last_became_active":"2023-10-08T13:49:43.780981+0000","last_became_peered":"2023-10-08T13:49:43.780981+0000","last_unstale":"2023-10-08T13:50:04.283647+0000","last_undegraded":"2023-10-08T13:50:04.283647+0000","last_fullsized":"2023-10-08T13:50:04.283647+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:22:18.130688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146867+0000","last_change":"2023-10-08T13:49:44.780935+0000","last_active":"2023-10-08T13:49:49.146867+0000","last_peered":"2023-10-08T13:49:49.146867+0000","last_clean":"2023-10-08T13:49:49.146867+0000","last_became_active":"2023-10-08T13:49:44.780613+0000","last_became_peered":"2023-10-08T13:49:44.780613+0000","last_unstale":"2023-10-08T13:49:49.146867+0000","last_undegraded":"2023-10-08T13:49:49.146867+0000","last_fullsized":"2023-10-08T13:49:49.146867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:57.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175911+0000","last_change":"2023-10-08T13:49:49.163652+0000","last_active":"2023-10-08T13:49:50.175911+0000","last_peered":"2023-10-08T13:49:50.175911+0000","last_clean":"2023-10-08T13:49:50.175911+0000","last_became_active":"2023-10-08T13:49:49.163140+0000","last_became_peered":"2023-10-08T13:49:49.163140+0000","last_unstale":"2023-10-08T13:49:50.175911+0000","last_undegraded":"2023-10-08T13:49:50.175911+0000","last_fullsized":"2023-10-08T13:49:50.175911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:48.733675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:06.314146+0000","last_change":"2023-10-08T13:49:43.775799+0000","last_active":"2023-10-08T13:50:06.314146+0000","last_peered":"2023-10-08T13:50:06.314146+0000","last_clean":"2023-10-08T13:50:06.314146+0000","last_became_active":"2023-10-08T13:49:43.775470+0000","last_became_peered":"2023-10-08T13:49:43.775470+0000","last_unstale":"2023-10-08T13:50:06.314146+0000","last_undegraded":"2023-10-08T13:50:06.314146+0000","last_fullsized":"2023-10-08T13:50:06.314146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:50.404554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149414+0000","last_change":"2023-10-08T13:49:44.789392+0000","last_active":"2023-10-08T13:49:49.149414+0000","last_peered":"2023-10-08T13:49:49.149414+0000","last_clean":"2023-10-08T13:49:49.149414+0000","last_became_active":"2023-10-08T13:49:44.789079+0000","last_became_peered":"2023-10-08T13:49:44.789079+0000","last_unstale":"2023-10-08T13:49:49.149414+0000","last_undegraded":"2023-10-08T13:49:49.149414+0000","last_fullsized":"2023-10-08T13:49:49.149414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:57:56.239854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175082+0000","last_change":"2023-10-08T13:49:49.166702+0000","last_active":"2023-10-08T13:49:50.175082+0000","last_peered":"2023-10-08T13:49:50.175082+0000","last_clean":"2023-10-08T13:49:50.175082+0000","last_became_active":"2023-10-08T13:49:49.166256+0000","last_became_peered":"2023-10-08T13:49:49.166256+0000","last_unstale":"2023-10-08T13:49:50.175082+0000","last_undegraded":"2023-10-08T13:49:50.175082+0000","last_fullsized":"2023-10-08T13:49:50.175082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:18:57.565040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:05.297390+0000","last_change":"2023-10-08T13:49:43.785085+0000","last_active":"2023-10-08T13:50:05.297390+0000","last_peered":"2023-10-08T13:50:05.297390+0000","last_clean":"2023-10-08T13:50:05.297390+0000","last_became_active":"2023-10-08T13:49:43.784746+0000","last_became_peered":"2023-10-08T13:49:43.784746+0000","last_unstale":"2023-10-08T13:50:05.297390+0000","last_undegraded":"2023-10-08T13:50:05.297390+0000","last_fullsized":"2023-10-08T13:50:05.297390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:13.471834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149099+0000","last_change":"2023-10-08T13:49:44.785670+0000","last_active":"2023-10-08T13:49:49.149099+0000","last_peered":"2023-10-08T13:49:49.149099+0000","last_clean":"2023-10-08T13:49:49.149099+0000","last_became_active":"2023-10-08T13:49:44.785221+0000","last_became_peered":"2023-10-08T13:49:44.785221+0000","last_unstale":"2023-10-08T13:49:49.149099+0000","last_undegraded":"2023-10-08T13:49:49.149099+0000","last_fullsized":"2023-10-08T13:49:49.149099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:25.415168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.389166+0000","last_change":"2023-10-08T13:49:49.156294+0000","last_active":"2023-10-08T13:49:50.389166+0000","last_peered":"2023-10-08T13:49:50.389166+0000","last_clean":"2023-10-08T13:49:50.389166+0000","last_became_active":"2023-10-08T13:49:49.155821+0000","last_became_peered":"2023-10-08T13:49:49.155821+0000","last_unstale":"2023-10-08T13:49:50.389166+0000","last_undegraded":"2023-10-08T13:49:50.389166+0000","last_fullsized":"2023-10-08T13:49:50.389166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:59:43.831634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174567+0000","last_change":"2023-10-08T13:49:43.780052+0000","last_active":"2023-10-08T13:49:50.174567+0000","last_peered":"2023-10-08T13:49:50.174567+0000","last_clean":"2023-10-08T13:49:50.174567+0000","last_became_active":"2023-10-08T13:49:43.779731+0000","last_became_peered":"2023-10-08T13:49:43.779731+0000","last_unstale":"2023-10-08T13:49:50.174567+0000","last_undegraded":"2023-10-08T13:49:50.174567+0000","last_fullsized":"2023-10-08T13:49:50.174567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:38.800783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.159663+0000","last_change":"2023-10-08T13:49:44.787880+0000","last_active":"2023-10-08T13:49:49.159663+0000","last_peered":"2023-10-08T13:49:49.159663+0000","last_clean":"2023-10-08T13:49:49.159663+0000","last_became_active":"2023-10-08T13:49:44.787591+0000","last_became_peered":"2023-10-08T13:49:44.787591+0000","last_unstale":"2023-10-08T13:49:49.159663+0000","last_undegraded":"2023-10-08T13:49:49.159663+0000","last_fullsized":"2023-10-08T13:49:49.159663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:53.078903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175396+0000","last_change":"2023-10-08T13:49:49.155004+0000","last_active":"2023-10-08T13:49:50.175396+0000","last_peered":"2023-10-08T13:49:50.175396+0000","last_clean":"2023-10-08T13:49:50.175396+0000","last_became_active":"2023-10-08T13:49:49.154503+0000","last_became_peered":"2023-10-08T13:49:49.154503+0000","last_unstale":"2023-10-08T13:49:50.175396+0000","last_undegraded":"2023-10-08T13:49:50.175396+0000","last_fullsized":"2023-10-08T13:49:50.175396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:21.565816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:04.283502+0000","last_change":"2023-10-08T13:49:43.775819+0000","last_active":"2023-10-08T13:50:04.283502+0000","last_peered":"2023-10-08T13:50:04.283502+0000","last_clean":"2023-10-08T13:50:04.283502+0000","last_became_active":"2023-10-08T13:49:43.775578+0000","last_became_peered":"2023-10-08T13:49:43.775578+0000","last_unstale":"2023-10-08T13:50:04.283502+0000","last_undegraded":"2023-10-08T13:50:04.283502+0000","last_fullsized":"2023-10-08T13:50:04.283502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:31.518191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148667+0000","last_change":"2023-10-08T13:49:44.785869+0000","last_active":"2023-10-08T13:49:49.148667+0000","last_peered":"2023-10-08T13:49:49.148667+0000","last_clean":"2023-10-08T13:49:49.148667+0000","last_became_active":"2023-10-08T13:49:44.785566+0000","last_became_peered":"2023-10-08T13:49:44.785566+0000","last_unstale":"2023-10-08T13:49:49.148667+0000","last_undegraded":"2023-10-08T13:49:49.148667+0000","last_fullsized":"2023-10-08T13:49:49.148667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:29.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.162217+0000","last_change":"2023-10-08T13:49:49.162075+0000","last_active":"2023-10-08T13:49:50.162217+0000","last_peered":"2023-10-08T13:49:50.162217+0000","last_clean":"2023-10-08T13:49:50.162217+0000","last_became_active":"2023-10-08T13:49:49.161485+0000","last_became_peered":"2023-10-08T13:49:49.161485+0000","last_unstale":"2023-10-08T13:49:50.162217+0000","last_undegraded":"2023-10-08T13:49:50.162217+0000","last_fullsized":"2023-10-08T13:49:50.162217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:16.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:07.323980+0000","last_change":"2023-10-08T13:49:43.776555+0000","last_active":"2023-10-08T13:50:07.323980+0000","last_peered":"2023-10-08T13:50:07.323980+0000","last_clean":"2023-10-08T13:50:07.323980+0000","last_became_active":"2023-10-08T13:49:43.776259+0000","last_became_peered":"2023-10-08T13:49:43.776259+0000","last_unstale":"2023-10-08T13:50:07.323980+0000","last_undegraded":"2023-10-08T13:50:07.323980+0000","last_fullsized":"2023-10-08T13:50:07.323980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:34.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148899+0000","last_change":"2023-10-08T13:49:44.779253+0000","last_active":"2023-10-08T13:49:49.148899+0000","last_peered":"2023-10-08T13:49:49.148899+0000","last_clean":"2023-10-08T13:49:49.148899+0000","last_became_active":"2023-10-08T13:49:44.778902+0000","last_became_peered":"2023-10-08T13:49:44.778902+0000","last_unstale":"2023-10-08T13:49:49.148899+0000","last_undegraded":"2023-10-08T13:49:49.148899+0000","last_fullsized":"2023-10-08T13:49:49.148899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:31:01.091424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164641+0000","last_change":"2023-10-08T13:49:49.164859+0000","last_active":"2023-10-08T13:49:50.164641+0000","last_peered":"2023-10-08T13:49:50.164641+0000","last_clean":"2023-10-08T13:49:50.164641+0000","last_became_active":"2023-10-08T13:49:49.164221+0000","last_became_peered":"2023-10-08T13:49:49.164221+0000","last_unstale":"2023-10-08T13:49:50.164641+0000","last_undegraded":"2023-10-08T13:49:50.164641+0000","last_fullsized":"2023-10-08T13:49:50.164641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:22.484934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175938+0000","last_change":"2023-10-08T13:49:43.788136+0000","last_active":"2023-10-08T13:49:50.175938+0000","last_peered":"2023-10-08T13:49:50.175938+0000","last_clean":"2023-10-08T13:49:50.175938+0000","last_became_active":"2023-10-08T13:49:43.787910+0000","last_became_peered":"2023-10-08T13:49:43.787910+0000","last_unstale":"2023-10-08T13:49:50.175938+0000","last_undegraded":"2023-10-08T13:49:50.175938+0000","last_fullsized":"2023-10-08T13:49:50.175938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:00:32.897972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148905+0000","last_change":"2023-10-08T13:49:44.786833+0000","last_active":"2023-10-08T13:49:49.148905+0000","last_peered":"2023-10-08T13:49:49.148905+0000","last_clean":"2023-10-08T13:49:49.148905+0000","last_became_active":"2023-10-08T13:49:44.786523+0000","last_became_peered":"2023-10-08T13:49:44.786523+0000","last_unstale":"2023-10-08T13:49:49.148905+0000","last_undegraded":"2023-10-08T13:49:49.148905+0000","last_fullsized":"2023-10-08T13:49:49.148905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:36:22.156252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.162461+0000","last_change":"2023-10-08T13:49:49.160653+0000","last_active":"2023-10-08T13:49:50.162461+0000","last_peered":"2023-10-08T13:49:50.162461+0000","last_clean":"2023-10-08T13:49:50.162461+0000","last_became_active":"2023-10-08T13:49:49.160253+0000","last_became_peered":"2023-10-08T13:49:49.160253+0000","last_unstale":"2023-10-08T13:49:50.162461+0000","last_undegraded":"2023-10-08T13:49:50.162461+0000","last_fullsized":"2023-10-08T13:49:50.162461+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:07.838617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'12","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:04.282958+0000","last_change":"2023-10-08T13:49:43.777736+0000","last_active":"2023-10-08T13:50:04.282958+0000","last_peered":"2023-10-08T13:50:04.282958+0000","last_clean":"2023-10-08T13:50:04.282958+0000","last_became_active":"2023-10-08T13:49:43.777431+0000","last_became_peered":"2023-10-08T13:49:43.777431+0000","last_unstale":"2023-10-08T13:50:04.282958+0000","last_undegraded":"2023-10-08T13:50:04.282958+0000","last_fullsized":"2023-10-08T13:50:04.282958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:12.907196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147573+0000","last_change":"2023-10-08T13:49:44.789122+0000","last_active":"2023-10-08T13:49:49.147573+0000","last_peered":"2023-10-08T13:49:49.147573+0000","last_clean":"2023-10-08T13:49:49.147573+0000","last_became_active":"2023-10-08T13:49:44.788831+0000","last_became_peered":"2023-10-08T13:49:44.788831+0000","last_unstale":"2023-10-08T13:49:49.147573+0000","last_undegraded":"2023-10-08T13:49:49.147573+0000","last_fullsized":"2023-10-08T13:49:49.147573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:45.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.176310+0000","last_change":"2023-10-08T13:49:49.157319+0000","last_active":"2023-10-08T13:49:50.176310+0000","last_peered":"2023-10-08T13:49:50.176310+0000","last_clean":"2023-10-08T13:49:50.176310+0000","last_became_active":"2023-10-08T13:49:49.156897+0000","last_became_peered":"2023-10-08T13:49:49.156897+0000","last_unstale":"2023-10-08T13:49:50.176310+0000","last_undegraded":"2023-10-08T13:49:50.176310+0000","last_fullsized":"2023-10-08T13:49:50.176310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:25.460707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:04.283662+0000","last_change":"2023-10-08T13:49:43.776699+0000","last_active":"2023-10-08T13:50:04.283662+0000","last_peered":"2023-10-08T13:50:04.283662+0000","last_clean":"2023-10-08T13:50:04.283662+0000","last_became_active":"2023-10-08T13:49:43.776293+0000","last_became_peered":"2023-10-08T13:49:43.776293+0000","last_unstale":"2023-10-08T13:50:04.283662+0000","last_undegraded":"2023-10-08T13:50:04.283662+0000","last_fullsized":"2023-10-08T13:50:04.283662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:19.056804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.141415+0000","last_change":"2023-10-08T13:49:44.789915+0000","last_active":"2023-10-08T13:49:49.141415+0000","last_peered":"2023-10-08T13:49:49.141415+0000","last_clean":"2023-10-08T13:49:49.141415+0000","last_became_active":"2023-10-08T13:49:44.789617+0000","last_became_peered":"2023-10-08T13:49:44.789617+0000","last_unstale":"2023-10-08T13:49:49.141415+0000","last_undegraded":"2023-10-08T13:49:49.141415+0000","last_fullsized":"2023-10-08T13:49:49.141415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:06.286155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175275+0000","last_change":"2023-10-08T13:49:49.165946+0000","last_active":"2023-10-08T13:49:50.175275+0000","last_peered":"2023-10-08T13:49:50.175275+0000","last_clean":"2023-10-08T13:49:50.175275+0000","last_became_active":"2023-10-08T13:49:49.165581+0000","last_became_peered":"2023-10-08T13:49:49.165581+0000","last_unstale":"2023-10-08T13:49:50.175275+0000","last_undegraded":"2023-10-08T13:49:50.175275+0000","last_fullsized":"2023-10-08T13:49:50.175275+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:13.772198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:04.283236+0000","last_change":"2023-10-08T13:49:43.779161+0000","last_active":"2023-10-08T13:50:04.283236+0000","last_peered":"2023-10-08T13:50:04.283236+0000","last_clean":"2023-10-08T13:50:04.283236+0000","last_became_active":"2023-10-08T13:49:43.778862+0000","last_became_peered":"2023-10-08T13:49:43.778862+0000","last_unstale":"2023-10-08T13:50:04.283236+0000","last_undegraded":"2023-10-08T13:50:04.283236+0000","last_fullsized":"2023-10-08T13:50:04.283236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:57:44.075322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147263+0000","last_change":"2023-10-08T13:49:44.782072+0000","last_active":"2023-10-08T13:49:49.147263+0000","last_peered":"2023-10-08T13:49:49.147263+0000","last_clean":"2023-10-08T13:49:49.147263+0000","last_became_active":"2023-10-08T13:49:44.781756+0000","last_became_peered":"2023-10-08T13:49:44.781756+0000","last_unstale":"2023-10-08T13:49:49.147263+0000","last_undegraded":"2023-10-08T13:49:49.147263+0000","last_fullsized":"2023-10-08T13:49:49.147263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:32.877818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.162940+0000","last_change":"2023-10-08T13:49:49.167457+0000","last_active":"2023-10-08T13:49:50.162940+0000","last_peered":"2023-10-08T13:49:50.162940+0000","last_clean":"2023-10-08T13:49:50.162940+0000","last_became_active":"2023-10-08T13:49:49.166861+0000","last_became_peered":"2023-10-08T13:49:49.166861+0000","last_unstale":"2023-10-08T13:49:50.162940+0000","last_undegraded":"2023-10-08T13:49:50.162940+0000","last_fullsized":"2023-10-08T13:49:50.162940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:55.805271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142732+0000","last_change":"2023-10-08T13:49:43.782822+0000","last_active":"2023-10-08T13:49:49.142732+0000","last_peered":"2023-10-08T13:49:49.142732+0000","last_clean":"2023-10-08T13:49:49.142732+0000","last_became_active":"2023-10-08T13:49:43.782524+0000","last_became_peered":"2023-10-08T13:49:43.782524+0000","last_unstale":"2023-10-08T13:49:49.142732+0000","last_undegraded":"2023-10-08T13:49:49.142732+0000","last_fullsized":"2023-10-08T13:49:49.142732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:11.847851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147586+0000","last_change":"2023-10-08T13:49:44.794296+0000","last_active":"2023-10-08T13:49:49.147586+0000","last_peered":"2023-10-08T13:49:49.147586+0000","last_clean":"2023-10-08T13:49:49.147586+0000","last_became_active":"2023-10-08T13:49:44.793987+0000","last_became_peered":"2023-10-08T13:49:44.793987+0000","last_unstale":"2023-10-08T13:49:49.147586+0000","last_undegraded":"2023-10-08T13:49:49.147586+0000","last_fullsized":"2023-10-08T13:49:49.147586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:41.562569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174829+0000","last_change":"2023-10-08T13:49:49.165016+0000","last_active":"2023-10-08T13:49:50.174829+0000","last_peered":"2023-10-08T13:49:50.174829+0000","last_clean":"2023-10-08T13:49:50.174829+0000","last_became_active":"2023-10-08T13:49:49.164531+0000","last_became_peered":"2023-10-08T13:49:49.164531+0000","last_unstale":"2023-10-08T13:49:50.174829+0000","last_undegraded":"2023-10-08T13:49:50.174829+0000","last_fullsized":"2023-10-08T13:49:50.174829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:45.014512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146511+0000","last_change":"2023-10-08T13:49:43.786929+0000","last_active":"2023-10-08T13:49:49.146511+0000","last_peered":"2023-10-08T13:49:49.146511+0000","last_clean":"2023-10-08T13:49:49.146511+0000","last_became_active":"2023-10-08T13:49:43.786641+0000","last_became_peered":"2023-10-08T13:49:43.786641+0000","last_unstale":"2023-10-08T13:49:49.146511+0000","last_undegraded":"2023-10-08T13:49:49.146511+0000","last_fullsized":"2023-10-08T13:49:49.146511+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:30.054067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148947+0000","last_change":"2023-10-08T13:49:44.786718+0000","last_active":"2023-10-08T13:49:49.148947+0000","last_peered":"2023-10-08T13:49:49.148947+0000","last_clean":"2023-10-08T13:49:49.148947+0000","last_became_active":"2023-10-08T13:49:44.786413+0000","last_became_peered":"2023-10-08T13:49:44.786413+0000","last_unstale":"2023-10-08T13:49:49.148947+0000","last_undegraded":"2023-10-08T13:49:49.148947+0000","last_fullsized":"2023-10-08T13:49:49.148947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.302426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174625+0000","last_change":"2023-10-08T13:49:49.152226+0000","last_active":"2023-10-08T13:49:50.174625+0000","last_peered":"2023-10-08T13:49:50.174625+0000","last_clean":"2023-10-08T13:49:50.174625+0000","last_became_active":"2023-10-08T13:49:49.151772+0000","last_became_peered":"2023-10-08T13:49:49.151772+0000","last_unstale":"2023-10-08T13:49:50.174625+0000","last_undegraded":"2023-10-08T13:49:50.174625+0000","last_fullsized":"2023-10-08T13:49:50.174625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:53.856496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146977+0000","last_change":"2023-10-08T13:49:43.777648+0000","last_active":"2023-10-08T13:49:49.146977+0000","last_peered":"2023-10-08T13:49:49.146977+0000","last_clean":"2023-10-08T13:49:49.146977+0000","last_became_active":"2023-10-08T13:49:43.777361+0000","last_became_peered":"2023-10-08T13:49:43.777361+0000","last_unstale":"2023-10-08T13:49:49.146977+0000","last_undegraded":"2023-10-08T13:49:49.146977+0000","last_fullsized":"2023-10-08T13:49:49.146977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:53:19.939752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.155523+0000","last_change":"2023-10-08T13:49:44.782805+0000","last_active":"2023-10-08T13:49:49.155523+0000","last_peered":"2023-10-08T13:49:49.155523+0000","last_clean":"2023-10-08T13:49:49.155523+0000","last_became_active":"2023-10-08T13:49:44.782485+0000","last_became_peered":"2023-10-08T13:49:44.782485+0000","last_unstale":"2023-10-08T13:49:49.155523+0000","last_undegraded":"2023-10-08T13:49:49.155523+0000","last_fullsized":"2023-10-08T13:49:49.155523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:19.805734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.167364+0000","last_change":"2023-10-08T13:49:49.546226+0000","last_active":"2023-10-08T13:49:50.167364+0000","last_peered":"2023-10-08T13:49:50.167364+0000","last_clean":"2023-10-08T13:49:50.167364+0000","last_became_active":"2023-10-08T13:49:49.545626+0000","last_became_peered":"2023-10-08T13:49:49.545626+0000","last_unstale":"2023-10-08T13:49:50.167364+0000","last_undegraded":"2023-10-08T13:49:50.167364+0000","last_fullsized":"2023-10-08T13:49:50.167364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:37.184944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.160970+0000","last_change":"2023-10-08T13:49:43.784705+0000","last_active":"2023-10-08T13:49:49.160970+0000","last_peered":"2023-10-08T13:49:49.160970+0000","last_clean":"2023-10-08T13:49:49.160970+0000","last_became_active":"2023-10-08T13:49:43.784391+0000","last_became_peered":"2023-10-08T13:49:43.784391+0000","last_unstale":"2023-10-08T13:49:49.160970+0000","last_undegraded":"2023-10-08T13:49:49.160970+0000","last_fullsized":"2023-10-08T13:49:49.160970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:45.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.545021+0000","last_change":"2023-10-08T13:49:44.789845+0000","last_active":"2023-10-08T13:49:49.545021+0000","last_peered":"2023-10-08T13:49:49.545021+0000","last_clean":"2023-10-08T13:49:49.545021+0000","last_became_active":"2023-10-08T13:49:44.789580+0000","last_became_peered":"2023-10-08T13:49:44.789580+0000","last_unstale":"2023-10-08T13:49:49.545021+0000","last_undegraded":"2023-10-08T13:49:49.545021+0000","last_fullsized":"2023-10-08T13:49:49.545021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:58.098757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175738+0000","last_change":"2023-10-08T13:49:49.153277+0000","last_active":"2023-10-08T13:49:50.175738+0000","last_peered":"2023-10-08T13:49:50.175738+0000","last_clean":"2023-10-08T13:49:50.175738+0000","last_became_active":"2023-10-08T13:49:49.152575+0000","last_became_peered":"2023-10-08T13:49:49.152575+0000","last_unstale":"2023-10-08T13:49:50.175738+0000","last_undegraded":"2023-10-08T13:49:50.175738+0000","last_fullsized":"2023-10-08T13:49:50.175738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:35.677823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:07.735809+0000","last_change":"2023-10-08T13:49:43.778442+0000","last_active":"2023-10-08T13:50:07.735809+0000","last_peered":"2023-10-08T13:50:07.735809+0000","last_clean":"2023-10-08T13:50:07.735809+0000","last_became_active":"2023-10-08T13:49:43.774799+0000","last_became_peered":"2023-10-08T13:49:43.774799+0000","last_unstale":"2023-10-08T13:50:07.735809+0000","last_undegraded":"2023-10-08T13:50:07.735809+0000","last_fullsized":"2023-10-08T13:50:07.735809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:15.464736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.544687+0000","last_change":"2023-10-08T13:49:44.780097+0000","last_active":"2023-10-08T13:49:49.544687+0000","last_peered":"2023-10-08T13:49:49.544687+0000","last_clean":"2023-10-08T13:49:49.544687+0000","last_became_active":"2023-10-08T13:49:44.779774+0000","last_became_peered":"2023-10-08T13:49:44.779774+0000","last_unstale":"2023-10-08T13:49:49.544687+0000","last_undegraded":"2023-10-08T13:49:49.544687+0000","last_fullsized":"2023-10-08T13:49:49.544687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:19.132295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175603+0000","last_change":"2023-10-08T13:49:49.547129+0000","last_active":"2023-10-08T13:49:50.175603+0000","last_peered":"2023-10-08T13:49:50.175603+0000","last_clean":"2023-10-08T13:49:50.175603+0000","last_became_active":"2023-10-08T13:49:49.546694+0000","last_became_peered":"2023-10-08T13:49:49.546694+0000","last_unstale":"2023-10-08T13:49:50.175603+0000","last_undegraded":"2023-10-08T13:49:50.175603+0000","last_fullsized":"2023-10-08T13:49:50.175603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:45.524650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149911+0000","last_change":"2023-10-08T13:49:44.779875+0000","last_active":"2023-10-08T13:49:49.149911+0000","last_peered":"2023-10-08T13:49:49.149911+0000","last_clean":"2023-10-08T13:49:49.149911+0000","last_became_active":"2023-10-08T13:49:44.779570+0000","last_became_peered":"2023-10-08T13:49:44.779570+0000","last_unstale":"2023-10-08T13:49:49.149911+0000","last_undegraded":"2023-10-08T13:49:49.149911+0000","last_fullsized":"2023-10-08T13:49:49.149911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:02.981998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:05.298544+0000","last_change":"2023-10-08T13:49:43.787574+0000","last_active":"2023-10-08T13:50:05.298544+0000","last_peered":"2023-10-08T13:50:05.298544+0000","last_clean":"2023-10-08T13:50:05.298544+0000","last_became_active":"2023-10-08T13:49:43.787261+0000","last_became_peered":"2023-10-08T13:49:43.787261+0000","last_unstale":"2023-10-08T13:50:05.298544+0000","last_undegraded":"2023-10-08T13:50:05.298544+0000","last_fullsized":"2023-10-08T13:50:05.298544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:28.676135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:52.935506+0000","last_change":"2023-10-08T13:49:49.163856+0000","last_active":"2023-10-08T13:49:52.935506+0000","last_peered":"2023-10-08T13:49:52.935506+0000","last_clean":"2023-10-08T13:49:52.935506+0000","last_became_active":"2023-10-08T13:49:49.163456+0000","last_became_peered":"2023-10-08T13:49:49.163456+0000","last_unstale":"2023-10-08T13:49:52.935506+0000","last_undegraded":"2023-10-08T13:49:52.935506+0000","last_fullsized":"2023-10-08T13:49:52.935506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:31.654309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149129+0000","last_change":"2023-10-08T13:49:44.792935+0000","last_active":"2023-10-08T13:49:49.149129+0000","last_peered":"2023-10-08T13:49:49.149129+0000","last_clean":"2023-10-08T13:49:49.149129+0000","last_became_active":"2023-10-08T13:49:44.792667+0000","last_became_peered":"2023-10-08T13:49:44.792667+0000","last_unstale":"2023-10-08T13:49:49.149129+0000","last_undegraded":"2023-10-08T13:49:49.149129+0000","last_fullsized":"2023-10-08T13:49:49.149129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:40.964452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142640+0000","last_change":"2023-10-08T13:49:43.781424+0000","last_active":"2023-10-08T13:49:49.142640+0000","last_peered":"2023-10-08T13:49:49.142640+0000","last_clean":"2023-10-08T13:49:49.142640+0000","last_became_active":"2023-10-08T13:49:43.780930+0000","last_became_peered":"2023-10-08T13:49:43.780930+0000","last_unstale":"2023-10-08T13:49:49.142640+0000","last_undegraded":"2023-10-08T13:49:49.142640+0000","last_fullsized":"2023-10-08T13:49:49.142640+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:56:50.397289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":323,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146439+0000","last_change":"2023-10-08T13:48:45.192219+0000","last_active":"2023-10-08T13:49:49.146439+0000","last_peered":"2023-10-08T13:49:49.146439+0000","last_clean":"2023-10-08T13:49:49.146439+0000","last_became_active":"2023-10-08T13:48:45.191813+0000","last_became_peered":"2023-10-08T13:48:45.191813+0000","last_unstale":"2023-10-08T13:49:49.146439+0000","last_undegraded":"2023-10-08T13:49:49.146439+0000","last_fullsized":"2023-10-08T13:49:49.146439+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_clean_scrub_stamp":"2023-10-08T13:45:51.040695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.303937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175811+0000","last_change":"2023-10-08T13:49:49.546181+0000","last_active":"2023-10-08T13:49:50.175811+0000","last_peered":"2023-10-08T13:49:50.175811+0000","last_clean":"2023-10-08T13:49:50.175811+0000","last_became_active":"2023-10-08T13:49:49.545726+0000","last_became_peered":"2023-10-08T13:49:49.545726+0000","last_unstale":"2023-10-08T13:49:50.175811+0000","last_undegraded":"2023-10-08T13:49:50.175811+0000","last_fullsized":"2023-10-08T13:49:50.175811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:45.370909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.141204+0000","last_change":"2023-10-08T13:49:44.785544+0000","last_active":"2023-10-08T13:49:49.141204+0000","last_peered":"2023-10-08T13:49:49.141204+0000","last_clean":"2023-10-08T13:49:49.141204+0000","last_became_active":"2023-10-08T13:49:44.785007+0000","last_became_peered":"2023-10-08T13:49:44.785007+0000","last_unstale":"2023-10-08T13:49:49.141204+0000","last_undegraded":"2023-10-08T13:49:49.141204+0000","last_fullsized":"2023-10-08T13:49:49.141204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:16:55.058693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:05.296736+0000","last_change":"2023-10-08T13:49:43.779207+0000","last_active":"2023-10-08T13:50:05.296736+0000","last_peered":"2023-10-08T13:50:05.296736+0000","last_clean":"2023-10-08T13:50:05.296736+0000","last_became_active":"2023-10-08T13:49:43.778846+0000","last_became_peered":"2023-10-08T13:49:43.778846+0000","last_unstale":"2023-10-08T13:50:05.296736+0000","last_undegraded":"2023-10-08T13:50:05.296736+0000","last_fullsized":"2023-10-08T13:50:05.296736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:45.868187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.175105+0000","last_change":"2023-10-08T13:49:49.162156+0000","last_active":"2023-10-08T13:49:50.175105+0000","last_peered":"2023-10-08T13:49:50.175105+0000","last_clean":"2023-10-08T13:49:50.175105+0000","last_became_active":"2023-10-08T13:49:49.161720+0000","last_became_peered":"2023-10-08T13:49:49.161720+0000","last_unstale":"2023-10-08T13:49:50.175105+0000","last_undegraded":"2023-10-08T13:49:50.175105+0000","last_fullsized":"2023-10-08T13:49:50.175105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:13.499915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147669+0000","last_change":"2023-10-08T13:49:43.782322+0000","last_active":"2023-10-08T13:49:49.147669+0000","last_peered":"2023-10-08T13:49:49.147669+0000","last_clean":"2023-10-08T13:49:49.147669+0000","last_became_active":"2023-10-08T13:49:43.781938+0000","last_became_peered":"2023-10-08T13:49:43.781938+0000","last_unstale":"2023-10-08T13:49:49.147669+0000","last_undegraded":"2023-10-08T13:49:49.147669+0000","last_fullsized":"2023-10-08T13:49:49.147669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:57.627507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142286+0000","last_change":"2023-10-08T13:49:44.778340+0000","last_active":"2023-10-08T13:49:49.142286+0000","last_peered":"2023-10-08T13:49:49.142286+0000","last_clean":"2023-10-08T13:49:49.142286+0000","last_became_active":"2023-10-08T13:49:44.778000+0000","last_became_peered":"2023-10-08T13:49:44.778000+0000","last_unstale":"2023-10-08T13:49:49.142286+0000","last_undegraded":"2023-10-08T13:49:49.142286+0000","last_fullsized":"2023-10-08T13:49:49.142286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:23.350449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174377+0000","last_change":"2023-10-08T13:49:49.545915+0000","last_active":"2023-10-08T13:49:50.174377+0000","last_peered":"2023-10-08T13:49:50.174377+0000","last_clean":"2023-10-08T13:49:50.174377+0000","last_became_active":"2023-10-08T13:49:49.545396+0000","last_became_peered":"2023-10-08T13:49:49.545396+0000","last_unstale":"2023-10-08T13:49:50.174377+0000","last_undegraded":"2023-10-08T13:49:50.174377+0000","last_fullsized":"2023-10-08T13:49:50.174377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:07:03.850707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.188889+0000","last_change":"2023-10-08T13:49:43.776479+0000","last_active":"2023-10-08T13:49:49.188889+0000","last_peered":"2023-10-08T13:49:49.188889+0000","last_clean":"2023-10-08T13:49:49.188889+0000","last_became_active":"2023-10-08T13:49:43.776130+0000","last_became_peered":"2023-10-08T13:49:43.776130+0000","last_unstale":"2023-10-08T13:49:49.188889+0000","last_undegraded":"2023-10-08T13:49:49.188889+0000","last_fullsized":"2023-10-08T13:49:49.188889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.760660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149144+0000","last_change":"2023-10-08T13:49:44.793864+0000","last_active":"2023-10-08T13:49:49.149144+0000","last_peered":"2023-10-08T13:49:49.149144+0000","last_clean":"2023-10-08T13:49:49.149144+0000","last_became_active":"2023-10-08T13:49:44.793469+0000","last_became_peered":"2023-10-08T13:49:44.793469+0000","last_unstale":"2023-10-08T13:49:49.149144+0000","last_undegraded":"2023-10-08T13:49:49.149144+0000","last_fullsized":"2023-10-08T13:49:49.149144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:28.669711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.176066+0000","last_change":"2023-10-08T13:49:49.150696+0000","last_active":"2023-10-08T13:49:50.176066+0000","last_peered":"2023-10-08T13:49:50.176066+0000","last_clean":"2023-10-08T13:49:50.176066+0000","last_became_active":"2023-10-08T13:49:49.150188+0000","last_became_peered":"2023-10-08T13:49:49.150188+0000","last_unstale":"2023-10-08T13:49:50.176066+0000","last_undegraded":"2023-10-08T13:49:50.176066+0000","last_fullsized":"2023-10-08T13:49:50.176066+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:51:35.775240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142893+0000","last_change":"2023-10-08T13:49:43.786245+0000","last_active":"2023-10-08T13:49:49.142893+0000","last_peered":"2023-10-08T13:49:49.142893+0000","last_clean":"2023-10-08T13:49:49.142893+0000","last_became_active":"2023-10-08T13:49:43.785986+0000","last_became_peered":"2023-10-08T13:49:43.785986+0000","last_unstale":"2023-10-08T13:49:49.142893+0000","last_undegraded":"2023-10-08T13:49:49.142893+0000","last_fullsized":"2023-10-08T13:49:49.142893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:53:59.643447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149418+0000","last_change":"2023-10-08T13:49:44.790494+0000","last_active":"2023-10-08T13:49:49.149418+0000","last_peered":"2023-10-08T13:49:49.149418+0000","last_clean":"2023-10-08T13:49:49.149418+0000","last_became_active":"2023-10-08T13:49:44.790091+0000","last_became_peered":"2023-10-08T13:49:44.790091+0000","last_unstale":"2023-10-08T13:49:49.149418+0000","last_undegraded":"2023-10-08T13:49:49.149418+0000","last_fullsized":"2023-10-08T13:49:49.149418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:49.309597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164913+0000","last_change":"2023-10-08T13:49:49.547271+0000","last_active":"2023-10-08T13:49:50.164913+0000","last_peered":"2023-10-08T13:49:50.164913+0000","last_clean":"2023-10-08T13:49:50.164913+0000","last_became_active":"2023-10-08T13:49:49.546603+0000","last_became_peered":"2023-10-08T13:49:49.546603+0000","last_unstale":"2023-10-08T13:49:50.164913+0000","last_undegraded":"2023-10-08T13:49:50.164913+0000","last_fullsized":"2023-10-08T13:49:50.164913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:29.908321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148090+0000","last_change":"2023-10-08T13:49:43.788638+0000","last_active":"2023-10-08T13:49:49.148090+0000","last_peered":"2023-10-08T13:49:49.148090+0000","last_clean":"2023-10-08T13:49:49.148090+0000","last_became_active":"2023-10-08T13:49:43.788303+0000","last_became_peered":"2023-10-08T13:49:43.788303+0000","last_unstale":"2023-10-08T13:49:49.148090+0000","last_undegraded":"2023-10-08T13:49:49.148090+0000","last_fullsized":"2023-10-08T13:49:49.148090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:52.637775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146530+0000","last_change":"2023-10-08T13:49:44.777546+0000","last_active":"2023-10-08T13:49:49.146530+0000","last_peered":"2023-10-08T13:49:49.146530+0000","last_clean":"2023-10-08T13:49:49.146530+0000","last_became_active":"2023-10-08T13:49:44.777261+0000","last_became_peered":"2023-10-08T13:49:44.777261+0000","last_unstale":"2023-10-08T13:49:49.146530+0000","last_undegraded":"2023-10-08T13:49:49.146530+0000","last_fullsized":"2023-10-08T13:49:49.146530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:26:36.368188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.163099+0000","last_change":"2023-10-08T13:49:49.163071+0000","last_active":"2023-10-08T13:49:50.163099+0000","last_peered":"2023-10-08T13:49:50.163099+0000","last_clean":"2023-10-08T13:49:50.163099+0000","last_became_active":"2023-10-08T13:49:49.162667+0000","last_became_peered":"2023-10-08T13:49:49.162667+0000","last_unstale":"2023-10-08T13:49:50.163099+0000","last_undegraded":"2023-10-08T13:49:50.163099+0000","last_fullsized":"2023-10-08T13:49:50.163099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:54:42.645837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:50:06.313233+0000","last_change":"2023-10-08T13:49:43.776693+0000","last_active":"2023-10-08T13:50:06.313233+0000","last_peered":"2023-10-08T13:50:06.313233+0000","last_clean":"2023-10-08T13:50:06.313233+0000","last_became_active":"2023-10-08T13:49:43.776367+0000","last_became_peered":"2023-10-08T13:49:43.776367+0000","last_unstale":"2023-10-08T13:50:06.313233+0000","last_undegraded":"2023-10-08T13:50:06.313233+0000","last_fullsized":"2023-10-08T13:50:06.313233+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:17.415011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.152599+0000","last_change":"2023-10-08T13:49:44.779472+0000","last_active":"2023-10-08T13:49:49.152599+0000","last_peered":"2023-10-08T13:49:49.152599+0000","last_clean":"2023-10-08T13:49:49.152599+0000","last_became_active":"2023-10-08T13:49:44.779172+0000","last_became_peered":"2023-10-08T13:49:44.779172+0000","last_unstale":"2023-10-08T13:49:49.152599+0000","last_undegraded":"2023-10-08T13:49:49.152599+0000","last_fullsized":"2023-10-08T13:49:49.152599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:17.314243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164634+0000","last_change":"2023-10-08T13:49:49.164535+0000","last_active":"2023-10-08T13:49:50.164634+0000","last_peered":"2023-10-08T13:49:50.164634+0000","last_clean":"2023-10-08T13:49:50.164634+0000","last_became_active":"2023-10-08T13:49:49.163467+0000","last_became_peered":"2023-10-08T13:49:49.163467+0000","last_unstale":"2023-10-08T13:49:50.164634+0000","last_undegraded":"2023-10-08T13:49:50.164634+0000","last_fullsized":"2023-10-08T13:49:50.164634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:55.058656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142401+0000","last_change":"2023-10-08T13:49:43.775311+0000","last_active":"2023-10-08T13:49:49.142401+0000","last_peered":"2023-10-08T13:49:49.142401+0000","last_clean":"2023-10-08T13:49:49.142401+0000","last_became_active":"2023-10-08T13:49:43.775005+0000","last_became_peered":"2023-10-08T13:49:43.775005+0000","last_unstale":"2023-10-08T13:49:49.142401+0000","last_undegraded":"2023-10-08T13:49:49.142401+0000","last_fullsized":"2023-10-08T13:49:49.142401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:33.713395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149584+0000","last_change":"2023-10-08T13:49:44.792051+0000","last_active":"2023-10-08T13:49:49.149584+0000","last_peered":"2023-10-08T13:49:49.149584+0000","last_clean":"2023-10-08T13:49:49.149584+0000","last_became_active":"2023-10-08T13:49:44.790045+0000","last_became_peered":"2023-10-08T13:49:44.790045+0000","last_unstale":"2023-10-08T13:49:49.149584+0000","last_undegraded":"2023-10-08T13:49:49.149584+0000","last_fullsized":"2023-10-08T13:49:49.149584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:28.699799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.164975+0000","last_change":"2023-10-08T13:49:49.546219+0000","last_active":"2023-10-08T13:49:50.164975+0000","last_peered":"2023-10-08T13:49:50.164975+0000","last_clean":"2023-10-08T13:49:50.164975+0000","last_became_active":"2023-10-08T13:49:49.545783+0000","last_became_peered":"2023-10-08T13:49:49.545783+0000","last_unstale":"2023-10-08T13:49:50.164975+0000","last_undegraded":"2023-10-08T13:49:50.164975+0000","last_fullsized":"2023-10-08T13:49:50.164975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:00.497908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146753+0000","last_change":"2023-10-08T13:49:43.776252+0000","last_active":"2023-10-08T13:49:49.146753+0000","last_peered":"2023-10-08T13:49:49.146753+0000","last_clean":"2023-10-08T13:49:49.146753+0000","last_became_active":"2023-10-08T13:49:43.775935+0000","last_became_peered":"2023-10-08T13:49:43.775935+0000","last_unstale":"2023-10-08T13:49:49.146753+0000","last_undegraded":"2023-10-08T13:49:49.146753+0000","last_fullsized":"2023-10-08T13:49:49.146753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:58.949365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.146922+0000","last_change":"2023-10-08T13:49:44.788628+0000","last_active":"2023-10-08T13:49:49.146922+0000","last_peered":"2023-10-08T13:49:49.146922+0000","last_clean":"2023-10-08T13:49:49.146922+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:49:49.146922+0000","last_undegraded":"2023-10-08T13:49:49.146922+0000","last_fullsized":"2023-10-08T13:49:49.146922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:46:42.311296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174221+0000","last_change":"2023-10-08T13:49:49.153641+0000","last_active":"2023-10-08T13:49:50.174221+0000","last_peered":"2023-10-08T13:49:50.174221+0000","last_clean":"2023-10-08T13:49:50.174221+0000","last_became_active":"2023-10-08T13:49:49.153172+0000","last_became_peered":"2023-10-08T13:49:49.153172+0000","last_unstale":"2023-10-08T13:49:50.174221+0000","last_undegraded":"2023-10-08T13:49:50.174221+0000","last_fullsized":"2023-10-08T13:49:50.174221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:40:51.726343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143719+0000","last_change":"2023-10-08T13:49:43.782683+0000","last_active":"2023-10-08T13:49:49.143719+0000","last_peered":"2023-10-08T13:49:49.143719+0000","last_clean":"2023-10-08T13:49:49.143719+0000","last_became_active":"2023-10-08T13:49:43.782269+0000","last_became_peered":"2023-10-08T13:49:43.782269+0000","last_unstale":"2023-10-08T13:49:49.143719+0000","last_undegraded":"2023-10-08T13:49:49.143719+0000","last_fullsized":"2023-10-08T13:49:49.143719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:50.850581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148276+0000","last_change":"2023-10-08T13:49:44.787894+0000","last_active":"2023-10-08T13:49:49.148276+0000","last_peered":"2023-10-08T13:49:49.148276+0000","last_clean":"2023-10-08T13:49:49.148276+0000","last_became_active":"2023-10-08T13:49:44.787599+0000","last_became_peered":"2023-10-08T13:49:44.787599+0000","last_unstale":"2023-10-08T13:49:49.148276+0000","last_undegraded":"2023-10-08T13:49:49.148276+0000","last_fullsized":"2023-10-08T13:49:49.148276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:17:15.131051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.389031+0000","last_change":"2023-10-08T13:49:49.164256+0000","last_active":"2023-10-08T13:49:50.389031+0000","last_peered":"2023-10-08T13:49:50.389031+0000","last_clean":"2023-10-08T13:49:50.389031+0000","last_became_active":"2023-10-08T13:49:49.163793+0000","last_became_peered":"2023-10-08T13:49:49.163793+0000","last_unstale":"2023-10-08T13:49:50.389031+0000","last_undegraded":"2023-10-08T13:49:50.389031+0000","last_fullsized":"2023-10-08T13:49:50.389031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:19.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149616+0000","last_change":"2023-10-08T13:49:43.778311+0000","last_active":"2023-10-08T13:49:49.149616+0000","last_peered":"2023-10-08T13:49:49.149616+0000","last_clean":"2023-10-08T13:49:49.149616+0000","last_became_active":"2023-10-08T13:49:43.777984+0000","last_became_peered":"2023-10-08T13:49:43.777984+0000","last_unstale":"2023-10-08T13:49:49.149616+0000","last_undegraded":"2023-10-08T13:49:49.149616+0000","last_fullsized":"2023-10-08T13:49:49.149616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:56.890729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.147428+0000","last_change":"2023-10-08T13:49:44.793424+0000","last_active":"2023-10-08T13:49:49.147428+0000","last_peered":"2023-10-08T13:49:49.147428+0000","last_clean":"2023-10-08T13:49:49.147428+0000","last_became_active":"2023-10-08T13:49:44.792825+0000","last_became_peered":"2023-10-08T13:49:44.792825+0000","last_unstale":"2023-10-08T13:49:49.147428+0000","last_undegraded":"2023-10-08T13:49:49.147428+0000","last_fullsized":"2023-10-08T13:49:49.147428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:39.463676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T13:49:50.174266+0000","last_change":"2023-10-08T13:49:49.151915+0000","last_active":"2023-10-08T13:49:50.174266+0000","last_peered":"2023-10-08T13:49:50.174266+0000","last_clean":"2023-10-08T13:49:50.174266+0000","last_became_active":"2023-10-08T13:49:49.151434+0000","last_became_peered":"2023-10-08T13:49:49.151434+0000","last_unstale":"2023-10-08T13:49:50.174266+0000","last_undegraded":"2023-10-08T13:49:50.174266+0000","last_fullsized":"2023-10-08T13:49:50.174266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:54:07.631498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142601+0000","last_change":"2023-10-08T13:49:43.772363+0000","last_active":"2023-10-08T13:49:49.142601+0000","last_peered":"2023-10-08T13:49:49.142601+0000","last_clean":"2023-10-08T13:49:49.142601+0000","last_became_active":"2023-10-08T13:49:43.772072+0000","last_became_peered":"2023-10-08T13:49:43.772072+0000","last_unstale":"2023-10-08T13:49:49.142601+0000","last_undegraded":"2023-10-08T13:49:49.142601+0000","last_fullsized":"2023-10-08T13:49:49.142601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:07:30.877217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148135+0000","last_change":"2023-10-08T13:49:44.790949+0000","last_active":"2023-10-08T13:49:49.148135+0000","last_peered":"2023-10-08T13:49:49.148135+0000","last_clean":"2023-10-08T13:49:49.148135+0000","last_became_active":"2023-10-08T13:49:44.790645+0000","last_became_peered":"2023-10-08T13:49:44.790645+0000","last_unstale":"2023-10-08T13:49:49.148135+0000","last_undegraded":"2023-10-08T13:49:49.148135+0000","last_fullsized":"2023-10-08T13:49:49.148135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:48.890730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.152709+0000","last_change":"2023-10-08T13:49:44.781165+0000","last_active":"2023-10-08T13:49:49.152709+0000","last_peered":"2023-10-08T13:49:49.152709+0000","last_clean":"2023-10-08T13:49:49.152709+0000","last_became_active":"2023-10-08T13:49:44.780828+0000","last_became_peered":"2023-10-08T13:49:44.780828+0000","last_unstale":"2023-10-08T13:49:49.152709+0000","last_undegraded":"2023-10-08T13:49:49.152709+0000","last_fullsized":"2023-10-08T13:49:49.152709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:04.259482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143422+0000","last_change":"2023-10-08T13:49:44.781271+0000","last_active":"2023-10-08T13:49:49.143422+0000","last_peered":"2023-10-08T13:49:49.143422+0000","last_clean":"2023-10-08T13:49:49.143422+0000","last_became_active":"2023-10-08T13:49:44.780912+0000","last_became_peered":"2023-10-08T13:49:44.780912+0000","last_unstale":"2023-10-08T13:49:49.143422+0000","last_undegraded":"2023-10-08T13:49:49.143422+0000","last_fullsized":"2023-10-08T13:49:49.143422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:17.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142288+0000","last_change":"2023-10-08T13:49:44.777613+0000","last_active":"2023-10-08T13:49:49.142288+0000","last_peered":"2023-10-08T13:49:49.142288+0000","last_clean":"2023-10-08T13:49:49.142288+0000","last_became_active":"2023-10-08T13:49:44.777304+0000","last_became_peered":"2023-10-08T13:49:44.777304+0000","last_unstale":"2023-10-08T13:49:49.142288+0000","last_undegraded":"2023-10-08T13:49:49.142288+0000","last_fullsized":"2023-10-08T13:49:49.142288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:45.482935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.150763+0000","last_change":"2023-10-08T13:49:44.782685+0000","last_active":"2023-10-08T13:49:49.150763+0000","last_peered":"2023-10-08T13:49:49.150763+0000","last_clean":"2023-10-08T13:49:49.150763+0000","last_became_active":"2023-10-08T13:49:44.781816+0000","last_became_peered":"2023-10-08T13:49:44.781816+0000","last_unstale":"2023-10-08T13:49:49.150763+0000","last_undegraded":"2023-10-08T13:49:49.150763+0000","last_fullsized":"2023-10-08T13:49:49.150763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:53.781237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.142383+0000","last_change":"2023-10-08T13:49:44.783973+0000","last_active":"2023-10-08T13:49:49.142383+0000","last_peered":"2023-10-08T13:49:49.142383+0000","last_clean":"2023-10-08T13:49:49.142383+0000","last_became_active":"2023-10-08T13:49:44.783648+0000","last_became_peered":"2023-10-08T13:49:44.783648+0000","last_unstale":"2023-10-08T13:49:49.142383+0000","last_undegraded":"2023-10-08T13:49:49.142383+0000","last_fullsized":"2023-10-08T13:49:49.142383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:01.253610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.153009+0000","last_change":"2023-10-08T13:49:44.781848+0000","last_active":"2023-10-08T13:49:49.153009+0000","last_peered":"2023-10-08T13:49:49.153009+0000","last_clean":"2023-10-08T13:49:49.153009+0000","last_became_active":"2023-10-08T13:49:44.781542+0000","last_became_peered":"2023-10-08T13:49:44.781542+0000","last_unstale":"2023-10-08T13:49:49.153009+0000","last_undegraded":"2023-10-08T13:49:49.153009+0000","last_fullsized":"2023-10-08T13:49:49.153009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:26:38.726298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.143093+0000","last_change":"2023-10-08T13:49:44.786541+0000","last_active":"2023-10-08T13:49:49.143093+0000","last_peered":"2023-10-08T13:49:49.143093+0000","last_clean":"2023-10-08T13:49:49.143093+0000","last_became_active":"2023-10-08T13:49:44.786214+0000","last_became_peered":"2023-10-08T13:49:44.786214+0000","last_unstale":"2023-10-08T13:49:49.143093+0000","last_undegraded":"2023-10-08T13:49:49.143093+0000","last_fullsized":"2023-10-08T13:49:49.143093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:30:18.342774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149227+0000","last_change":"2023-10-08T13:49:44.788445+0000","last_active":"2023-10-08T13:49:49.149227+0000","last_peered":"2023-10-08T13:49:49.149227+0000","last_clean":"2023-10-08T13:49:49.149227+0000","last_became_active":"2023-10-08T13:49:44.788075+0000","last_became_peered":"2023-10-08T13:49:44.788075+0000","last_unstale":"2023-10-08T13:49:49.149227+0000","last_undegraded":"2023-10-08T13:49:49.149227+0000","last_fullsized":"2023-10-08T13:49:49.149227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:08:20.701231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149588+0000","last_change":"2023-10-08T13:49:44.783755+0000","last_active":"2023-10-08T13:49:49.149588+0000","last_peered":"2023-10-08T13:49:49.149588+0000","last_clean":"2023-10-08T13:49:49.149588+0000","last_became_active":"2023-10-08T13:49:44.783467+0000","last_became_peered":"2023-10-08T13:49:44.783467+0000","last_unstale":"2023-10-08T13:49:49.149588+0000","last_undegraded":"2023-10-08T13:49:49.149588+0000","last_fullsized":"2023-10-08T13:49:49.149588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:39.838940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.149966+0000","last_change":"2023-10-08T13:49:44.779342+0000","last_active":"2023-10-08T13:49:49.149966+0000","last_peered":"2023-10-08T13:49:49.149966+0000","last_clean":"2023-10-08T13:49:49.149966+0000","last_became_active":"2023-10-08T13:49:44.778518+0000","last_became_peered":"2023-10-08T13:49:44.778518+0000","last_unstale":"2023-10-08T13:49:49.149966+0000","last_undegraded":"2023-10-08T13:49:49.149966+0000","last_fullsized":"2023-10-08T13:49:49.149966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:33.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.148573+0000","last_change":"2023-10-08T13:49:44.792955+0000","last_active":"2023-10-08T13:49:49.148573+0000","last_peered":"2023-10-08T13:49:49.148573+0000","last_clean":"2023-10-08T13:49:49.148573+0000","last_became_active":"2023-10-08T13:49:44.792637+0000","last_became_peered":"2023-10-08T13:49:44.792637+0000","last_unstale":"2023-10-08T13:49:49.148573+0000","last_undegraded":"2023-10-08T13:49:49.148573+0000","last_fullsized":"2023-10-08T13:49:49.148573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:40.635913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.141628+0000","last_change":"2023-10-08T13:49:44.786817+0000","last_active":"2023-10-08T13:49:49.141628+0000","last_peered":"2023-10-08T13:49:49.141628+0000","last_clean":"2023-10-08T13:49:49.141628+0000","last_became_active":"2023-10-08T13:49:44.786439+0000","last_became_peered":"2023-10-08T13:49:44.786439+0000","last_unstale":"2023-10-08T13:49:49.141628+0000","last_undegraded":"2023-10-08T13:49:49.141628+0000","last_fullsized":"2023-10-08T13:49:49.141628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:45.308600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T13:49:49.152693+0000","last_change":"2023-10-08T13:49:44.782776+0000","last_active":"2023-10-08T13:49:49.152693+0000","last_peered":"2023-10-08T13:49:49.152693+0000","last_clean":"2023-10-08T13:49:49.152693+0000","last_became_active":"2023-10-08T13:49:44.782460+0000","last_became_peered":"2023-10-08T13:49:44.782460+0000","last_unstale":"2023-10-08T13:49:49.152693+0000","last_undegraded":"2023-10-08T13:49:49.152693+0000","last_fullsized":"2023-10-08T13:49:49.152693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:15:12.580688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6792,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":102,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":22866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":486499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":10,"up_from":71,"seq":304942678036,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1083117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.41999999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68300000000000005}]},{"osd":3,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65500000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48499999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78300000000000003}]},{"osd":6,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.70699999999999996}]},{"osd":7,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55300000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.72199999999999998}]},{"osd":9,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75800000000000001}]},{"osd":11,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77000000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841559,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":487989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.35499999999999998}]},{"osd":1,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.442}]},{"osd":3,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.375},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64300000000000002}]},{"osd":5,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.42899999999999999}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.9570000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.345}]},{"osd":7,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.66600000000000004}]},{"osd":8,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.498}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.65500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":8,"up_from":58,"seq":249108103195,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":987136,"data_stored":487809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.57399999999999995,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":5.5860000000000003,"5min":5.5860000000000003,"15min":5.5860000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.55700000000000005,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.80300000000000005}]},{"osd":1,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.628,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.77700000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.51300000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56799999999999995,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84199999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70499999999999996}]},{"osd":4,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.876}]},{"osd":5,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.59799999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91500000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.69399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.89200000000000002}]},{"osd":7,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":0.96999999999999997}]},{"osd":9,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.747},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.92800000000000005}]},{"osd":10,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.99099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364833,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1085201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.439}]},{"osd":1,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.27000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.438,"15min":0.42799999999999999},"min":{"1min":0.28299999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45600000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.497},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57299999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.40699999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.627,"15min":0.62},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58599999999999997}]},{"osd":6,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67400000000000004,"15min":0.66900000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.69999999999999996}]},{"osd":8,"last update":"Sun Oct 8 13:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67000000000000004}]},{"osd":9,"last update":"Sun Oct 8 13:49:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.65800000000000003}]},{"osd":10,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.626}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528357,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":487205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.27300000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.41599999999999998,"15min":0.40699999999999997},"min":{"1min":0.29399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.61199999999999999}]},{"osd":1,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.38200000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.433,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.752}]},{"osd":2,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89000000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.91200000000000003,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.50900000000000001,"15min":0.50600000000000001},"min":{"1min":0.29699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.877,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.35399999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.91500000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69499999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58599999999999997,"15min":0.57199999999999995},"min":{"1min":0.435,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.627,"15min":0.61899999999999999},"min":{"1min":0.47999999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.84399999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0549999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.71999999999999997}]},{"osd":7,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73799999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:50:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1080000000000001,"5min":1.147,"15min":1.147},"last":1.0069999999999999}]},{"osd":9,"last update":"Sun Oct 8 13:49:20 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.89600000000000002}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":1.0469999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":491047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.503},"min":{"1min":0.34599999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.51600000000000001},"min":{"1min":0.36399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.88900000000000001,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52400000000000002,"15min":0.50700000000000001},"min":{"1min":0.441,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80300000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65600000000000003}]},{"osd":3,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.873,"5min":3.7959999999999998,"15min":3.7959999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.39000000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94099999999999995,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.60799999999999998}]},{"osd":4,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.316,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.378,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.58599999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94599999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77100000000000002}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92400000000000004,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.75}]},{"osd":7,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.35499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.93500000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89500000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.71899999999999997}]},{"osd":8,"last update":"Sun Oct 8 13:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84299999999999997}]},{"osd":9,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.82799999999999996}]},{"osd":10,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705727,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":489271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 13:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.54500000000000004,"15min":0.53000000000000003},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56299999999999994,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85299999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.625}]},{"osd":3,"last update":"Sun Oct 8 13:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.47099999999999997,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 13:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57999999999999996,"15min":0.56000000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.43099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.46700000000000003}]},{"osd":5,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85999999999999999,"5min":1.012,"15min":1.012},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.60099999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.069,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Sun Oct 8 13:49:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.5}]},{"osd":7,"last update":"Sun Oct 8 13:49:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.50700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.80800000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76600000000000001,"15min":0.77800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1419999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.57499999999999996}]},{"osd":9,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.74199999999999999}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.78600000000000003}]},{"osd":11,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.64700000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378677,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":491081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.55200000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.56799999999999995}]},{"osd":1,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.501},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80600000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78800000000000003}]},{"osd":3,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82099999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.69799999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73699999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.61399999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44700000000000001,"15min":0.42899999999999999},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.38200000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.71899999999999997}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.45700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.44900000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.80500000000000005}]},{"osd":7,"last update":"Sun Oct 8 13:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.93200000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.68500000000000005}]},{"osd":9,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.96599999999999997}]},{"osd":10,"last update":"Sun Oct 8 13:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149746,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":487489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.33600000000000002},"min":{"1min":0.252,"5min":0.22,"15min":0.22},"max":{"1min":0.54900000000000004,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.441,"15min":0.40899999999999997},"min":{"1min":0.316,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77900000000000003,"5min":0.87,"15min":0.87},"last":0.372}]},{"osd":1,"last update":"Sun Oct 8 13:49:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.52100000000000002},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.41499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83299999999999996,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.57199999999999995}]},{"osd":2,"last update":"Sun Oct 8 13:49:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.42199999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.621},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59299999999999997,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97399999999999998,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.44600000000000001}]},{"osd":4,"last update":"Sun Oct 8 13:49:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59999999999999998,"15min":0.56200000000000006},"min":{"1min":0.47299999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91300000000000003,"5min":2.077,"15min":2.077},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59099999999999997,"15min":0.55100000000000005},"min":{"1min":0.49299999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81100000000000005,"5min":2.129,"15min":2.129},"last":0.52300000000000002}]},{"osd":5,"last update":"Sun Oct 8 13:49:59 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.60599999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96999999999999997,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Oct 8 13:49:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63900000000000001,"15min":0.626},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.50600000000000001}]},{"osd":7,"last update":"Sun Oct 8 13:49:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.55300000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99399999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.67200000000000004}]},{"osd":8,"last update":"Sun Oct 8 13:48:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.46800000000000003}]},{"osd":9,"last update":"Sun Oct 8 13:49:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63100000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.68600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920813,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":487227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.51800000000000002,"15min":0.47999999999999998},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.371,"15min":0.35199999999999998},"min":{"1min":0.32400000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.38100000000000001,"15min":0.35699999999999998},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.437}]},{"osd":2,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.504,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65800000000000003}]},{"osd":3,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.67700000000000005}]},{"osd":5,"last update":"Sun Oct 8 13:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.48099999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.621},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.44900000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Oct 8 13:49:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53100000000000003}]},{"osd":7,"last update":"Sun Oct 8 13:49:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.41599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.51400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.73499999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.755},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69299999999999995}]},{"osd":9,"last update":"Sun Oct 8 13:49:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70599999999999996}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757289,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1078718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48599999999999999,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.49299999999999999,"15min":0.46600000000000003},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.871,"15min":0.871},"last":0.71299999999999997}]},{"osd":1,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.55000000000000004,"15min":0.53000000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84999999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61499999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.38600000000000001,"15min":0.36899999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58699999999999997,"5min":0.629,"15min":0.629},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.40300000000000002,"15min":0.38300000000000001},"min":{"1min":0.29599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.45200000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.313,"5min":0.309,"15min":0.309},"max":{"1min":0.71499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.40699999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.77400000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.67500000000000004}]},{"osd":4,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.42999999999999999}]},{"osd":6,"last update":"Sun Oct 8 13:49:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.502,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.58199999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84799999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65700000000000003,"15min":0.65500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77900000000000003}]},{"osd":8,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.57699999999999996}]},{"osd":9,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.749},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.86699999999999999}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.83999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T13:50:10.132 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T13:50:10.132 INFO:tasks.cephadm:Setting up client nodes... 2023-10-08T13:50:10.133 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-08T13:50:10.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:09 smithi115 ceph-mon[104408]: pgmap v253: 129 pgs: 129 active+clean; 583 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.2 KiB/s wr, 4 op/s 2023-10-08T13:50:10.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:09 smithi115 ceph-mon[104408]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:10.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:09 smithi115 ceph-mon[104408]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:50:10.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:09 smithi115 ceph-mon[104408]: mgrmap e21: x(active, since 5m), standbys: z, y 2023-10-08T13:50:10.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:09 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/782714939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:10.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:09 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/171048795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:10.730 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:11.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:10 smithi099 ceph-mon[103849]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:11.118 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:10 smithi057 ceph-mon[99476]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:11.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:10 smithi115 ceph-mon[104408]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:11.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 -- 172.21.15.57:0/2466658621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7750068310 msgr2=0x7f7750068770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:11.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 --2- 172.21.15.57:0/2466658621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7750068310 0x7f7750068770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7744009990 tx=0x7f7744009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:11.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 -- 172.21.15.57:0/2466658621 shutdown_connections 2023-10-08T13:50:11.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 --2- 172.21.15.57:0/2466658621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7750068cb0 0x7f775010d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:11.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 --2- 172.21.15.57:0/2466658621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7750106770 0x7f7750106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:11.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 --2- 172.21.15.57:0/2466658621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7750068310 0x7f7750068770 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:11.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 -- 172.21.15.57:0/2466658621 >> 172.21.15.57:0/2466658621 conn(0x7f7750075710 msgr2=0x7f7750075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:11.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 -- 172.21.15.57:0/2466658621 shutdown_connections 2023-10-08T13:50:11.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 -- 172.21.15.57:0/2466658621 wait complete. 2023-10-08T13:50:11.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 Processor -- start 2023-10-08T13:50:11.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.818+0000 7f77556d9700 1 -- start start 2023-10-08T13:50:11.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f77556d9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7750068310 0x7f775019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:11.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f77556d9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7750068cb0 0x7f775019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f77556d9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7750106770 0x7f77501969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f77556d9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750197040 con 0x7f7750068cb0 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f77556d9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77501971b0 con 0x7f7750068310 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f77556d9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750197370 con 0x7f7750106770 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7750068cb0 0x7f775019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7750068cb0 0x7f775019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54574/0 (socket says 172.21.15.57:54574) 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774e7fc700 1 -- 172.21.15.57:0/3767121858 learned_addr learned my addr 172.21.15.57:0/3767121858 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774f7fe700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7750106770 0x7f77501969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774effd700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7750068310 0x7f775019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774e7fc700 1 -- 172.21.15.57:0/3767121858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7750106770 msgr2=0x7f77501969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:11.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774e7fc700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7750106770 0x7f77501969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:11.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774e7fc700 1 -- 172.21.15.57:0/3767121858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7750068310 msgr2=0x7f775019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:11.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774e7fc700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7750068310 0x7f775019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:11.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774e7fc700 1 -- 172.21.15.57:0/3767121858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77440095f0 con 0x7f7750068cb0 2023-10-08T13:50:11.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774effd700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7750068310 0x7f775019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:11.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774f7fe700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7750106770 0x7f77501969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:11.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.819+0000 7f774e7fc700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7750068cb0 0x7f775019cde0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7744000c00 tx=0x7f774400f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:11.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.820+0000 7f7737fff700 1 -- 172.21.15.57:0/3767121858 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774401d070 con 0x7f7750068cb0 2023-10-08T13:50:11.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.820+0000 7f77556d9700 1 -- 172.21.15.57:0/3767121858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77501975f0 con 0x7f7750068cb0 2023-10-08T13:50:11.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.820+0000 7f7737fff700 1 -- 172.21.15.57:0/3767121858 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f774400fde0 con 0x7f7750068cb0 2023-10-08T13:50:11.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.820+0000 7f7737fff700 1 -- 172.21.15.57:0/3767121858 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744017770 con 0x7f7750068cb0 2023-10-08T13:50:11.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.820+0000 7f77556d9700 1 -- 172.21.15.57:0/3767121858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77501a3890 con 0x7f7750068cb0 2023-10-08T13:50:11.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.821+0000 7f77556d9700 1 -- 172.21.15.57:0/3767121858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7750108d20 con 0x7f7750068cb0 2023-10-08T13:50:11.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.822+0000 7f7737fff700 1 -- 172.21.15.57:0/3767121858 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f7744022aa0 con 0x7f7750068cb0 2023-10-08T13:50:11.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.822+0000 7f7737fff700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77380a1560 0x7f77380a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:11.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.822+0000 7f774effd700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77380a1560 0x7f77380a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:11.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.822+0000 7f7737fff700 1 -- 172.21.15.57:0/3767121858 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7f77440ce610 con 0x7f7750068cb0 2023-10-08T13:50:11.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.823+0000 7f774effd700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77380a1560 0x7f77380a3a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7740005fd0 tx=0x7f7740005d40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:11.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:11.827+0000 7f7737fff700 1 -- 172.21.15.57:0/3767121858 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f774405d100 con 0x7f7750068cb0 2023-10-08T13:50:11.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:11 smithi057 ceph-mon[99476]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:11.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:11 smithi057 ceph-mon[99476]: pgmap v254: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2023-10-08T13:50:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:11 smithi099 ceph-mon[103849]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:12.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:11 smithi099 ceph-mon[103849]: pgmap v254: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2023-10-08T13:50:12.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.112+0000 7f77556d9700 1 -- 172.21.15.57:0/3767121858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f7750197ea0 con 0x7f7750068cb0 2023-10-08T13:50:12.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.115+0000 7f7737fff700 1 -- 172.21.15.57:0/3767121858 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f77440978d0 con 0x7f7750068cb0 2023-10-08T13:50:12.115 INFO:teuthology.orchestra.run.smithi057.stdout:[client.0] 2023-10-08T13:50:12.115 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCUsyJlx8a3BhAAkqYaDHLFsk38/B2NZmR7Dw== 2023-10-08T13:50:12.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 -- 172.21.15.57:0/3767121858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77380a1560 msgr2=0x7f77380a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:12.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77380a1560 0x7f77380a3a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7740005fd0 tx=0x7f7740005d40 comp rx=0 tx=0).stop 2023-10-08T13:50:12.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 -- 172.21.15.57:0/3767121858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7750068cb0 msgr2=0x7f775019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:12.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7750068cb0 0x7f775019cde0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7744000c00 tx=0x7f774400f880 comp rx=0 tx=0).stop 2023-10-08T13:50:12.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 -- 172.21.15.57:0/3767121858 shutdown_connections 2023-10-08T13:50:12.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7750106770 0x7f77501969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:12.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77380a1560 0x7f77380a3a20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:12.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7750068310 0x7f775019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:12.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 --2- 172.21.15.57:0/3767121858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7750068cb0 0x7f775019cde0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:12.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 -- 172.21.15.57:0/3767121858 >> 172.21.15.57:0/3767121858 conn(0x7f7750075710 msgr2=0x7f77500fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:12.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 -- 172.21.15.57:0/3767121858 shutdown_connections 2023-10-08T13:50:12.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:12.119+0000 7f77556d9700 1 -- 172.21.15.57:0/3767121858 wait complete. 2023-10-08T13:50:12.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:11 smithi115 ceph-mon[104408]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:11 smithi115 ceph-mon[104408]: pgmap v254: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2023-10-08T13:50:12.397 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:50:12.397 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-08T13:50:12.397 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-08T13:50:12.440 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph config log 1 --format=json 2023-10-08T13:50:13.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:12 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3767121858' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-08T13:50:13.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:12 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3767121858' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-08T13:50:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:13.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:12 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3767121858' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-08T13:50:13.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:12 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3767121858' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-08T13:50:13.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:12 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3767121858' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-08T13:50:13.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:12 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3767121858' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-08T13:50:13.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:13 smithi057 ceph-mon[99476]: pgmap v255: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.5 KiB/s wr, 17 op/s 2023-10-08T13:50:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:13 smithi099 ceph-mon[103849]: pgmap v255: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.5 KiB/s wr, 17 op/s 2023-10-08T13:50:14.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:13 smithi115 ceph-mon[104408]: pgmap v255: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 4.5 KiB/s wr, 17 op/s 2023-10-08T13:50:14.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.218+0000 7ff6599e7700 1 -- 172.21.15.57:0/3887153207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff654101260 msgr2=0x7ff65410d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:14.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.218+0000 7ff6599e7700 1 --2- 172.21.15.57:0/3887153207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff654101260 0x7ff65410d690 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00b0a0 tx=0x7ff64c00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.218+0000 7ff6599e7700 1 -- 172.21.15.57:0/3887153207 shutdown_connections 2023-10-08T13:50:14.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.218+0000 7ff6599e7700 1 --2- 172.21.15.57:0/3887153207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff654101260 0x7ff65410d690 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.218+0000 7ff6599e7700 1 --2- 172.21.15.57:0/3887153207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff654106830 0x7ff654106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.218+0000 7ff6599e7700 1 --2- 172.21.15.57:0/3887153207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6541008a0 0x7ff654100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.218+0000 7ff6599e7700 1 -- 172.21.15.57:0/3887153207 >> 172.21.15.57:0/3887153207 conn(0x7ff6540779e0 msgr2=0x7ff654077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:14.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.218+0000 7ff6599e7700 1 -- 172.21.15.57:0/3887153207 shutdown_connections 2023-10-08T13:50:14.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.218+0000 7ff6599e7700 1 -- 172.21.15.57:0/3887153207 wait complete. 2023-10-08T13:50:14.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.219+0000 7ff6599e7700 1 Processor -- start 2023-10-08T13:50:14.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.219+0000 7ff6599e7700 1 -- start start 2023-10-08T13:50:14.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.219+0000 7ff6599e7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6541008a0 0x7ff654072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:14.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.219+0000 7ff6599e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff654101260 0x7ff654073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:14.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.219+0000 7ff6599e7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff654106830 0x7ff654073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:14.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.219+0000 7ff6599e7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541a5090 con 0x7ff6541008a0 2023-10-08T13:50:14.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.219+0000 7ff6599e7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541a5200 con 0x7ff654101260 2023-10-08T13:50:14.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.219+0000 7ff6599e7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541a53c0 con 0x7ff654106830 2023-10-08T13:50:14.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff652ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6541008a0 0x7ff654072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:14.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff652ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6541008a0 0x7ff654072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54586/0 (socket says 172.21.15.57:54586) 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff652ffd700 1 -- 172.21.15.57:0/278905991 learned_addr learned my addr 172.21.15.57:0/278905991 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff6527fc700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff654101260 0x7ff654073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff6537fe700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff654106830 0x7ff654073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff652ffd700 1 -- 172.21.15.57:0/278905991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff654106830 msgr2=0x7ff654073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff652ffd700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff654106830 0x7ff654073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff652ffd700 1 -- 172.21.15.57:0/278905991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff654101260 msgr2=0x7ff654073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff652ffd700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff654101260 0x7ff654073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff652ffd700 1 -- 172.21.15.57:0/278905991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c009d00 con 0x7ff6541008a0 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff652ffd700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6541008a0 0x7ff654072af0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff64800e9a0 tx=0x7ff64800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff6589e5700 1 -- 172.21.15.57:0/278905991 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648009790 con 0x7ff6541008a0 2023-10-08T13:50:14.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff6599e7700 1 -- 172.21.15.57:0/278905991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6541a56a0 con 0x7ff6541008a0 2023-10-08T13:50:14.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff6589e5700 1 -- 172.21.15.57:0/278905991 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff648010430 con 0x7ff6541008a0 2023-10-08T13:50:14.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff6589e5700 1 -- 172.21.15.57:0/278905991 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648018680 con 0x7ff6541008a0 2023-10-08T13:50:14.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.220+0000 7ff6599e7700 1 -- 172.21.15.57:0/278905991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6541a5ba0 con 0x7ff6541008a0 2023-10-08T13:50:14.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.221+0000 7ff6599e7700 1 -- 172.21.15.57:0/278905991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff654108d70 con 0x7ff6541008a0 2023-10-08T13:50:14.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.223+0000 7ff6589e5700 1 -- 172.21.15.57:0/278905991 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7ff6480105a0 con 0x7ff6541008a0 2023-10-08T13:50:14.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.223+0000 7ff6589e5700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff63c0a1560 0x7ff63c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:14.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.223+0000 7ff6527fc700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff63c0a1560 0x7ff63c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:14.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.223+0000 7ff6589e5700 1 -- 172.21.15.57:0/278905991 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8945+0+0 (secure 0 0 0) 0x7ff648014070 con 0x7ff6541008a0 2023-10-08T13:50:14.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.224+0000 7ff6527fc700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff63c0a1560 0x7ff63c0a3a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff654074220 tx=0x7ff644009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:14.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.227+0000 7ff6589e5700 1 -- 172.21.15.57:0/278905991 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff648096710 con 0x7ff6541008a0 2023-10-08T13:50:14.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.426+0000 7ff6599e7700 1 -- 172.21.15.57:0/278905991 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7ff654101f60 con 0x7ff6541008a0 2023-10-08T13:50:14.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.427+0000 7ff6589e5700 1 -- 172.21.15.57:0/278905991 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v46) v1 ==== 86+0+133 (secure 0 0 0) 0x7ff648099d30 con 0x7ff6541008a0 2023-10-08T13:50:14.427 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:14.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 -- 172.21.15.57:0/278905991 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff63c0a1560 msgr2=0x7ff63c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:14.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff63c0a1560 0x7ff63c0a3a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff654074220 tx=0x7ff644009260 comp rx=0 tx=0).stop 2023-10-08T13:50:14.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 -- 172.21.15.57:0/278905991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6541008a0 msgr2=0x7ff654072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:14.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6541008a0 0x7ff654072af0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff64800e9a0 tx=0x7ff64800ecb0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 -- 172.21.15.57:0/278905991 shutdown_connections 2023-10-08T13:50:14.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff654106830 0x7ff654073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6541008a0 0x7ff654072af0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff63c0a1560 0x7ff63c0a3a20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 --2- 172.21.15.57:0/278905991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff654101260 0x7ff654073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:14.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 -- 172.21.15.57:0/278905991 >> 172.21.15.57:0/278905991 conn(0x7ff6540779e0 msgr2=0x7ff6540ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:14.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 -- 172.21.15.57:0/278905991 shutdown_connections 2023-10-08T13:50:14.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:14.429+0000 7ff6599e7700 1 -- 172.21.15.57:0/278905991 wait complete. 2023-10-08T13:50:14.809 INFO:teuthology.orchestra.run.smithi057.stdout:[{"version":46,"timestamp":"2023-10-08T13:49:39.371090+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2023-10-08T13:50:14.809 INFO:tasks.ceph_manager:config epoch is 46 2023-10-08T13:50:14.809 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-08T13:50:14.809 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-08T13:50:14.810 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-08T13:50:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-08T13:50:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-10-08T13:50:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-10-08T13:50:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-10-08T13:50:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2023-10-08T13:50:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-10-08T13:50:14.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-10-08T13:50:14.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2023-10-08T13:50:14.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-10-08T13:50:14.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2023-10-08T13:50:14.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/278905991' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-08T13:50:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-08T13:50:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-10-08T13:50:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-10-08T13:50:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-10-08T13:50:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2023-10-08T13:50:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-10-08T13:50:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-10-08T13:50:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2023-10-08T13:50:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-10-08T13:50:15.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2023-10-08T13:50:15.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:14 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/278905991' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-08T13:50:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-08T13:50:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-10-08T13:50:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-10-08T13:50:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-10-08T13:50:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2023-10-08T13:50:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-10-08T13:50:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-10-08T13:50:15.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2023-10-08T13:50:15.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-10-08T13:50:15.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2023-10-08T13:50:15.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:14 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/278905991' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-08T13:50:15.373 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:15.860 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: pgmap v256: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.0 KiB/s wr, 21 op/s 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2023-10-08T13:50:15.861 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:15 smithi057 ceph-mon[99476]: osdmap e89: 12 total, 12 up, 12 in 2023-10-08T13:50:16.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: pgmap v256: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.0 KiB/s wr, 21 op/s 2023-10-08T13:50:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-08T13:50:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-10-08T13:50:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-10-08T13:50:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-10-08T13:50:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2023-10-08T13:50:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-10-08T13:50:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-10-08T13:50:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2023-10-08T13:50:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-10-08T13:50:16.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2023-10-08T13:50:16.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:15 smithi115 ceph-mon[104408]: osdmap e89: 12 total, 12 up, 12 in 2023-10-08T13:50:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: pgmap v256: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.0 KiB/s wr, 21 op/s 2023-10-08T13:50:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-08T13:50:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-10-08T13:50:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-10-08T13:50:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-10-08T13:50:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2023-10-08T13:50:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-10-08T13:50:16.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-10-08T13:50:16.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2023-10-08T13:50:16.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-10-08T13:50:16.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2023-10-08T13:50:16.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:15 smithi099 ceph-mon[103849]: osdmap e89: 12 total, 12 up, 12 in 2023-10-08T13:50:16.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.547+0000 7f718d1d1700 1 -- 172.21.15.57:0/458967246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f71881008a0 msgr2=0x7f7188100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:16.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.547+0000 7f718d1d1700 1 --2- 172.21.15.57:0/458967246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f71881008a0 0x7f7188100d20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7178009990 tx=0x7f7178009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:16.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.547+0000 7f718d1d1700 1 -- 172.21.15.57:0/458967246 shutdown_connections 2023-10-08T13:50:16.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.547+0000 7f718d1d1700 1 --2- 172.21.15.57:0/458967246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7188101260 0x7f718810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:16.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.547+0000 7f718d1d1700 1 --2- 172.21.15.57:0/458967246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71881057c0 0x7f7188105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:16.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.547+0000 7f718d1d1700 1 --2- 172.21.15.57:0/458967246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f71881008a0 0x7f7188100d20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:16.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.547+0000 7f718d1d1700 1 -- 172.21.15.57:0/458967246 >> 172.21.15.57:0/458967246 conn(0x7f71880779e0 msgr2=0x7f7188077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:16.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.547+0000 7f718d1d1700 1 -- 172.21.15.57:0/458967246 shutdown_connections 2023-10-08T13:50:16.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.547+0000 7f718d1d1700 1 -- 172.21.15.57:0/458967246 wait complete. 2023-10-08T13:50:16.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.548+0000 7f718d1d1700 1 Processor -- start 2023-10-08T13:50:16.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.548+0000 7f718d1d1700 1 -- start start 2023-10-08T13:50:16.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.548+0000 7f718d1d1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71881008a0 0x7f718819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:16.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.548+0000 7f718d1d1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7188101260 0x7f718819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.548+0000 7f718d1d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71881057c0 0x7f7188196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.548+0000 7f718d1d1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7188197240 con 0x7f7188101260 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.548+0000 7f718d1d1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71881973b0 con 0x7f71881057c0 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.548+0000 7f718d1d1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7188197570 con 0x7f71881008a0 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7188101260 0x7f718819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7188101260 0x7f718819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54600/0 (socket says 172.21.15.57:54600) 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718659c700 1 -- 172.21.15.57:0/600407297 learned_addr learned my addr 172.21.15.57:0/600407297 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f7186d9d700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71881008a0 0x7f718819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718759e700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71881057c0 0x7f7188196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718659c700 1 -- 172.21.15.57:0/600407297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71881008a0 msgr2=0x7f718819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:16.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718659c700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71881008a0 0x7f718819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:16.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718659c700 1 -- 172.21.15.57:0/600407297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71881057c0 msgr2=0x7f7188196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:16.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718659c700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71881057c0 0x7f7188196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:16.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718659c700 1 -- 172.21.15.57:0/600407297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71780095f0 con 0x7f7188101260 2023-10-08T13:50:16.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718659c700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7188101260 0x7f718819cf20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7178009fd0 tx=0x7f717800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:16.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f716ffff700 1 -- 172.21.15.57:0/600407297 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717801d070 con 0x7f7188101260 2023-10-08T13:50:16.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f716ffff700 1 -- 172.21.15.57:0/600407297 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7178005510 con 0x7f7188101260 2023-10-08T13:50:16.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f716ffff700 1 -- 172.21.15.57:0/600407297 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178004a90 con 0x7f7188101260 2023-10-08T13:50:16.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718d1d1700 1 -- 172.21.15.57:0/600407297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71881977f0 con 0x7f7188101260 2023-10-08T13:50:16.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.549+0000 7f718d1d1700 1 -- 172.21.15.57:0/600407297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71881a3960 con 0x7f7188101260 2023-10-08T13:50:16.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.550+0000 7f718d1d1700 1 -- 172.21.15.57:0/600407297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7188108da0 con 0x7f7188101260 2023-10-08T13:50:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.552+0000 7f716ffff700 1 -- 172.21.15.57:0/600407297 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f7178004c50 con 0x7f7188101260 2023-10-08T13:50:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.553+0000 7f716ffff700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71700a15e0 0x7f71700a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.553+0000 7f7186d9d700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71700a15e0 0x7f71700a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:16.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.553+0000 7f716ffff700 1 -- 172.21.15.57:0/600407297 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f71780ce3d0 con 0x7f7188101260 2023-10-08T13:50:16.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.553+0000 7f7186d9d700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71700a15e0 0x7f71700a3aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7188101930 tx=0x7f717c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:16.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.556+0000 7f716ffff700 1 -- 172.21.15.57:0/600407297 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7178097570 con 0x7f7188101260 2023-10-08T13:50:16.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.819+0000 7f718d1d1700 1 -- 172.21.15.57:0/600407297 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7188197f70 con 0x7f7188101260 2023-10-08T13:50:16.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.824+0000 7f716ffff700 1 -- 172.21.15.57:0/600407297 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) v1 ==== 74+0+260424 (secure 0 0 0) 0x7f717809ab90 con 0x7f7188101260 2023-10-08T13:50:16.828 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:16.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 -- 172.21.15.57:0/600407297 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71700a15e0 msgr2=0x7f71700a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:16.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71700a15e0 0x7f71700a3aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7188101930 tx=0x7f717c00b410 comp rx=0 tx=0).stop 2023-10-08T13:50:16.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 -- 172.21.15.57:0/600407297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7188101260 msgr2=0x7f718819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:16.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7188101260 0x7f718819cf20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7178009fd0 tx=0x7f717800f670 comp rx=0 tx=0).stop 2023-10-08T13:50:16.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 -- 172.21.15.57:0/600407297 shutdown_connections 2023-10-08T13:50:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f71700a15e0 0x7f71700a3aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71881057c0 0x7f7188196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71881008a0 0x7f718819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 --2- 172.21.15.57:0/600407297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7188101260 0x7f718819cf20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 -- 172.21.15.57:0/600407297 >> 172.21.15.57:0/600407297 conn(0x7f71880779e0 msgr2=0x7f71880fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 -- 172.21.15.57:0/600407297 shutdown_connections 2023-10-08T13:50:16.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:16.827+0000 7f718d1d1700 1 -- 172.21.15.57:0/600407297 wait complete. 2023-10-08T13:50:17.127 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:16 smithi057 ceph-mon[99476]: osdmap e90: 12 total, 12 up, 12 in 2023-10-08T13:50:17.129 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":21,"active_gid":14217,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":2547096577},{"type":"v1","addr":"172.21.15.57:6801","nonce":2547096577}]},"active_addr":"172.21.15.57:6801/2547096577","active_change":"2023-10-08T13:44:14.025199+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24148,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24164,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.57:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":1007973081}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":1204879539}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2777153216}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":3719711945}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":3634059504}]}]} 2023-10-08T13:50:17.140 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-08T13:50:17.140 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-08T13:50:17.140 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-08T13:50:17.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:16 smithi115 ceph-mon[104408]: osdmap e90: 12 total, 12 up, 12 in 2023-10-08T13:50:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:16 smithi099 ceph-mon[103849]: osdmap e90: 12 total, 12 up, 12 in 2023-10-08T13:50:17.707 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:18.092 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:17 smithi057 ceph-mon[99476]: pgmap v259: 129 pgs: 4 peering, 125 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 26 op/s 2023-10-08T13:50:18.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:17 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/600407297' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-08T13:50:18.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:17 smithi115 ceph-mon[104408]: pgmap v259: 129 pgs: 4 peering, 125 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 26 op/s 2023-10-08T13:50:18.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:17 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/600407297' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-08T13:50:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:17 smithi099 ceph-mon[103849]: pgmap v259: 129 pgs: 4 peering, 125 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 26 op/s 2023-10-08T13:50:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:17 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/600407297' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-08T13:50:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.876+0000 7f875311d700 1 -- 172.21.15.57:0/3994425214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f874c069410 msgr2=0x7f874c10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.876+0000 7f875311d700 1 --2- 172.21.15.57:0/3994425214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f874c069410 0x7f874c10d650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f874400b0d0 tx=0x7f874400b3e0 comp rx=0 tx=0).stop 2023-10-08T13:50:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.876+0000 7f875311d700 1 -- 172.21.15.57:0/3994425214 shutdown_connections 2023-10-08T13:50:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.876+0000 7f875311d700 1 --2- 172.21.15.57:0/3994425214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f874c069410 0x7f874c10d650 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.876+0000 7f875311d700 1 --2- 172.21.15.57:0/3994425214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f874c105670 0x7f874c105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.876+0000 7f875311d700 1 --2- 172.21.15.57:0/3994425214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f874c068a50 0x7f874c068ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.876+0000 7f875311d700 1 -- 172.21.15.57:0/3994425214 >> 172.21.15.57:0/3994425214 conn(0x7f874c0766c0 msgr2=0x7f874c076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:18.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.876+0000 7f875311d700 1 -- 172.21.15.57:0/3994425214 shutdown_connections 2023-10-08T13:50:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.876+0000 7f875311d700 1 -- 172.21.15.57:0/3994425214 wait complete. 2023-10-08T13:50:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.876+0000 7f875311d700 1 Processor -- start 2023-10-08T13:50:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f875311d700 1 -- start start 2023-10-08T13:50:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f875311d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f874c068a50 0x7f874c0ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f875311d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f874c069410 0x7f874c0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f875311d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f874c105670 0x7f874c100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f875311d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a5060 con 0x7f874c068a50 2023-10-08T13:50:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f875311d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a51d0 con 0x7f874c105670 2023-10-08T13:50:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f875311d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a5390 con 0x7f874c069410 2023-10-08T13:50:18.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f8750eb9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f874c068a50 0x7f874c0ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:18.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f8750eb9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f874c068a50 0x7f874c0ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54612/0 (socket says 172.21.15.57:54612) 2023-10-08T13:50:18.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f8750eb9700 1 -- 172.21.15.57:0/1533284284 learned_addr learned my addr 172.21.15.57:0/1533284284 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:18.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f874bfff700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f874c069410 0x7f874c0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:18.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f87516ba700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f874c105670 0x7f874c100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:18.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f8750eb9700 1 -- 172.21.15.57:0/1533284284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f874c069410 msgr2=0x7f874c0ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:18.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f8750eb9700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f874c069410 0x7f874c0ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:18.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f8750eb9700 1 -- 172.21.15.57:0/1533284284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f874c105670 msgr2=0x7f874c100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:18.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f8750eb9700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f874c105670 0x7f874c100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f8750eb9700 1 -- 172.21.15.57:0/1533284284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8744009d00 con 0x7f874c068a50 2023-10-08T13:50:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f87516ba700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f874c105670 0x7f874c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.877+0000 7f8750eb9700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f874c068a50 0x7f874c0ff7b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f874000d950 tx=0x7f874000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.878+0000 7f8749ffb700 1 -- 172.21.15.57:0/1533284284 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740009940 con 0x7f874c068a50 2023-10-08T13:50:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.878+0000 7f8749ffb700 1 -- 172.21.15.57:0/1533284284 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8740010460 con 0x7f874c068a50 2023-10-08T13:50:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.878+0000 7f875311d700 1 -- 172.21.15.57:0/1533284284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c1a5670 con 0x7f874c068a50 2023-10-08T13:50:18.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.878+0000 7f8749ffb700 1 -- 172.21.15.57:0/1533284284 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874000f620 con 0x7f874c068a50 2023-10-08T13:50:18.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.878+0000 7f875311d700 1 -- 172.21.15.57:0/1533284284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c1a5b70 con 0x7f874c068a50 2023-10-08T13:50:18.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.879+0000 7f875311d700 1 -- 172.21.15.57:0/1533284284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c108d30 con 0x7f874c068a50 2023-10-08T13:50:18.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.881+0000 7f8749ffb700 1 -- 172.21.15.57:0/1533284284 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f87400108d0 con 0x7f874c068a50 2023-10-08T13:50:18.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.881+0000 7f8749ffb700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f873c0a1560 0x7f873c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:18.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.881+0000 7f8749ffb700 1 -- 172.21.15.57:0/1533284284 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f87400cd0d0 con 0x7f874c068a50 2023-10-08T13:50:18.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.881+0000 7f874bfff700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f873c0a1560 0x7f873c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:18.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.882+0000 7f874bfff700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f873c0a1560 0x7f873c0a3a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f874c100ee0 tx=0x7f873800a300 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:18.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:18.885+0000 7f8749ffb700 1 -- 172.21.15.57:0/1533284284 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f87400961e0 con 0x7f874c068a50 2023-10-08T13:50:19.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.084+0000 7f875311d700 1 -- 172.21.15.57:0/1533284284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f874c100c90 con 0x7f874c068a50 2023-10-08T13:50:19.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.086+0000 7f8749ffb700 1 -- 172.21.15.57:0/1533284284 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24753 (secure 0 0 0) 0x7f8740016090 con 0x7f874c068a50 2023-10-08T13:50:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:19.087 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:50:15.770421+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:50:19.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 -- 172.21.15.57:0/1533284284 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f873c0a1560 msgr2=0x7f873c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:19.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f873c0a1560 0x7f873c0a3a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f874c100ee0 tx=0x7f873800a300 comp rx=0 tx=0).stop 2023-10-08T13:50:19.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 -- 172.21.15.57:0/1533284284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f874c068a50 msgr2=0x7f874c0ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:19.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f874c068a50 0x7f874c0ff7b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f874000d950 tx=0x7f874000dc60 comp rx=0 tx=0).stop 2023-10-08T13:50:19.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 -- 172.21.15.57:0/1533284284 shutdown_connections 2023-10-08T13:50:19.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f873c0a1560 0x7f873c0a3a20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:19.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f874c105670 0x7f874c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:19.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f874c068a50 0x7f874c0ff7b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:19.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 --2- 172.21.15.57:0/1533284284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f874c069410 0x7f874c0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:19.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 -- 172.21.15.57:0/1533284284 >> 172.21.15.57:0/1533284284 conn(0x7f874c0766c0 msgr2=0x7f874c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:19.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 -- 172.21.15.57:0/1533284284 shutdown_connections 2023-10-08T13:50:19.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:19.088+0000 7f875311d700 1 -- 172.21.15.57:0/1533284284 wait complete. 2023-10-08T13:50:19.363 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-08T13:50:19.364 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-08T13:50:19.929 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:20.115 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:19 smithi057 ceph-mon[99476]: pgmap v260: 129 pgs: 4 peering, 125 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2023-10-08T13:50:20.116 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:19 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1533284284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:20.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:19 smithi115 ceph-mon[104408]: pgmap v260: 129 pgs: 4 peering, 125 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2023-10-08T13:50:20.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:19 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1533284284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:19 smithi099 ceph-mon[103849]: pgmap v260: 129 pgs: 4 peering, 125 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2023-10-08T13:50:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:19 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1533284284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:21.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.080+0000 7fcc20094700 1 -- 172.21.15.57:0/4009413132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc18100110 msgr2=0x7fcc1810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:21.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.080+0000 7fcc20094700 1 --2- 172.21.15.57:0/4009413132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc18100110 0x7fcc1810c580 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000b0a0 tx=0x7fcc1000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:21.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.080+0000 7fcc20094700 1 -- 172.21.15.57:0/4009413132 shutdown_connections 2023-10-08T13:50:21.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.080+0000 7fcc20094700 1 --2- 172.21.15.57:0/4009413132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc18100110 0x7fcc1810c580 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:21.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.080+0000 7fcc20094700 1 --2- 172.21.15.57:0/4009413132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc181056b0 0x7fcc18105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.080+0000 7fcc20094700 1 --2- 172.21.15.57:0/4009413132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc180ff750 0x7fcc180ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.080+0000 7fcc20094700 1 -- 172.21.15.57:0/4009413132 >> 172.21.15.57:0/4009413132 conn(0x7fcc18075790 msgr2=0x7fcc18075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.080+0000 7fcc20094700 1 -- 172.21.15.57:0/4009413132 shutdown_connections 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.080+0000 7fcc20094700 1 -- 172.21.15.57:0/4009413132 wait complete. 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc20094700 1 Processor -- start 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc20094700 1 -- start start 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc20094700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc180ff750 0x7fcc18197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc20094700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc18100110 0x7fcc18198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc20094700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181056b0 0x7fcc18198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc20094700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc181a2eb0 con 0x7fcc18100110 2023-10-08T13:50:21.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc20094700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc181a3020 con 0x7fcc180ff750 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc20094700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc181a31e0 con 0x7fcc181056b0 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc1d62f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc18100110 0x7fcc18198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc1d62f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc18100110 0x7fcc18198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54626/0 (socket says 172.21.15.57:54626) 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.081+0000 7fcc1d62f700 1 -- 172.21.15.57:0/752893646 learned_addr learned my addr 172.21.15.57:0/752893646 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc1e631700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181056b0 0x7fcc18198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc1de30700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc180ff750 0x7fcc18197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc1d62f700 1 -- 172.21.15.57:0/752893646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181056b0 msgr2=0x7fcc18198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc1d62f700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181056b0 0x7fcc18198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc1d62f700 1 -- 172.21.15.57:0/752893646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc180ff750 msgr2=0x7fcc18197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc1d62f700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc180ff750 0x7fcc18197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:21.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc1d62f700 1 -- 172.21.15.57:0/752893646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10009d00 con 0x7fcc18100110 2023-10-08T13:50:21.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc1e631700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181056b0 0x7fcc18198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:21.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc1de30700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc180ff750 0x7fcc18197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:21.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc1d62f700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc18100110 0x7fcc18198100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00d980 tx=0x7fcc0c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:21.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc0affd700 1 -- 172.21.15.57:0/752893646 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c009b40 con 0x7fcc18100110 2023-10-08T13:50:21.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc0affd700 1 -- 172.21.15.57:0/752893646 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc0c00de60 con 0x7fcc18100110 2023-10-08T13:50:21.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc0affd700 1 -- 172.21.15.57:0/752893646 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c005580 con 0x7fcc18100110 2023-10-08T13:50:21.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc20094700 1 -- 172.21.15.57:0/752893646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc181a34c0 con 0x7fcc18100110 2023-10-08T13:50:21.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.082+0000 7fcc20094700 1 -- 172.21.15.57:0/752893646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc181a3970 con 0x7fcc18100110 2023-10-08T13:50:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.083+0000 7fcc20094700 1 -- 172.21.15.57:0/752893646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc18107c40 con 0x7fcc18100110 2023-10-08T13:50:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.084+0000 7fcc0affd700 1 -- 172.21.15.57:0/752893646 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fcc0c004750 con 0x7fcc18100110 2023-10-08T13:50:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.085+0000 7fcc0affd700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc040a1560 0x7fcc040a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.085+0000 7fcc1de30700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc040a1560 0x7fcc040a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.085+0000 7fcc0affd700 1 -- 172.21.15.57:0/752893646 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fcc0c0cd2f0 con 0x7fcc18100110 2023-10-08T13:50:21.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.085+0000 7fcc1de30700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc040a1560 0x7fcc040a3a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcc14005e10 tx=0x7fcc14005d60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:21.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.089+0000 7fcc0affd700 1 -- 172.21.15.57:0/752893646 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcc0c096400 con 0x7fcc18100110 2023-10-08T13:50:21.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.294+0000 7fcc20094700 1 -- 172.21.15.57:0/752893646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc18100df0 con 0x7fcc18100110 2023-10-08T13:50:21.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.295+0000 7fcc0affd700 1 -- 172.21.15.57:0/752893646 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24753 (secure 0 0 0) 0x7fcc0c099a20 con 0x7fcc18100110 2023-10-08T13:50:21.297 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:21.297 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:50:15.770421+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:50:21.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 -- 172.21.15.57:0/752893646 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc040a1560 msgr2=0x7fcc040a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:21.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc040a1560 0x7fcc040a3a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcc14005e10 tx=0x7fcc14005d60 comp rx=0 tx=0).stop 2023-10-08T13:50:21.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 -- 172.21.15.57:0/752893646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc18100110 msgr2=0x7fcc18198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:21.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc18100110 0x7fcc18198100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00d980 tx=0x7fcc0c00dd40 comp rx=0 tx=0).stop 2023-10-08T13:50:21.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 -- 172.21.15.57:0/752893646 shutdown_connections 2023-10-08T13:50:21.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181056b0 0x7fcc18198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:21.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc040a1560 0x7fcc040a3a20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:21.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc180ff750 0x7fcc18197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:21.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 --2- 172.21.15.57:0/752893646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc18100110 0x7fcc18198100 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:21.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 -- 172.21.15.57:0/752893646 >> 172.21.15.57:0/752893646 conn(0x7fcc18075790 msgr2=0x7fcc180fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:21.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 -- 172.21.15.57:0/752893646 shutdown_connections 2023-10-08T13:50:21.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:21.298+0000 7fcc20094700 1 -- 172.21.15.57:0/752893646 wait complete. 2023-10-08T13:50:21.596 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-08T13:50:21.596 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-08T13:50:21.596 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-08T13:50:21.597 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-08T13:50:21.597 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-08T13:50:21.598 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-08T13:50:21.598 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-08T13:50:21.598 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-08T13:50:21.599 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-08T13:50:21.599 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-08T13:50:21.600 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-08T13:50:21.600 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-08T13:50:22.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:21 smithi115 ceph-mon[104408]: pgmap v261: 129 pgs: 1 peering, 128 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 14 op/s; 153 B/s, 0 objects/s recovering 2023-10-08T13:50:22.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:21 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/752893646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:22.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:21 smithi057 ceph-mon[99476]: pgmap v261: 129 pgs: 1 peering, 128 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 14 op/s; 153 B/s, 0 objects/s recovering 2023-10-08T13:50:22.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:21 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/752893646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:22.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:21 smithi099 ceph-mon[103849]: pgmap v261: 129 pgs: 1 peering, 128 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 14 op/s; 153 B/s, 0 objects/s recovering 2023-10-08T13:50:22.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:21 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/752893646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:22.443 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:22.455 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:22.498 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:22.518 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:22.594 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:22.599 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:22.600 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:22.734 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:23.376 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:24.100 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:23 smithi057 ceph-mon[99476]: pgmap v262: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s; 153 B/s, 1 objects/s recovering 2023-10-08T13:50:24.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:23 smithi115 ceph-mon[104408]: pgmap v262: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s; 153 B/s, 1 objects/s recovering 2023-10-08T13:50:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:23 smithi099 ceph-mon[103849]: pgmap v262: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s; 153 B/s, 1 objects/s recovering 2023-10-08T13:50:25.079 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:25.886 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:25 smithi057 ceph-mon[99476]: pgmap v263: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s; 131 B/s, 0 objects/s recovering 2023-10-08T13:50:26.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:25 smithi115 ceph-mon[104408]: pgmap v263: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s; 131 B/s, 0 objects/s recovering 2023-10-08T13:50:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:25 smithi099 ceph-mon[103849]: pgmap v263: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s; 131 B/s, 0 objects/s recovering 2023-10-08T13:50:26.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.818+0000 7fa063482700 1 -- 172.21.15.57:0/4216662478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c068190 msgr2=0x7fa05c0685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:26.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.818+0000 7fa063482700 1 --2- 172.21.15.57:0/4216662478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c068190 0x7fa05c0685f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009990 tx=0x7fa04c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:26.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.818+0000 7fa063482700 1 -- 172.21.15.57:0/4216662478 shutdown_connections 2023-10-08T13:50:26.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.818+0000 7fa063482700 1 --2- 172.21.15.57:0/4216662478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa05c068b30 0x7fa05c10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.818+0000 7fa063482700 1 --2- 172.21.15.57:0/4216662478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa05c1067c0 0x7fa05c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.818+0000 7fa063482700 1 --2- 172.21.15.57:0/4216662478 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c068190 0x7fa05c0685f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.818+0000 7fa063482700 1 -- 172.21.15.57:0/4216662478 >> 172.21.15.57:0/4216662478 conn(0x7fa05c075790 msgr2=0x7fa05c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.818+0000 7fa063482700 1 -- 172.21.15.57:0/4216662478 shutdown_connections 2023-10-08T13:50:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.818+0000 7fa063482700 1 -- 172.21.15.57:0/4216662478 wait complete. 2023-10-08T13:50:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa063482700 1 Processor -- start 2023-10-08T13:50:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa063482700 1 -- start start 2023-10-08T13:50:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa063482700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c068190 0x7fa05c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa063482700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa05c068b30 0x7fa05c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa063482700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa05c1067c0 0x7fa05c196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa063482700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c197080 con 0x7fa05c068190 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa063482700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c1971f0 con 0x7fa05c1067c0 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa063482700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c1973b0 con 0x7fa05c068b30 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa06121e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c068190 0x7fa05c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa06121e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c068190 0x7fa05c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54636/0 (socket says 172.21.15.57:54636) 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa06121e700 1 -- 172.21.15.57:0/450184177 learned_addr learned my addr 172.21.15.57:0/450184177 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.819+0000 7fa060a1d700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa05c068b30 0x7fa05c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa061a1f700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa05c1067c0 0x7fa05c196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa06121e700 1 -- 172.21.15.57:0/450184177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa05c068b30 msgr2=0x7fa05c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:26.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa06121e700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa05c068b30 0x7fa05c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:26.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa06121e700 1 -- 172.21.15.57:0/450184177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa05c1067c0 msgr2=0x7fa05c196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:26.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa06121e700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa05c1067c0 0x7fa05c196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:26.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa06121e700 1 -- 172.21.15.57:0/450184177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c0095f0 con 0x7fa05c068190 2023-10-08T13:50:26.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa061a1f700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa05c1067c0 0x7fa05c196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:26.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa06121e700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c068190 0x7fa05c19c8e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa05800d950 tx=0x7fa05800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:26.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa0527fc700 1 -- 172.21.15.57:0/450184177 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058009940 con 0x7fa05c068190 2023-10-08T13:50:26.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa0527fc700 1 -- 172.21.15.57:0/450184177 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa058010460 con 0x7fa05c068190 2023-10-08T13:50:26.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05c197690 con 0x7fa05c068190 2023-10-08T13:50:26.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa0527fc700 1 -- 172.21.15.57:0/450184177 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05800f620 con 0x7fa05c068190 2023-10-08T13:50:26.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.820+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05c1a38d0 con 0x7fa05c068190 2023-10-08T13:50:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.821+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa05c1109a0 con 0x7fa05c068190 2023-10-08T13:50:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.822+0000 7fa0527fc700 1 -- 172.21.15.57:0/450184177 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fa0580108d0 con 0x7fa05c068190 2023-10-08T13:50:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.823+0000 7fa0527fc700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0480a15e0 0x7fa0480a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.823+0000 7fa060a1d700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0480a15e0 0x7fa0480a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:26.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.823+0000 7fa0527fc700 1 -- 172.21.15.57:0/450184177 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fa0580cd570 con 0x7fa05c068190 2023-10-08T13:50:26.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.823+0000 7fa0527fc700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] conn(0x7fa0480ac790 0x7fa0480aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:26.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.823+0000 7fa0527fc700 1 -- 172.21.15.57:0/450184177 --> [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0480af300 con 0x7fa0480ac790 2023-10-08T13:50:26.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.823+0000 7fa0527fc700 1 -- 172.21.15.57:0/450184177 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0580cd980 con 0x7fa05c068190 2023-10-08T13:50:26.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.824+0000 7fa061a1f700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] conn(0x7fa0480ac790 0x7fa0480aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:26.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.824+0000 7fa060a1d700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0480a15e0 0x7fa0480a3aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00b5c0 tx=0x7fa04c005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.824+0000 7fa061a1f700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] conn(0x7fa0480ac790 0x7fa0480aec30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.824+0000 7fa0527fc700 1 -- 172.21.15.57:0/450184177 <== osd.3 v2:172.21.15.57:6810/966286851 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa0480af300 con 0x7fa0480ac790 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 --> [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa05c060ef0 con 0x7fa0480ac790 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa0527fc700 1 -- 172.21.15.57:0/450184177 <== osd.3 v2:172.21.15.57:6810/966286851 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa05c060ef0 con 0x7fa0480ac790 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] conn(0x7fa0480ac790 msgr2=0x7fa0480aec30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa063482700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] conn(0x7fa0480ac790 0x7fa0480aec30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0480a15e0 msgr2=0x7fa0480a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa063482700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0480a15e0 0x7fa0480a3aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00b5c0 tx=0x7fa04c005960 comp rx=0 tx=0).stop 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c068190 msgr2=0x7fa05c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa063482700 1 --2- 172.21.15.57:0/450184177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c068190 0x7fa05c19c8e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa05800d950 tx=0x7fa05800dc60 comp rx=0 tx=0).stop 2023-10-08T13:50:26.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa061a1f700 1 -- 172.21.15.57:0/450184177 reap_dead start 2023-10-08T13:50:26.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 shutdown_connections 2023-10-08T13:50:26.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.841+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 >> 172.21.15.57:0/450184177 conn(0x7fa05c075790 msgr2=0x7fa05c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:26.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.842+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 shutdown_connections 2023-10-08T13:50:26.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:26.842+0000 7fa063482700 1 -- 172.21.15.57:0/450184177 wait complete. 2023-10-08T13:50:27.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.432+0000 7ff7c9317700 1 -- 172.21.15.57:0/1485502516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7c40ff9d0 msgr2=0x7ff7c40ffe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.432+0000 7ff7c9317700 1 --2- 172.21.15.57:0/1485502516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7c40ff9d0 0x7ff7c40ffe50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009990 tx=0x7ff7b4009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.432+0000 7ff7c9317700 1 -- 172.21.15.57:0/1485502516 shutdown_connections 2023-10-08T13:50:27.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.432+0000 7ff7c9317700 1 --2- 172.21.15.57:0/1485502516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7c4100390 0x7ff7c410c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.432+0000 7ff7c9317700 1 --2- 172.21.15.57:0/1485502516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7c41046b0 0x7ff7c4104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.432+0000 7ff7c9317700 1 --2- 172.21.15.57:0/1485502516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7c40ff9d0 0x7ff7c40ffe50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.432+0000 7ff7c9317700 1 -- 172.21.15.57:0/1485502516 >> 172.21.15.57:0/1485502516 conn(0x7ff7c40765e0 msgr2=0x7ff7c40769f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:27.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.432+0000 7ff7c9317700 1 -- 172.21.15.57:0/1485502516 shutdown_connections 2023-10-08T13:50:27.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.432+0000 7ff7c9317700 1 -- 172.21.15.57:0/1485502516 wait complete. 2023-10-08T13:50:27.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.433+0000 7ff7c9317700 1 Processor -- start 2023-10-08T13:50:27.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.433+0000 7ff7c9317700 1 -- start start 2023-10-08T13:50:27.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.433+0000 7ff7c9317700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7c40ff9d0 0x7ff7c419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:27.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c9317700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7c4100390 0x7ff7c419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:27.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c9317700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7c41046b0 0x7ff7c4195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:27.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c9317700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4195f90 con 0x7ff7c41046b0 2023-10-08T13:50:27.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c9317700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4196100 con 0x7ff7c40ff9d0 2023-10-08T13:50:27.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c9317700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c41962c0 con 0x7ff7c4100390 2023-10-08T13:50:27.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7c41046b0 0x7ff7c4195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:27.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7c41046b0 0x7ff7c4195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54656/0 (socket says 172.21.15.57:54656) 2023-10-08T13:50:27.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c37fe700 1 -- 172.21.15.57:0/1530086098 learned_addr learned my addr 172.21.15.57:0/1530086098 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:27.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c2ffd700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7c40ff9d0 0x7ff7c419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:27.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c27fc700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7c4100390 0x7ff7c419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:27.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c37fe700 1 -- 172.21.15.57:0/1530086098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7c4100390 msgr2=0x7ff7c419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c37fe700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7c4100390 0x7ff7c419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c37fe700 1 -- 172.21.15.57:0/1530086098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7c40ff9d0 msgr2=0x7ff7c419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c37fe700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7c40ff9d0 0x7ff7c419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c37fe700 1 -- 172.21.15.57:0/1530086098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40095f0 con 0x7ff7c41046b0 2023-10-08T13:50:27.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c37fe700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7c41046b0 0x7ff7c4195930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00ef30 tx=0x7ff7bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:27.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7abfff700 1 -- 172.21.15.57:0/1530086098 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc018070 con 0x7ff7c41046b0 2023-10-08T13:50:27.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c4196540 con 0x7ff7c41046b0 2023-10-08T13:50:27.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7abfff700 1 -- 172.21.15.57:0/1530086098 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7bc00f040 con 0x7ff7c41046b0 2023-10-08T13:50:27.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.434+0000 7ff7abfff700 1 -- 172.21.15.57:0/1530086098 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc00cd00 con 0x7ff7c41046b0 2023-10-08T13:50:27.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.435+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c41a27e0 con 0x7ff7c41046b0 2023-10-08T13:50:27.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.436+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7c410f8d0 con 0x7ff7c41046b0 2023-10-08T13:50:27.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.437+0000 7ff7abfff700 1 -- 172.21.15.57:0/1530086098 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7ff7bc003a00 con 0x7ff7c41046b0 2023-10-08T13:50:27.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.437+0000 7ff7abfff700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff7ac0a1560 0x7ff7ac0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:27.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.437+0000 7ff7abfff700 1 -- 172.21.15.57:0/1530086098 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7ff7bc0cce50 con 0x7ff7c41046b0 2023-10-08T13:50:27.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.437+0000 7ff7c2ffd700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff7ac0a1560 0x7ff7ac0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:27.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.437+0000 7ff7abfff700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] conn(0x7ff7ac0ac710 0x7ff7ac0aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:27.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.437+0000 7ff7abfff700 1 -- 172.21.15.57:0/1530086098 --> [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac0af280 con 0x7ff7ac0ac710 2023-10-08T13:50:27.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.437+0000 7ff7abfff700 1 -- 172.21.15.57:0/1530086098 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7bc0cf4d0 con 0x7ff7c41046b0 2023-10-08T13:50:27.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.438+0000 7ff7c27fc700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] conn(0x7ff7ac0ac710 0x7ff7ac0aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:27.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.438+0000 7ff7c2ffd700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff7ac0a1560 0x7ff7ac0a3a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009be0 tx=0x7ff7b8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:27.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.438+0000 7ff7c27fc700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] conn(0x7ff7ac0ac710 0x7ff7ac0aebb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:27.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.439+0000 7ff7abfff700 1 -- 172.21.15.57:0/1530086098 <== osd.2 v2:172.21.15.115:6800/3640540626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff7ac0af280 con 0x7ff7ac0ac710 2023-10-08T13:50:27.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.455+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 --> [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff7c4060ef0 con 0x7ff7ac0ac710 2023-10-08T13:50:27.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.456+0000 7ff7abfff700 1 -- 172.21.15.57:0/1530086098 <== osd.2 v2:172.21.15.115:6800/3640540626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff7c4060ef0 con 0x7ff7ac0ac710 2023-10-08T13:50:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.456+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 >> [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] conn(0x7ff7ac0ac710 msgr2=0x7ff7ac0aebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.456+0000 7ff7c9317700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] conn(0x7ff7ac0ac710 0x7ff7ac0aebb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.456+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff7ac0a1560 msgr2=0x7ff7ac0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.456+0000 7ff7c9317700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff7ac0a1560 0x7ff7ac0a3a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009be0 tx=0x7ff7b8009260 comp rx=0 tx=0).stop 2023-10-08T13:50:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.456+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7c41046b0 msgr2=0x7ff7c4195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.457+0000 7ff7c9317700 1 --2- 172.21.15.57:0/1530086098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7c41046b0 0x7ff7c4195930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00ef30 tx=0x7ff7bc00c3a0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.457+0000 7ff7c37fe700 1 -- 172.21.15.57:0/1530086098 reap_dead start 2023-10-08T13:50:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.457+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 shutdown_connections 2023-10-08T13:50:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.457+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 >> 172.21.15.57:0/1530086098 conn(0x7ff7c40765e0 msgr2=0x7ff7c40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.457+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 shutdown_connections 2023-10-08T13:50:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.457+0000 7ff7c9317700 1 -- 172.21.15.57:0/1530086098 wait complete. 2023-10-08T13:50:27.518 INFO:teuthology.orchestra.run.smithi057.stdout:111669149751 2023-10-08T13:50:27.519 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-08T13:50:27.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.757+0000 7ff75474f700 1 -- 172.21.15.57:0/1137277910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff74c100c60 msgr2=0x7ff74c10c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.757+0000 7ff75474f700 1 --2- 172.21.15.57:0/1137277910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff74c100c60 0x7ff74c10c5c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b0a0 tx=0x7ff74400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.757+0000 7ff75474f700 1 -- 172.21.15.57:0/1137277910 shutdown_connections 2023-10-08T13:50:27.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.757+0000 7ff75474f700 1 --2- 172.21.15.57:0/1137277910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff74c100c60 0x7ff74c10c5c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.757+0000 7ff75474f700 1 --2- 172.21.15.57:0/1137277910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c104720 0x7ff74c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.757+0000 7ff75474f700 1 --2- 172.21.15.57:0/1137277910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff74c1002a0 0x7ff74c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.757+0000 7ff75474f700 1 -- 172.21.15.57:0/1137277910 >> 172.21.15.57:0/1137277910 conn(0x7ff74c074a10 msgr2=0x7ff74c074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:27.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.757+0000 7ff75474f700 1 -- 172.21.15.57:0/1137277910 shutdown_connections 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.757+0000 7ff75474f700 1 -- 172.21.15.57:0/1137277910 wait complete. 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff75474f700 1 Processor -- start 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff75474f700 1 -- start start 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff75474f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff74c1002a0 0x7ff74c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff75474f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c100c60 0x7ff74c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff75474f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff74c104720 0x7ff74c196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff75474f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c197300 con 0x7ff74c104720 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff75474f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c197470 con 0x7ff74c100c60 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff75474f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c197630 con 0x7ff74c1002a0 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff752cec700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff74c104720 0x7ff74c196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff752cec700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff74c104720 0x7ff74c196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54672/0 (socket says 172.21.15.57:54672) 2023-10-08T13:50:27.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff752cec700 1 -- 172.21.15.57:0/2789979538 learned_addr learned my addr 172.21.15.57:0/2789979538 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff7524eb700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff74c1002a0 0x7ff74c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff751cea700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c100c60 0x7ff74c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff752cec700 1 -- 172.21.15.57:0/2789979538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff74c1002a0 msgr2=0x7ff74c19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.758+0000 7ff752cec700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff74c1002a0 0x7ff74c19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.759+0000 7ff752cec700 1 -- 172.21.15.57:0/2789979538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c100c60 msgr2=0x7ff74c19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.759+0000 7ff752cec700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c100c60 0x7ff74c19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.759+0000 7ff752cec700 1 -- 172.21.15.57:0/2789979538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff744009d00 con 0x7ff74c104720 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.759+0000 7ff752cec700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff74c104720 0x7ff74c196b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b830 tx=0x7ff7440078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.759+0000 7ff7437fe700 1 -- 172.21.15.57:0/2789979538 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74400e040 con 0x7ff74c104720 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.759+0000 7ff7437fe700 1 -- 172.21.15.57:0/2789979538 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff744004e60 con 0x7ff74c104720 2023-10-08T13:50:27.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.759+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74c1978b0 con 0x7ff74c104720 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.759+0000 7ff7437fe700 1 -- 172.21.15.57:0/2789979538 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744005420 con 0x7ff74c104720 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.759+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a39d0 con 0x7ff74c104720 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.760+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff74c110a70 con 0x7ff74c104720 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.761+0000 7ff7437fe700 1 -- 172.21.15.57:0/2789979538 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7ff74401bb30 con 0x7ff74c104720 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.761+0000 7ff7437fe700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff7380a15e0 0x7ff7380a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.762+0000 7ff7437fe700 1 -- 172.21.15.57:0/2789979538 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7ff7440cf280 con 0x7ff74c104720 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.762+0000 7ff7524eb700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff7380a15e0 0x7ff7380a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.762+0000 7ff7437fe700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] conn(0x7ff7380ac790 0x7ff7380aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.762+0000 7ff7437fe700 1 -- 172.21.15.57:0/2789979538 --> [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7380af300 con 0x7ff7380ac790 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.762+0000 7ff7437fe700 1 -- 172.21.15.57:0/2789979538 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7440cf690 con 0x7ff74c104720 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.762+0000 7ff7524eb700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff7380a15e0 0x7ff7380a3aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff748005fd0 tx=0x7ff74800a560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:27.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.762+0000 7ff751cea700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] conn(0x7ff7380ac790 0x7ff7380aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.763+0000 7ff751cea700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] conn(0x7ff7380ac790 0x7ff7380aec30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.764+0000 7ff7437fe700 1 -- 172.21.15.57:0/2789979538 <== osd.7 v2:172.21.15.99:6816/4070534570 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff7380af300 con 0x7ff7380ac790 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.780+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 --> [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff74c060e70 con 0x7ff7380ac790 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.781+0000 7ff7437fe700 1 -- 172.21.15.57:0/2789979538 <== osd.7 v2:172.21.15.99:6816/4070534570 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff74c060e70 con 0x7ff7380ac790 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.781+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 >> [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] conn(0x7ff7380ac790 msgr2=0x7ff7380aec30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.781+0000 7ff75474f700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] conn(0x7ff7380ac790 0x7ff7380aec30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.781+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff7380a15e0 msgr2=0x7ff7380a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.781+0000 7ff75474f700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff7380a15e0 0x7ff7380a3aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff748005fd0 tx=0x7ff74800a560 comp rx=0 tx=0).stop 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.781+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff74c104720 msgr2=0x7ff74c196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.781+0000 7ff75474f700 1 --2- 172.21.15.57:0/2789979538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff74c104720 0x7ff74c196b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b830 tx=0x7ff7440078a0 comp rx=0 tx=0).stop 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.781+0000 7ff752cec700 1 -- 172.21.15.57:0/2789979538 reap_dead start 2023-10-08T13:50:27.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.781+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 shutdown_connections 2023-10-08T13:50:27.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.781+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 >> 172.21.15.57:0/2789979538 conn(0x7ff74c074a10 msgr2=0x7ff74c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:27.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.782+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 shutdown_connections 2023-10-08T13:50:27.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:27.782+0000 7ff75474f700 1 -- 172.21.15.57:0/2789979538 wait complete. 2023-10-08T13:50:27.989 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:28.015 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:27 smithi057 ceph-mon[99476]: pgmap v264: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s; 118 B/s, 0 objects/s recovering 2023-10-08T13:50:28.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:27 smithi115 ceph-mon[104408]: pgmap v264: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s; 118 B/s, 0 objects/s recovering 2023-10-08T13:50:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:27 smithi099 ceph-mon[103849]: pgmap v264: 129 pgs: 129 active+clean; 636 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s; 118 B/s, 0 objects/s recovering 2023-10-08T13:50:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.441+0000 7f346c771700 1 -- 172.21.15.57:0/1809580326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34640ff750 msgr2=0x7f34640ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.441+0000 7f346c771700 1 --2- 172.21.15.57:0/1809580326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34640ff750 0x7f34640ffbd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3454009990 tx=0x7f3454009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.441+0000 7f346c771700 1 -- 172.21.15.57:0/1809580326 shutdown_connections 2023-10-08T13:50:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.441+0000 7f346c771700 1 --2- 172.21.15.57:0/1809580326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3464100110 0x7f346410c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.441+0000 7f346c771700 1 --2- 172.21.15.57:0/1809580326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3464105690 0x7f3464105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.441+0000 7f346c771700 1 --2- 172.21.15.57:0/1809580326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34640ff750 0x7f34640ffbd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.441+0000 7f346c771700 1 -- 172.21.15.57:0/1809580326 >> 172.21.15.57:0/1809580326 conn(0x7f34640757d0 msgr2=0x7f3464075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:28.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.441+0000 7f346c771700 1 -- 172.21.15.57:0/1809580326 shutdown_connections 2023-10-08T13:50:28.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.441+0000 7f346c771700 1 -- 172.21.15.57:0/1809580326 wait complete. 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346c771700 1 Processor -- start 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346c771700 1 -- start start 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346c771700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34640ff750 0x7f3464101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346c771700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3464100110 0x7f34641017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346c771700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3464105690 0x7f3464101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346c771700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3464103220 con 0x7f34640ff750 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346c771700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34641a9570 con 0x7f3464100110 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346c771700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34641a9730 con 0x7f3464105690 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346a50d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34640ff750 0x7f3464101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346a50d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34640ff750 0x7f3464101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54700/0 (socket says 172.21.15.57:54700) 2023-10-08T13:50:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.442+0000 7f346a50d700 1 -- 172.21.15.57:0/1522501571 learned_addr learned my addr 172.21.15.57:0/1522501571 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f346ad0e700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3464105690 0x7f3464101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f3469d0c700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3464100110 0x7f34641017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f346a50d700 1 -- 172.21.15.57:0/1522501571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3464105690 msgr2=0x7f3464101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f346a50d700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3464105690 0x7f3464101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f346a50d700 1 -- 172.21.15.57:0/1522501571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3464100110 msgr2=0x7f34641017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f346a50d700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3464100110 0x7f34641017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f346a50d700 1 -- 172.21.15.57:0/1522501571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34540095f0 con 0x7f34640ff750 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f346ad0e700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3464105690 0x7f3464101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f3469d0c700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3464100110 0x7f34641017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f346a50d700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34640ff750 0x7f3464101260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f346000c970 tx=0x7f346000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f345b7fe700 1 -- 172.21.15.57:0/1522501571 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460007aa0 con 0x7f34640ff750 2023-10-08T13:50:28.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34641a9a10 con 0x7f34640ff750 2023-10-08T13:50:28.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f345b7fe700 1 -- 172.21.15.57:0/1522501571 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3460004cf0 con 0x7f34640ff750 2023-10-08T13:50:28.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f345b7fe700 1 -- 172.21.15.57:0/1522501571 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34600055a0 con 0x7f34640ff750 2023-10-08T13:50:28.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.443+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34641a9ec0 con 0x7f34640ff750 2023-10-08T13:50:28.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.444+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f346410f8c0 con 0x7f34640ff750 2023-10-08T13:50:28.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.446+0000 7f345b7fe700 1 -- 172.21.15.57:0/1522501571 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f3460007c00 con 0x7f34640ff750 2023-10-08T13:50:28.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.446+0000 7f345b7fe700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34500a1560 0x7f34500a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.447+0000 7f3469d0c700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34500a1560 0x7f34500a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.447+0000 7f345b7fe700 1 -- 172.21.15.57:0/1522501571 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f3460013070 con 0x7f34640ff750 2023-10-08T13:50:28.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.447+0000 7f345b7fe700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] conn(0x7f34500ac710 0x7f34500aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.447+0000 7f345b7fe700 1 -- 172.21.15.57:0/1522501571 --> [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34500af280 con 0x7f34500ac710 2023-10-08T13:50:28.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.447+0000 7f345b7fe700 1 -- 172.21.15.57:0/1522501571 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34600cd890 con 0x7f34640ff750 2023-10-08T13:50:28.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.447+0000 7f3469d0c700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34500a1560 0x7f34500a3a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3454000c00 tx=0x7f3454005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:28.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.447+0000 7f346ad0e700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] conn(0x7f34500ac710 0x7f34500aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.448+0000 7f346ad0e700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] conn(0x7f34500ac710 0x7f34500aebb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:28.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.449+0000 7f345b7fe700 1 -- 172.21.15.57:0/1522501571 <== osd.4 v2:172.21.15.99:6808/1502873104 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f34500af280 con 0x7f34500ac710 2023-10-08T13:50:28.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.466+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 --> [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3464100dd0 con 0x7f34500ac710 2023-10-08T13:50:28.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.467+0000 7f345b7fe700 1 -- 172.21.15.57:0/1522501571 <== osd.4 v2:172.21.15.99:6808/1502873104 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3464100dd0 con 0x7f34500ac710 2023-10-08T13:50:28.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 >> [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] conn(0x7f34500ac710 msgr2=0x7f34500aebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346c771700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] conn(0x7f34500ac710 0x7f34500aebb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34500a1560 msgr2=0x7f34500a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346c771700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34500a1560 0x7f34500a3a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3454000c00 tx=0x7f3454005fb0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34640ff750 msgr2=0x7f3464101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346c771700 1 --2- 172.21.15.57:0/1522501571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34640ff750 0x7f3464101260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f346000c970 tx=0x7f346000cd30 comp rx=0 tx=0).stop 2023-10-08T13:50:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346ad0e700 1 -- 172.21.15.57:0/1522501571 reap_dead start 2023-10-08T13:50:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 shutdown_connections 2023-10-08T13:50:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 >> 172.21.15.57:0/1522501571 conn(0x7f34640757d0 msgr2=0x7f3464103500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 shutdown_connections 2023-10-08T13:50:28.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.468+0000 7f346c771700 1 -- 172.21.15.57:0/1522501571 wait complete. 2023-10-08T13:50:28.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.607+0000 7f5b64f8f700 1 -- 172.21.15.57:0/3263223297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b60072a20 msgr2=0x7f5b60072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.607+0000 7f5b64f8f700 1 --2- 172.21.15.57:0/3263223297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b60072a20 0x7f5b60072e00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009990 tx=0x7f5b50009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.607+0000 7f5b64f8f700 1 -- 172.21.15.57:0/3263223297 shutdown_connections 2023-10-08T13:50:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.607+0000 7f5b64f8f700 1 --2- 172.21.15.57:0/3263223297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b6010d250 0x7f5b6010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.607+0000 7f5b64f8f700 1 --2- 172.21.15.57:0/3263223297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b60072a20 0x7f5b60072e00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.607+0000 7f5b64f8f700 1 --2- 172.21.15.57:0/3263223297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b60073340 0x7f5b6010cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.607+0000 7f5b64f8f700 1 -- 172.21.15.57:0/3263223297 >> 172.21.15.57:0/3263223297 conn(0x7f5b600779e0 msgr2=0x7f5b60077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.607+0000 7f5b64f8f700 1 -- 172.21.15.57:0/3263223297 shutdown_connections 2023-10-08T13:50:28.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.607+0000 7f5b64f8f700 1 -- 172.21.15.57:0/3263223297 wait complete. 2023-10-08T13:50:28.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b64f8f700 1 Processor -- start 2023-10-08T13:50:28.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b64f8f700 1 -- start start 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b64f8f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b60072a20 0x7f5b60198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b64f8f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b60073340 0x7f5b60199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b64f8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b6010d250 0x7f5b6019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b64f8f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6019de00 con 0x7f5b60072a20 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b64f8f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6019df70 con 0x7f5b6010d250 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b64f8f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6019e130 con 0x7f5b60073340 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b5e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b60072a20 0x7f5b60198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b5e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b60072a20 0x7f5b60198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54716/0 (socket says 172.21.15.57:54716) 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b5e59c700 1 -- 172.21.15.57:0/1410380800 learned_addr learned my addr 172.21.15.57:0/1410380800 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b5ed9d700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b6010d250 0x7f5b6019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.608+0000 7f5b5dd9b700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b60073340 0x7f5b60199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b5e59c700 1 -- 172.21.15.57:0/1410380800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b60073340 msgr2=0x7f5b60199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b5e59c700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b60073340 0x7f5b60199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b5e59c700 1 -- 172.21.15.57:0/1410380800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b6010d250 msgr2=0x7f5b6019d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b5e59c700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b6010d250 0x7f5b6019d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b5e59c700 1 -- 172.21.15.57:0/1410380800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b500095f0 con 0x7f5b60072a20 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b5ed9d700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b6010d250 0x7f5b6019d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b5e59c700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b60072a20 0x7f5b60198f50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5b50000c00 tx=0x7f5b50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b4f7fe700 1 -- 172.21.15.57:0/1410380800 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5001d070 con 0x7f5b60072a20 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b4f7fe700 1 -- 172.21.15.57:0/1410380800 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b50022470 con 0x7f5b60072a20 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6019e3b0 con 0x7f5b60072a20 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b4f7fe700 1 -- 172.21.15.57:0/1410380800 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50017650 con 0x7f5b60072a20 2023-10-08T13:50:28.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.609+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b601a5c70 con 0x7f5b60072a20 2023-10-08T13:50:28.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.610+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b60112ca0 con 0x7f5b60072a20 2023-10-08T13:50:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.611+0000 7f5b4f7fe700 1 -- 172.21.15.57:0/1410380800 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f5b500225e0 con 0x7f5b60072a20 2023-10-08T13:50:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.612+0000 7f5b4f7fe700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b480a1560 0x7f5b480a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.612+0000 7f5b5dd9b700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b480a1560 0x7f5b480a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.612+0000 7f5b4f7fe700 1 -- 172.21.15.57:0/1410380800 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f5b500ced70 con 0x7f5b60072a20 2023-10-08T13:50:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.612+0000 7f5b4f7fe700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] conn(0x7f5b480ac710 0x7f5b480aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.612+0000 7f5b4f7fe700 1 -- 172.21.15.57:0/1410380800 --> [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b480af280 con 0x7f5b480ac710 2023-10-08T13:50:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.612+0000 7f5b4f7fe700 1 -- 172.21.15.57:0/1410380800 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b500cf0e0 con 0x7f5b60072a20 2023-10-08T13:50:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.612+0000 7f5b5dd9b700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b480a1560 0x7f5b480a3a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5b6019a460 tx=0x7f5b54009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:28.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.612+0000 7f5b5ed9d700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] conn(0x7f5b480ac710 0x7f5b480aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.613+0000 7f5b5ed9d700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] conn(0x7f5b480ac710 0x7f5b480aebb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:28.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.614+0000 7f5b4f7fe700 1 -- 172.21.15.57:0/1410380800 <== osd.10 v2:172.21.15.99:6824/317517288 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f5b480af280 con 0x7f5b480ac710 2023-10-08T13:50:28.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.630+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 --> [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5b6019e680 con 0x7f5b480ac710 2023-10-08T13:50:28.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b4f7fe700 1 -- 172.21.15.57:0/1410380800 <== osd.10 v2:172.21.15.99:6824/317517288 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5b58010040 con 0x7f5b480ac710 2023-10-08T13:50:28.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 >> [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] conn(0x7f5b480ac710 msgr2=0x7f5b480aebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b64f8f700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] conn(0x7f5b480ac710 0x7f5b480aebb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b480a1560 msgr2=0x7f5b480a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b64f8f700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b480a1560 0x7f5b480a3a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5b6019a460 tx=0x7f5b54009260 comp rx=0 tx=0).stop 2023-10-08T13:50:28.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b60072a20 msgr2=0x7f5b60198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b64f8f700 1 --2- 172.21.15.57:0/1410380800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b60072a20 0x7f5b60198f50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5b50000c00 tx=0x7f5b50004730 comp rx=0 tx=0).stop 2023-10-08T13:50:28.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b5ed9d700 1 -- 172.21.15.57:0/1410380800 reap_dead start 2023-10-08T13:50:28.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 shutdown_connections 2023-10-08T13:50:28.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 >> 172.21.15.57:0/1410380800 conn(0x7f5b600779e0 msgr2=0x7f5b600fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:28.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 shutdown_connections 2023-10-08T13:50:28.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.631+0000 7f5b64f8f700 1 -- 172.21.15.57:0/1410380800 wait complete. 2023-10-08T13:50:28.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.867+0000 7fc4188a1700 1 -- 172.21.15.57:0/854817866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4141067a0 msgr2=0x7fc414106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.867+0000 7fc4188a1700 1 --2- 172.21.15.57:0/854817866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4141067a0 0x7fc414106b80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc404009990 tx=0x7fc404009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.867+0000 7fc4188a1700 1 -- 172.21.15.57:0/854817866 shutdown_connections 2023-10-08T13:50:28.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.867+0000 7fc4188a1700 1 --2- 172.21.15.57:0/854817866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc414068b30 0x7fc41410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.867+0000 7fc4188a1700 1 --2- 172.21.15.57:0/854817866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4141067a0 0x7fc414106b80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.867+0000 7fc4188a1700 1 --2- 172.21.15.57:0/854817866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc414068190 0x7fc4140685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.867+0000 7fc4188a1700 1 -- 172.21.15.57:0/854817866 >> 172.21.15.57:0/854817866 conn(0x7fc414075790 msgr2=0x7fc414075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:28.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.867+0000 7fc4188a1700 1 -- 172.21.15.57:0/854817866 shutdown_connections 2023-10-08T13:50:28.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.867+0000 7fc4188a1700 1 -- 172.21.15.57:0/854817866 wait complete. 2023-10-08T13:50:28.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.868+0000 7fc4188a1700 1 Processor -- start 2023-10-08T13:50:28.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.868+0000 7fc4188a1700 1 -- start start 2023-10-08T13:50:28.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.868+0000 7fc4188a1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc414068190 0x7fc4140ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.868+0000 7fc4188a1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc414068b30 0x7fc4140ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.868+0000 7fc4188a1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4141067a0 0x7fc4141002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.868+0000 7fc4188a1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141a50c0 con 0x7fc414068190 2023-10-08T13:50:28.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.868+0000 7fc4188a1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141a5230 con 0x7fc414068b30 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.868+0000 7fc4188a1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141a53f0 con 0x7fc4141067a0 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.868+0000 7fc41259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc414068190 0x7fc4140ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc41259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc414068190 0x7fc4140ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54732/0 (socket says 172.21.15.57:54732) 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc41259c700 1 -- 172.21.15.57:0/502793681 learned_addr learned my addr 172.21.15.57:0/502793681 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc412d9d700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4141067a0 0x7fc4141002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc40bfff700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc414068b30 0x7fc4140ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc41259c700 1 -- 172.21.15.57:0/502793681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4141067a0 msgr2=0x7fc4141002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc41259c700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4141067a0 0x7fc4141002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc41259c700 1 -- 172.21.15.57:0/502793681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc414068b30 msgr2=0x7fc4140ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc41259c700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc414068b30 0x7fc4140ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc41259c700 1 -- 172.21.15.57:0/502793681 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4040095f0 con 0x7fc414068190 2023-10-08T13:50:28.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc41259c700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc414068190 0x7fc4140ff820 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc404000c00 tx=0x7fc404004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:28.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc40b7fe700 1 -- 172.21.15.57:0/502793681 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40401d070 con 0x7fc414068190 2023-10-08T13:50:28.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4141a5670 con 0x7fc414068190 2023-10-08T13:50:28.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc40b7fe700 1 -- 172.21.15.57:0/502793681 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc404022470 con 0x7fc414068190 2023-10-08T13:50:28.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc40b7fe700 1 -- 172.21.15.57:0/502793681 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404017650 con 0x7fc414068190 2023-10-08T13:50:28.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.869+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4141a5a80 con 0x7fc414068190 2023-10-08T13:50:28.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.870+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc414110970 con 0x7fc414068190 2023-10-08T13:50:28.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.871+0000 7fc40b7fe700 1 -- 172.21.15.57:0/502793681 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fc4040225e0 con 0x7fc414068190 2023-10-08T13:50:28.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.872+0000 7fc40b7fe700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4000a1560 0x7fc4000a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.872+0000 7fc40bfff700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4000a1560 0x7fc4000a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.872+0000 7fc40b7fe700 1 -- 172.21.15.57:0/502793681 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fc4040cf010 con 0x7fc414068190 2023-10-08T13:50:28.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.872+0000 7fc40b7fe700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] conn(0x7fc4000ac710 0x7fc4000aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:28.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.872+0000 7fc40b7fe700 1 -- 172.21.15.57:0/502793681 --> [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4000af280 con 0x7fc4000ac710 2023-10-08T13:50:28.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.872+0000 7fc40b7fe700 1 -- 172.21.15.57:0/502793681 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4040cf420 con 0x7fc414068190 2023-10-08T13:50:28.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.873+0000 7fc40bfff700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4000a1560 0x7fc4000a3a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc414100f50 tx=0x7fc3fc0073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:28.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.873+0000 7fc412d9d700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] conn(0x7fc4000ac710 0x7fc4000aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:28.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.873+0000 7fc412d9d700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] conn(0x7fc4000ac710 0x7fc4000aebb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:28.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.874+0000 7fc40b7fe700 1 -- 172.21.15.57:0/502793681 <== osd.8 v2:172.21.15.115:6816/3107603962 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc4000af280 con 0x7fc4000ac710 2023-10-08T13:50:28.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.890+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 --> [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc414101ee0 con 0x7fc4000ac710 2023-10-08T13:50:28.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc40b7fe700 1 -- 172.21.15.57:0/502793681 <== osd.8 v2:172.21.15.115:6816/3107603962 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc414101ee0 con 0x7fc4000ac710 2023-10-08T13:50:28.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 >> [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] conn(0x7fc4000ac710 msgr2=0x7fc4000aebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc4188a1700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] conn(0x7fc4000ac710 0x7fc4000aebb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4000a1560 msgr2=0x7fc4000a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc4188a1700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4000a1560 0x7fc4000a3a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc414100f50 tx=0x7fc3fc0073d0 comp rx=0 tx=0).stop 2023-10-08T13:50:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc414068190 msgr2=0x7fc4140ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc4188a1700 1 --2- 172.21.15.57:0/502793681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc414068190 0x7fc4140ff820 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc404000c00 tx=0x7fc404004730 comp rx=0 tx=0).stop 2023-10-08T13:50:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc412d9d700 1 -- 172.21.15.57:0/502793681 reap_dead start 2023-10-08T13:50:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 shutdown_connections 2023-10-08T13:50:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 >> 172.21.15.57:0/502793681 conn(0x7fc414075790 msgr2=0x7fc4140fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.891+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 shutdown_connections 2023-10-08T13:50:28.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.892+0000 7fc4188a1700 1 -- 172.21.15.57:0/502793681 wait complete. 2023-10-08T13:50:29.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.145+0000 7fb89694d700 1 -- 172.21.15.57:0/3609921994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8900ffa40 msgr2=0x7fb8900ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.145+0000 7fb89694d700 1 --2- 172.21.15.57:0/3609921994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8900ffa40 0x7fb8900ffec0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb884009990 tx=0x7fb884009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.145+0000 7fb89694d700 1 -- 172.21.15.57:0/3609921994 shutdown_connections 2023-10-08T13:50:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.145+0000 7fb89694d700 1 --2- 172.21.15.57:0/3609921994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb890100400 0x7fb89010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.145+0000 7fb89694d700 1 --2- 172.21.15.57:0/3609921994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8901057b0 0x7fb890105b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.145+0000 7fb89694d700 1 --2- 172.21.15.57:0/3609921994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8900ffa40 0x7fb8900ffec0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.145+0000 7fb89694d700 1 -- 172.21.15.57:0/3609921994 >> 172.21.15.57:0/3609921994 conn(0x7fb8900764a0 msgr2=0x7fb8900768b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.145+0000 7fb89694d700 1 -- 172.21.15.57:0/3609921994 shutdown_connections 2023-10-08T13:50:29.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.145+0000 7fb89694d700 1 -- 172.21.15.57:0/3609921994 wait complete. 2023-10-08T13:50:29.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.146+0000 7fb89694d700 1 Processor -- start 2023-10-08T13:50:29.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.146+0000 7fb89694d700 1 -- start start 2023-10-08T13:50:29.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb89694d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8900ffa40 0x7fb890197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb89694d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb890100400 0x7fb890197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb89694d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8901057b0 0x7fb8901a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb89694d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb890198860 con 0x7fb8900ffa40 2023-10-08T13:50:29.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb89694d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8901a2ff0 con 0x7fb8901057b0 2023-10-08T13:50:29.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb89694d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8901a3160 con 0x7fb890100400 2023-10-08T13:50:29.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8900ffa40 0x7fb890197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8900ffa40 0x7fb890197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54748/0 (socket says 172.21.15.57:54748) 2023-10-08T13:50:29.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88ffff700 1 -- 172.21.15.57:0/1855632375 learned_addr learned my addr 172.21.15.57:0/1855632375 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:29.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88f7fe700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb890100400 0x7fb890197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb894eea700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8901057b0 0x7fb8901a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88ffff700 1 -- 172.21.15.57:0/1855632375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb890100400 msgr2=0x7fb890197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88ffff700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb890100400 0x7fb890197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88ffff700 1 -- 172.21.15.57:0/1855632375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8901057b0 msgr2=0x7fb8901a2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88ffff700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8901057b0 0x7fb8901a2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88ffff700 1 -- 172.21.15.57:0/1855632375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8840095f0 con 0x7fb8900ffa40 2023-10-08T13:50:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb894eea700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8901057b0 0x7fb8901a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88ffff700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8900ffa40 0x7fb890197a30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e9a0 tx=0x7fb88000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.147+0000 7fb88d7fa700 1 -- 172.21.15.57:0/1855632375 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880009790 con 0x7fb8900ffa40 2023-10-08T13:50:29.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.148+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8901a3440 con 0x7fb8900ffa40 2023-10-08T13:50:29.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.148+0000 7fb88d7fa700 1 -- 172.21.15.57:0/1855632375 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb88000cbd0 con 0x7fb8900ffa40 2023-10-08T13:50:29.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.148+0000 7fb88d7fa700 1 -- 172.21.15.57:0/1855632375 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880018680 con 0x7fb8900ffa40 2023-10-08T13:50:29.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.148+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8901a3960 con 0x7fb8900ffa40 2023-10-08T13:50:29.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.148+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb89010f910 con 0x7fb8900ffa40 2023-10-08T13:50:29.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.150+0000 7fb88d7fa700 1 -- 172.21.15.57:0/1855632375 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fb88000cd40 con 0x7fb8900ffa40 2023-10-08T13:50:29.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.150+0000 7fb88d7fa700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb8780a1560 0x7fb8780a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.151+0000 7fb88d7fa700 1 -- 172.21.15.57:0/1855632375 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fb880014070 con 0x7fb8900ffa40 2023-10-08T13:50:29.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.151+0000 7fb88f7fe700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb8780a1560 0x7fb8780a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.151+0000 7fb88d7fa700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] conn(0x7fb8780ac710 0x7fb8780aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.151+0000 7fb88d7fa700 1 -- 172.21.15.57:0/1855632375 --> [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8780af280 con 0x7fb8780ac710 2023-10-08T13:50:29.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.151+0000 7fb88d7fa700 1 -- 172.21.15.57:0/1855632375 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8800cdc80 con 0x7fb8900ffa40 2023-10-08T13:50:29.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.151+0000 7fb88f7fe700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb8780a1560 0x7fb8780a3a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb8840098e0 tx=0x7fb884005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.151+0000 7fb894eea700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] conn(0x7fb8780ac710 0x7fb8780aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.152+0000 7fb894eea700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] conn(0x7fb8780ac710 0x7fb8780aebb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.153+0000 7fb88d7fa700 1 -- 172.21.15.57:0/1855632375 <== osd.1 v2:172.21.15.99:6800/4047402511 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb8780af280 con 0x7fb8780ac710 2023-10-08T13:50:29.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.171+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 --> [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb890100df0 con 0x7fb8780ac710 2023-10-08T13:50:29.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb88d7fa700 1 -- 172.21.15.57:0/1855632375 <== osd.1 v2:172.21.15.99:6800/4047402511 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb890100df0 con 0x7fb8780ac710 2023-10-08T13:50:29.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 >> [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] conn(0x7fb8780ac710 msgr2=0x7fb8780aebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb89694d700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] conn(0x7fb8780ac710 0x7fb8780aebb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb8780a1560 msgr2=0x7fb8780a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb89694d700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb8780a1560 0x7fb8780a3a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb8840098e0 tx=0x7fb884005960 comp rx=0 tx=0).stop 2023-10-08T13:50:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8900ffa40 msgr2=0x7fb890197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb89694d700 1 --2- 172.21.15.57:0/1855632375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb8900ffa40 0x7fb890197a30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e9a0 tx=0x7fb88000ecb0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb894eea700 1 -- 172.21.15.57:0/1855632375 reap_dead start 2023-10-08T13:50:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 shutdown_connections 2023-10-08T13:50:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 >> 172.21.15.57:0/1855632375 conn(0x7fb8900764a0 msgr2=0x7fb8900fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 shutdown_connections 2023-10-08T13:50:29.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.172+0000 7fb89694d700 1 -- 172.21.15.57:0/1855632375 wait complete. 2023-10-08T13:50:29.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.297+0000 7f41f637f700 1 -- 172.21.15.57:0/484520772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f01057a0 msgr2=0x7f41f0105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.297+0000 7f41f637f700 1 --2- 172.21.15.57:0/484520772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f01057a0 0x7f41f0105b80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009990 tx=0x7f41e0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.298+0000 7f41f637f700 1 -- 172.21.15.57:0/484520772 shutdown_connections 2023-10-08T13:50:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.298+0000 7f41f637f700 1 --2- 172.21.15.57:0/484520772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f0101200 0x7f41f010d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.298+0000 7f41f637f700 1 --2- 172.21.15.57:0/484520772 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f01057a0 0x7f41f0105b80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.298+0000 7f41f637f700 1 --2- 172.21.15.57:0/484520772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f0100840 0x7f41f0100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.298+0000 7f41f637f700 1 -- 172.21.15.57:0/484520772 >> 172.21.15.57:0/484520772 conn(0x7f41f0077950 msgr2=0x7f41f0077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.298+0000 7f41f637f700 1 -- 172.21.15.57:0/484520772 shutdown_connections 2023-10-08T13:50:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.298+0000 7f41f637f700 1 -- 172.21.15.57:0/484520772 wait complete. 2023-10-08T13:50:29.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.298+0000 7f41f637f700 1 Processor -- start 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.298+0000 7f41f637f700 1 -- start start 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41f637f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f0100840 0x7f41f0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41f637f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f0101200 0x7f41f0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41f637f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f01057a0 0x7f41f0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41f637f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f01a4ff0 con 0x7f41f0101200 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41f637f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f01a5160 con 0x7f41f0100840 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41f637f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f01a5320 con 0x7f41f01057a0 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41ef7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f0101200 0x7f41f0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41ef7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f0101200 0x7f41f0073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54686/0 (socket says 172.21.15.57:54686) 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41ef7fe700 1 -- 172.21.15.57:0/1129652828 learned_addr learned my addr 172.21.15.57:0/1129652828 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41effff700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f0100840 0x7f41f0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41f491c700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f01057a0 0x7f41f0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41ef7fe700 1 -- 172.21.15.57:0/1129652828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f01057a0 msgr2=0x7f41f0073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41ef7fe700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f01057a0 0x7f41f0073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41ef7fe700 1 -- 172.21.15.57:0/1129652828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f0100840 msgr2=0x7f41f0072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41ef7fe700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f0100840 0x7f41f0072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41ef7fe700 1 -- 172.21.15.57:0/1129652828 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00095f0 con 0x7f41f0101200 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41f491c700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f01057a0 0x7f41f0073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.299+0000 7f41ef7fe700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f0101200 0x7f41f0073030 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c920 tx=0x7f41e400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.300+0000 7f41ed7fa700 1 -- 172.21.15.57:0/1129652828 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4007b00 con 0x7f41f0101200 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.300+0000 7f41ed7fa700 1 -- 172.21.15.57:0/1129652828 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41e400ce70 con 0x7f41f0101200 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.300+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f01a5600 con 0x7f41f0101200 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.300+0000 7f41ed7fa700 1 -- 172.21.15.57:0/1129652828 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e40054e0 con 0x7f41f0101200 2023-10-08T13:50:29.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.300+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f01a5b00 con 0x7f41f0101200 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.300+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41f0110a00 con 0x7f41f0101200 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.302+0000 7f41ed7fa700 1 -- 172.21.15.57:0/1129652828 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f41e4004750 con 0x7f41f0101200 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.303+0000 7f41ed7fa700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f41d80a1560 0x7f41d80a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.303+0000 7f41ed7fa700 1 -- 172.21.15.57:0/1129652828 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f41e40cd2b0 con 0x7f41f0101200 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.303+0000 7f41effff700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f41d80a1560 0x7f41d80a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.303+0000 7f41ed7fa700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] conn(0x7f41d80ac710 0x7f41d80aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.303+0000 7f41ed7fa700 1 -- 172.21.15.57:0/1129652828 --> [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41d80af280 con 0x7f41d80ac710 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.303+0000 7f41ed7fa700 1 -- 172.21.15.57:0/1129652828 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41e40cd6c0 con 0x7f41f0101200 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.303+0000 7f41effff700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f41d80a1560 0x7f41d80a3a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b5c0 tx=0x7f41e0005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.304+0000 7f41f491c700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] conn(0x7f41d80ac710 0x7f41d80aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.304+0000 7f41f491c700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] conn(0x7f41d80ac710 0x7f41d80aebb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.305+0000 7f41ed7fa700 1 -- 172.21.15.57:0/1129652828 <== osd.5 v2:172.21.15.115:6808/2071180529 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f41d80af280 con 0x7f41d80ac710 2023-10-08T13:50:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.325+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 --> [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f41f0060e70 con 0x7f41d80ac710 2023-10-08T13:50:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.325+0000 7f41ed7fa700 1 -- 172.21.15.57:0/1129652828 <== osd.5 v2:172.21.15.115:6808/2071180529 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f41f0060e70 con 0x7f41d80ac710 2023-10-08T13:50:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 >> [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] conn(0x7f41d80ac710 msgr2=0x7f41d80aebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f637f700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] conn(0x7f41d80ac710 0x7f41d80aebb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f41d80a1560 msgr2=0x7f41d80a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f637f700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f41d80a1560 0x7f41d80a3a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b5c0 tx=0x7f41e0005960 comp rx=0 tx=0).stop 2023-10-08T13:50:29.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f0101200 msgr2=0x7f41f0073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f637f700 1 --2- 172.21.15.57:0/1129652828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f0101200 0x7f41f0073030 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c920 tx=0x7f41e400cc30 comp rx=0 tx=0).stop 2023-10-08T13:50:29.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f491c700 1 -- 172.21.15.57:0/1129652828 reap_dead start 2023-10-08T13:50:29.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 shutdown_connections 2023-10-08T13:50:29.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 >> 172.21.15.57:0/1129652828 conn(0x7f41f0077950 msgr2=0x7f41f00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:29.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 shutdown_connections 2023-10-08T13:50:29.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:28.326+0000 7f41f637f700 1 -- 172.21.15.57:0/1129652828 wait complete. 2023-10-08T13:50:29.337 INFO:teuthology.orchestra.run.smithi057.stdout:81604378682 2023-10-08T13:50:29.337 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-08T13:50:29.353 INFO:teuthology.orchestra.run.smithi057.stdout:141733920818 2023-10-08T13:50:29.353 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-08T13:50:29.609 INFO:teuthology.orchestra.run.smithi057.stdout:214748364837 2023-10-08T13:50:29.609 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-08T13:50:29.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.920+0000 7f32cdb79700 1 -- 172.21.15.57:0/3066223050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32c8068b30 msgr2=0x7f32c810d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.920+0000 7f32cdb79700 1 --2- 172.21.15.57:0/3066223050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32c8068b30 0x7f32c810d670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f32c000b0a0 tx=0x7f32c000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.920+0000 7f32cdb79700 1 -- 172.21.15.57:0/3066223050 shutdown_connections 2023-10-08T13:50:29.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.920+0000 7f32cdb79700 1 --2- 172.21.15.57:0/3066223050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32c8068b30 0x7f32c810d670 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.920+0000 7f32cdb79700 1 --2- 172.21.15.57:0/3066223050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c81067a0 0x7f32c8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.920+0000 7f32cdb79700 1 --2- 172.21.15.57:0/3066223050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8068190 0x7f32c80685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.920+0000 7f32cdb79700 1 -- 172.21.15.57:0/3066223050 >> 172.21.15.57:0/3066223050 conn(0x7f32c8075790 msgr2=0x7f32c8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:29.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.920+0000 7f32cdb79700 1 -- 172.21.15.57:0/3066223050 shutdown_connections 2023-10-08T13:50:29.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.920+0000 7f32cdb79700 1 -- 172.21.15.57:0/3066223050 wait complete. 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.921+0000 7f32cdb79700 1 Processor -- start 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32cdb79700 1 -- start start 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32cdb79700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32c8068190 0x7f32c819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32cdb79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c8068b30 0x7f32c819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32cdb79700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c81067a0 0x7f32c8196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32cdb79700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81971b0 con 0x7f32c8068190 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32cdb79700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8197320 con 0x7f32c81067a0 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32cdb79700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81974e0 con 0x7f32c8068b30 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32c77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32c8068190 0x7f32c819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32c77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32c8068190 0x7f32c819c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54764/0 (socket says 172.21.15.57:54764) 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32c77fe700 1 -- 172.21.15.57:0/752202342 learned_addr learned my addr 172.21.15.57:0/752202342 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:29.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32c6ffd700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c8068b30 0x7f32c819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32c7fff700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c81067a0 0x7f32c8196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32c77fe700 1 -- 172.21.15.57:0/752202342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c8068b30 msgr2=0x7f32c819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32c77fe700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c8068b30 0x7f32c819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32c77fe700 1 -- 172.21.15.57:0/752202342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c81067a0 msgr2=0x7f32c8196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32c77fe700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c81067a0 0x7f32c8196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.922+0000 7f32c77fe700 1 -- 172.21.15.57:0/752202342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c0009d00 con 0x7f32c8068190 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.923+0000 7f32c77fe700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32c8068190 0x7f32c819c9c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32b800e9a0 tx=0x7f32b800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.923+0000 7f32c4ff9700 1 -- 172.21.15.57:0/752202342 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8009790 con 0x7f32c8068190 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.923+0000 7f32c4ff9700 1 -- 172.21.15.57:0/752202342 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32b800cbd0 con 0x7f32c8068190 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.923+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c81977c0 con 0x7f32c8068190 2023-10-08T13:50:29.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.923+0000 7f32c4ff9700 1 -- 172.21.15.57:0/752202342 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8018680 con 0x7f32c8068190 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.923+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c80ff920 con 0x7f32c8068190 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.925+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32c8110930 con 0x7f32c8068190 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.926+0000 7f32c4ff9700 1 -- 172.21.15.57:0/752202342 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f32b800cd40 con 0x7f32c8068190 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.927+0000 7f32c4ff9700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32b00a1560 0x7f32b00a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.927+0000 7f32c6ffd700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32b00a1560 0x7f32b00a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.927+0000 7f32c4ff9700 1 -- 172.21.15.57:0/752202342 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f32b8014070 con 0x7f32c8068190 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.927+0000 7f32c4ff9700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] conn(0x7f32b00ac710 0x7f32b00aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.927+0000 7f32c4ff9700 1 -- 172.21.15.57:0/752202342 --> [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32b00af280 con 0x7f32b00ac710 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.927+0000 7f32c4ff9700 1 -- 172.21.15.57:0/752202342 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32b80cfa60 con 0x7f32c8068190 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.927+0000 7f32c6ffd700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32b00a1560 0x7f32b00a3a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f32bc005950 tx=0x7f32bc0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.927+0000 7f32c7fff700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] conn(0x7f32b00ac710 0x7f32b00aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.928+0000 7f32c7fff700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] conn(0x7f32b00ac710 0x7f32b00aebb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.929+0000 7f32c4ff9700 1 -- 172.21.15.57:0/752202342 <== osd.11 v2:172.21.15.115:6824/901134235 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f32b00af280 con 0x7f32b00ac710 2023-10-08T13:50:29.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.945+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 --> [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f32c8060ef0 con 0x7f32b00ac710 2023-10-08T13:50:29.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32c4ff9700 1 -- 172.21.15.57:0/752202342 <== osd.11 v2:172.21.15.115:6824/901134235 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f32c8060ef0 con 0x7f32b00ac710 2023-10-08T13:50:29.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 >> [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] conn(0x7f32b00ac710 msgr2=0x7f32b00aebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32cdb79700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] conn(0x7f32b00ac710 0x7f32b00aebb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32b00a1560 msgr2=0x7f32b00a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32cdb79700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32b00a1560 0x7f32b00a3a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f32bc005950 tx=0x7f32bc0058e0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32c8068190 msgr2=0x7f32c819c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32cdb79700 1 --2- 172.21.15.57:0/752202342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32c8068190 0x7f32c819c9c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32b800e9a0 tx=0x7f32b800ed60 comp rx=0 tx=0).stop 2023-10-08T13:50:29.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32c7fff700 1 -- 172.21.15.57:0/752202342 reap_dead start 2023-10-08T13:50:29.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 shutdown_connections 2023-10-08T13:50:29.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 >> 172.21.15.57:0/752202342 conn(0x7f32c8075790 msgr2=0x7f32c80fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:29.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 shutdown_connections 2023-10-08T13:50:29.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f32cdb79700 1 -- 172.21.15.57:0/752202342 wait complete. 2023-10-08T13:50:29.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f70cc2a8700 1 -- 172.21.15.57:0/3109243832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70c41004c0 msgr2=0x7f70c41008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f70cc2a8700 1 --2- 172.21.15.57:0/3109243832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70c41004c0 0x7f70c41008a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f70c000b0a0 tx=0x7f70c000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f70cc2a8700 1 -- 172.21.15.57:0/3109243832 shutdown_connections 2023-10-08T13:50:29.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f70cc2a8700 1 --2- 172.21.15.57:0/3109243832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c4104ae0 0x7f70c410c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f70cc2a8700 1 --2- 172.21.15.57:0/3109243832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70c41004c0 0x7f70c41008a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.946+0000 7f70cc2a8700 1 --2- 172.21.15.57:0/3109243832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c4100e70 0x7f70c41043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.947+0000 7f70cc2a8700 1 -- 172.21.15.57:0/3109243832 >> 172.21.15.57:0/3109243832 conn(0x7f70c4074aa0 msgr2=0x7f70c4074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:29.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.947+0000 7f70cc2a8700 1 -- 172.21.15.57:0/3109243832 shutdown_connections 2023-10-08T13:50:29.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.947+0000 7f70cc2a8700 1 -- 172.21.15.57:0/3109243832 wait complete. 2023-10-08T13:50:29.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.947+0000 7f70cc2a8700 1 Processor -- start 2023-10-08T13:50:29.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.947+0000 7f70cc2a8700 1 -- start start 2023-10-08T13:50:29.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.947+0000 7f70cc2a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41004c0 0x7f70c4195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70cc2a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c4100e70 0x7f70c4195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70cc2a8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70c4104ae0 0x7f70c41964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70cc2a8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c41a5190 con 0x7f70c4104ae0 2023-10-08T13:50:29.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70cc2a8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c41a5300 con 0x7f70c41004c0 2023-10-08T13:50:29.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70cc2a8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c41a54c0 con 0x7f70c4100e70 2023-10-08T13:50:29.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca845700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70c4104ae0 0x7f70c41964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca845700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70c4104ae0 0x7f70c41964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54784/0 (socket says 172.21.15.57:54784) 2023-10-08T13:50:29.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70c9843700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c4100e70 0x7f70c4195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca044700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41004c0 0x7f70c4195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca845700 1 -- 172.21.15.57:0/2565091476 learned_addr learned my addr 172.21.15.57:0/2565091476 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca845700 1 -- 172.21.15.57:0/2565091476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c4100e70 msgr2=0x7f70c4195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca845700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c4100e70 0x7f70c4195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca845700 1 -- 172.21.15.57:0/2565091476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41004c0 msgr2=0x7f70c4195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca845700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41004c0 0x7f70c4195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca845700 1 -- 172.21.15.57:0/2565091476 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c0009d00 con 0x7f70c4104ae0 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca044700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41004c0 0x7f70c4195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70ca845700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70c4104ae0 0x7f70c41964c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00e940 tx=0x7f70bc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70bb7fe700 1 -- 172.21.15.57:0/2565091476 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0097b0 con 0x7f70c4104ae0 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70bb7fe700 1 -- 172.21.15.57:0/2565091476 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70bc005020 con 0x7f70c4104ae0 2023-10-08T13:50:29.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.948+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c41a57a0 con 0x7f70c4104ae0 2023-10-08T13:50:29.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.949+0000 7f70bb7fe700 1 -- 172.21.15.57:0/2565091476 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00cc40 con 0x7f70c4104ae0 2023-10-08T13:50:29.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.949+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c41a5ca0 con 0x7f70c4104ae0 2023-10-08T13:50:29.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.949+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70c410f990 con 0x7f70c4104ae0 2023-10-08T13:50:29.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.951+0000 7f70bb7fe700 1 -- 172.21.15.57:0/2565091476 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f70bc005400 con 0x7f70c4104ae0 2023-10-08T13:50:29.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.951+0000 7f70bb7fe700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70b00a1560 0x7f70b00a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.952+0000 7f70bb7fe700 1 -- 172.21.15.57:0/2565091476 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f70bc014070 con 0x7f70c4104ae0 2023-10-08T13:50:29.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.952+0000 7f70ca044700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70b00a1560 0x7f70b00a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.952+0000 7f70bb7fe700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] conn(0x7f70b00ac710 0x7f70b00aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.952+0000 7f70bb7fe700 1 -- 172.21.15.57:0/2565091476 --> [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70b00af280 con 0x7f70b00ac710 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.952+0000 7f70bb7fe700 1 -- 172.21.15.57:0/2565091476 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70bc0cdaa0 con 0x7f70c4104ae0 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.952+0000 7f70c9843700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] conn(0x7f70b00ac710 0x7f70b00aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.952+0000 7f70ca044700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70b00a1560 0x7f70b00a3a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009fd0 tx=0x7f70c000b720 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.952+0000 7f70c9843700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] conn(0x7f70b00ac710 0x7f70b00aebb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.953+0000 7f70bb7fe700 1 -- 172.21.15.57:0/2565091476 <== osd.9 v2:172.21.15.57:6826/1074674274 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f70b00af280 con 0x7f70b00ac710 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.970+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 --> [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70c4060ef0 con 0x7f70b00ac710 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.970+0000 7f70bb7fe700 1 -- 172.21.15.57:0/2565091476 <== osd.9 v2:172.21.15.57:6826/1074674274 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f70c4060ef0 con 0x7f70b00ac710 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.970+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] conn(0x7f70b00ac710 msgr2=0x7f70b00aebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.970+0000 7f70cc2a8700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] conn(0x7f70b00ac710 0x7f70b00aebb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:29.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.971+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70b00a1560 msgr2=0x7f70b00a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.971+0000 7f70cc2a8700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70b00a1560 0x7f70b00a3a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009fd0 tx=0x7f70c000b720 comp rx=0 tx=0).stop 2023-10-08T13:50:29.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.971+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70c4104ae0 msgr2=0x7f70c41964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:29.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.971+0000 7f70cc2a8700 1 --2- 172.21.15.57:0/2565091476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f70c4104ae0 0x7f70c41964c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00e940 tx=0x7f70bc00ec50 comp rx=0 tx=0).stop 2023-10-08T13:50:29.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.971+0000 7f70ca845700 1 -- 172.21.15.57:0/2565091476 reap_dead start 2023-10-08T13:50:29.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.971+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 shutdown_connections 2023-10-08T13:50:29.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.971+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 >> 172.21.15.57:0/2565091476 conn(0x7f70c4074aa0 msgr2=0x7f70c40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:29.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.971+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 shutdown_connections 2023-10-08T13:50:29.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:29.971+0000 7f70cc2a8700 1 -- 172.21.15.57:0/2565091476 wait complete. 2023-10-08T13:50:30.045 INFO:teuthology.orchestra.run.smithi057.stdout:163208757294 2023-10-08T13:50:30.045 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-08T13:50:30.130 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:29 smithi057 ceph-mon[99476]: pgmap v265: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 102 B/s, 0 objects/s recovering 2023-10-08T13:50:30.174 INFO:teuthology.orchestra.run.smithi057.stdout:304942678041 2023-10-08T13:50:30.174 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-08T13:50:30.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:29 smithi115 ceph-mon[104408]: pgmap v265: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 102 B/s, 0 objects/s recovering 2023-10-08T13:50:30.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:29 smithi099 ceph-mon[103849]: pgmap v265: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 102 B/s, 0 objects/s recovering 2023-10-08T13:50:30.688 INFO:teuthology.orchestra.run.smithi057.stdout:339302416405 2023-10-08T13:50:30.688 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-08T13:50:30.728 INFO:teuthology.orchestra.run.smithi057.stdout:249108103201 2023-10-08T13:50:30.729 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-08T13:50:31.022 INFO:teuthology.orchestra.run.smithi057.stdout:60129542208 2023-10-08T13:50:31.023 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-08T13:50:31.137 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:31.591 INFO:teuthology.orchestra.run.smithi057.stdout:283467841564 2023-10-08T13:50:31.592 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-08T13:50:31.682 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:31.778 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:32.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.031+0000 7f117d6cd700 1 -- 172.21.15.57:0/229781487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11781056b0 msgr2=0x7f1178105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.031+0000 7f117d6cd700 1 --2- 172.21.15.57:0/229781487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11781056b0 0x7f1178105a90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1168009990 tx=0x7f1168009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.031+0000 7f117d6cd700 1 -- 172.21.15.57:0/229781487 shutdown_connections 2023-10-08T13:50:32.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.031+0000 7f117d6cd700 1 --2- 172.21.15.57:0/229781487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1178100110 0x7f117810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.032+0000 7f117d6cd700 1 --2- 172.21.15.57:0/229781487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11781056b0 0x7f1178105a90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.032+0000 7f117d6cd700 1 --2- 172.21.15.57:0/229781487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11780ff750 0x7f11780ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.032+0000 7f117d6cd700 1 -- 172.21.15.57:0/229781487 >> 172.21.15.57:0/229781487 conn(0x7f1178075790 msgr2=0x7f1178075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:32.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.032+0000 7f117d6cd700 1 -- 172.21.15.57:0/229781487 shutdown_connections 2023-10-08T13:50:32.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.032+0000 7f117d6cd700 1 -- 172.21.15.57:0/229781487 wait complete. 2023-10-08T13:50:32.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.032+0000 7f117d6cd700 1 Processor -- start 2023-10-08T13:50:32.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f117d6cd700 1 -- start start 2023-10-08T13:50:32.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f117d6cd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11780ff750 0x7f117819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:32.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f117d6cd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1178100110 0x7f117819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:32.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f117d6cd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11781056b0 0x7f1178195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:32.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f117d6cd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1178195f70 con 0x7f1178100110 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f117d6cd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11781960e0 con 0x7f11781056b0 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f117d6cd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11781962a0 con 0x7f11780ff750 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f11767fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1178100110 0x7f117819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f11767fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1178100110 0x7f117819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54798/0 (socket says 172.21.15.57:54798) 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f11767fc700 1 -- 172.21.15.57:0/2164401948 learned_addr learned my addr 172.21.15.57:0/2164401948 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f1176ffd700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11780ff750 0x7f117819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f11777fe700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11781056b0 0x7f1178195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f11767fc700 1 -- 172.21.15.57:0/2164401948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11780ff750 msgr2=0x7f117819b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f11767fc700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11780ff750 0x7f117819b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f11767fc700 1 -- 172.21.15.57:0/2164401948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11781056b0 msgr2=0x7f1178195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f11767fc700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11781056b0 0x7f1178195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.033+0000 7f11767fc700 1 -- 172.21.15.57:0/2164401948 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11680095f0 con 0x7f1178100110 2023-10-08T13:50:32.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.034+0000 7f11777fe700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11781056b0 0x7f1178195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:32.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.034+0000 7f11767fc700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1178100110 0x7f117819bd10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f116c00d950 tx=0x7f116c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:32.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.034+0000 7f115ffff700 1 -- 172.21.15.57:0/2164401948 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c009b40 con 0x7f1178100110 2023-10-08T13:50:32.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.034+0000 7f115ffff700 1 -- 172.21.15.57:0/2164401948 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f116c009ca0 con 0x7f1178100110 2023-10-08T13:50:32.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.034+0000 7f115ffff700 1 -- 172.21.15.57:0/2164401948 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00f620 con 0x7f1178100110 2023-10-08T13:50:32.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.034+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1178196580 con 0x7f1178100110 2023-10-08T13:50:32.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.034+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11781a27c0 con 0x7f1178100110 2023-10-08T13:50:32.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.035+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f117810f910 con 0x7f1178100110 2023-10-08T13:50:32.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:31 smithi057 ceph-mon[99476]: pgmap v266: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s; 102 B/s, 0 objects/s recovering 2023-10-08T13:50:32.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.037+0000 7f115ffff700 1 -- 172.21.15.57:0/2164401948 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f116c010460 con 0x7f1178100110 2023-10-08T13:50:32.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.037+0000 7f115ffff700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11600a15e0 0x7f11600a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:32.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.037+0000 7f1176ffd700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11600a15e0 0x7f11600a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:32.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.037+0000 7f115ffff700 1 -- 172.21.15.57:0/2164401948 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f116c0cd7f0 con 0x7f1178100110 2023-10-08T13:50:32.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.038+0000 7f115ffff700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] conn(0x7f11600ac790 0x7f11600aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:32.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.038+0000 7f115ffff700 1 -- 172.21.15.57:0/2164401948 --> [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11600af300 con 0x7f11600ac790 2023-10-08T13:50:32.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.038+0000 7f115ffff700 1 -- 172.21.15.57:0/2164401948 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f116c0cdba0 con 0x7f1178100110 2023-10-08T13:50:32.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.038+0000 7f11777fe700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] conn(0x7f11600ac790 0x7f11600aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:32.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.038+0000 7f1176ffd700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11600a15e0 0x7f11600a3aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1168005850 tx=0x7f116801a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:32.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.038+0000 7f11777fe700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] conn(0x7f11600ac790 0x7f11600aec30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:32.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.039+0000 7f115ffff700 1 -- 172.21.15.57:0/2164401948 <== osd.0 v2:172.21.15.57:6802/1388588372 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f11600af300 con 0x7f11600ac790 2023-10-08T13:50:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:32.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.056+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 --> [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1178060ef0 con 0x7f11600ac790 2023-10-08T13:50:32.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.058+0000 7f115ffff700 1 -- 172.21.15.57:0/2164401948 <== osd.0 v2:172.21.15.57:6802/1388588372 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1178060ef0 con 0x7f11600ac790 2023-10-08T13:50:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.058+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] conn(0x7f11600ac790 msgr2=0x7f11600aec30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.058+0000 7f117d6cd700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] conn(0x7f11600ac790 0x7f11600aec30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.058+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11600a15e0 msgr2=0x7f11600a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.058+0000 7f117d6cd700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11600a15e0 0x7f11600a3aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1168005850 tx=0x7f116801a040 comp rx=0 tx=0).stop 2023-10-08T13:50:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.058+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1178100110 msgr2=0x7f117819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.058+0000 7f117d6cd700 1 --2- 172.21.15.57:0/2164401948 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1178100110 0x7f117819bd10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f116c00d950 tx=0x7f116c00dd10 comp rx=0 tx=0).stop 2023-10-08T13:50:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.058+0000 7f11777fe700 1 -- 172.21.15.57:0/2164401948 reap_dead start 2023-10-08T13:50:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.059+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 shutdown_connections 2023-10-08T13:50:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.059+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 >> 172.21.15.57:0/2164401948 conn(0x7f1178075790 msgr2=0x7f11780fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:32.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.059+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 shutdown_connections 2023-10-08T13:50:32.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.059+0000 7f117d6cd700 1 -- 172.21.15.57:0/2164401948 wait complete. 2023-10-08T13:50:32.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:31 smithi115 ceph-mon[104408]: pgmap v266: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s; 102 B/s, 0 objects/s recovering 2023-10-08T13:50:32.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:31 smithi099 ceph-mon[103849]: pgmap v266: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s; 102 B/s, 0 objects/s recovering 2023-10-08T13:50:32.365 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:32.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.408+0000 7fe194f96700 1 -- 172.21.15.57:0/2361022740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe190100c60 msgr2=0x7fe19010c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.408+0000 7fe194f96700 1 --2- 172.21.15.57:0/2361022740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe190100c60 0x7fe19010c5c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe18800b0a0 tx=0x7fe18800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.408+0000 7fe194f96700 1 -- 172.21.15.57:0/2361022740 shutdown_connections 2023-10-08T13:50:32.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.408+0000 7fe194f96700 1 --2- 172.21.15.57:0/2361022740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe190100c60 0x7fe19010c5c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.408+0000 7fe194f96700 1 --2- 172.21.15.57:0/2361022740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190104720 0x7fe190104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.408+0000 7fe194f96700 1 --2- 172.21.15.57:0/2361022740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1901002a0 0x7fe190100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.408+0000 7fe194f96700 1 -- 172.21.15.57:0/2361022740 >> 172.21.15.57:0/2361022740 conn(0x7fe1900749b0 msgr2=0x7fe190074dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.408+0000 7fe194f96700 1 -- 172.21.15.57:0/2361022740 shutdown_connections 2023-10-08T13:50:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.408+0000 7fe194f96700 1 -- 172.21.15.57:0/2361022740 wait complete. 2023-10-08T13:50:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.409+0000 7fe194f96700 1 Processor -- start 2023-10-08T13:50:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.409+0000 7fe194f96700 1 -- start start 2023-10-08T13:50:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.409+0000 7fe194f96700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1901002a0 0x7fe190195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.409+0000 7fe194f96700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190100c60 0x7fe190195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.409+0000 7fe194f96700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe190104720 0x7fe1901963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:32.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.409+0000 7fe194f96700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901a5050 con 0x7fe190104720 2023-10-08T13:50:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.409+0000 7fe194f96700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901a5190 con 0x7fe190100c60 2023-10-08T13:50:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.409+0000 7fe194f96700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901a5350 con 0x7fe1901002a0 2023-10-08T13:50:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.409+0000 7fe18ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe190104720 0x7fe1901963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.409+0000 7fe18ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe190104720 0x7fe1901963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54814/0 (socket says 172.21.15.57:54814) 2023-10-08T13:50:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.410+0000 7fe18ed9d700 1 -- 172.21.15.57:0/1361221663 learned_addr learned my addr 172.21.15.57:0/1361221663 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.410+0000 7fe18e59c700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1901002a0 0x7fe190195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:32.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.410+0000 7fe18dd9b700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190100c60 0x7fe190195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.410+0000 7fe18e59c700 1 -- 172.21.15.57:0/1361221663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190100c60 msgr2=0x7fe190195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.410+0000 7fe18e59c700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe190100c60 0x7fe190195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.410+0000 7fe18e59c700 1 -- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe190104720 msgr2=0x7fe1901963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.410+0000 7fe18e59c700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe190104720 0x7fe1901963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.410+0000 7fe18e59c700 1 -- 172.21.15.57:0/1361221663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe188009d00 con 0x7fe1901002a0 2023-10-08T13:50:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.410+0000 7fe18e59c700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1901002a0 0x7fe190195940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe18000e9f0 tx=0x7fe18000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.411+0000 7fe17f7fe700 1 -- 172.21.15.57:0/1361221663 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18000eee0 con 0x7fe1901002a0 2023-10-08T13:50:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.411+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1901a5630 con 0x7fe1901002a0 2023-10-08T13:50:32.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.411+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1901a5b80 con 0x7fe1901002a0 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.411+0000 7fe17f7fe700 1 -- 172.21.15.57:0/1361221663 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe18000cbd0 con 0x7fe1901002a0 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.411+0000 7fe17f7fe700 1 -- 172.21.15.57:0/1361221663 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe180018680 con 0x7fe1901002a0 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.413+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe170000fc0 con 0x7fe1901002a0 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.414+0000 7fe17f7fe700 1 -- 172.21.15.57:0/1361221663 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fe18000cd40 con 0x7fe1901002a0 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.415+0000 7fe17f7fe700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1780a15e0 0x7fe1780a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.415+0000 7fe18dd9b700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1780a15e0 0x7fe1780a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.415+0000 7fe17f7fe700 1 -- 172.21.15.57:0/1361221663 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fe180014070 con 0x7fe1901002a0 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.415+0000 7fe17f7fe700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] conn(0x7fe1780ac790 0x7fe1780aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.415+0000 7fe17f7fe700 1 -- 172.21.15.57:0/1361221663 --> [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1780af300 con 0x7fe1780ac790 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.415+0000 7fe17f7fe700 1 -- 172.21.15.57:0/1361221663 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1800cf7e0 con 0x7fe1901002a0 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.415+0000 7fe18ed9d700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] conn(0x7fe1780ac790 0x7fe1780aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:32.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.415+0000 7fe18dd9b700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1780a15e0 0x7fe1780a3aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe184005ea0 tx=0x7fe184005e30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.416+0000 7fe18ed9d700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] conn(0x7fe1780ac790 0x7fe1780aec30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.416+0000 7fe17f7fe700 1 -- 172.21.15.57:0/1361221663 <== osd.6 v2:172.21.15.57:6818/3636365210 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe1780af300 con 0x7fe1780ac790 2023-10-08T13:50:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.432+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 --> [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe170002d40 con 0x7fe1780ac790 2023-10-08T13:50:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe17f7fe700 1 -- 172.21.15.57:0/1361221663 <== osd.6 v2:172.21.15.57:6818/3636365210 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe170002d40 con 0x7fe1780ac790 2023-10-08T13:50:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] conn(0x7fe1780ac790 msgr2=0x7fe1780aec30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe194f96700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] conn(0x7fe1780ac790 0x7fe1780aec30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1780a15e0 msgr2=0x7fe1780a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe194f96700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1780a15e0 0x7fe1780a3aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe184005ea0 tx=0x7fe184005e30 comp rx=0 tx=0).stop 2023-10-08T13:50:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1901002a0 msgr2=0x7fe190195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe194f96700 1 --2- 172.21.15.57:0/1361221663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1901002a0 0x7fe190195940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe18000e9f0 tx=0x7fe18000ed00 comp rx=0 tx=0).stop 2023-10-08T13:50:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe18ed9d700 1 -- 172.21.15.57:0/1361221663 reap_dead start 2023-10-08T13:50:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 shutdown_connections 2023-10-08T13:50:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 >> 172.21.15.57:0/1361221663 conn(0x7fe1900749b0 msgr2=0x7fe1900fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 shutdown_connections 2023-10-08T13:50:32.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:32.433+0000 7fe194f96700 1 -- 172.21.15.57:0/1361221663 wait complete. 2023-10-08T13:50:32.769 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:32.787 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:32.963 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:33.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:33 smithi057 ceph-mon[99476]: pgmap v267: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-10-08T13:50:34.071 INFO:teuthology.orchestra.run.smithi057.stdout:38654705733 2023-10-08T13:50:34.071 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-08T13:50:34.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:33 smithi115 ceph-mon[104408]: pgmap v267: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-10-08T13:50:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:33 smithi099 ceph-mon[103849]: pgmap v267: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-10-08T13:50:34.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.688+0000 7ffa848af700 1 -- 172.21.15.57:0/988974043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa80069410 msgr2=0x7ffa8010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:34.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.688+0000 7ffa848af700 1 --2- 172.21.15.57:0/988974043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa80069410 0x7ffa8010d650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b0d0 tx=0x7ffa7800b3e0 comp rx=0 tx=0).stop 2023-10-08T13:50:34.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.689+0000 7ffa848af700 1 -- 172.21.15.57:0/988974043 shutdown_connections 2023-10-08T13:50:34.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.689+0000 7ffa848af700 1 --2- 172.21.15.57:0/988974043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa80069410 0x7ffa8010d650 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:34.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.689+0000 7ffa848af700 1 --2- 172.21.15.57:0/988974043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80105670 0x7ffa80105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:34.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.689+0000 7ffa848af700 1 --2- 172.21.15.57:0/988974043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa80068a50 0x7ffa80068ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:34.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.689+0000 7ffa848af700 1 -- 172.21.15.57:0/988974043 >> 172.21.15.57:0/988974043 conn(0x7ffa800766c0 msgr2=0x7ffa80076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:34.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.689+0000 7ffa848af700 1 -- 172.21.15.57:0/988974043 shutdown_connections 2023-10-08T13:50:34.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.689+0000 7ffa848af700 1 -- 172.21.15.57:0/988974043 wait complete. 2023-10-08T13:50:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.689+0000 7ffa848af700 1 Processor -- start 2023-10-08T13:50:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.689+0000 7ffa848af700 1 -- start start 2023-10-08T13:50:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa848af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa80068a50 0x7ffa8019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa848af700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80069410 0x7ffa8019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa848af700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa80105670 0x7ffa80196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa848af700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa801971b0 con 0x7ffa80068a50 2023-10-08T13:50:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa848af700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa801972f0 con 0x7ffa80069410 2023-10-08T13:50:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa848af700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa801974b0 con 0x7ffa80105670 2023-10-08T13:50:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa80068a50 0x7ffa8019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa80068a50 0x7ffa8019c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54828/0 (socket says 172.21.15.57:54828) 2023-10-08T13:50:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7e59c700 1 -- 172.21.15.57:0/4063745588 learned_addr learned my addr 172.21.15.57:0/4063745588 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7dd9b700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80069410 0x7ffa8019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7ed9d700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa80105670 0x7ffa80196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7e59c700 1 -- 172.21.15.57:0/4063745588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa80105670 msgr2=0x7ffa80196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7e59c700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa80105670 0x7ffa80196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7e59c700 1 -- 172.21.15.57:0/4063745588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80069410 msgr2=0x7ffa8019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7e59c700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80069410 0x7ffa8019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7e59c700 1 -- 172.21.15.57:0/4063745588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa78009d00 con 0x7ffa80068a50 2023-10-08T13:50:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.690+0000 7ffa7e59c700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa80068a50 0x7ffa8019c9b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000d950 tx=0x7ffa7000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:34.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.691+0000 7ffa6f7fe700 1 -- 172.21.15.57:0/4063745588 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa70009b40 con 0x7ffa80068a50 2023-10-08T13:50:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.691+0000 7ffa6f7fe700 1 -- 172.21.15.57:0/4063745588 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa70009ca0 con 0x7ffa80068a50 2023-10-08T13:50:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.691+0000 7ffa848af700 1 -- 172.21.15.57:0/4063745588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa80197790 con 0x7ffa80068a50 2023-10-08T13:50:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.691+0000 7ffa6f7fe700 1 -- 172.21.15.57:0/4063745588 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7000f620 con 0x7ffa80068a50 2023-10-08T13:50:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.691+0000 7ffa848af700 1 -- 172.21.15.57:0/4063745588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa801a3a50 con 0x7ffa80068a50 2023-10-08T13:50:34.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.692+0000 7ffa848af700 1 -- 172.21.15.57:0/4063745588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa80108d30 con 0x7ffa80068a50 2023-10-08T13:50:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.693+0000 7ffa6f7fe700 1 -- 172.21.15.57:0/4063745588 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7ffa70010460 con 0x7ffa80068a50 2023-10-08T13:50:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.693+0000 7ffa6f7fe700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa680a15e0 0x7ffa680a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.693+0000 7ffa6f7fe700 1 -- 172.21.15.57:0/4063745588 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7ffa700cd580 con 0x7ffa80068a50 2023-10-08T13:50:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.694+0000 7ffa7dd9b700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa680a15e0 0x7ffa680a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.694+0000 7ffa7dd9b700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa680a15e0 0x7ffa680a3aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa74005fd0 tx=0x7ffa7400a560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.697+0000 7ffa6f7fe700 1 -- 172.21.15.57:0/4063745588 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffa70096690 con 0x7ffa80068a50 2023-10-08T13:50:34.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.926+0000 7ffa848af700 1 -- 172.21.15.57:0/4063745588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffa80101ee0 con 0x7ffa80068a50 2023-10-08T13:50:34.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.926+0000 7ffa6f7fe700 1 -- 172.21.15.57:0/4063745588 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa70020020 con 0x7ffa80068a50 2023-10-08T13:50:34.927 INFO:teuthology.orchestra.run.smithi057.stdout:111669149752 2023-10-08T13:50:34.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 -- 172.21.15.57:0/4063745588 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa680a15e0 msgr2=0x7ffa680a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:34.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa680a15e0 0x7ffa680a3aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa74005fd0 tx=0x7ffa7400a560 comp rx=0 tx=0).stop 2023-10-08T13:50:34.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 -- 172.21.15.57:0/4063745588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa80068a50 msgr2=0x7ffa8019c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:34.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa80068a50 0x7ffa8019c9b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000d950 tx=0x7ffa7000dd10 comp rx=0 tx=0).stop 2023-10-08T13:50:34.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 -- 172.21.15.57:0/4063745588 shutdown_connections 2023-10-08T13:50:34.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa80105670 0x7ffa80196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:34.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa80068a50 0x7ffa8019c9b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:34.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa680a15e0 0x7ffa680a3aa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:34.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 --2- 172.21.15.57:0/4063745588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80069410 0x7ffa8019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:34.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 -- 172.21.15.57:0/4063745588 >> 172.21.15.57:0/4063745588 conn(0x7ffa800766c0 msgr2=0x7ffa800fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:34.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 -- 172.21.15.57:0/4063745588 shutdown_connections 2023-10-08T13:50:34.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:34.928+0000 7ffa848af700 1 -- 172.21.15.57:0/4063745588 wait complete. 2023-10-08T13:50:35.970 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149751 got 111669149752 for osd.3 2023-10-08T13:50:35.970 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:36.008 INFO:teuthology.orchestra.run.smithi057.stdout:193273528363 2023-10-08T13:50:36.009 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-08T13:50:36.077 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:35 smithi057 ceph-mon[99476]: pgmap v268: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2023-10-08T13:50:36.077 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:35 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4063745588' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-08T13:50:36.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:35 smithi115 ceph-mon[104408]: pgmap v268: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2023-10-08T13:50:36.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:35 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4063745588' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-08T13:50:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.220+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2224565556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd40ffb30 msgr2=0x7f5cd410d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.220+0000 7f5cd92e2700 1 --2- 172.21.15.57:0/2224565556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd40ffb30 0x7f5cd410d5c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00b0a0 tx=0x7f5ccc00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.220+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2224565556 shutdown_connections 2023-10-08T13:50:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.220+0000 7f5cd92e2700 1 --2- 172.21.15.57:0/2224565556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd40ffb30 0x7f5cd410d5c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.220+0000 7f5cd92e2700 1 --2- 172.21.15.57:0/2224565556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd41058d0 0x7f5cd4105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.220+0000 7f5cd92e2700 1 --2- 172.21.15.57:0/2224565556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd40ff190 0x7f5cd40ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:36.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.220+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2224565556 >> 172.21.15.57:0/2224565556 conn(0x7f5cd4074a10 msgr2=0x7f5cd4074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:36.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.220+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2224565556 shutdown_connections 2023-10-08T13:50:36.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.220+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2224565556 wait complete. 2023-10-08T13:50:36.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd92e2700 1 Processor -- start 2023-10-08T13:50:36.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd92e2700 1 -- start start 2023-10-08T13:50:36.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd92e2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd40ff190 0x7f5cd4100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd92e2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd40ffb30 0x7f5cd41008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd92e2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd41058d0 0x7f5cd4100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd92e2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41a5090 con 0x7f5cd40ff190 2023-10-08T13:50:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd92e2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41a5200 con 0x7f5cd40ffb30 2023-10-08T13:50:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd92e2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41a53c0 con 0x7f5cd41058d0 2023-10-08T13:50:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd40ff190 0x7f5cd4100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd40ff190 0x7f5cd4100370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54840/0 (socket says 172.21.15.57:54840) 2023-10-08T13:50:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.221+0000 7f5cd2ffd700 1 -- 172.21.15.57:0/2949986885 learned_addr learned my addr 172.21.15.57:0/2949986885 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cd37fe700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd41058d0 0x7f5cd4100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cd27fc700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd40ffb30 0x7f5cd41008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cd2ffd700 1 -- 172.21.15.57:0/2949986885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd41058d0 msgr2=0x7f5cd4100df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cd2ffd700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd41058d0 0x7f5cd4100df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cd2ffd700 1 -- 172.21.15.57:0/2949986885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd40ffb30 msgr2=0x7f5cd41008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cd2ffd700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd40ffb30 0x7f5cd41008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cd2ffd700 1 -- 172.21.15.57:0/2949986885 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc009d00 con 0x7f5cd40ff190 2023-10-08T13:50:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cd2ffd700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd40ff190 0x7f5cd4100370 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400d980 tx=0x7f5cc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cbbfff700 1 -- 172.21.15.57:0/2949986885 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4004020 con 0x7f5cd40ff190 2023-10-08T13:50:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2949986885 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd41a56a0 con 0x7f5cd40ff190 2023-10-08T13:50:36.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cbbfff700 1 -- 172.21.15.57:0/2949986885 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cc400f690 con 0x7f5cd40ff190 2023-10-08T13:50:36.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cbbfff700 1 -- 172.21.15.57:0/2949986885 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4003a60 con 0x7f5cd40ff190 2023-10-08T13:50:36.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.222+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2949986885 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd41a5ba0 con 0x7f5cd40ff190 2023-10-08T13:50:36.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.223+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2949986885 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd4108d30 con 0x7f5cd40ff190 2023-10-08T13:50:36.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.225+0000 7f5cbbfff700 1 -- 172.21.15.57:0/2949986885 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f5cc400fb00 con 0x7f5cd40ff190 2023-10-08T13:50:36.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.225+0000 7f5cbbfff700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cbc0a1560 0x7f5cbc0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:36.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.225+0000 7f5cd27fc700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cbc0a1560 0x7f5cbc0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:36.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.225+0000 7f5cbbfff700 1 -- 172.21.15.57:0/2949986885 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f5cc40cceb0 con 0x7f5cd40ff190 2023-10-08T13:50:36.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.226+0000 7f5cd27fc700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cbc0a1560 0x7f5cbc0a3a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4101a50 tx=0x7f5cc800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.230+0000 7f5cbbfff700 1 -- 172.21.15.57:0/2949986885 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5cc4095fc0 con 0x7f5cd40ff190 2023-10-08T13:50:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:35 smithi099 ceph-mon[103849]: pgmap v268: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2023-10-08T13:50:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:35 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4063745588' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-08T13:50:36.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.434+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2949986885 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5cd4067c30 con 0x7f5cd40ff190 2023-10-08T13:50:36.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.435+0000 7f5cbbfff700 1 -- 172.21.15.57:0/2949986885 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5cc40995e0 con 0x7f5cd40ff190 2023-10-08T13:50:36.436 INFO:teuthology.orchestra.run.smithi057.stdout:81604378683 2023-10-08T13:50:36.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2949986885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cbc0a1560 msgr2=0x7f5cbc0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:36.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cbc0a1560 0x7f5cbc0a3a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4101a50 tx=0x7f5cc800b410 comp rx=0 tx=0).stop 2023-10-08T13:50:36.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2949986885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd40ff190 msgr2=0x7f5cd4100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:36.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd40ff190 0x7f5cd4100370 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400d980 tx=0x7f5cc400dc90 comp rx=0 tx=0).stop 2023-10-08T13:50:36.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2949986885 shutdown_connections 2023-10-08T13:50:36.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd41058d0 0x7f5cd4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd40ff190 0x7f5cd4100370 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cbc0a1560 0x7f5cbc0a3a20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 --2- 172.21.15.57:0/2949986885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd40ffb30 0x7f5cd41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2949986885 >> 172.21.15.57:0/2949986885 conn(0x7f5cd4074a10 msgr2=0x7f5cd40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2949986885 shutdown_connections 2023-10-08T13:50:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:36.437+0000 7f5cd92e2700 1 -- 172.21.15.57:0/2949986885 wait complete. 2023-10-08T13:50:36.636 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.101+0000 7f435fc07700 1 -- 172.21.15.57:0/1186067415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43581067a0 msgr2=0x7f4358106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.101+0000 7f435fc07700 1 --2- 172.21.15.57:0/1186067415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43581067a0 0x7f4358106b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4354009990 tx=0x7f4354009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.101+0000 7f435fc07700 1 -- 172.21.15.57:0/1186067415 shutdown_connections 2023-10-08T13:50:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.101+0000 7f435fc07700 1 --2- 172.21.15.57:0/1186067415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4358068b30 0x7f435810d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.101+0000 7f435fc07700 1 --2- 172.21.15.57:0/1186067415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43581067a0 0x7f4358106b80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.101+0000 7f435fc07700 1 --2- 172.21.15.57:0/1186067415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4358068190 0x7f43580685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.101+0000 7f435fc07700 1 -- 172.21.15.57:0/1186067415 >> 172.21.15.57:0/1186067415 conn(0x7f4358075790 msgr2=0x7f4358075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.101+0000 7f435fc07700 1 -- 172.21.15.57:0/1186067415 shutdown_connections 2023-10-08T13:50:37.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.102+0000 7f435fc07700 1 -- 172.21.15.57:0/1186067415 wait complete. 2023-10-08T13:50:37.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.102+0000 7f435fc07700 1 Processor -- start 2023-10-08T13:50:37.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.102+0000 7f435fc07700 1 -- start start 2023-10-08T13:50:37.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435fc07700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4358068190 0x7f435819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435fc07700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4358068b30 0x7f435819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435fc07700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43581067a0 0x7f4358196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435fc07700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4358197150 con 0x7f43581067a0 2023-10-08T13:50:37.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435fc07700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43581972c0 con 0x7f4358068b30 2023-10-08T13:50:37.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435fc07700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4358197480 con 0x7f4358068190 2023-10-08T13:50:37.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435e1a4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43581067a0 0x7f4358196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435e1a4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43581067a0 0x7f4358196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54856/0 (socket says 172.21.15.57:54856) 2023-10-08T13:50:37.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435e1a4700 1 -- 172.21.15.57:0/1503350146 learned_addr learned my addr 172.21.15.57:0/1503350146 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:37.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435d1a2700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4358068b30 0x7f435819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435d9a3700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4358068190 0x7f435819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435e1a4700 1 -- 172.21.15.57:0/1503350146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4358068190 msgr2=0x7f435819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435e1a4700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4358068190 0x7f435819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435e1a4700 1 -- 172.21.15.57:0/1503350146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4358068b30 msgr2=0x7f435819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435e1a4700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4358068b30 0x7f435819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.103+0000 7f435e1a4700 1 -- 172.21.15.57:0/1503350146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43540095f0 con 0x7f43581067a0 2023-10-08T13:50:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.104+0000 7f435e1a4700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43581067a0 0x7f4358196af0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4350007e60 tx=0x7f435000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.104+0000 7f434effd700 1 -- 172.21.15.57:0/1503350146 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350004bb0 con 0x7f43581067a0 2023-10-08T13:50:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.104+0000 7f435fc07700 1 -- 172.21.15.57:0/1503350146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4358197760 con 0x7f43581067a0 2023-10-08T13:50:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.104+0000 7f434effd700 1 -- 172.21.15.57:0/1503350146 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4350007b30 con 0x7f43581067a0 2023-10-08T13:50:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.104+0000 7f434effd700 1 -- 172.21.15.57:0/1503350146 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350011b30 con 0x7f43581067a0 2023-10-08T13:50:37.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.104+0000 7f435fc07700 1 -- 172.21.15.57:0/1503350146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43580ff920 con 0x7f43581067a0 2023-10-08T13:50:37.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.105+0000 7f435fc07700 1 -- 172.21.15.57:0/1503350146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4358108d50 con 0x7f43581067a0 2023-10-08T13:50:37.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.107+0000 7f434effd700 1 -- 172.21.15.57:0/1503350146 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f4350004d10 con 0x7f43581067a0 2023-10-08T13:50:37.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.108+0000 7f434effd700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43440a1560 0x7f43440a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.108+0000 7f434effd700 1 -- 172.21.15.57:0/1503350146 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f43500d0e20 con 0x7f43581067a0 2023-10-08T13:50:37.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.108+0000 7f435d9a3700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43440a1560 0x7f43440a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.109+0000 7f435d9a3700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43440a1560 0x7f43440a3a20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f435400b5c0 tx=0x7f4354005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:37.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.111+0000 7f434effd700 1 -- 172.21.15.57:0/1503350146 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4350099fe0 con 0x7f43581067a0 2023-10-08T13:50:37.137 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:36 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2949986885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-08T13:50:37.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:36 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2949986885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-08T13:50:37.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:36 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2949986885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-08T13:50:37.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.330+0000 7f435fc07700 1 -- 172.21.15.57:0/1503350146 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4358197fb0 con 0x7f43581067a0 2023-10-08T13:50:37.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.330+0000 7f434effd700 1 -- 172.21.15.57:0/1503350146 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f435009d600 con 0x7f43581067a0 2023-10-08T13:50:37.331 INFO:teuthology.orchestra.run.smithi057.stdout:163208757296 2023-10-08T13:50:37.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 -- 172.21.15.57:0/1503350146 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43440a1560 msgr2=0x7f43440a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43440a1560 0x7f43440a3a20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f435400b5c0 tx=0x7f4354005960 comp rx=0 tx=0).stop 2023-10-08T13:50:37.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 -- 172.21.15.57:0/1503350146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43581067a0 msgr2=0x7f4358196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43581067a0 0x7f4358196af0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4350007e60 tx=0x7f435000c330 comp rx=0 tx=0).stop 2023-10-08T13:50:37.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 -- 172.21.15.57:0/1503350146 shutdown_connections 2023-10-08T13:50:37.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43581067a0 0x7f4358196af0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4358068190 0x7f435819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43440a1560 0x7f43440a3a20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 --2- 172.21.15.57:0/1503350146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4358068b30 0x7f435819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 -- 172.21.15.57:0/1503350146 >> 172.21.15.57:0/1503350146 conn(0x7f4358075790 msgr2=0x7f43580fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:37.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 -- 172.21.15.57:0/1503350146 shutdown_connections 2023-10-08T13:50:37.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.332+0000 7f435fc07700 1 -- 172.21.15.57:0/1503350146 wait complete. 2023-10-08T13:50:37.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.457+0000 7fc4b070e700 1 -- 172.21.15.57:0/3940374580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a80ff750 msgr2=0x7fc4a80ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.457+0000 7fc4b070e700 1 --2- 172.21.15.57:0/3940374580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a80ff750 0x7fc4a80ffbd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc498009990 tx=0x7fc498009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.457+0000 7fc4b070e700 1 -- 172.21.15.57:0/3940374580 shutdown_connections 2023-10-08T13:50:37.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.457+0000 7fc4b070e700 1 --2- 172.21.15.57:0/3940374580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a8100110 0x7fc4a810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.457+0000 7fc4b070e700 1 --2- 172.21.15.57:0/3940374580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4a81056b0 0x7fc4a8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.457+0000 7fc4b070e700 1 --2- 172.21.15.57:0/3940374580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a80ff750 0x7fc4a80ffbd0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.457+0000 7fc4b070e700 1 -- 172.21.15.57:0/3940374580 >> 172.21.15.57:0/3940374580 conn(0x7fc4a8075790 msgr2=0x7fc4a8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:37.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.458+0000 7fc4b070e700 1 -- 172.21.15.57:0/3940374580 shutdown_connections 2023-10-08T13:50:37.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.458+0000 7fc4b070e700 1 -- 172.21.15.57:0/3940374580 wait complete. 2023-10-08T13:50:37.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.458+0000 7fc4b070e700 1 Processor -- start 2023-10-08T13:50:37.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.458+0000 7fc4b070e700 1 -- start start 2023-10-08T13:50:37.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4b070e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a80ff750 0x7fc4a8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4b070e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a8100110 0x7fc4a8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4b070e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4a81056b0 0x7fc4a81a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4b070e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8198820 con 0x7fc4a8100110 2023-10-08T13:50:37.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4b070e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a81a3050 con 0x7fc4a81056b0 2023-10-08T13:50:37.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4b070e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a81a3190 con 0x7fc4a80ff750 2023-10-08T13:50:37.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4adca9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a8100110 0x7fc4a8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4adca9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a8100110 0x7fc4a8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54870/0 (socket says 172.21.15.57:54870) 2023-10-08T13:50:37.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4adca9700 1 -- 172.21.15.57:0/1767814484 learned_addr learned my addr 172.21.15.57:0/1767814484 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:37.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4aecab700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4a81056b0 0x7fc4a81a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4ae4aa700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a80ff750 0x7fc4a8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4adca9700 1 -- 172.21.15.57:0/1767814484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a80ff750 msgr2=0x7fc4a8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4adca9700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a80ff750 0x7fc4a8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4adca9700 1 -- 172.21.15.57:0/1767814484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4a81056b0 msgr2=0x7fc4a81a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4adca9700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4a81056b0 0x7fc4a81a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4adca9700 1 -- 172.21.15.57:0/1767814484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980095f0 con 0x7fc4a8100110 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc4adca9700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a8100110 0x7fc4a8197fd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc498000c00 tx=0x7fc49800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.459+0000 7fc49f7fe700 1 -- 172.21.15.57:0/1767814484 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49801d070 con 0x7fc4a8100110 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.460+0000 7fc49f7fe700 1 -- 172.21.15.57:0/1767814484 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc498005030 con 0x7fc4a8100110 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.460+0000 7fc4b070e700 1 -- 172.21.15.57:0/1767814484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a81a3390 con 0x7fc4a8100110 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.460+0000 7fc49f7fe700 1 -- 172.21.15.57:0/1767814484 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498004a90 con 0x7fc4a8100110 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.460+0000 7fc4b070e700 1 -- 172.21.15.57:0/1767814484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a81a37a0 con 0x7fc4a8100110 2023-10-08T13:50:37.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.460+0000 7fc4b070e700 1 -- 172.21.15.57:0/1767814484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a8107cd0 con 0x7fc4a8100110 2023-10-08T13:50:37.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.462+0000 7fc49f7fe700 1 -- 172.21.15.57:0/1767814484 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fc498004bf0 con 0x7fc4a8100110 2023-10-08T13:50:37.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.462+0000 7fc49f7fe700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4940a1560 0x7fc4940a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.462+0000 7fc49f7fe700 1 -- 172.21.15.57:0/1767814484 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fc4980ce3a0 con 0x7fc4a8100110 2023-10-08T13:50:37.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.462+0000 7fc4ae4aa700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4940a1560 0x7fc4940a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.463+0000 7fc4ae4aa700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4940a1560 0x7fc4940a3a20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4005950 tx=0x7fc4a400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:37.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.466+0000 7fc49f7fe700 1 -- 172.21.15.57:0/1767814484 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc4980975e0 con 0x7fc4a8100110 2023-10-08T13:50:37.562 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378682 got 81604378683 for osd.2 2023-10-08T13:50:37.562 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:37.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.688+0000 7fc4b070e700 1 -- 172.21.15.57:0/1767814484 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc4a8100df0 con 0x7fc4a8100110 2023-10-08T13:50:37.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.689+0000 7fc49f7fe700 1 -- 172.21.15.57:0/1767814484 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc498026020 con 0x7fc4a8100110 2023-10-08T13:50:37.690 INFO:teuthology.orchestra.run.smithi057.stdout:214748364839 2023-10-08T13:50:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 -- 172.21.15.57:0/1767814484 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4940a1560 msgr2=0x7fc4940a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4940a1560 0x7fc4940a3a20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4005950 tx=0x7fc4a400b410 comp rx=0 tx=0).stop 2023-10-08T13:50:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 -- 172.21.15.57:0/1767814484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a8100110 msgr2=0x7fc4a8197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a8100110 0x7fc4a8197fd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc498000c00 tx=0x7fc49800f690 comp rx=0 tx=0).stop 2023-10-08T13:50:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 -- 172.21.15.57:0/1767814484 shutdown_connections 2023-10-08T13:50:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4940a1560 0x7fc4940a3a20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4a81056b0 0x7fc4a81a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a80ff750 0x7fc4a8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 --2- 172.21.15.57:0/1767814484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4a8100110 0x7fc4a8197fd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 -- 172.21.15.57:0/1767814484 >> 172.21.15.57:0/1767814484 conn(0x7fc4a8075790 msgr2=0x7fc4a80fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 -- 172.21.15.57:0/1767814484 shutdown_connections 2023-10-08T13:50:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.691+0000 7fc4b070e700 1 -- 172.21.15.57:0/1767814484 wait complete. 2023-10-08T13:50:37.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.825+0000 7f6c1230b700 1 -- 172.21.15.57:0/1742205739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c0c068b30 msgr2=0x7f6c0c10d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.825+0000 7f6c1230b700 1 --2- 172.21.15.57:0/1742205739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c0c068b30 0x7f6c0c10d670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400b0a0 tx=0x7f6c0400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.825+0000 7f6c1230b700 1 -- 172.21.15.57:0/1742205739 shutdown_connections 2023-10-08T13:50:37.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.825+0000 7f6c1230b700 1 --2- 172.21.15.57:0/1742205739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c0c068b30 0x7f6c0c10d670 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.825+0000 7f6c1230b700 1 --2- 172.21.15.57:0/1742205739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c0c1067a0 0x7f6c0c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.825+0000 7f6c1230b700 1 --2- 172.21.15.57:0/1742205739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c0c068190 0x7f6c0c0685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.825+0000 7f6c1230b700 1 -- 172.21.15.57:0/1742205739 >> 172.21.15.57:0/1742205739 conn(0x7f6c0c075790 msgr2=0x7f6c0c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:37.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.825+0000 7f6c1230b700 1 -- 172.21.15.57:0/1742205739 shutdown_connections 2023-10-08T13:50:37.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.825+0000 7f6c1230b700 1 -- 172.21.15.57:0/1742205739 wait complete. 2023-10-08T13:50:37.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c1230b700 1 Processor -- start 2023-10-08T13:50:37.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c1230b700 1 -- start start 2023-10-08T13:50:37.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c1230b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c0c068190 0x7f6c0c0ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c1230b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c0c068b30 0x7f6c0c0ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c1230b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c0c1067a0 0x7f6c0c1002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c1230b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1a5080 con 0x7f6c0c1067a0 2023-10-08T13:50:37.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c1230b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1a51f0 con 0x7f6c0c068190 2023-10-08T13:50:37.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c1230b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1a53b0 con 0x7f6c0c068b30 2023-10-08T13:50:37.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c108a8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c0c1067a0 0x7f6c0c1002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c108a8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c0c1067a0 0x7f6c0c1002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54886/0 (socket says 172.21.15.57:54886) 2023-10-08T13:50:37.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c108a8700 1 -- 172.21.15.57:0/410336344 learned_addr learned my addr 172.21.15.57:0/410336344 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:37.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.826+0000 7f6c0bfff700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c0c068190 0x7f6c0c0ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c0b7fe700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c0c068b30 0x7f6c0c0ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c108a8700 1 -- 172.21.15.57:0/410336344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c0c068b30 msgr2=0x7f6c0c0ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c108a8700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c0c068b30 0x7f6c0c0ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c108a8700 1 -- 172.21.15.57:0/410336344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c0c068190 msgr2=0x7f6c0c0ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c108a8700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c0c068190 0x7f6c0c0ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c108a8700 1 -- 172.21.15.57:0/410336344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04009d00 con 0x7f6c0c1067a0 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c0b7fe700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c0c068b30 0x7f6c0c0ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c108a8700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c0c1067a0 0x7f6c0c1002a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400b070 tx=0x7f6c04007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c097fa700 1 -- 172.21.15.57:0/410336344 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400e040 con 0x7f6c0c1067a0 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c097fa700 1 -- 172.21.15.57:0/410336344 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c04009440 con 0x7f6c0c1067a0 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c1230b700 1 -- 172.21.15.57:0/410336344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c1a5630 con 0x7f6c0c1067a0 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c097fa700 1 -- 172.21.15.57:0/410336344 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0401b450 con 0x7f6c0c1067a0 2023-10-08T13:50:37.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.827+0000 7f6c1230b700 1 -- 172.21.15.57:0/410336344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c1a5aa0 con 0x7f6c0c1067a0 2023-10-08T13:50:37.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.828+0000 7f6c1230b700 1 -- 172.21.15.57:0/410336344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c108d50 con 0x7f6c0c1067a0 2023-10-08T13:50:37.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.830+0000 7f6c097fa700 1 -- 172.21.15.57:0/410336344 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f6c04004cf0 con 0x7f6c0c1067a0 2023-10-08T13:50:37.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.830+0000 7f6c097fa700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6bf40a1560 0x7f6bf40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:37.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.830+0000 7f6c0bfff700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6bf40a1560 0x7f6bf40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:37.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.830+0000 7f6c097fa700 1 -- 172.21.15.57:0/410336344 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f6c040cef40 con 0x7f6c0c1067a0 2023-10-08T13:50:37.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.831+0000 7f6c0bfff700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6bf40a1560 0x7f6bf40a3a20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc007b60 tx=0x7f6bfc0073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:37.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:37.834+0000 7f6c097fa700 1 -- 172.21.15.57:0/410336344 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6c040980e0 con 0x7f6c0c1067a0 2023-10-08T13:50:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.023+0000 7fdd651d0700 1 -- 172.21.15.57:0/1553578735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd60069450 msgr2=0x7fdd6010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.023+0000 7fdd651d0700 1 --2- 172.21.15.57:0/1553578735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd60069450 0x7fdd6010d650 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800b0d0 tx=0x7fdd5800b3e0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.023+0000 7fdd651d0700 1 -- 172.21.15.57:0/1553578735 shutdown_connections 2023-10-08T13:50:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.023+0000 7fdd651d0700 1 --2- 172.21.15.57:0/1553578735 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd60069450 0x7fdd6010d650 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.023+0000 7fdd651d0700 1 --2- 172.21.15.57:0/1553578735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60105670 0x7fdd60105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.023+0000 7fdd651d0700 1 --2- 172.21.15.57:0/1553578735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd60068a90 0x7fdd60068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.023+0000 7fdd651d0700 1 -- 172.21.15.57:0/1553578735 >> 172.21.15.57:0/1553578735 conn(0x7fdd600766c0 msgr2=0x7fdd60076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:38.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.024+0000 7fdd651d0700 1 -- 172.21.15.57:0/1553578735 shutdown_connections 2023-10-08T13:50:38.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.024+0000 7fdd651d0700 1 -- 172.21.15.57:0/1553578735 wait complete. 2023-10-08T13:50:38.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.024+0000 7fdd651d0700 1 Processor -- start 2023-10-08T13:50:38.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.024+0000 7fdd651d0700 1 -- start start 2023-10-08T13:50:38.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd651d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd60068a90 0x7fdd600ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd651d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60069450 0x7fdd600ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd651d0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd60105670 0x7fdd60100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd651d0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601a5140 con 0x7fdd60068a90 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd651d0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601a52b0 con 0x7fdd60069450 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd651d0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601a5470 con 0x7fdd60105670 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd60068a90 0x7fdd600ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd60068a90 0x7fdd600ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54896/0 (socket says 172.21.15.57:54896) 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5ed9d700 1 -- 172.21.15.57:0/2847112112 learned_addr learned my addr 172.21.15.57:0/2847112112 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5e59c700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60069450 0x7fdd600ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5f59e700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd60105670 0x7fdd60100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5ed9d700 1 -- 172.21.15.57:0/2847112112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd60105670 msgr2=0x7fdd60100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5ed9d700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd60105670 0x7fdd60100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5ed9d700 1 -- 172.21.15.57:0/2847112112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60069450 msgr2=0x7fdd600ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5ed9d700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60069450 0x7fdd600ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5ed9d700 1 -- 172.21.15.57:0/2847112112 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58009d00 con 0x7fdd60068a90 2023-10-08T13:50:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5f59e700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd60105670 0x7fdd60100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.025+0000 7fdd5ed9d700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd60068a90 0x7fdd600ff7b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000d950 tx=0x7fdd5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.026+0000 7fdd47fff700 1 -- 172.21.15.57:0/2847112112 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50009940 con 0x7fdd60068a90 2023-10-08T13:50:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.026+0000 7fdd651d0700 1 -- 172.21.15.57:0/2847112112 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd601a5750 con 0x7fdd60068a90 2023-10-08T13:50:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.026+0000 7fdd47fff700 1 -- 172.21.15.57:0/2847112112 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd50010460 con 0x7fdd60068a90 2023-10-08T13:50:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.026+0000 7fdd47fff700 1 -- 172.21.15.57:0/2847112112 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5000f620 con 0x7fdd60068a90 2023-10-08T13:50:38.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.026+0000 7fdd651d0700 1 -- 172.21.15.57:0/2847112112 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd601a5c50 con 0x7fdd60068a90 2023-10-08T13:50:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.027+0000 7fdd651d0700 1 -- 172.21.15.57:0/2847112112 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd60108d30 con 0x7fdd60068a90 2023-10-08T13:50:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.028+0000 7fdd47fff700 1 -- 172.21.15.57:0/2847112112 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fdd500108d0 con 0x7fdd60068a90 2023-10-08T13:50:38.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.028+0000 7fdd47fff700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd480a1560 0x7fdd480a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.029+0000 7fdd5e59c700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd480a1560 0x7fdd480a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.029+0000 7fdd47fff700 1 -- 172.21.15.57:0/2847112112 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fdd500cd0d0 con 0x7fdd60068a90 2023-10-08T13:50:38.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.029+0000 7fdd5e59c700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd480a1560 0x7fdd480a3a20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdd60100ee0 tx=0x7fdd540073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.032+0000 7fdd47fff700 1 -- 172.21.15.57:0/2847112112 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdd500961e0 con 0x7fdd60068a90 2023-10-08T13:50:38.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.039+0000 7f6c1230b700 1 -- 172.21.15.57:0/410336344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6c0c100d00 con 0x7f6c0c1067a0 2023-10-08T13:50:38.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.039+0000 7f6c097fa700 1 -- 172.21.15.57:0/410336344 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c04011390 con 0x7f6c0c1067a0 2023-10-08T13:50:38.040 INFO:teuthology.orchestra.run.smithi057.stdout:141733920820 2023-10-08T13:50:38.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 -- 172.21.15.57:0/410336344 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6bf40a1560 msgr2=0x7f6bf40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6bf40a1560 0x7f6bf40a3a20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc007b60 tx=0x7f6bfc0073d0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 -- 172.21.15.57:0/410336344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c0c1067a0 msgr2=0x7f6c0c1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c0c1067a0 0x7f6c0c1002a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400b070 tx=0x7f6c04007870 comp rx=0 tx=0).stop 2023-10-08T13:50:38.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 -- 172.21.15.57:0/410336344 shutdown_connections 2023-10-08T13:50:38.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c0c1067a0 0x7f6c0c1002a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6bf40a1560 0x7f6bf40a3a20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c0c068190 0x7f6c0c0ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 --2- 172.21.15.57:0/410336344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c0c068b30 0x7f6c0c0ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 -- 172.21.15.57:0/410336344 >> 172.21.15.57:0/410336344 conn(0x7f6c0c075790 msgr2=0x7f6c0c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:38.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.041+0000 7f6c1230b700 1 -- 172.21.15.57:0/410336344 shutdown_connections 2023-10-08T13:50:38.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.042+0000 7f6c1230b700 1 -- 172.21.15.57:0/410336344 wait complete. 2023-10-08T13:50:38.088 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757294 got 163208757296 for osd.5 2023-10-08T13:50:38.088 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:38.169 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:37 smithi057 ceph-mon[99476]: pgmap v269: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2023-10-08T13:50:38.169 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:37 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1503350146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-08T13:50:38.169 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:37 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1767814484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-08T13:50:38.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:37 smithi115 ceph-mon[104408]: pgmap v269: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2023-10-08T13:50:38.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:37 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1503350146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-08T13:50:38.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:37 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1767814484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-08T13:50:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.262+0000 7fdd651d0700 1 -- 172.21.15.57:0/2847112112 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdd60100c90 con 0x7fdd60068a90 2023-10-08T13:50:38.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.263+0000 7fdd47fff700 1 -- 172.21.15.57:0/2847112112 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdd50016090 con 0x7fdd60068a90 2023-10-08T13:50:38.264 INFO:teuthology.orchestra.run.smithi057.stdout:60129542209 2023-10-08T13:50:38.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 -- 172.21.15.57:0/2847112112 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd480a1560 msgr2=0x7fdd480a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd480a1560 0x7fdd480a3a20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdd60100ee0 tx=0x7fdd540073d0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 -- 172.21.15.57:0/2847112112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd60068a90 msgr2=0x7fdd600ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd60068a90 0x7fdd600ff7b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000d950 tx=0x7fdd5000dc60 comp rx=0 tx=0).stop 2023-10-08T13:50:38.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 -- 172.21.15.57:0/2847112112 shutdown_connections 2023-10-08T13:50:38.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd60105670 0x7fdd60100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd60068a90 0x7fdd600ff7b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd480a1560 0x7fdd480a3a20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 --2- 172.21.15.57:0/2847112112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd60069450 0x7fdd600ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 -- 172.21.15.57:0/2847112112 >> 172.21.15.57:0/2847112112 conn(0x7fdd600766c0 msgr2=0x7fdd600fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:38.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 -- 172.21.15.57:0/2847112112 shutdown_connections 2023-10-08T13:50:38.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.265+0000 7fdd651d0700 1 -- 172.21.15.57:0/2847112112 wait complete. 2023-10-08T13:50:38.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:37 smithi099 ceph-mon[103849]: pgmap v269: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2023-10-08T13:50:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:37 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1503350146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-08T13:50:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:37 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1767814484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-08T13:50:38.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.456+0000 7f5be2c49700 1 -- 172.21.15.57:0/1987225080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bdc068190 msgr2=0x7f5bdc0685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.456+0000 7f5be2c49700 1 --2- 172.21.15.57:0/1987225080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bdc068190 0x7f5bdc0685f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009990 tx=0x7f5bc8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.456+0000 7f5be2c49700 1 -- 172.21.15.57:0/1987225080 shutdown_connections 2023-10-08T13:50:38.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.456+0000 7f5be2c49700 1 --2- 172.21.15.57:0/1987225080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bdc068b30 0x7f5bdc10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.456+0000 7f5be2c49700 1 --2- 172.21.15.57:0/1987225080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bdc1067c0 0x7f5bdc106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.456+0000 7f5be2c49700 1 --2- 172.21.15.57:0/1987225080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bdc068190 0x7f5bdc0685f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.456+0000 7f5be2c49700 1 -- 172.21.15.57:0/1987225080 >> 172.21.15.57:0/1987225080 conn(0x7f5bdc075790 msgr2=0x7f5bdc075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.456+0000 7f5be2c49700 1 -- 172.21.15.57:0/1987225080 shutdown_connections 2023-10-08T13:50:38.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.456+0000 7f5be2c49700 1 -- 172.21.15.57:0/1987225080 wait complete. 2023-10-08T13:50:38.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.457+0000 7f5be2c49700 1 Processor -- start 2023-10-08T13:50:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.457+0000 7f5be2c49700 1 -- start start 2023-10-08T13:50:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be2c49700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bdc068190 0x7f5bdc19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be2c49700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bdc068b30 0x7f5bdc19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be2c49700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bdc1067c0 0x7f5bdc196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be2c49700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc197080 con 0x7f5bdc1067c0 2023-10-08T13:50:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be2c49700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc1971f0 con 0x7f5bdc068b30 2023-10-08T13:50:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be2c49700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc1973b0 con 0x7f5bdc068190 2023-10-08T13:50:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be11e6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bdc1067c0 0x7f5bdc196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be11e6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bdc1067c0 0x7f5bdc196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54912/0 (socket says 172.21.15.57:54912) 2023-10-08T13:50:38.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be11e6700 1 -- 172.21.15.57:0/3394387211 learned_addr learned my addr 172.21.15.57:0/3394387211 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be09e5700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bdc068190 0x7f5bdc19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5bdbfff700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bdc068b30 0x7f5bdc19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be11e6700 1 -- 172.21.15.57:0/3394387211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bdc068190 msgr2=0x7f5bdc19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be11e6700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bdc068190 0x7f5bdc19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be11e6700 1 -- 172.21.15.57:0/3394387211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bdc068b30 msgr2=0x7f5bdc19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be11e6700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bdc068b30 0x7f5bdc19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be11e6700 1 -- 172.21.15.57:0/3394387211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80095f0 con 0x7f5bdc1067c0 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5be11e6700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bdc1067c0 0x7f5bdc196a20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400ef30 tx=0x7f5bd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5bd9ffb700 1 -- 172.21.15.57:0/3394387211 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4018070 con 0x7f5bdc1067c0 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5bd9ffb700 1 -- 172.21.15.57:0/3394387211 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bd400f040 con 0x7f5bdc1067c0 2023-10-08T13:50:38.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.458+0000 7f5bd9ffb700 1 -- 172.21.15.57:0/3394387211 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd400cdb0 con 0x7f5bdc1067c0 2023-10-08T13:50:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.459+0000 7f5be2c49700 1 -- 172.21.15.57:0/3394387211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc197630 con 0x7f5bdc1067c0 2023-10-08T13:50:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.459+0000 7f5be2c49700 1 -- 172.21.15.57:0/3394387211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc1a38d0 con 0x7f5bdc1067c0 2023-10-08T13:50:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.460+0000 7f5be2c49700 1 -- 172.21.15.57:0/3394387211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc108de0 con 0x7f5bdc1067c0 2023-10-08T13:50:38.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.461+0000 7f5bd9ffb700 1 -- 172.21.15.57:0/3394387211 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f5bd4003a00 con 0x7f5bdc1067c0 2023-10-08T13:50:38.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.461+0000 7f5bd9ffb700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bcc0a1560 0x7f5bcc0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.461+0000 7f5be09e5700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bcc0a1560 0x7f5bcc0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.462+0000 7f5bd9ffb700 1 -- 172.21.15.57:0/3394387211 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f5bd40cced0 con 0x7f5bdc1067c0 2023-10-08T13:50:38.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.462+0000 7f5be09e5700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bcc0a1560 0x7f5bcc0a3a20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0005950 tx=0x7f5bd000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:38.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.466+0000 7f5bd9ffb700 1 -- 172.21.15.57:0/3394387211 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5bd4096070 con 0x7f5bdc1067c0 2023-10-08T13:50:38.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.619+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2380186516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a40105750 msgr2=0x7f6a40105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.619+0000 7f6a46d5c700 1 --2- 172.21.15.57:0/2380186516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a40105750 0x7f6a40105b30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009990 tx=0x7f6a34009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.619+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2380186516 shutdown_connections 2023-10-08T13:50:38.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.619+0000 7f6a46d5c700 1 --2- 172.21.15.57:0/2380186516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a40100390 0x7f6a4010c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.619+0000 7f6a46d5c700 1 --2- 172.21.15.57:0/2380186516 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a40105750 0x7f6a40105b30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.619+0000 7f6a46d5c700 1 --2- 172.21.15.57:0/2380186516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a400ff9d0 0x7f6a400ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.619+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2380186516 >> 172.21.15.57:0/2380186516 conn(0x7f6a40076580 msgr2=0x7f6a40076990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:38.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.620+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2380186516 shutdown_connections 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.620+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2380186516 wait complete. 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.620+0000 7f6a46d5c700 1 Processor -- start 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.620+0000 7f6a46d5c700 1 -- start start 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.620+0000 7f6a46d5c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a400ff9d0 0x7f6a4019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.620+0000 7f6a46d5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a40100390 0x7f6a4019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.620+0000 7f6a46d5c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a40105750 0x7f6a401958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.620+0000 7f6a46d5c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40195f20 con 0x7f6a40105750 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.620+0000 7f6a46d5c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40196090 con 0x7f6a40100390 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.620+0000 7f6a46d5c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40196250 con 0x7f6a400ff9d0 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a452f9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a40105750 0x7f6a401958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a44af8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a400ff9d0 0x7f6a4019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a452f9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a40105750 0x7f6a401958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54926/0 (socket says 172.21.15.57:54926) 2023-10-08T13:50:38.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a452f9700 1 -- 172.21.15.57:0/2612990364 learned_addr learned my addr 172.21.15.57:0/2612990364 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a3ffff700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a40100390 0x7f6a4019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a44af8700 1 -- 172.21.15.57:0/2612990364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a40100390 msgr2=0x7f6a4019bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a44af8700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a40100390 0x7f6a4019bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a44af8700 1 -- 172.21.15.57:0/2612990364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a40105750 msgr2=0x7f6a401958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a44af8700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a40105750 0x7f6a401958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a44af8700 1 -- 172.21.15.57:0/2612990364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340095f0 con 0x7f6a400ff9d0 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a452f9700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a40105750 0x7f6a401958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a44af8700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a400ff9d0 0x7f6a4019b780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a34000c00 tx=0x7f6a34009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a3dffb700 1 -- 172.21.15.57:0/2612990364 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3401d070 con 0x7f6a400ff9d0 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2612990364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a401964d0 con 0x7f6a400ff9d0 2023-10-08T13:50:38.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.621+0000 7f6a3dffb700 1 -- 172.21.15.57:0/2612990364 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a34021410 con 0x7f6a400ff9d0 2023-10-08T13:50:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.622+0000 7f6a3dffb700 1 -- 172.21.15.57:0/2612990364 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34017650 con 0x7f6a400ff9d0 2023-10-08T13:50:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.622+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2612990364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a401a2770 con 0x7f6a400ff9d0 2023-10-08T13:50:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.623+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2612990364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a40107c00 con 0x7f6a400ff9d0 2023-10-08T13:50:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.624+0000 7f6a3dffb700 1 -- 172.21.15.57:0/2612990364 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f6a34021580 con 0x7f6a400ff9d0 2023-10-08T13:50:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.625+0000 7f6a3dffb700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a300a1560 0x7f6a300a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.625+0000 7f6a3ffff700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a300a1560 0x7f6a300a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.625+0000 7f6a3dffb700 1 -- 172.21.15.57:0/2612990364 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f6a340cec80 con 0x7f6a400ff9d0 2023-10-08T13:50:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.625+0000 7f6a3ffff700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a300a1560 0x7f6a300a3a20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a40196a80 tx=0x7f6a2c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:38.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.628+0000 7f6a3dffb700 1 -- 172.21.15.57:0/2612990364 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6a34097e20 con 0x7f6a400ff9d0 2023-10-08T13:50:38.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.708+0000 7f5be2c49700 1 -- 172.21.15.57:0/3394387211 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f5bdc101f00 con 0x7f5bdc1067c0 2023-10-08T13:50:38.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.708+0000 7f5bd9ffb700 1 -- 172.21.15.57:0/3394387211 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5bd4099690 con 0x7f5bdc1067c0 2023-10-08T13:50:38.709 INFO:teuthology.orchestra.run.smithi057.stdout:249108103202 2023-10-08T13:50:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.710+0000 7f5be2c49700 1 -- 172.21.15.57:0/3394387211 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bcc0a1560 msgr2=0x7f5bcc0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.710+0000 7f5be2c49700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bcc0a1560 0x7f5bcc0a3a20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0005950 tx=0x7f5bd000b410 comp rx=0 tx=0).stop 2023-10-08T13:50:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.710+0000 7f5be2c49700 1 -- 172.21.15.57:0/3394387211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bdc1067c0 msgr2=0x7f5bdc196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.710+0000 7f5be2c49700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bdc1067c0 0x7f5bdc196a20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400ef30 tx=0x7f5bd400c3a0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.710+0000 7f5be2c49700 1 -- 172.21.15.57:0/3394387211 shutdown_connections 2023-10-08T13:50:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.710+0000 7f5be2c49700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5bdc1067c0 0x7f5bdc196a20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.710+0000 7f5be2c49700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5bdc068190 0x7f5bdc19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.710+0000 7f5be2c49700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bcc0a1560 0x7f5bcc0a3a20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.710+0000 7f5be2c49700 1 --2- 172.21.15.57:0/3394387211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bdc068b30 0x7f5bdc19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.710+0000 7f5be2c49700 1 -- 172.21.15.57:0/3394387211 >> 172.21.15.57:0/3394387211 conn(0x7f5bdc075790 msgr2=0x7f5bdc0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:38.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.711+0000 7f5be2c49700 1 -- 172.21.15.57:0/3394387211 shutdown_connections 2023-10-08T13:50:38.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.711+0000 7f5be2c49700 1 -- 172.21.15.57:0/3394387211 wait complete. 2023-10-08T13:50:38.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.884+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2612990364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f6a40196d80 con 0x7f6a400ff9d0 2023-10-08T13:50:38.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.884+0000 7f6a3dffb700 1 -- 172.21.15.57:0/2612990364 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f6a3409b440 con 0x7f6a400ff9d0 2023-10-08T13:50:38.885 INFO:teuthology.orchestra.run.smithi057.stdout:304942678042 2023-10-08T13:50:38.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.887+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2612990364 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a300a1560 msgr2=0x7f6a300a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.887+0000 7f6a46d5c700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a300a1560 0x7f6a300a3a20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a40196a80 tx=0x7f6a2c00b410 comp rx=0 tx=0).stop 2023-10-08T13:50:38.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.887+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2612990364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a400ff9d0 msgr2=0x7f6a4019b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:38.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.887+0000 7f6a46d5c700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a400ff9d0 0x7f6a4019b780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a34000c00 tx=0x7f6a34009580 comp rx=0 tx=0).stop 2023-10-08T13:50:38.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.889+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2612990364 shutdown_connections 2023-10-08T13:50:38.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.889+0000 7f6a46d5c700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a40105750 0x7f6a401958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.889+0000 7f6a46d5c700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a400ff9d0 0x7f6a4019b780 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.889+0000 7f6a46d5c700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6a300a1560 0x7f6a300a3a20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.889+0000 7f6a46d5c700 1 --2- 172.21.15.57:0/2612990364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a40100390 0x7f6a4019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:38.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.889+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2612990364 >> 172.21.15.57:0/2612990364 conn(0x7f6a40076580 msgr2=0x7f6a400fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:38.891 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:38.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.890+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2612990364 shutdown_connections 2023-10-08T13:50:38.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.890+0000 7f6a46d5c700 1 -- 172.21.15.57:0/2612990364 wait complete. 2023-10-08T13:50:38.926 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542208 got 60129542209 for osd.1 2023-10-08T13:50:38.926 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:38.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:38 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/410336344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-08T13:50:38.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:38 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2847112112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-08T13:50:38.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:38 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3394387211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-08T13:50:38.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:38 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2612990364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-08T13:50:39.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:38 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/410336344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-08T13:50:39.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:38 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2847112112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-08T13:50:39.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:38 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3394387211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-08T13:50:39.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:38 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2612990364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-08T13:50:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.846+0000 7fb98cb0e700 1 -- 172.21.15.57:0/1728651342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9880ff0f0 msgr2=0x7fb9880ff4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.846+0000 7fb98cb0e700 1 --2- 172.21.15.57:0/1728651342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9880ff0f0 0x7fb9880ff4d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb97800aaa0 tx=0x7fb97800adb0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.846+0000 7fb98cb0e700 1 -- 172.21.15.57:0/1728651342 shutdown_connections 2023-10-08T13:50:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.846+0000 7fb98cb0e700 1 --2- 172.21.15.57:0/1728651342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb988104180 0x7fb98810bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.846+0000 7fb98cb0e700 1 --2- 172.21.15.57:0/1728651342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9880ff0f0 0x7fb9880ff4d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.846+0000 7fb98cb0e700 1 --2- 172.21.15.57:0/1728651342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9880ffaa0 0x7fb988103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.846+0000 7fb98cb0e700 1 -- 172.21.15.57:0/1728651342 >> 172.21.15.57:0/1728651342 conn(0x7fb988075cf0 msgr2=0x7fb9880fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.846+0000 7fb98cb0e700 1 -- 172.21.15.57:0/1728651342 shutdown_connections 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.846+0000 7fb98cb0e700 1 -- 172.21.15.57:0/1728651342 wait complete. 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.847+0000 7fb98cb0e700 1 Processor -- start 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.847+0000 7fb98cb0e700 1 -- start start 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb98cb0e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9880ff0f0 0x7fb98819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb98cb0e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9880ffaa0 0x7fb98819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb98cb0e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb988104180 0x7fb988196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb98cb0e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb988197130 con 0x7fb988104180 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb98cb0e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9881972a0 con 0x7fb9880ff0f0 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb98cb0e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb988197460 con 0x7fb9880ffaa0 2023-10-08T13:50:39.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb986d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb988104180 0x7fb988196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb98659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9880ff0f0 0x7fb98819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb986d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb988104180 0x7fb988196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54940/0 (socket says 172.21.15.57:54940) 2023-10-08T13:50:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb986d9d700 1 -- 172.21.15.57:0/3701351139 learned_addr learned my addr 172.21.15.57:0/3701351139 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb985d9b700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9880ffaa0 0x7fb98819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb986d9d700 1 -- 172.21.15.57:0/3701351139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9880ffaa0 msgr2=0x7fb98819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb986d9d700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9880ffaa0 0x7fb98819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb986d9d700 1 -- 172.21.15.57:0/3701351139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9880ff0f0 msgr2=0x7fb98819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:39.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb986d9d700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9880ff0f0 0x7fb98819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb986d9d700 1 -- 172.21.15.57:0/3701351139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97800a700 con 0x7fb988104180 2023-10-08T13:50:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb98659c700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9880ff0f0 0x7fb98819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.848+0000 7fb986d9d700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb988104180 0x7fb988196ad0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb980007ee0 tx=0x7fb98000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.849+0000 7fb9777fe700 1 -- 172.21.15.57:0/3701351139 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980017070 con 0x7fb988104180 2023-10-08T13:50:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.849+0000 7fb98cb0e700 1 -- 172.21.15.57:0/3701351139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb988197740 con 0x7fb988104180 2023-10-08T13:50:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.849+0000 7fb9777fe700 1 -- 172.21.15.57:0/3701351139 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb98000f040 con 0x7fb988104180 2023-10-08T13:50:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.849+0000 7fb9777fe700 1 -- 172.21.15.57:0/3701351139 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980005550 con 0x7fb988104180 2023-10-08T13:50:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.849+0000 7fb98cb0e700 1 -- 172.21.15.57:0/3701351139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9881a3930 con 0x7fb988104180 2023-10-08T13:50:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.850+0000 7fb98cb0e700 1 -- 172.21.15.57:0/3701351139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb988107440 con 0x7fb988104180 2023-10-08T13:50:39.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.852+0000 7fb9777fe700 1 -- 172.21.15.57:0/3701351139 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fb98000db20 con 0x7fb988104180 2023-10-08T13:50:39.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.852+0000 7fb9777fe700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9700a1560 0x7fb9700a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:39.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.852+0000 7fb98659c700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9700a1560 0x7fb9700a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:39.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.852+0000 7fb9777fe700 1 -- 172.21.15.57:0/3701351139 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fb9800cc940 con 0x7fb988104180 2023-10-08T13:50:39.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.853+0000 7fb98659c700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9700a1560 0x7fb9700a3a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c6d0 tx=0x7fb9780058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:39.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:38.857+0000 7fb9777fe700 1 -- 172.21.15.57:0/3701351139 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb980095a50 con 0x7fb988104180 2023-10-08T13:50:39.253 INFO:teuthology.orchestra.run.smithi057.stdout:339302416406 2023-10-08T13:50:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.078+0000 7fb98cb0e700 1 -- 172.21.15.57:0/3701351139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fb988100580 con 0x7fb988104180 2023-10-08T13:50:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.078+0000 7fb9777fe700 1 -- 172.21.15.57:0/3701351139 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fb980099070 con 0x7fb988104180 2023-10-08T13:50:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.080+0000 7fb98cb0e700 1 -- 172.21.15.57:0/3701351139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9700a1560 msgr2=0x7fb9700a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.080+0000 7fb98cb0e700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9700a1560 0x7fb9700a3a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c6d0 tx=0x7fb9780058e0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.080+0000 7fb98cb0e700 1 -- 172.21.15.57:0/3701351139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb988104180 msgr2=0x7fb988196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.080+0000 7fb98cb0e700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb988104180 0x7fb988196ad0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb980007ee0 tx=0x7fb98000d3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.080+0000 7fb98cb0e700 1 -- 172.21.15.57:0/3701351139 shutdown_connections 2023-10-08T13:50:39.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.080+0000 7fb98cb0e700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb988104180 0x7fb988196ad0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.080+0000 7fb98cb0e700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb9700a1560 0x7fb9700a3a20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.080+0000 7fb98cb0e700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9880ff0f0 0x7fb98819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.080+0000 7fb98cb0e700 1 --2- 172.21.15.57:0/3701351139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9880ffaa0 0x7fb98819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.080+0000 7fb98cb0e700 1 -- 172.21.15.57:0/3701351139 >> 172.21.15.57:0/3701351139 conn(0x7fb988075cf0 msgr2=0x7fb9880fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.081+0000 7fb98cb0e700 1 -- 172.21.15.57:0/3701351139 shutdown_connections 2023-10-08T13:50:39.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:39.081+0000 7fb98cb0e700 1 -- 172.21.15.57:0/3701351139 wait complete. 2023-10-08T13:50:39.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:38 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/410336344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-08T13:50:39.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:38 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2847112112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-08T13:50:39.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:38 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3394387211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-08T13:50:39.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:38 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2612990364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-08T13:50:39.405 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678041 got 304942678042 for osd.10 2023-10-08T13:50:39.405 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:39.587 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364837 got 214748364839 for osd.7 2023-10-08T13:50:39.587 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:39.643 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103201 got 249108103202 for osd.8 2023-10-08T13:50:39.643 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:39.872 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920818 got 141733920820 for osd.4 2023-10-08T13:50:39.873 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:39.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:39 smithi057 ceph-mon[99476]: pgmap v270: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2023-10-08T13:50:39.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:39 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3701351139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-08T13:50:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:39 smithi115 ceph-mon[104408]: pgmap v270: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2023-10-08T13:50:40.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:39 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3701351139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-08T13:50:40.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:39 smithi099 ceph-mon[103849]: pgmap v270: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2023-10-08T13:50:40.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:39 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3701351139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-08T13:50:40.310 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:40.784 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416405 got 339302416406 for osd.11 2023-10-08T13:50:40.784 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:41.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.636+0000 7fb0fdc04700 1 -- 172.21.15.57:0/805692577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f8068190 msgr2=0x7fb0f80685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.636+0000 7fb0fdc04700 1 --2- 172.21.15.57:0/805692577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f8068190 0x7fb0f80685f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009990 tx=0x7fb0e0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.636+0000 7fb0fdc04700 1 -- 172.21.15.57:0/805692577 shutdown_connections 2023-10-08T13:50:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.636+0000 7fb0fdc04700 1 --2- 172.21.15.57:0/805692577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0f8068b30 0x7fb0f810d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.636+0000 7fb0fdc04700 1 --2- 172.21.15.57:0/805692577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0f81067a0 0x7fb0f8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.636+0000 7fb0fdc04700 1 --2- 172.21.15.57:0/805692577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f8068190 0x7fb0f80685f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.636+0000 7fb0fdc04700 1 -- 172.21.15.57:0/805692577 >> 172.21.15.57:0/805692577 conn(0x7fb0f8075790 msgr2=0x7fb0f8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.636+0000 7fb0fdc04700 1 -- 172.21.15.57:0/805692577 shutdown_connections 2023-10-08T13:50:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.636+0000 7fb0fdc04700 1 -- 172.21.15.57:0/805692577 wait complete. 2023-10-08T13:50:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.637+0000 7fb0fdc04700 1 Processor -- start 2023-10-08T13:50:41.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.637+0000 7fb0fdc04700 1 -- start start 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.637+0000 7fb0fdc04700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0f8068190 0x7fb0f819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.637+0000 7fb0fdc04700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f8068b30 0x7fb0f819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.637+0000 7fb0fdc04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0f81067a0 0x7fb0f8196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.637+0000 7fb0fdc04700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f81970d0 con 0x7fb0f8068b30 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.637+0000 7fb0fdc04700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f8197240 con 0x7fb0f81067a0 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.637+0000 7fb0fdc04700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f8197400 con 0x7fb0f8068190 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f8068b30 0x7fb0f819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f8068b30 0x7fb0f819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54950/0 (socket says 172.21.15.57:54950) 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f6ffd700 1 -- 172.21.15.57:0/3175325795 learned_addr learned my addr 172.21.15.57:0/3175325795 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f77fe700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0f8068190 0x7fb0f819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:41.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f7fff700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0f81067a0 0x7fb0f8196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f6ffd700 1 -- 172.21.15.57:0/3175325795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0f8068190 msgr2=0x7fb0f819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f6ffd700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0f8068190 0x7fb0f819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f6ffd700 1 -- 172.21.15.57:0/3175325795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0f81067a0 msgr2=0x7fb0f8196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f6ffd700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0f81067a0 0x7fb0f8196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f6ffd700 1 -- 172.21.15.57:0/3175325795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00095f0 con 0x7fb0f8068b30 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f6ffd700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f8068b30 0x7fb0f819ce70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00098e0 tx=0x7fb0e000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f4ff9700 1 -- 172.21.15.57:0/3175325795 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e001d070 con 0x7fb0f8068b30 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0fdc04700 1 -- 172.21.15.57:0/3175325795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f8197680 con 0x7fb0f8068b30 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f4ff9700 1 -- 172.21.15.57:0/3175325795 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e0022470 con 0x7fb0f8068b30 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0f4ff9700 1 -- 172.21.15.57:0/3175325795 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0017650 con 0x7fb0f8068b30 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.638+0000 7fb0fdc04700 1 -- 172.21.15.57:0/3175325795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f81a3920 con 0x7fb0f8068b30 2023-10-08T13:50:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.639+0000 7fb0fdc04700 1 -- 172.21.15.57:0/3175325795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f8108d50 con 0x7fb0f8068b30 2023-10-08T13:50:41.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.640+0000 7fb0f4ff9700 1 -- 172.21.15.57:0/3175325795 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fb0e00225e0 con 0x7fb0f8068b30 2023-10-08T13:50:41.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.641+0000 7fb0f4ff9700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0e40a1560 0x7fb0e40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:41.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.641+0000 7fb0f4ff9700 1 -- 172.21.15.57:0/3175325795 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fb0e00ce330 con 0x7fb0f8068b30 2023-10-08T13:50:41.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.641+0000 7fb0f77fe700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0e40a1560 0x7fb0e40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:41.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.641+0000 7fb0f77fe700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0e40a1560 0x7fb0e40a3a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005950 tx=0x7fb0e800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:41.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.645+0000 7fb0f4ff9700 1 -- 172.21.15.57:0/3175325795 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb0e0097470 con 0x7fb0f8068b30 2023-10-08T13:50:41.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.752+0000 7fe340504700 1 -- 172.21.15.57:0/1120447892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe338068a20 msgr2=0x7fe338068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:41.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.752+0000 7fe340504700 1 --2- 172.21.15.57:0/1120447892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe338068a20 0x7fe338068e80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe3280099c0 tx=0x7fe328009cd0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.752+0000 7fe340504700 1 -- 172.21.15.57:0/1120447892 shutdown_connections 2023-10-08T13:50:41.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.752+0000 7fe340504700 1 --2- 172.21.15.57:0/1120447892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3380693c0 0x7fe33810d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.752+0000 7fe340504700 1 --2- 172.21.15.57:0/1120447892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3381067d0 0x7fe338106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.752+0000 7fe340504700 1 --2- 172.21.15.57:0/1120447892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe338068a20 0x7fe338068e80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.752+0000 7fe340504700 1 -- 172.21.15.57:0/1120447892 >> 172.21.15.57:0/1120447892 conn(0x7fe338075790 msgr2=0x7fe338075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.752+0000 7fe340504700 1 -- 172.21.15.57:0/1120447892 shutdown_connections 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.752+0000 7fe340504700 1 -- 172.21.15.57:0/1120447892 wait complete. 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe340504700 1 Processor -- start 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe340504700 1 -- start start 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe340504700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe338068a20 0x7fe338109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe340504700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3380693c0 0x7fe33810a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe340504700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3381067d0 0x7fe3381a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe340504700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3381a94a0 con 0x7fe3380693c0 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe340504700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3381a95e0 con 0x7fe338068a20 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe340504700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3381a9720 con 0x7fe3381067d0 2023-10-08T13:50:41.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33da9f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3380693c0 0x7fe33810a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33da9f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3380693c0 0x7fe33810a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54964/0 (socket says 172.21.15.57:54964) 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33da9f700 1 -- 172.21.15.57:0/4088323241 learned_addr learned my addr 172.21.15.57:0/4088323241 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33eaa1700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3381067d0 0x7fe3381a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33e2a0700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe338068a20 0x7fe338109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33da9f700 1 -- 172.21.15.57:0/4088323241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3381067d0 msgr2=0x7fe3381a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33da9f700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3381067d0 0x7fe3381a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33da9f700 1 -- 172.21.15.57:0/4088323241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe338068a20 msgr2=0x7fe338109eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33da9f700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe338068a20 0x7fe338109eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33da9f700 1 -- 172.21.15.57:0/4088323241 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3280095f0 con 0x7fe3380693c0 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.753+0000 7fe33e2a0700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe338068a20 0x7fe338109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.754+0000 7fe33da9f700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3380693c0 0x7fe33810a3f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe32800b580 tx=0x7fe328004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.754+0000 7fe32f7fe700 1 -- 172.21.15.57:0/4088323241 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32801d070 con 0x7fe3380693c0 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.754+0000 7fe32f7fe700 1 -- 172.21.15.57:0/4088323241 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe32800fb30 con 0x7fe3380693c0 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.754+0000 7fe32f7fe700 1 -- 172.21.15.57:0/4088323241 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328022c60 con 0x7fe3380693c0 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.754+0000 7fe340504700 1 -- 172.21.15.57:0/4088323241 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3381a9950 con 0x7fe3380693c0 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.754+0000 7fe340504700 1 -- 172.21.15.57:0/4088323241 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3381a9cb0 con 0x7fe3380693c0 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.756+0000 7fe340504700 1 -- 172.21.15.57:0/4088323241 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe338101f10 con 0x7fe3380693c0 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.757+0000 7fe32f7fe700 1 -- 172.21.15.57:0/4088323241 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fe32800fca0 con 0x7fe3380693c0 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.757+0000 7fe32f7fe700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3240a1560 0x7fe3240a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.757+0000 7fe32f7fe700 1 -- 172.21.15.57:0/4088323241 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fe3280ce3a0 con 0x7fe3380693c0 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.757+0000 7fe33e2a0700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3240a1560 0x7fe3240a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.758+0000 7fe33e2a0700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3240a1560 0x7fe3240a3a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe334005e10 tx=0x7fe334005d60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.762+0000 7fe32f7fe700 1 -- 172.21.15.57:0/4088323241 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe32805d100 con 0x7fe3380693c0 2023-10-08T13:50:41.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.857+0000 7fb0fdc04700 1 -- 172.21.15.57:0/3175325795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fb0f8197810 con 0x7fb0f8068b30 2023-10-08T13:50:41.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.857+0000 7fb0f4ff9700 1 -- 172.21.15.57:0/3175325795 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0e009aa90 con 0x7fb0f8068b30 2023-10-08T13:50:41.859 INFO:teuthology.orchestra.run.smithi057.stdout:283467841567 2023-10-08T13:50:41.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 -- 172.21.15.57:0/3175325795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0e40a1560 msgr2=0x7fb0e40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:41.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0e40a1560 0x7fb0e40a3a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005950 tx=0x7fb0e800b410 comp rx=0 tx=0).stop 2023-10-08T13:50:41.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 -- 172.21.15.57:0/3175325795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f8068b30 msgr2=0x7fb0f819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:41.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f8068b30 0x7fb0f819ce70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00098e0 tx=0x7fb0e000f670 comp rx=0 tx=0).stop 2023-10-08T13:50:41.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 -- 172.21.15.57:0/3175325795 shutdown_connections 2023-10-08T13:50:41.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0e40a1560 0x7fb0e40a3a20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0f81067a0 0x7fb0f8196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0f8068190 0x7fb0f819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 --2- 172.21.15.57:0/3175325795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0f8068b30 0x7fb0f819ce70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 -- 172.21.15.57:0/3175325795 >> 172.21.15.57:0/3175325795 conn(0x7fb0f8075790 msgr2=0x7fb0f80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:41.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 -- 172.21.15.57:0/3175325795 shutdown_connections 2023-10-08T13:50:41.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.860+0000 7fb0fdc04700 1 -- 172.21.15.57:0/3175325795 wait complete. 2023-10-08T13:50:41.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.978+0000 7fe340504700 1 -- 172.21.15.57:0/4088323241 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe33804e4c0 con 0x7fe3380693c0 2023-10-08T13:50:41.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.979+0000 7fe32f7fe700 1 -- 172.21.15.57:0/4088323241 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe328097540 con 0x7fe3380693c0 2023-10-08T13:50:41.980 INFO:teuthology.orchestra.run.smithi057.stdout:38654705734 2023-10-08T13:50:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 -- 172.21.15.57:0/4088323241 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3240a1560 msgr2=0x7fe3240a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3240a1560 0x7fe3240a3a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe334005e10 tx=0x7fe334005d60 comp rx=0 tx=0).stop 2023-10-08T13:50:41.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 -- 172.21.15.57:0/4088323241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3380693c0 msgr2=0x7fe33810a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:41.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3380693c0 0x7fe33810a3f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe32800b580 tx=0x7fe328004730 comp rx=0 tx=0).stop 2023-10-08T13:50:41.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 -- 172.21.15.57:0/4088323241 shutdown_connections 2023-10-08T13:50:41.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3381067d0 0x7fe3381a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3240a1560 0x7fe3240a3a20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe338068a20 0x7fe338109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 --2- 172.21.15.57:0/4088323241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3380693c0 0x7fe33810a3f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:41.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 -- 172.21.15.57:0/4088323241 >> 172.21.15.57:0/4088323241 conn(0x7fe338075790 msgr2=0x7fe3380fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:41.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.981+0000 7fe340504700 1 -- 172.21.15.57:0/4088323241 shutdown_connections 2023-10-08T13:50:41.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:41.982+0000 7fe340504700 1 -- 172.21.15.57:0/4088323241 wait complete. 2023-10-08T13:50:42.008 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:41 smithi057 ceph-mon[99476]: pgmap v271: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T13:50:42.008 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:41 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3175325795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-08T13:50:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:41 smithi115 ceph-mon[104408]: pgmap v271: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T13:50:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:41 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3175325795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-08T13:50:42.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:41 smithi099 ceph-mon[103849]: pgmap v271: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T13:50:42.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:41 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3175325795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-08T13:50:42.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.381+0000 7fcc1e19b700 1 -- 172.21.15.57:0/2490514206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc181057c0 msgr2=0x7fcc18105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:42.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.381+0000 7fcc1e19b700 1 --2- 172.21.15.57:0/2490514206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc181057c0 0x7fcc18105ba0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009990 tx=0x7fcc08009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:42.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.381+0000 7fcc1e19b700 1 -- 172.21.15.57:0/2490514206 shutdown_connections 2023-10-08T13:50:42.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.381+0000 7fcc1e19b700 1 --2- 172.21.15.57:0/2490514206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc18101260 0x7fcc1810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:42.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.381+0000 7fcc1e19b700 1 --2- 172.21.15.57:0/2490514206 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc181057c0 0x7fcc18105ba0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:42.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.381+0000 7fcc1e19b700 1 --2- 172.21.15.57:0/2490514206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181008a0 0x7fcc18100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:42.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.381+0000 7fcc1e19b700 1 -- 172.21.15.57:0/2490514206 >> 172.21.15.57:0/2490514206 conn(0x7fcc180779e0 msgr2=0x7fcc18077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.381+0000 7fcc1e19b700 1 -- 172.21.15.57:0/2490514206 shutdown_connections 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.381+0000 7fcc1e19b700 1 -- 172.21.15.57:0/2490514206 wait complete. 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.382+0000 7fcc1e19b700 1 Processor -- start 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.382+0000 7fcc1e19b700 1 -- start start 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.382+0000 7fcc1e19b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc181008a0 0x7fcc18196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.382+0000 7fcc1e19b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc18101260 0x7fcc18197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.382+0000 7fcc1e19b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181057c0 0x7fcc1819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.382+0000 7fcc1e19b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819ab50 con 0x7fcc181008a0 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.382+0000 7fcc1e19b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819acc0 con 0x7fcc18101260 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.382+0000 7fcc1e19b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819ae80 con 0x7fcc181057c0 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc177fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc181008a0 0x7fcc18196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:42.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc177fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc181008a0 0x7fcc18196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54976/0 (socket says 172.21.15.57:54976) 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc177fe700 1 -- 172.21.15.57:0/3745810238 learned_addr learned my addr 172.21.15.57:0/3745810238 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc16ffd700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc18101260 0x7fcc18197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc17fff700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181057c0 0x7fcc1819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc177fe700 1 -- 172.21.15.57:0/3745810238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181057c0 msgr2=0x7fcc1819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc177fe700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181057c0 0x7fcc1819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc177fe700 1 -- 172.21.15.57:0/3745810238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc18101260 msgr2=0x7fcc18197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc177fe700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc18101260 0x7fcc18197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc177fe700 1 -- 172.21.15.57:0/3745810238 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc080095f0 con 0x7fcc181008a0 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc177fe700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc181008a0 0x7fcc18196d10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcc08003730 tx=0x7fcc0800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc14ff9700 1 -- 172.21.15.57:0/3745810238 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0801d070 con 0x7fcc181008a0 2023-10-08T13:50:42.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc1e19b700 1 -- 172.21.15.57:0/3745810238 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1819b100 con 0x7fcc181008a0 2023-10-08T13:50:42.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc14ff9700 1 -- 172.21.15.57:0/3745810238 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc08005030 con 0x7fcc181008a0 2023-10-08T13:50:42.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc14ff9700 1 -- 172.21.15.57:0/3745810238 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08004a40 con 0x7fcc181008a0 2023-10-08T13:50:42.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.383+0000 7fcc1e19b700 1 -- 172.21.15.57:0/3745810238 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1819b3d0 con 0x7fcc181008a0 2023-10-08T13:50:42.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.384+0000 7fcc1e19b700 1 -- 172.21.15.57:0/3745810238 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc18108d80 con 0x7fcc181008a0 2023-10-08T13:50:42.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.385+0000 7fcc14ff9700 1 -- 172.21.15.57:0/3745810238 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fcc08004bc0 con 0x7fcc181008a0 2023-10-08T13:50:42.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.386+0000 7fcc14ff9700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc000a1560 0x7fcc000a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:42.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.386+0000 7fcc14ff9700 1 -- 172.21.15.57:0/3745810238 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fcc080ce0c0 con 0x7fcc181008a0 2023-10-08T13:50:42.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.386+0000 7fcc16ffd700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc000a1560 0x7fcc000a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:42.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.386+0000 7fcc16ffd700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc000a1560 0x7fcc000a3a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc18198230 tx=0x7fcc0c009190 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:42.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.390+0000 7fcc14ff9700 1 -- 172.21.15.57:0/3745810238 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcc08097260 con 0x7fcc181008a0 2023-10-08T13:50:42.500 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705733 got 38654705734 for osd.0 2023-10-08T13:50:42.500 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:42.574 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841564 got 283467841567 for osd.9 2023-10-08T13:50:42.575 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:42.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.623+0000 7fcc1e19b700 1 -- 172.21.15.57:0/3745810238 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fcc18197fe0 con 0x7fcc181008a0 2023-10-08T13:50:42.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.623+0000 7fcc14ff9700 1 -- 172.21.15.57:0/3745810238 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc0809a880 con 0x7fcc181008a0 2023-10-08T13:50:42.625 INFO:teuthology.orchestra.run.smithi057.stdout:193273528364 2023-10-08T13:50:42.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.625+0000 7fcc1e19b700 1 -- 172.21.15.57:0/3745810238 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc000a1560 msgr2=0x7fcc000a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:42.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.625+0000 7fcc1e19b700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc000a1560 0x7fcc000a3a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc18198230 tx=0x7fcc0c009190 comp rx=0 tx=0).stop 2023-10-08T13:50:42.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.625+0000 7fcc1e19b700 1 -- 172.21.15.57:0/3745810238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc181008a0 msgr2=0x7fcc18196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:42.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.625+0000 7fcc1e19b700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc181008a0 0x7fcc18196d10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcc08003730 tx=0x7fcc0800f690 comp rx=0 tx=0).stop 2023-10-08T13:50:42.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.626+0000 7fcc1e19b700 1 -- 172.21.15.57:0/3745810238 shutdown_connections 2023-10-08T13:50:42.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.626+0000 7fcc1e19b700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc181057c0 0x7fcc1819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:42.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.626+0000 7fcc1e19b700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc181008a0 0x7fcc18196d10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:42.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.626+0000 7fcc1e19b700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcc000a1560 0x7fcc000a3a20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:42.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.626+0000 7fcc1e19b700 1 --2- 172.21.15.57:0/3745810238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc18101260 0x7fcc18197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:42.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.626+0000 7fcc1e19b700 1 -- 172.21.15.57:0/3745810238 >> 172.21.15.57:0/3745810238 conn(0x7fcc180779e0 msgr2=0x7fcc180fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:42.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.626+0000 7fcc1e19b700 1 -- 172.21.15.57:0/3745810238 shutdown_connections 2023-10-08T13:50:42.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:42.626+0000 7fcc1e19b700 1 -- 172.21.15.57:0/3745810238 wait complete. 2023-10-08T13:50:43.040 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:42 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4088323241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-08T13:50:43.040 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:42 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3745810238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-08T13:50:43.041 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528363 got 193273528364 for osd.6 2023-10-08T13:50:43.041 DEBUG:teuthology.parallel:result is None 2023-10-08T13:50:43.041 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-08T13:50:43.041 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-08T13:50:43.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:42 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4088323241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-08T13:50:43.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:42 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3745810238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-08T13:50:43.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:42 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4088323241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-08T13:50:43.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:42 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3745810238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-08T13:50:43.609 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:44.063 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:43 smithi057 ceph-mon[99476]: pgmap v272: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2023-10-08T13:50:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:43 smithi115 ceph-mon[104408]: pgmap v272: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2023-10-08T13:50:44.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:43 smithi099 ceph-mon[103849]: pgmap v272: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2023-10-08T13:50:44.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 -- 172.21.15.57:0/1397816195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e080ff750 msgr2=0x7f8e080ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:44.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 --2- 172.21.15.57:0/1397816195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e080ff750 0x7f8e080ffbd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009990 tx=0x7f8df8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:44.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 -- 172.21.15.57:0/1397816195 shutdown_connections 2023-10-08T13:50:44.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 --2- 172.21.15.57:0/1397816195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e08100110 0x7f8e0810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:44.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 --2- 172.21.15.57:0/1397816195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e081056b0 0x7f8e08105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:44.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 --2- 172.21.15.57:0/1397816195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e080ff750 0x7f8e080ffbd0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:44.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 -- 172.21.15.57:0/1397816195 >> 172.21.15.57:0/1397816195 conn(0x7f8e08075790 msgr2=0x7f8e08075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:44.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 -- 172.21.15.57:0/1397816195 shutdown_connections 2023-10-08T13:50:44.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 -- 172.21.15.57:0/1397816195 wait complete. 2023-10-08T13:50:44.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 Processor -- start 2023-10-08T13:50:44.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.693+0000 7f8e0f433700 1 -- start start 2023-10-08T13:50:44.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0f433700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e080ff750 0x7f8e08197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:44.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0f433700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e08100110 0x7f8e08197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:44.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0f433700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e081056b0 0x7f8e08198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:44.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0f433700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e081a2d90 con 0x7f8e080ff750 2023-10-08T13:50:44.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0f433700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e081a2f00 con 0x7f8e081056b0 2023-10-08T13:50:44.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0f433700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e081a30c0 con 0x7f8e08100110 2023-10-08T13:50:44.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0d1cf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e080ff750 0x7f8e08197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:44.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0c9ce700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e08100110 0x7f8e08197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:44.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0d1cf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e080ff750 0x7f8e08197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54986/0 (socket says 172.21.15.57:54986) 2023-10-08T13:50:44.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0c9ce700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e08100110 0x7f8e08197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:55800/0 (socket says 172.21.15.57:55800) 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0d1cf700 1 -- 172.21.15.57:0/3370564078 learned_addr learned my addr 172.21.15.57:0/3370564078 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0d9d0700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e081056b0 0x7f8e08198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0d1cf700 1 -- 172.21.15.57:0/3370564078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e08100110 msgr2=0x7f8e08197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0d1cf700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e08100110 0x7f8e08197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0d1cf700 1 -- 172.21.15.57:0/3370564078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e081056b0 msgr2=0x7f8e08198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0d1cf700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e081056b0 0x7f8e08198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.694+0000 7f8e0d1cf700 1 -- 172.21.15.57:0/3370564078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df80095f0 con 0x7f8e080ff750 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.695+0000 7f8e0d1cf700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e080ff750 0x7f8e08197a80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400c920 tx=0x7f8e0400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.695+0000 7f8dfe7fc700 1 -- 172.21.15.57:0/3370564078 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04004a30 con 0x7f8e080ff750 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.695+0000 7f8e0f433700 1 -- 172.21.15.57:0/3370564078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e081a33a0 con 0x7f8e080ff750 2023-10-08T13:50:44.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.695+0000 7f8dfe7fc700 1 -- 172.21.15.57:0/3370564078 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e040054e0 con 0x7f8e080ff750 2023-10-08T13:50:44.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.695+0000 7f8dfe7fc700 1 -- 172.21.15.57:0/3370564078 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04007c10 con 0x7f8e080ff750 2023-10-08T13:50:44.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.695+0000 7f8e0f433700 1 -- 172.21.15.57:0/3370564078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e081a38f0 con 0x7f8e080ff750 2023-10-08T13:50:44.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.696+0000 7f8e0f433700 1 -- 172.21.15.57:0/3370564078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e08107cd0 con 0x7f8e080ff750 2023-10-08T13:50:44.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.697+0000 7f8dfe7fc700 1 -- 172.21.15.57:0/3370564078 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f8e04005000 con 0x7f8e080ff750 2023-10-08T13:50:44.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.698+0000 7f8dfe7fc700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8df40a1560 0x7f8df40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:44.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.698+0000 7f8e0c9ce700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8df40a1560 0x7f8df40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:44.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.698+0000 7f8dfe7fc700 1 -- 172.21.15.57:0/3370564078 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f8e040cd150 con 0x7f8e080ff750 2023-10-08T13:50:44.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.699+0000 7f8e0c9ce700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8df40a1560 0x7f8df40a3a20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8df8003800 tx=0x7f8df800b560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:44.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.701+0000 7f8dfe7fc700 1 -- 172.21.15.57:0/3370564078 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8e04096260 con 0x7f8e080ff750 2023-10-08T13:50:44.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.898+0000 7f8e0f433700 1 -- 172.21.15.57:0/3370564078 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e08068700 con 0x7f8df40a1560 2023-10-08T13:50:44.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.913+0000 7f8dfe7fc700 1 -- 172.21.15.57:0/3370564078 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422726 (secure 0 0 0) 0x7f8e08068700 con 0x7f8df40a1560 2023-10-08T13:50:44.919 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 -- 172.21.15.57:0/3370564078 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8df40a1560 msgr2=0x7f8df40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8df40a1560 0x7f8df40a3a20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8df8003800 tx=0x7f8df800b560 comp rx=0 tx=0).stop 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 -- 172.21.15.57:0/3370564078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e080ff750 msgr2=0x7f8e08197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e080ff750 0x7f8e08197a80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400c920 tx=0x7f8e0400cc30 comp rx=0 tx=0).stop 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 -- 172.21.15.57:0/3370564078 shutdown_connections 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8df40a1560 0x7f8df40a3a20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e081056b0 0x7f8e08198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e080ff750 0x7f8e08197a80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 --2- 172.21.15.57:0/3370564078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e08100110 0x7f8e08197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 -- 172.21.15.57:0/3370564078 >> 172.21.15.57:0/3370564078 conn(0x7f8e08075790 msgr2=0x7f8e080fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:44.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 -- 172.21.15.57:0/3370564078 shutdown_connections 2023-10-08T13:50:44.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:44.917+0000 7f8e0f433700 1 -- 172.21.15.57:0/3370564078 wait complete. 2023-10-08T13:50:44.924 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-08T13:50:45.377 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2023-10-08T13:50:44.117943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656009,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":507,"num_read_kb":378,"num_write":416,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338420,"kb_used_data":14172,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603404,"statfs":{"total":1151940427776,"available":1151593885696,"internally_reserved":0,"allocated":14512128,"data_stored":8079456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":64,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004006"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.781367+0000","last_change":"2023-10-08T13:49:44.786130+0000","last_active":"2023-10-08T13:50:14.781367+0000","last_peered":"2023-10-08T13:50:14.781367+0000","last_clean":"2023-10-08T13:50:14.781367+0000","last_became_active":"2023-10-08T13:49:44.785792+0000","last_became_peered":"2023-10-08T13:49:44.785792+0000","last_unstale":"2023-10-08T13:50:14.781367+0000","last_undegraded":"2023-10-08T13:50:14.781367+0000","last_fullsized":"2023-10-08T13:50:14.781367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:17.046800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780058+0000","last_change":"2023-10-08T13:49:44.777601+0000","last_active":"2023-10-08T13:50:14.780058+0000","last_peered":"2023-10-08T13:50:14.780058+0000","last_clean":"2023-10-08T13:50:14.780058+0000","last_became_active":"2023-10-08T13:49:44.777287+0000","last_became_peered":"2023-10-08T13:49:44.777287+0000","last_unstale":"2023-10-08T13:50:14.780058+0000","last_undegraded":"2023-10-08T13:50:14.780058+0000","last_fullsized":"2023-10-08T13:50:14.780058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:09:17.317629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779588+0000","last_change":"2023-10-08T13:49:44.787035+0000","last_active":"2023-10-08T13:50:14.779588+0000","last_peered":"2023-10-08T13:50:14.779588+0000","last_clean":"2023-10-08T13:50:14.779588+0000","last_became_active":"2023-10-08T13:49:44.785834+0000","last_became_peered":"2023-10-08T13:49:44.785834+0000","last_unstale":"2023-10-08T13:50:14.779588+0000","last_undegraded":"2023-10-08T13:50:14.779588+0000","last_fullsized":"2023-10-08T13:50:14.779588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:11.611404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773779+0000","last_change":"2023-10-08T13:49:44.781937+0000","last_active":"2023-10-08T13:50:14.773779+0000","last_peered":"2023-10-08T13:50:14.773779+0000","last_clean":"2023-10-08T13:50:14.773779+0000","last_became_active":"2023-10-08T13:49:44.780712+0000","last_became_peered":"2023-10-08T13:49:44.780712+0000","last_unstale":"2023-10-08T13:50:14.773779+0000","last_undegraded":"2023-10-08T13:50:14.773779+0000","last_fullsized":"2023-10-08T13:50:14.773779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:02.514007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.772899+0000","last_change":"2023-10-08T13:49:44.788661+0000","last_active":"2023-10-08T13:50:14.772899+0000","last_peered":"2023-10-08T13:50:14.772899+0000","last_clean":"2023-10-08T13:50:14.772899+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:50:14.772899+0000","last_undegraded":"2023-10-08T13:50:14.772899+0000","last_fullsized":"2023-10-08T13:50:14.772899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:35.770287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776011+0000","last_change":"2023-10-08T13:49:44.778519+0000","last_active":"2023-10-08T13:50:14.776011+0000","last_peered":"2023-10-08T13:50:14.776011+0000","last_clean":"2023-10-08T13:50:14.776011+0000","last_became_active":"2023-10-08T13:49:44.778193+0000","last_became_peered":"2023-10-08T13:49:44.778193+0000","last_unstale":"2023-10-08T13:50:14.776011+0000","last_undegraded":"2023-10-08T13:50:14.776011+0000","last_fullsized":"2023-10-08T13:50:14.776011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:12.916019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777538+0000","last_change":"2023-10-08T13:49:44.791759+0000","last_active":"2023-10-08T13:50:14.777538+0000","last_peered":"2023-10-08T13:50:14.777538+0000","last_clean":"2023-10-08T13:50:14.777538+0000","last_became_active":"2023-10-08T13:49:44.791494+0000","last_became_peered":"2023-10-08T13:49:44.791494+0000","last_unstale":"2023-10-08T13:50:14.777538+0000","last_undegraded":"2023-10-08T13:50:14.777538+0000","last_fullsized":"2023-10-08T13:50:14.777538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:55.859609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775547+0000","last_change":"2023-10-08T13:49:44.786831+0000","last_active":"2023-10-08T13:50:14.775547+0000","last_peered":"2023-10-08T13:50:14.775547+0000","last_clean":"2023-10-08T13:50:14.775547+0000","last_became_active":"2023-10-08T13:49:44.786516+0000","last_became_peered":"2023-10-08T13:49:44.786516+0000","last_unstale":"2023-10-08T13:50:14.775547+0000","last_undegraded":"2023-10-08T13:50:14.775547+0000","last_fullsized":"2023-10-08T13:50:14.775547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:07.616821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780365+0000","last_change":"2023-10-08T13:49:44.789827+0000","last_active":"2023-10-08T13:50:14.780365+0000","last_peered":"2023-10-08T13:50:14.780365+0000","last_clean":"2023-10-08T13:50:14.780365+0000","last_became_active":"2023-10-08T13:49:44.786635+0000","last_became_peered":"2023-10-08T13:49:44.786635+0000","last_unstale":"2023-10-08T13:50:14.780365+0000","last_undegraded":"2023-10-08T13:50:14.780365+0000","last_fullsized":"2023-10-08T13:50:14.780365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:35.492437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776603+0000","last_change":"2023-10-08T13:49:44.784240+0000","last_active":"2023-10-08T13:50:14.776603+0000","last_peered":"2023-10-08T13:50:14.776603+0000","last_clean":"2023-10-08T13:50:14.776603+0000","last_became_active":"2023-10-08T13:49:44.783475+0000","last_became_peered":"2023-10-08T13:49:44.783475+0000","last_unstale":"2023-10-08T13:50:14.776603+0000","last_undegraded":"2023-10-08T13:50:14.776603+0000","last_fullsized":"2023-10-08T13:50:14.776603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:39.730679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777907+0000","last_change":"2023-10-08T13:49:44.790901+0000","last_active":"2023-10-08T13:50:14.777907+0000","last_peered":"2023-10-08T13:50:14.777907+0000","last_clean":"2023-10-08T13:50:14.777907+0000","last_became_active":"2023-10-08T13:49:44.790611+0000","last_became_peered":"2023-10-08T13:49:44.790611+0000","last_unstale":"2023-10-08T13:50:14.777907+0000","last_undegraded":"2023-10-08T13:50:14.777907+0000","last_fullsized":"2023-10-08T13:50:14.777907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:19:21.764105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773901+0000","last_change":"2023-10-08T13:49:44.775226+0000","last_active":"2023-10-08T13:50:14.773901+0000","last_peered":"2023-10-08T13:50:14.773901+0000","last_clean":"2023-10-08T13:50:14.773901+0000","last_became_active":"2023-10-08T13:49:44.774756+0000","last_became_peered":"2023-10-08T13:49:44.774756+0000","last_unstale":"2023-10-08T13:50:14.773901+0000","last_undegraded":"2023-10-08T13:50:14.773901+0000","last_fullsized":"2023-10-08T13:50:14.773901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:57:14.749066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778685+0000","last_change":"2023-10-08T13:49:44.781610+0000","last_active":"2023-10-08T13:50:14.778685+0000","last_peered":"2023-10-08T13:50:14.778685+0000","last_clean":"2023-10-08T13:50:14.778685+0000","last_became_active":"2023-10-08T13:49:44.781281+0000","last_became_peered":"2023-10-08T13:49:44.781281+0000","last_unstale":"2023-10-08T13:50:14.778685+0000","last_undegraded":"2023-10-08T13:50:14.778685+0000","last_fullsized":"2023-10-08T13:50:14.778685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:50.823909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779251+0000","last_change":"2023-10-08T13:49:44.775836+0000","last_active":"2023-10-08T13:50:14.779251+0000","last_peered":"2023-10-08T13:50:14.779251+0000","last_clean":"2023-10-08T13:50:14.779251+0000","last_became_active":"2023-10-08T13:49:44.775206+0000","last_became_peered":"2023-10-08T13:49:44.775206+0000","last_unstale":"2023-10-08T13:50:14.779251+0000","last_undegraded":"2023-10-08T13:50:14.779251+0000","last_fullsized":"2023-10-08T13:50:14.779251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:51.566800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779976+0000","last_change":"2023-10-08T13:49:44.778495+0000","last_active":"2023-10-08T13:50:14.779976+0000","last_peered":"2023-10-08T13:50:14.779976+0000","last_clean":"2023-10-08T13:50:14.779976+0000","last_became_active":"2023-10-08T13:49:44.778196+0000","last_became_peered":"2023-10-08T13:49:44.778196+0000","last_unstale":"2023-10-08T13:50:14.779976+0000","last_undegraded":"2023-10-08T13:50:14.779976+0000","last_fullsized":"2023-10-08T13:50:14.779976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:57:38.531853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779796+0000","last_change":"2023-10-08T13:49:44.792960+0000","last_active":"2023-10-08T13:50:14.779796+0000","last_peered":"2023-10-08T13:50:14.779796+0000","last_clean":"2023-10-08T13:50:14.779796+0000","last_became_active":"2023-10-08T13:49:44.792674+0000","last_became_peered":"2023-10-08T13:49:44.792674+0000","last_unstale":"2023-10-08T13:50:14.779796+0000","last_undegraded":"2023-10-08T13:50:14.779796+0000","last_fullsized":"2023-10-08T13:50:14.779796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:13.934954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.781270+0000","last_change":"2023-10-08T13:49:44.785971+0000","last_active":"2023-10-08T13:50:14.781270+0000","last_peered":"2023-10-08T13:50:14.781270+0000","last_clean":"2023-10-08T13:50:14.781270+0000","last_became_active":"2023-10-08T13:49:44.785680+0000","last_became_peered":"2023-10-08T13:49:44.785680+0000","last_unstale":"2023-10-08T13:50:14.781270+0000","last_undegraded":"2023-10-08T13:50:14.781270+0000","last_fullsized":"2023-10-08T13:50:14.781270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:35.447519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773879+0000","last_change":"2023-10-08T13:49:44.787553+0000","last_active":"2023-10-08T13:50:14.773879+0000","last_peered":"2023-10-08T13:50:14.773879+0000","last_clean":"2023-10-08T13:50:14.773879+0000","last_became_active":"2023-10-08T13:49:44.787288+0000","last_became_peered":"2023-10-08T13:49:44.787288+0000","last_unstale":"2023-10-08T13:50:14.773879+0000","last_undegraded":"2023-10-08T13:50:14.773879+0000","last_fullsized":"2023-10-08T13:50:14.773879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:48.386120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780408+0000","last_change":"2023-10-08T13:49:44.789252+0000","last_active":"2023-10-08T13:50:14.780408+0000","last_peered":"2023-10-08T13:50:14.780408+0000","last_clean":"2023-10-08T13:50:14.780408+0000","last_became_active":"2023-10-08T13:49:44.788947+0000","last_became_peered":"2023-10-08T13:49:44.788947+0000","last_unstale":"2023-10-08T13:50:14.780408+0000","last_undegraded":"2023-10-08T13:50:14.780408+0000","last_fullsized":"2023-10-08T13:50:14.780408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:30.266913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779610+0000","last_change":"2023-10-08T13:49:49.163619+0000","last_active":"2023-10-08T13:50:14.779610+0000","last_peered":"2023-10-08T13:50:14.779610+0000","last_clean":"2023-10-08T13:50:14.779610+0000","last_became_active":"2023-10-08T13:49:49.163020+0000","last_became_peered":"2023-10-08T13:49:49.163020+0000","last_unstale":"2023-10-08T13:50:14.779610+0000","last_undegraded":"2023-10-08T13:50:14.779610+0000","last_fullsized":"2023-10-08T13:50:14.779610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:56.092404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:38.794493+0000","last_change":"2023-10-08T13:49:43.782863+0000","last_active":"2023-10-08T13:50:38.794493+0000","last_peered":"2023-10-08T13:50:38.794493+0000","last_clean":"2023-10-08T13:50:38.794493+0000","last_became_active":"2023-10-08T13:49:43.782477+0000","last_became_peered":"2023-10-08T13:49:43.782477+0000","last_unstale":"2023-10-08T13:50:38.794493+0000","last_undegraded":"2023-10-08T13:50:38.794493+0000","last_fullsized":"2023-10-08T13:50:38.794493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:10.134907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775780+0000","last_change":"2023-10-08T13:49:44.788452+0000","last_active":"2023-10-08T13:50:14.775780+0000","last_peered":"2023-10-08T13:50:14.775780+0000","last_clean":"2023-10-08T13:50:14.775780+0000","last_became_active":"2023-10-08T13:49:44.788122+0000","last_became_peered":"2023-10-08T13:49:44.788122+0000","last_unstale":"2023-10-08T13:50:14.775780+0000","last_undegraded":"2023-10-08T13:50:14.775780+0000","last_fullsized":"2023-10-08T13:50:14.775780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:25:45.766329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773267+0000","last_change":"2023-10-08T13:49:49.164248+0000","last_active":"2023-10-08T13:50:14.773267+0000","last_peered":"2023-10-08T13:50:14.773267+0000","last_clean":"2023-10-08T13:50:14.773267+0000","last_became_active":"2023-10-08T13:49:49.163624+0000","last_became_peered":"2023-10-08T13:49:49.163624+0000","last_unstale":"2023-10-08T13:50:14.773267+0000","last_undegraded":"2023-10-08T13:50:14.773267+0000","last_fullsized":"2023-10-08T13:50:14.773267+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:04.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776273+0000","last_change":"2023-10-08T13:49:43.788026+0000","last_active":"2023-10-08T13:50:14.776273+0000","last_peered":"2023-10-08T13:50:14.776273+0000","last_clean":"2023-10-08T13:50:14.776273+0000","last_became_active":"2023-10-08T13:49:43.787639+0000","last_became_peered":"2023-10-08T13:49:43.787639+0000","last_unstale":"2023-10-08T13:50:14.776273+0000","last_undegraded":"2023-10-08T13:50:14.776273+0000","last_fullsized":"2023-10-08T13:50:14.776273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:25:48.367765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779569+0000","last_change":"2023-10-08T13:49:44.786322+0000","last_active":"2023-10-08T13:50:14.779569+0000","last_peered":"2023-10-08T13:50:14.779569+0000","last_clean":"2023-10-08T13:50:14.779569+0000","last_became_active":"2023-10-08T13:49:44.785953+0000","last_became_peered":"2023-10-08T13:49:44.785953+0000","last_unstale":"2023-10-08T13:50:14.779569+0000","last_undegraded":"2023-10-08T13:50:14.779569+0000","last_fullsized":"2023-10-08T13:50:14.779569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:39.866785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.976961+0000","last_change":"2023-10-08T13:50:15.976961+0000","last_active":"2023-10-08T13:50:15.976961+0000","last_peered":"2023-10-08T13:50:15.976961+0000","last_clean":"2023-10-08T13:50:15.976961+0000","last_became_active":"2023-10-08T13:50:15.976492+0000","last_became_peered":"2023-10-08T13:50:15.976492+0000","last_unstale":"2023-10-08T13:50:15.976961+0000","last_undegraded":"2023-10-08T13:50:15.976961+0000","last_fullsized":"2023-10-08T13:50:15.976961+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:05:48.947259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.774448+0000","last_change":"2023-10-08T13:49:43.776239+0000","last_active":"2023-10-08T13:50:14.774448+0000","last_peered":"2023-10-08T13:50:14.774448+0000","last_clean":"2023-10-08T13:50:14.774448+0000","last_became_active":"2023-10-08T13:49:43.775969+0000","last_became_peered":"2023-10-08T13:49:43.775969+0000","last_unstale":"2023-10-08T13:50:14.774448+0000","last_undegraded":"2023-10-08T13:50:14.774448+0000","last_fullsized":"2023-10-08T13:50:14.774448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:24.011655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780166+0000","last_change":"2023-10-08T13:49:44.794912+0000","last_active":"2023-10-08T13:50:14.780166+0000","last_peered":"2023-10-08T13:50:14.780166+0000","last_clean":"2023-10-08T13:50:14.780166+0000","last_became_active":"2023-10-08T13:49:44.794590+0000","last_became_peered":"2023-10-08T13:49:44.794590+0000","last_unstale":"2023-10-08T13:50:14.780166+0000","last_undegraded":"2023-10-08T13:50:14.780166+0000","last_fullsized":"2023-10-08T13:50:14.780166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:51.631386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773315+0000","last_change":"2023-10-08T13:49:49.152149+0000","last_active":"2023-10-08T13:50:14.773315+0000","last_peered":"2023-10-08T13:50:14.773315+0000","last_clean":"2023-10-08T13:50:14.773315+0000","last_became_active":"2023-10-08T13:49:49.151764+0000","last_became_peered":"2023-10-08T13:49:49.151764+0000","last_unstale":"2023-10-08T13:50:14.773315+0000","last_undegraded":"2023-10-08T13:50:14.773315+0000","last_fullsized":"2023-10-08T13:50:14.773315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:25:08.232556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.800831+0000","last_change":"2023-10-08T13:50:15.800467+0000","last_active":"2023-10-08T13:50:15.800831+0000","last_peered":"2023-10-08T13:50:15.800831+0000","last_clean":"2023-10-08T13:50:15.800831+0000","last_became_active":"2023-10-08T13:50:15.788865+0000","last_became_peered":"2023-10-08T13:50:15.788865+0000","last_unstale":"2023-10-08T13:50:15.800831+0000","last_undegraded":"2023-10-08T13:50:15.800831+0000","last_fullsized":"2023-10-08T13:50:15.800831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:11:01.887072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776118+0000","last_change":"2023-10-08T13:49:44.783915+0000","last_active":"2023-10-08T13:50:14.776118+0000","last_peered":"2023-10-08T13:50:14.776118+0000","last_clean":"2023-10-08T13:50:14.776118+0000","last_became_active":"2023-10-08T13:49:44.783628+0000","last_became_peered":"2023-10-08T13:49:44.783628+0000","last_unstale":"2023-10-08T13:50:14.776118+0000","last_undegraded":"2023-10-08T13:50:14.776118+0000","last_fullsized":"2023-10-08T13:50:14.776118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:43.759681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780342+0000","last_change":"2023-10-08T13:49:49.156241+0000","last_active":"2023-10-08T13:50:14.780342+0000","last_peered":"2023-10-08T13:50:14.780342+0000","last_clean":"2023-10-08T13:50:14.780342+0000","last_became_active":"2023-10-08T13:49:49.155652+0000","last_became_peered":"2023-10-08T13:49:49.155652+0000","last_unstale":"2023-10-08T13:50:14.780342+0000","last_undegraded":"2023-10-08T13:50:14.780342+0000","last_fullsized":"2023-10-08T13:50:14.780342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:38.823059+0000","last_change":"2023-10-08T13:50:15.795816+0000","last_active":"2023-10-08T13:50:38.823059+0000","last_peered":"2023-10-08T13:50:38.823059+0000","last_clean":"2023-10-08T13:50:38.823059+0000","last_became_active":"2023-10-08T13:50:15.789936+0000","last_became_peered":"2023-10-08T13:50:15.789936+0000","last_unstale":"2023-10-08T13:50:38.823059+0000","last_undegraded":"2023-10-08T13:50:38.823059+0000","last_fullsized":"2023-10-08T13:50:38.823059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:21.496490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773190+0000","last_change":"2023-10-08T13:49:44.780935+0000","last_active":"2023-10-08T13:50:14.773190+0000","last_peered":"2023-10-08T13:50:14.773190+0000","last_clean":"2023-10-08T13:50:14.773190+0000","last_became_active":"2023-10-08T13:49:44.780613+0000","last_became_peered":"2023-10-08T13:49:44.780613+0000","last_unstale":"2023-10-08T13:50:14.773190+0000","last_undegraded":"2023-10-08T13:50:14.773190+0000","last_fullsized":"2023-10-08T13:50:14.773190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:57.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779985+0000","last_change":"2023-10-08T13:49:49.163652+0000","last_active":"2023-10-08T13:50:14.779985+0000","last_peered":"2023-10-08T13:50:14.779985+0000","last_clean":"2023-10-08T13:50:14.779985+0000","last_became_active":"2023-10-08T13:49:49.163140+0000","last_became_peered":"2023-10-08T13:49:49.163140+0000","last_unstale":"2023-10-08T13:50:14.779985+0000","last_undegraded":"2023-10-08T13:50:14.779985+0000","last_fullsized":"2023-10-08T13:50:14.779985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:48.733675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775983+0000","last_change":"2023-10-08T13:49:43.775799+0000","last_active":"2023-10-08T13:50:14.775983+0000","last_peered":"2023-10-08T13:50:14.775983+0000","last_clean":"2023-10-08T13:50:14.775983+0000","last_became_active":"2023-10-08T13:49:43.775470+0000","last_became_peered":"2023-10-08T13:49:43.775470+0000","last_unstale":"2023-10-08T13:50:14.775983+0000","last_undegraded":"2023-10-08T13:50:14.775983+0000","last_fullsized":"2023-10-08T13:50:14.775983+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:50.404554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.785798+0000","last_change":"2023-10-08T13:50:15.785798+0000","last_active":"2023-10-08T13:50:15.785798+0000","last_peered":"2023-10-08T13:50:15.785798+0000","last_clean":"2023-10-08T13:50:15.785798+0000","last_became_active":"2023-10-08T13:50:15.785343+0000","last_became_peered":"2023-10-08T13:50:15.785343+0000","last_unstale":"2023-10-08T13:50:15.785798+0000","last_undegraded":"2023-10-08T13:50:15.785798+0000","last_fullsized":"2023-10-08T13:50:15.785798+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:36.717707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.781065+0000","last_change":"2023-10-08T13:49:49.166702+0000","last_active":"2023-10-08T13:50:14.781065+0000","last_peered":"2023-10-08T13:50:14.781065+0000","last_clean":"2023-10-08T13:50:14.781065+0000","last_became_active":"2023-10-08T13:49:49.166256+0000","last_became_peered":"2023-10-08T13:49:49.166256+0000","last_unstale":"2023-10-08T13:50:14.781065+0000","last_undegraded":"2023-10-08T13:50:14.781065+0000","last_fullsized":"2023-10-08T13:50:14.781065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:18:57.565040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778550+0000","last_change":"2023-10-08T13:49:43.785085+0000","last_active":"2023-10-08T13:50:14.778550+0000","last_peered":"2023-10-08T13:50:14.778550+0000","last_clean":"2023-10-08T13:50:14.778550+0000","last_became_active":"2023-10-08T13:49:43.784746+0000","last_became_peered":"2023-10-08T13:49:43.784746+0000","last_unstale":"2023-10-08T13:50:14.778550+0000","last_undegraded":"2023-10-08T13:50:14.778550+0000","last_fullsized":"2023-10-08T13:50:14.778550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:13.471834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775477+0000","last_change":"2023-10-08T13:49:44.785670+0000","last_active":"2023-10-08T13:50:14.775477+0000","last_peered":"2023-10-08T13:50:14.775477+0000","last_clean":"2023-10-08T13:50:14.775477+0000","last_became_active":"2023-10-08T13:49:44.785221+0000","last_became_peered":"2023-10-08T13:49:44.785221+0000","last_unstale":"2023-10-08T13:50:14.775477+0000","last_undegraded":"2023-10-08T13:50:14.775477+0000","last_fullsized":"2023-10-08T13:50:14.775477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:25.415168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780015+0000","last_change":"2023-10-08T13:49:49.156294+0000","last_active":"2023-10-08T13:50:14.780015+0000","last_peered":"2023-10-08T13:50:14.780015+0000","last_clean":"2023-10-08T13:50:14.780015+0000","last_became_active":"2023-10-08T13:49:49.155821+0000","last_became_peered":"2023-10-08T13:49:49.155821+0000","last_unstale":"2023-10-08T13:50:14.780015+0000","last_undegraded":"2023-10-08T13:50:14.780015+0000","last_fullsized":"2023-10-08T13:50:14.780015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:59:43.831634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775524+0000","last_change":"2023-10-08T13:49:43.780052+0000","last_active":"2023-10-08T13:50:14.775524+0000","last_peered":"2023-10-08T13:50:14.775524+0000","last_clean":"2023-10-08T13:50:14.775524+0000","last_became_active":"2023-10-08T13:49:43.779731+0000","last_became_peered":"2023-10-08T13:49:43.779731+0000","last_unstale":"2023-10-08T13:50:14.775524+0000","last_undegraded":"2023-10-08T13:50:14.775524+0000","last_fullsized":"2023-10-08T13:50:14.775524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:38.800783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773039+0000","last_change":"2023-10-08T13:49:44.787880+0000","last_active":"2023-10-08T13:50:14.773039+0000","last_peered":"2023-10-08T13:50:14.773039+0000","last_clean":"2023-10-08T13:50:14.773039+0000","last_became_active":"2023-10-08T13:49:44.787591+0000","last_became_peered":"2023-10-08T13:49:44.787591+0000","last_unstale":"2023-10-08T13:50:14.773039+0000","last_undegraded":"2023-10-08T13:50:14.773039+0000","last_fullsized":"2023-10-08T13:50:14.773039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:53.078903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775715+0000","last_change":"2023-10-08T13:49:49.155004+0000","last_active":"2023-10-08T13:50:14.775715+0000","last_peered":"2023-10-08T13:50:14.775715+0000","last_clean":"2023-10-08T13:50:14.775715+0000","last_became_active":"2023-10-08T13:49:49.154503+0000","last_became_peered":"2023-10-08T13:49:49.154503+0000","last_unstale":"2023-10-08T13:50:14.775715+0000","last_undegraded":"2023-10-08T13:50:14.775715+0000","last_fullsized":"2023-10-08T13:50:14.775715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:21.565816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:41.785193+0000","last_change":"2023-10-08T13:49:43.775819+0000","last_active":"2023-10-08T13:50:41.785193+0000","last_peered":"2023-10-08T13:50:41.785193+0000","last_clean":"2023-10-08T13:50:41.785193+0000","last_became_active":"2023-10-08T13:49:43.775578+0000","last_became_peered":"2023-10-08T13:49:43.775578+0000","last_unstale":"2023-10-08T13:50:41.785193+0000","last_undegraded":"2023-10-08T13:50:41.785193+0000","last_fullsized":"2023-10-08T13:50:41.785193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:31.518191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780366+0000","last_change":"2023-10-08T13:49:44.785869+0000","last_active":"2023-10-08T13:50:14.780366+0000","last_peered":"2023-10-08T13:50:14.780366+0000","last_clean":"2023-10-08T13:50:14.780366+0000","last_became_active":"2023-10-08T13:49:44.785566+0000","last_became_peered":"2023-10-08T13:49:44.785566+0000","last_unstale":"2023-10-08T13:50:14.780366+0000","last_undegraded":"2023-10-08T13:50:14.780366+0000","last_fullsized":"2023-10-08T13:50:14.780366+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:29.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.772921+0000","last_change":"2023-10-08T13:49:49.162075+0000","last_active":"2023-10-08T13:50:14.772921+0000","last_peered":"2023-10-08T13:50:14.772921+0000","last_clean":"2023-10-08T13:50:14.772921+0000","last_became_active":"2023-10-08T13:49:49.161485+0000","last_became_peered":"2023-10-08T13:49:49.161485+0000","last_unstale":"2023-10-08T13:50:14.772921+0000","last_undegraded":"2023-10-08T13:50:14.772921+0000","last_fullsized":"2023-10-08T13:50:14.772921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:16.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778043+0000","last_change":"2023-10-08T13:49:43.776555+0000","last_active":"2023-10-08T13:50:14.778043+0000","last_peered":"2023-10-08T13:50:14.778043+0000","last_clean":"2023-10-08T13:50:14.778043+0000","last_became_active":"2023-10-08T13:49:43.776259+0000","last_became_peered":"2023-10-08T13:49:43.776259+0000","last_unstale":"2023-10-08T13:50:14.778043+0000","last_undegraded":"2023-10-08T13:50:14.778043+0000","last_fullsized":"2023-10-08T13:50:14.778043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:34.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779744+0000","last_change":"2023-10-08T13:49:44.779253+0000","last_active":"2023-10-08T13:50:14.779744+0000","last_peered":"2023-10-08T13:50:14.779744+0000","last_clean":"2023-10-08T13:50:14.779744+0000","last_became_active":"2023-10-08T13:49:44.778902+0000","last_became_peered":"2023-10-08T13:49:44.778902+0000","last_unstale":"2023-10-08T13:50:14.779744+0000","last_undegraded":"2023-10-08T13:50:14.779744+0000","last_fullsized":"2023-10-08T13:50:14.779744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:31:01.091424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773632+0000","last_change":"2023-10-08T13:49:49.164859+0000","last_active":"2023-10-08T13:50:14.773632+0000","last_peered":"2023-10-08T13:50:14.773632+0000","last_clean":"2023-10-08T13:50:14.773632+0000","last_became_active":"2023-10-08T13:49:49.164221+0000","last_became_peered":"2023-10-08T13:49:49.164221+0000","last_unstale":"2023-10-08T13:50:14.773632+0000","last_undegraded":"2023-10-08T13:50:14.773632+0000","last_fullsized":"2023-10-08T13:50:14.773632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:22.484934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775988+0000","last_change":"2023-10-08T13:49:43.788136+0000","last_active":"2023-10-08T13:50:14.775988+0000","last_peered":"2023-10-08T13:50:14.775988+0000","last_clean":"2023-10-08T13:50:14.775988+0000","last_became_active":"2023-10-08T13:49:43.787910+0000","last_became_peered":"2023-10-08T13:49:43.787910+0000","last_unstale":"2023-10-08T13:50:14.775988+0000","last_undegraded":"2023-10-08T13:50:14.775988+0000","last_fullsized":"2023-10-08T13:50:14.775988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:00:32.897972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780262+0000","last_change":"2023-10-08T13:49:44.786833+0000","last_active":"2023-10-08T13:50:14.780262+0000","last_peered":"2023-10-08T13:50:14.780262+0000","last_clean":"2023-10-08T13:50:14.780262+0000","last_became_active":"2023-10-08T13:49:44.786523+0000","last_became_peered":"2023-10-08T13:49:44.786523+0000","last_unstale":"2023-10-08T13:50:14.780262+0000","last_undegraded":"2023-10-08T13:50:14.780262+0000","last_fullsized":"2023-10-08T13:50:14.780262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:36:22.156252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.977115+0000","last_change":"2023-10-08T13:50:15.977115+0000","last_active":"2023-10-08T13:50:15.977115+0000","last_peered":"2023-10-08T13:50:15.977115+0000","last_clean":"2023-10-08T13:50:15.977115+0000","last_became_active":"2023-10-08T13:50:15.976617+0000","last_became_peered":"2023-10-08T13:50:15.976617+0000","last_unstale":"2023-10-08T13:50:15.977115+0000","last_undegraded":"2023-10-08T13:50:15.977115+0000","last_fullsized":"2023-10-08T13:50:15.977115+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:57.124054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":128,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:41.826537+0000","last_change":"2023-10-08T13:49:43.777736+0000","last_active":"2023-10-08T13:50:41.826537+0000","last_peered":"2023-10-08T13:50:41.826537+0000","last_clean":"2023-10-08T13:50:41.826537+0000","last_became_active":"2023-10-08T13:49:43.777431+0000","last_became_peered":"2023-10-08T13:49:43.777431+0000","last_unstale":"2023-10-08T13:50:41.826537+0000","last_undegraded":"2023-10-08T13:50:41.826537+0000","last_fullsized":"2023-10-08T13:50:41.826537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:12.907196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58583,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":95,"num_write":35,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.774165+0000","last_change":"2023-10-08T13:49:44.789122+0000","last_active":"2023-10-08T13:50:14.774165+0000","last_peered":"2023-10-08T13:50:14.774165+0000","last_clean":"2023-10-08T13:50:14.774165+0000","last_became_active":"2023-10-08T13:49:44.788831+0000","last_became_peered":"2023-10-08T13:49:44.788831+0000","last_unstale":"2023-10-08T13:50:14.774165+0000","last_undegraded":"2023-10-08T13:50:14.774165+0000","last_fullsized":"2023-10-08T13:50:14.774165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:45.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776462+0000","last_change":"2023-10-08T13:49:49.157319+0000","last_active":"2023-10-08T13:50:14.776462+0000","last_peered":"2023-10-08T13:50:14.776462+0000","last_clean":"2023-10-08T13:50:14.776462+0000","last_became_active":"2023-10-08T13:49:49.156897+0000","last_became_peered":"2023-10-08T13:49:49.156897+0000","last_unstale":"2023-10-08T13:50:14.776462+0000","last_undegraded":"2023-10-08T13:50:14.776462+0000","last_fullsized":"2023-10-08T13:50:14.776462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:25.460707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780243+0000","last_change":"2023-10-08T13:49:43.776699+0000","last_active":"2023-10-08T13:50:14.780243+0000","last_peered":"2023-10-08T13:50:14.780243+0000","last_clean":"2023-10-08T13:50:14.780243+0000","last_became_active":"2023-10-08T13:49:43.776293+0000","last_became_peered":"2023-10-08T13:49:43.776293+0000","last_unstale":"2023-10-08T13:50:14.780243+0000","last_undegraded":"2023-10-08T13:50:14.780243+0000","last_fullsized":"2023-10-08T13:50:14.780243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:19.056804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778762+0000","last_change":"2023-10-08T13:49:44.789915+0000","last_active":"2023-10-08T13:50:14.778762+0000","last_peered":"2023-10-08T13:50:14.778762+0000","last_clean":"2023-10-08T13:50:14.778762+0000","last_became_active":"2023-10-08T13:49:44.789617+0000","last_became_peered":"2023-10-08T13:49:44.789617+0000","last_unstale":"2023-10-08T13:50:14.778762+0000","last_undegraded":"2023-10-08T13:50:14.778762+0000","last_fullsized":"2023-10-08T13:50:14.778762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:06.286155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777143+0000","last_change":"2023-10-08T13:49:49.165946+0000","last_active":"2023-10-08T13:50:14.777143+0000","last_peered":"2023-10-08T13:50:14.777143+0000","last_clean":"2023-10-08T13:50:14.777143+0000","last_became_active":"2023-10-08T13:49:49.165581+0000","last_became_peered":"2023-10-08T13:49:49.165581+0000","last_unstale":"2023-10-08T13:50:14.777143+0000","last_undegraded":"2023-10-08T13:50:14.777143+0000","last_fullsized":"2023-10-08T13:50:14.777143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:13.772198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777873+0000","last_change":"2023-10-08T13:49:43.779161+0000","last_active":"2023-10-08T13:50:14.777873+0000","last_peered":"2023-10-08T13:50:14.777873+0000","last_clean":"2023-10-08T13:50:14.777873+0000","last_became_active":"2023-10-08T13:49:43.778862+0000","last_became_peered":"2023-10-08T13:49:43.778862+0000","last_unstale":"2023-10-08T13:50:14.777873+0000","last_undegraded":"2023-10-08T13:50:14.777873+0000","last_fullsized":"2023-10-08T13:50:14.777873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:57:44.075322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778206+0000","last_change":"2023-10-08T13:49:44.782072+0000","last_active":"2023-10-08T13:50:14.778206+0000","last_peered":"2023-10-08T13:50:14.778206+0000","last_clean":"2023-10-08T13:50:14.778206+0000","last_became_active":"2023-10-08T13:49:44.781756+0000","last_became_peered":"2023-10-08T13:49:44.781756+0000","last_unstale":"2023-10-08T13:50:14.778206+0000","last_undegraded":"2023-10-08T13:50:14.778206+0000","last_fullsized":"2023-10-08T13:50:14.778206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:32.877818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:16.100169+0000","last_change":"2023-10-08T13:50:16.099755+0000","last_active":"2023-10-08T13:50:16.100169+0000","last_peered":"2023-10-08T13:50:16.100169+0000","last_clean":"2023-10-08T13:50:16.100169+0000","last_became_active":"2023-10-08T13:50:16.099267+0000","last_became_peered":"2023-10-08T13:50:16.099267+0000","last_unstale":"2023-10-08T13:50:16.100169+0000","last_undegraded":"2023-10-08T13:50:16.100169+0000","last_fullsized":"2023-10-08T13:50:16.100169+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:19.949246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:41.799215+0000","last_change":"2023-10-08T13:49:43.782822+0000","last_active":"2023-10-08T13:50:41.799215+0000","last_peered":"2023-10-08T13:50:41.799215+0000","last_clean":"2023-10-08T13:50:41.799215+0000","last_became_active":"2023-10-08T13:49:43.782524+0000","last_became_peered":"2023-10-08T13:49:43.782524+0000","last_unstale":"2023-10-08T13:50:41.799215+0000","last_undegraded":"2023-10-08T13:50:41.799215+0000","last_fullsized":"2023-10-08T13:50:41.799215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:11.847851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777500+0000","last_change":"2023-10-08T13:49:44.794296+0000","last_active":"2023-10-08T13:50:14.777500+0000","last_peered":"2023-10-08T13:50:14.777500+0000","last_clean":"2023-10-08T13:50:14.777500+0000","last_became_active":"2023-10-08T13:49:44.793987+0000","last_became_peered":"2023-10-08T13:49:44.793987+0000","last_unstale":"2023-10-08T13:50:14.777500+0000","last_undegraded":"2023-10-08T13:50:14.777500+0000","last_fullsized":"2023-10-08T13:50:14.777500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:41.562569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780499+0000","last_change":"2023-10-08T13:49:49.165016+0000","last_active":"2023-10-08T13:50:14.780499+0000","last_peered":"2023-10-08T13:50:14.780499+0000","last_clean":"2023-10-08T13:50:14.780499+0000","last_became_active":"2023-10-08T13:49:49.164531+0000","last_became_peered":"2023-10-08T13:49:49.164531+0000","last_unstale":"2023-10-08T13:50:14.780499+0000","last_undegraded":"2023-10-08T13:50:14.780499+0000","last_fullsized":"2023-10-08T13:50:14.780499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:45.014512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.772750+0000","last_change":"2023-10-08T13:49:43.786929+0000","last_active":"2023-10-08T13:50:14.772750+0000","last_peered":"2023-10-08T13:50:14.772750+0000","last_clean":"2023-10-08T13:50:14.772750+0000","last_became_active":"2023-10-08T13:49:43.786641+0000","last_became_peered":"2023-10-08T13:49:43.786641+0000","last_unstale":"2023-10-08T13:50:14.772750+0000","last_undegraded":"2023-10-08T13:50:14.772750+0000","last_fullsized":"2023-10-08T13:50:14.772750+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:30.054067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775577+0000","last_change":"2023-10-08T13:49:44.786718+0000","last_active":"2023-10-08T13:50:14.775577+0000","last_peered":"2023-10-08T13:50:14.775577+0000","last_clean":"2023-10-08T13:50:14.775577+0000","last_became_active":"2023-10-08T13:49:44.786413+0000","last_became_peered":"2023-10-08T13:49:44.786413+0000","last_unstale":"2023-10-08T13:50:14.775577+0000","last_undegraded":"2023-10-08T13:50:14.775577+0000","last_fullsized":"2023-10-08T13:50:14.775577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.302426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775695+0000","last_change":"2023-10-08T13:49:49.152226+0000","last_active":"2023-10-08T13:50:14.775695+0000","last_peered":"2023-10-08T13:50:14.775695+0000","last_clean":"2023-10-08T13:50:14.775695+0000","last_became_active":"2023-10-08T13:49:49.151772+0000","last_became_peered":"2023-10-08T13:49:49.151772+0000","last_unstale":"2023-10-08T13:50:14.775695+0000","last_undegraded":"2023-10-08T13:50:14.775695+0000","last_fullsized":"2023-10-08T13:50:14.775695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:53.856496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.793964+0000","last_change":"2023-10-08T13:50:15.793598+0000","last_active":"2023-10-08T13:50:15.793964+0000","last_peered":"2023-10-08T13:50:15.793964+0000","last_clean":"2023-10-08T13:50:15.793964+0000","last_became_active":"2023-10-08T13:50:15.786421+0000","last_became_peered":"2023-10-08T13:50:15.786421+0000","last_unstale":"2023-10-08T13:50:15.793964+0000","last_undegraded":"2023-10-08T13:50:15.793964+0000","last_fullsized":"2023-10-08T13:50:15.793964+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:09:37.586245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776958+0000","last_change":"2023-10-08T13:49:44.782805+0000","last_active":"2023-10-08T13:50:14.776958+0000","last_peered":"2023-10-08T13:50:14.776958+0000","last_clean":"2023-10-08T13:50:14.776958+0000","last_became_active":"2023-10-08T13:49:44.782485+0000","last_became_peered":"2023-10-08T13:49:44.782485+0000","last_unstale":"2023-10-08T13:50:14.776958+0000","last_undegraded":"2023-10-08T13:50:14.776958+0000","last_fullsized":"2023-10-08T13:50:14.776958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:19.805734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773088+0000","last_change":"2023-10-08T13:49:49.546226+0000","last_active":"2023-10-08T13:50:14.773088+0000","last_peered":"2023-10-08T13:50:14.773088+0000","last_clean":"2023-10-08T13:50:14.773088+0000","last_became_active":"2023-10-08T13:49:49.545626+0000","last_became_peered":"2023-10-08T13:49:49.545626+0000","last_unstale":"2023-10-08T13:50:14.773088+0000","last_undegraded":"2023-10-08T13:50:14.773088+0000","last_fullsized":"2023-10-08T13:50:14.773088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:37.184944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773503+0000","last_change":"2023-10-08T13:49:43.784705+0000","last_active":"2023-10-08T13:50:14.773503+0000","last_peered":"2023-10-08T13:50:14.773503+0000","last_clean":"2023-10-08T13:50:14.773503+0000","last_became_active":"2023-10-08T13:49:43.784391+0000","last_became_peered":"2023-10-08T13:49:43.784391+0000","last_unstale":"2023-10-08T13:50:14.773503+0000","last_undegraded":"2023-10-08T13:50:14.773503+0000","last_fullsized":"2023-10-08T13:50:14.773503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:45.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776355+0000","last_change":"2023-10-08T13:49:44.789845+0000","last_active":"2023-10-08T13:50:14.776355+0000","last_peered":"2023-10-08T13:50:14.776355+0000","last_clean":"2023-10-08T13:50:14.776355+0000","last_became_active":"2023-10-08T13:49:44.789580+0000","last_became_peered":"2023-10-08T13:49:44.789580+0000","last_unstale":"2023-10-08T13:50:14.776355+0000","last_undegraded":"2023-10-08T13:50:14.776355+0000","last_fullsized":"2023-10-08T13:50:14.776355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:58.098757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778757+0000","last_change":"2023-10-08T13:49:49.153277+0000","last_active":"2023-10-08T13:50:14.778757+0000","last_peered":"2023-10-08T13:50:14.778757+0000","last_clean":"2023-10-08T13:50:14.778757+0000","last_became_active":"2023-10-08T13:49:49.152575+0000","last_became_peered":"2023-10-08T13:49:49.152575+0000","last_unstale":"2023-10-08T13:50:14.778757+0000","last_undegraded":"2023-10-08T13:50:14.778757+0000","last_fullsized":"2023-10-08T13:50:14.778757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:35.677823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:41.799278+0000","last_change":"2023-10-08T13:49:43.778442+0000","last_active":"2023-10-08T13:50:41.799278+0000","last_peered":"2023-10-08T13:50:41.799278+0000","last_clean":"2023-10-08T13:50:41.799278+0000","last_became_active":"2023-10-08T13:49:43.774799+0000","last_became_peered":"2023-10-08T13:49:43.774799+0000","last_unstale":"2023-10-08T13:50:41.799278+0000","last_undegraded":"2023-10-08T13:50:41.799278+0000","last_fullsized":"2023-10-08T13:50:41.799278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:15.464736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776195+0000","last_change":"2023-10-08T13:49:44.780097+0000","last_active":"2023-10-08T13:50:14.776195+0000","last_peered":"2023-10-08T13:50:14.776195+0000","last_clean":"2023-10-08T13:50:14.776195+0000","last_became_active":"2023-10-08T13:49:44.779774+0000","last_became_peered":"2023-10-08T13:49:44.779774+0000","last_unstale":"2023-10-08T13:50:14.776195+0000","last_undegraded":"2023-10-08T13:50:14.776195+0000","last_fullsized":"2023-10-08T13:50:14.776195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:19.132295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779203+0000","last_change":"2023-10-08T13:49:49.547129+0000","last_active":"2023-10-08T13:50:14.779203+0000","last_peered":"2023-10-08T13:50:14.779203+0000","last_clean":"2023-10-08T13:50:14.779203+0000","last_became_active":"2023-10-08T13:49:49.546694+0000","last_became_peered":"2023-10-08T13:49:49.546694+0000","last_unstale":"2023-10-08T13:50:14.779203+0000","last_undegraded":"2023-10-08T13:50:14.779203+0000","last_fullsized":"2023-10-08T13:50:14.779203+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:45.524650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775627+0000","last_change":"2023-10-08T13:49:44.779875+0000","last_active":"2023-10-08T13:50:14.775627+0000","last_peered":"2023-10-08T13:50:14.775627+0000","last_clean":"2023-10-08T13:50:14.775627+0000","last_became_active":"2023-10-08T13:49:44.779570+0000","last_became_peered":"2023-10-08T13:49:44.779570+0000","last_unstale":"2023-10-08T13:50:14.775627+0000","last_undegraded":"2023-10-08T13:50:14.775627+0000","last_fullsized":"2023-10-08T13:50:14.775627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:02.981998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:38.794493+0000","last_change":"2023-10-08T13:49:43.787574+0000","last_active":"2023-10-08T13:50:38.794493+0000","last_peered":"2023-10-08T13:50:38.794493+0000","last_clean":"2023-10-08T13:50:38.794493+0000","last_became_active":"2023-10-08T13:49:43.787261+0000","last_became_peered":"2023-10-08T13:49:43.787261+0000","last_unstale":"2023-10-08T13:50:38.794493+0000","last_undegraded":"2023-10-08T13:50:38.794493+0000","last_fullsized":"2023-10-08T13:50:38.794493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:28.676135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777768+0000","last_change":"2023-10-08T13:49:49.163856+0000","last_active":"2023-10-08T13:50:14.777768+0000","last_peered":"2023-10-08T13:50:14.777768+0000","last_clean":"2023-10-08T13:50:14.777768+0000","last_became_active":"2023-10-08T13:49:49.163456+0000","last_became_peered":"2023-10-08T13:49:49.163456+0000","last_unstale":"2023-10-08T13:50:14.777768+0000","last_undegraded":"2023-10-08T13:50:14.777768+0000","last_fullsized":"2023-10-08T13:50:14.777768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:31.654309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775665+0000","last_change":"2023-10-08T13:49:44.792935+0000","last_active":"2023-10-08T13:50:14.775665+0000","last_peered":"2023-10-08T13:50:14.775665+0000","last_clean":"2023-10-08T13:50:14.775665+0000","last_became_active":"2023-10-08T13:49:44.792667+0000","last_became_peered":"2023-10-08T13:49:44.792667+0000","last_unstale":"2023-10-08T13:50:14.775665+0000","last_undegraded":"2023-10-08T13:50:14.775665+0000","last_fullsized":"2023-10-08T13:50:14.775665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:40.964452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773491+0000","last_change":"2023-10-08T13:49:43.781424+0000","last_active":"2023-10-08T13:50:14.773491+0000","last_peered":"2023-10-08T13:50:14.773491+0000","last_clean":"2023-10-08T13:50:14.773491+0000","last_became_active":"2023-10-08T13:49:43.780930+0000","last_became_peered":"2023-10-08T13:49:43.780930+0000","last_unstale":"2023-10-08T13:50:14.773491+0000","last_undegraded":"2023-10-08T13:50:14.773491+0000","last_fullsized":"2023-10-08T13:50:14.773491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:56:50.397289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":325,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777396+0000","last_change":"2023-10-08T13:48:45.192219+0000","last_active":"2023-10-08T13:50:14.777396+0000","last_peered":"2023-10-08T13:50:14.777396+0000","last_clean":"2023-10-08T13:50:14.777396+0000","last_became_active":"2023-10-08T13:48:45.191813+0000","last_became_peered":"2023-10-08T13:48:45.191813+0000","last_unstale":"2023-10-08T13:50:14.777396+0000","last_undegraded":"2023-10-08T13:50:14.777396+0000","last_fullsized":"2023-10-08T13:50:14.777396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_clean_scrub_stamp":"2023-10-08T13:45:51.040695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.303937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776642+0000","last_change":"2023-10-08T13:49:49.546181+0000","last_active":"2023-10-08T13:50:14.776642+0000","last_peered":"2023-10-08T13:50:14.776642+0000","last_clean":"2023-10-08T13:50:14.776642+0000","last_became_active":"2023-10-08T13:49:49.545726+0000","last_became_peered":"2023-10-08T13:49:49.545726+0000","last_unstale":"2023-10-08T13:50:14.776642+0000","last_undegraded":"2023-10-08T13:50:14.776642+0000","last_fullsized":"2023-10-08T13:50:14.776642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:45.370909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778348+0000","last_change":"2023-10-08T13:49:44.785544+0000","last_active":"2023-10-08T13:50:14.778348+0000","last_peered":"2023-10-08T13:50:14.778348+0000","last_clean":"2023-10-08T13:50:14.778348+0000","last_became_active":"2023-10-08T13:49:44.785007+0000","last_became_peered":"2023-10-08T13:49:44.785007+0000","last_unstale":"2023-10-08T13:50:14.778348+0000","last_undegraded":"2023-10-08T13:50:14.778348+0000","last_fullsized":"2023-10-08T13:50:14.778348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:16:55.058693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777478+0000","last_change":"2023-10-08T13:49:43.779207+0000","last_active":"2023-10-08T13:50:14.777478+0000","last_peered":"2023-10-08T13:50:14.777478+0000","last_clean":"2023-10-08T13:50:14.777478+0000","last_became_active":"2023-10-08T13:49:43.778846+0000","last_became_peered":"2023-10-08T13:49:43.778846+0000","last_unstale":"2023-10-08T13:50:14.777478+0000","last_undegraded":"2023-10-08T13:50:14.777478+0000","last_fullsized":"2023-10-08T13:50:14.777478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:45.868187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776400+0000","last_change":"2023-10-08T13:49:49.162156+0000","last_active":"2023-10-08T13:50:14.776400+0000","last_peered":"2023-10-08T13:50:14.776400+0000","last_clean":"2023-10-08T13:50:14.776400+0000","last_became_active":"2023-10-08T13:49:49.161720+0000","last_became_peered":"2023-10-08T13:49:49.161720+0000","last_unstale":"2023-10-08T13:50:14.776400+0000","last_undegraded":"2023-10-08T13:50:14.776400+0000","last_fullsized":"2023-10-08T13:50:14.776400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:13.499915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.774269+0000","last_change":"2023-10-08T13:49:43.782322+0000","last_active":"2023-10-08T13:50:14.774269+0000","last_peered":"2023-10-08T13:50:14.774269+0000","last_clean":"2023-10-08T13:50:14.774269+0000","last_became_active":"2023-10-08T13:49:43.781938+0000","last_became_peered":"2023-10-08T13:49:43.781938+0000","last_unstale":"2023-10-08T13:50:14.774269+0000","last_undegraded":"2023-10-08T13:50:14.774269+0000","last_fullsized":"2023-10-08T13:50:14.774269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:57.627507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778880+0000","last_change":"2023-10-08T13:49:44.778340+0000","last_active":"2023-10-08T13:50:14.778880+0000","last_peered":"2023-10-08T13:50:14.778880+0000","last_clean":"2023-10-08T13:50:14.778880+0000","last_became_active":"2023-10-08T13:49:44.778000+0000","last_became_peered":"2023-10-08T13:49:44.778000+0000","last_unstale":"2023-10-08T13:50:14.778880+0000","last_undegraded":"2023-10-08T13:50:14.778880+0000","last_fullsized":"2023-10-08T13:50:14.778880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:23.350449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775457+0000","last_change":"2023-10-08T13:49:49.545915+0000","last_active":"2023-10-08T13:50:14.775457+0000","last_peered":"2023-10-08T13:50:14.775457+0000","last_clean":"2023-10-08T13:50:14.775457+0000","last_became_active":"2023-10-08T13:49:49.545396+0000","last_became_peered":"2023-10-08T13:49:49.545396+0000","last_unstale":"2023-10-08T13:50:14.775457+0000","last_undegraded":"2023-10-08T13:50:14.775457+0000","last_fullsized":"2023-10-08T13:50:14.775457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:07:03.850707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773842+0000","last_change":"2023-10-08T13:49:43.776479+0000","last_active":"2023-10-08T13:50:14.773842+0000","last_peered":"2023-10-08T13:50:14.773842+0000","last_clean":"2023-10-08T13:50:14.773842+0000","last_became_active":"2023-10-08T13:49:43.776130+0000","last_became_peered":"2023-10-08T13:49:43.776130+0000","last_unstale":"2023-10-08T13:50:14.773842+0000","last_undegraded":"2023-10-08T13:50:14.773842+0000","last_fullsized":"2023-10-08T13:50:14.773842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.760660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776442+0000","last_change":"2023-10-08T13:49:44.793864+0000","last_active":"2023-10-08T13:50:14.776442+0000","last_peered":"2023-10-08T13:50:14.776442+0000","last_clean":"2023-10-08T13:50:14.776442+0000","last_became_active":"2023-10-08T13:49:44.793469+0000","last_became_peered":"2023-10-08T13:49:44.793469+0000","last_unstale":"2023-10-08T13:50:14.776442+0000","last_undegraded":"2023-10-08T13:50:14.776442+0000","last_fullsized":"2023-10-08T13:50:14.776442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:28.669711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779427+0000","last_change":"2023-10-08T13:49:49.150696+0000","last_active":"2023-10-08T13:50:14.779427+0000","last_peered":"2023-10-08T13:50:14.779427+0000","last_clean":"2023-10-08T13:50:14.779427+0000","last_became_active":"2023-10-08T13:49:49.150188+0000","last_became_peered":"2023-10-08T13:49:49.150188+0000","last_unstale":"2023-10-08T13:50:14.779427+0000","last_undegraded":"2023-10-08T13:50:14.779427+0000","last_fullsized":"2023-10-08T13:50:14.779427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:51:35.775240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.774208+0000","last_change":"2023-10-08T13:49:43.786245+0000","last_active":"2023-10-08T13:50:14.774208+0000","last_peered":"2023-10-08T13:50:14.774208+0000","last_clean":"2023-10-08T13:50:14.774208+0000","last_became_active":"2023-10-08T13:49:43.785986+0000","last_became_peered":"2023-10-08T13:49:43.785986+0000","last_unstale":"2023-10-08T13:50:14.774208+0000","last_undegraded":"2023-10-08T13:50:14.774208+0000","last_fullsized":"2023-10-08T13:50:14.774208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:53:59.643447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780099+0000","last_change":"2023-10-08T13:49:44.790494+0000","last_active":"2023-10-08T13:50:14.780099+0000","last_peered":"2023-10-08T13:50:14.780099+0000","last_clean":"2023-10-08T13:50:14.780099+0000","last_became_active":"2023-10-08T13:49:44.790091+0000","last_became_peered":"2023-10-08T13:49:44.790091+0000","last_unstale":"2023-10-08T13:50:14.780099+0000","last_undegraded":"2023-10-08T13:50:14.780099+0000","last_fullsized":"2023-10-08T13:50:14.780099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:49.309597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773799+0000","last_change":"2023-10-08T13:49:49.547271+0000","last_active":"2023-10-08T13:50:14.773799+0000","last_peered":"2023-10-08T13:50:14.773799+0000","last_clean":"2023-10-08T13:50:14.773799+0000","last_became_active":"2023-10-08T13:49:49.546603+0000","last_became_peered":"2023-10-08T13:49:49.546603+0000","last_unstale":"2023-10-08T13:50:14.773799+0000","last_undegraded":"2023-10-08T13:50:14.773799+0000","last_fullsized":"2023-10-08T13:50:14.773799+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:29.908321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779809+0000","last_change":"2023-10-08T13:49:43.788638+0000","last_active":"2023-10-08T13:50:14.779809+0000","last_peered":"2023-10-08T13:50:14.779809+0000","last_clean":"2023-10-08T13:50:14.779809+0000","last_became_active":"2023-10-08T13:49:43.788303+0000","last_became_peered":"2023-10-08T13:49:43.788303+0000","last_unstale":"2023-10-08T13:50:14.779809+0000","last_undegraded":"2023-10-08T13:50:14.779809+0000","last_fullsized":"2023-10-08T13:50:14.779809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:52.637775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.772743+0000","last_change":"2023-10-08T13:49:44.777546+0000","last_active":"2023-10-08T13:50:14.772743+0000","last_peered":"2023-10-08T13:50:14.772743+0000","last_clean":"2023-10-08T13:50:14.772743+0000","last_became_active":"2023-10-08T13:49:44.777261+0000","last_became_peered":"2023-10-08T13:49:44.777261+0000","last_unstale":"2023-10-08T13:50:14.772743+0000","last_undegraded":"2023-10-08T13:50:14.772743+0000","last_fullsized":"2023-10-08T13:50:14.772743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:26:36.368188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.790468+0000","last_change":"2023-10-08T13:50:15.787317+0000","last_active":"2023-10-08T13:50:15.790468+0000","last_peered":"2023-10-08T13:50:15.790468+0000","last_clean":"2023-10-08T13:50:15.790468+0000","last_became_active":"2023-10-08T13:50:15.786816+0000","last_became_peered":"2023-10-08T13:50:15.786816+0000","last_unstale":"2023-10-08T13:50:15.790468+0000","last_undegraded":"2023-10-08T13:50:15.790468+0000","last_fullsized":"2023-10-08T13:50:15.790468+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:53.766803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775828+0000","last_change":"2023-10-08T13:49:43.776693+0000","last_active":"2023-10-08T13:50:14.775828+0000","last_peered":"2023-10-08T13:50:14.775828+0000","last_clean":"2023-10-08T13:50:14.775828+0000","last_became_active":"2023-10-08T13:49:43.776367+0000","last_became_peered":"2023-10-08T13:49:43.776367+0000","last_unstale":"2023-10-08T13:50:14.775828+0000","last_undegraded":"2023-10-08T13:50:14.775828+0000","last_fullsized":"2023-10-08T13:50:14.775828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:17.415011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776044+0000","last_change":"2023-10-08T13:49:44.779472+0000","last_active":"2023-10-08T13:50:14.776044+0000","last_peered":"2023-10-08T13:50:14.776044+0000","last_clean":"2023-10-08T13:50:14.776044+0000","last_became_active":"2023-10-08T13:49:44.779172+0000","last_became_peered":"2023-10-08T13:49:44.779172+0000","last_unstale":"2023-10-08T13:50:14.776044+0000","last_undegraded":"2023-10-08T13:50:14.776044+0000","last_fullsized":"2023-10-08T13:50:14.776044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:17.314243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773591+0000","last_change":"2023-10-08T13:49:49.164535+0000","last_active":"2023-10-08T13:50:14.773591+0000","last_peered":"2023-10-08T13:50:14.773591+0000","last_clean":"2023-10-08T13:50:14.773591+0000","last_became_active":"2023-10-08T13:49:49.163467+0000","last_became_peered":"2023-10-08T13:49:49.163467+0000","last_unstale":"2023-10-08T13:50:14.773591+0000","last_undegraded":"2023-10-08T13:50:14.773591+0000","last_fullsized":"2023-10-08T13:50:14.773591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:55.058656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780711+0000","last_change":"2023-10-08T13:49:43.775311+0000","last_active":"2023-10-08T13:50:14.780711+0000","last_peered":"2023-10-08T13:50:14.780711+0000","last_clean":"2023-10-08T13:50:14.780711+0000","last_became_active":"2023-10-08T13:49:43.775005+0000","last_became_peered":"2023-10-08T13:49:43.775005+0000","last_unstale":"2023-10-08T13:50:14.780711+0000","last_undegraded":"2023-10-08T13:50:14.780711+0000","last_fullsized":"2023-10-08T13:50:14.780711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:33.713395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776317+0000","last_change":"2023-10-08T13:49:44.792051+0000","last_active":"2023-10-08T13:50:14.776317+0000","last_peered":"2023-10-08T13:50:14.776317+0000","last_clean":"2023-10-08T13:50:14.776317+0000","last_became_active":"2023-10-08T13:49:44.790045+0000","last_became_peered":"2023-10-08T13:49:44.790045+0000","last_unstale":"2023-10-08T13:50:14.776317+0000","last_undegraded":"2023-10-08T13:50:14.776317+0000","last_fullsized":"2023-10-08T13:50:14.776317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:28.699799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773740+0000","last_change":"2023-10-08T13:49:49.546219+0000","last_active":"2023-10-08T13:50:14.773740+0000","last_peered":"2023-10-08T13:50:14.773740+0000","last_clean":"2023-10-08T13:50:14.773740+0000","last_became_active":"2023-10-08T13:49:49.545783+0000","last_became_peered":"2023-10-08T13:49:49.545783+0000","last_unstale":"2023-10-08T13:50:14.773740+0000","last_undegraded":"2023-10-08T13:50:14.773740+0000","last_fullsized":"2023-10-08T13:50:14.773740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:00.497908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.772903+0000","last_change":"2023-10-08T13:49:43.776252+0000","last_active":"2023-10-08T13:50:14.772903+0000","last_peered":"2023-10-08T13:50:14.772903+0000","last_clean":"2023-10-08T13:50:14.772903+0000","last_became_active":"2023-10-08T13:49:43.775935+0000","last_became_peered":"2023-10-08T13:49:43.775935+0000","last_unstale":"2023-10-08T13:50:14.772903+0000","last_undegraded":"2023-10-08T13:50:14.772903+0000","last_fullsized":"2023-10-08T13:50:14.772903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:58.949365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778127+0000","last_change":"2023-10-08T13:49:44.788628+0000","last_active":"2023-10-08T13:50:14.778127+0000","last_peered":"2023-10-08T13:50:14.778127+0000","last_clean":"2023-10-08T13:50:14.778127+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:50:14.778127+0000","last_undegraded":"2023-10-08T13:50:14.778127+0000","last_fullsized":"2023-10-08T13:50:14.778127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:46:42.311296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779858+0000","last_change":"2023-10-08T13:49:49.153641+0000","last_active":"2023-10-08T13:50:14.779858+0000","last_peered":"2023-10-08T13:50:14.779858+0000","last_clean":"2023-10-08T13:50:14.779858+0000","last_became_active":"2023-10-08T13:49:49.153172+0000","last_became_peered":"2023-10-08T13:49:49.153172+0000","last_unstale":"2023-10-08T13:50:14.779858+0000","last_undegraded":"2023-10-08T13:50:14.779858+0000","last_fullsized":"2023-10-08T13:50:14.779858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:40:51.726343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:37.780504+0000","last_change":"2023-10-08T13:49:43.782683+0000","last_active":"2023-10-08T13:50:37.780504+0000","last_peered":"2023-10-08T13:50:37.780504+0000","last_clean":"2023-10-08T13:50:37.780504+0000","last_became_active":"2023-10-08T13:49:43.782269+0000","last_became_peered":"2023-10-08T13:49:43.782269+0000","last_unstale":"2023-10-08T13:50:37.780504+0000","last_undegraded":"2023-10-08T13:50:37.780504+0000","last_fullsized":"2023-10-08T13:50:37.780504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:50.850581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777949+0000","last_change":"2023-10-08T13:49:44.787894+0000","last_active":"2023-10-08T13:50:14.777949+0000","last_peered":"2023-10-08T13:50:14.777949+0000","last_clean":"2023-10-08T13:50:14.777949+0000","last_became_active":"2023-10-08T13:49:44.787599+0000","last_became_peered":"2023-10-08T13:49:44.787599+0000","last_unstale":"2023-10-08T13:50:14.777949+0000","last_undegraded":"2023-10-08T13:50:14.777949+0000","last_fullsized":"2023-10-08T13:50:14.777949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:17:15.131051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779473+0000","last_change":"2023-10-08T13:49:49.164256+0000","last_active":"2023-10-08T13:50:14.779473+0000","last_peered":"2023-10-08T13:50:14.779473+0000","last_clean":"2023-10-08T13:50:14.779473+0000","last_became_active":"2023-10-08T13:49:49.163793+0000","last_became_peered":"2023-10-08T13:49:49.163793+0000","last_unstale":"2023-10-08T13:50:14.779473+0000","last_undegraded":"2023-10-08T13:50:14.779473+0000","last_fullsized":"2023-10-08T13:50:14.779473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:19.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776219+0000","last_change":"2023-10-08T13:49:43.778311+0000","last_active":"2023-10-08T13:50:14.776219+0000","last_peered":"2023-10-08T13:50:14.776219+0000","last_clean":"2023-10-08T13:50:14.776219+0000","last_became_active":"2023-10-08T13:49:43.777984+0000","last_became_peered":"2023-10-08T13:49:43.777984+0000","last_unstale":"2023-10-08T13:50:14.776219+0000","last_undegraded":"2023-10-08T13:50:14.776219+0000","last_fullsized":"2023-10-08T13:50:14.776219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:56.890729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777769+0000","last_change":"2023-10-08T13:49:44.793424+0000","last_active":"2023-10-08T13:50:14.777769+0000","last_peered":"2023-10-08T13:50:14.777769+0000","last_clean":"2023-10-08T13:50:14.777769+0000","last_became_active":"2023-10-08T13:49:44.792825+0000","last_became_peered":"2023-10-08T13:49:44.792825+0000","last_unstale":"2023-10-08T13:50:14.777769+0000","last_undegraded":"2023-10-08T13:50:14.777769+0000","last_fullsized":"2023-10-08T13:50:14.777769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:39.463676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775289+0000","last_change":"2023-10-08T13:49:49.151915+0000","last_active":"2023-10-08T13:50:14.775289+0000","last_peered":"2023-10-08T13:50:14.775289+0000","last_clean":"2023-10-08T13:50:14.775289+0000","last_became_active":"2023-10-08T13:49:49.151434+0000","last_became_peered":"2023-10-08T13:49:49.151434+0000","last_unstale":"2023-10-08T13:50:14.775289+0000","last_undegraded":"2023-10-08T13:50:14.775289+0000","last_fullsized":"2023-10-08T13:50:14.775289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:54:07.631498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.787243+0000","last_change":"2023-10-08T13:50:15.787243+0000","last_active":"2023-10-08T13:50:15.787243+0000","last_peered":"2023-10-08T13:50:15.787243+0000","last_clean":"2023-10-08T13:50:15.787243+0000","last_became_active":"2023-10-08T13:50:15.782305+0000","last_became_peered":"2023-10-08T13:50:15.782305+0000","last_unstale":"2023-10-08T13:50:15.787243+0000","last_undegraded":"2023-10-08T13:50:15.787243+0000","last_fullsized":"2023-10-08T13:50:15.787243+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:08.520010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778420+0000","last_change":"2023-10-08T13:49:44.790949+0000","last_active":"2023-10-08T13:50:14.778420+0000","last_peered":"2023-10-08T13:50:14.778420+0000","last_clean":"2023-10-08T13:50:14.778420+0000","last_became_active":"2023-10-08T13:49:44.790645+0000","last_became_peered":"2023-10-08T13:49:44.790645+0000","last_unstale":"2023-10-08T13:50:14.778420+0000","last_undegraded":"2023-10-08T13:50:14.778420+0000","last_fullsized":"2023-10-08T13:50:14.778420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:48.890730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776140+0000","last_change":"2023-10-08T13:49:44.781165+0000","last_active":"2023-10-08T13:50:14.776140+0000","last_peered":"2023-10-08T13:50:14.776140+0000","last_clean":"2023-10-08T13:50:14.776140+0000","last_became_active":"2023-10-08T13:49:44.780828+0000","last_became_peered":"2023-10-08T13:49:44.780828+0000","last_unstale":"2023-10-08T13:50:14.776140+0000","last_undegraded":"2023-10-08T13:50:14.776140+0000","last_fullsized":"2023-10-08T13:50:14.776140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:04.259482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779799+0000","last_change":"2023-10-08T13:49:44.781271+0000","last_active":"2023-10-08T13:50:14.779799+0000","last_peered":"2023-10-08T13:50:14.779799+0000","last_clean":"2023-10-08T13:50:14.779799+0000","last_became_active":"2023-10-08T13:49:44.780912+0000","last_became_peered":"2023-10-08T13:49:44.780912+0000","last_unstale":"2023-10-08T13:50:14.779799+0000","last_undegraded":"2023-10-08T13:50:14.779799+0000","last_fullsized":"2023-10-08T13:50:14.779799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:17.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780636+0000","last_change":"2023-10-08T13:49:44.777613+0000","last_active":"2023-10-08T13:50:14.780636+0000","last_peered":"2023-10-08T13:50:14.780636+0000","last_clean":"2023-10-08T13:50:14.780636+0000","last_became_active":"2023-10-08T13:49:44.777304+0000","last_became_peered":"2023-10-08T13:49:44.777304+0000","last_unstale":"2023-10-08T13:50:14.780636+0000","last_undegraded":"2023-10-08T13:50:14.780636+0000","last_fullsized":"2023-10-08T13:50:14.780636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:45.482935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779823+0000","last_change":"2023-10-08T13:49:44.782685+0000","last_active":"2023-10-08T13:50:14.779823+0000","last_peered":"2023-10-08T13:50:14.779823+0000","last_clean":"2023-10-08T13:50:14.779823+0000","last_became_active":"2023-10-08T13:49:44.781816+0000","last_became_peered":"2023-10-08T13:49:44.781816+0000","last_unstale":"2023-10-08T13:50:14.779823+0000","last_undegraded":"2023-10-08T13:50:14.779823+0000","last_fullsized":"2023-10-08T13:50:14.779823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:53.781237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778962+0000","last_change":"2023-10-08T13:49:44.783973+0000","last_active":"2023-10-08T13:50:14.778962+0000","last_peered":"2023-10-08T13:50:14.778962+0000","last_clean":"2023-10-08T13:50:14.778962+0000","last_became_active":"2023-10-08T13:49:44.783648+0000","last_became_peered":"2023-10-08T13:49:44.783648+0000","last_unstale":"2023-10-08T13:50:14.778962+0000","last_undegraded":"2023-10-08T13:50:14.778962+0000","last_fullsized":"2023-10-08T13:50:14.778962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:01.253610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776705+0000","last_change":"2023-10-08T13:49:44.781848+0000","last_active":"2023-10-08T13:50:14.776705+0000","last_peered":"2023-10-08T13:50:14.776705+0000","last_clean":"2023-10-08T13:50:14.776705+0000","last_became_active":"2023-10-08T13:49:44.781542+0000","last_became_peered":"2023-10-08T13:49:44.781542+0000","last_unstale":"2023-10-08T13:50:14.776705+0000","last_undegraded":"2023-10-08T13:50:14.776705+0000","last_fullsized":"2023-10-08T13:50:14.776705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:26:38.726298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780659+0000","last_change":"2023-10-08T13:49:44.786541+0000","last_active":"2023-10-08T13:50:14.780659+0000","last_peered":"2023-10-08T13:50:14.780659+0000","last_clean":"2023-10-08T13:50:14.780659+0000","last_became_active":"2023-10-08T13:49:44.786214+0000","last_became_peered":"2023-10-08T13:49:44.786214+0000","last_unstale":"2023-10-08T13:50:14.780659+0000","last_undegraded":"2023-10-08T13:50:14.780659+0000","last_fullsized":"2023-10-08T13:50:14.780659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:30:18.342774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776997+0000","last_change":"2023-10-08T13:49:44.788445+0000","last_active":"2023-10-08T13:50:14.776997+0000","last_peered":"2023-10-08T13:50:14.776997+0000","last_clean":"2023-10-08T13:50:14.776997+0000","last_became_active":"2023-10-08T13:49:44.788075+0000","last_became_peered":"2023-10-08T13:49:44.788075+0000","last_unstale":"2023-10-08T13:50:14.776997+0000","last_undegraded":"2023-10-08T13:50:14.776997+0000","last_fullsized":"2023-10-08T13:50:14.776997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:08:20.701231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775451+0000","last_change":"2023-10-08T13:49:44.783755+0000","last_active":"2023-10-08T13:50:14.775451+0000","last_peered":"2023-10-08T13:50:14.775451+0000","last_clean":"2023-10-08T13:50:14.775451+0000","last_became_active":"2023-10-08T13:49:44.783467+0000","last_became_peered":"2023-10-08T13:49:44.783467+0000","last_unstale":"2023-10-08T13:50:14.775451+0000","last_undegraded":"2023-10-08T13:50:14.775451+0000","last_fullsized":"2023-10-08T13:50:14.775451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:39.838940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775876+0000","last_change":"2023-10-08T13:49:44.779342+0000","last_active":"2023-10-08T13:50:14.775876+0000","last_peered":"2023-10-08T13:50:14.775876+0000","last_clean":"2023-10-08T13:50:14.775876+0000","last_became_active":"2023-10-08T13:49:44.778518+0000","last_became_peered":"2023-10-08T13:49:44.778518+0000","last_unstale":"2023-10-08T13:50:14.775876+0000","last_undegraded":"2023-10-08T13:50:14.775876+0000","last_fullsized":"2023-10-08T13:50:14.775876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:33.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779999+0000","last_change":"2023-10-08T13:49:44.792955+0000","last_active":"2023-10-08T13:50:14.779999+0000","last_peered":"2023-10-08T13:50:14.779999+0000","last_clean":"2023-10-08T13:50:14.779999+0000","last_became_active":"2023-10-08T13:49:44.792637+0000","last_became_peered":"2023-10-08T13:49:44.792637+0000","last_unstale":"2023-10-08T13:50:14.779999+0000","last_undegraded":"2023-10-08T13:50:14.779999+0000","last_fullsized":"2023-10-08T13:50:14.779999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:40.635913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778825+0000","last_change":"2023-10-08T13:49:44.786817+0000","last_active":"2023-10-08T13:50:14.778825+0000","last_peered":"2023-10-08T13:50:14.778825+0000","last_clean":"2023-10-08T13:50:14.778825+0000","last_became_active":"2023-10-08T13:49:44.786439+0000","last_became_peered":"2023-10-08T13:49:44.786439+0000","last_unstale":"2023-10-08T13:50:14.778825+0000","last_undegraded":"2023-10-08T13:50:14.778825+0000","last_fullsized":"2023-10-08T13:50:14.778825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:45.308600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.788414+0000","last_change":"2023-10-08T13:50:15.788414+0000","last_active":"2023-10-08T13:50:15.788414+0000","last_peered":"2023-10-08T13:50:15.788414+0000","last_clean":"2023-10-08T13:50:15.788414+0000","last_became_active":"2023-10-08T13:50:15.788079+0000","last_became_peered":"2023-10-08T13:50:15.788079+0000","last_unstale":"2023-10-08T13:50:15.788414+0000","last_undegraded":"2023-10-08T13:50:15.788414+0000","last_fullsized":"2023-10-08T13:50:15.788414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:43.723339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65395,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":371,"num_read_kb":259,"num_write":233,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":196185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416408,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":511889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77700000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.754},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.88100000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65400000000000003}]},{"osd":6,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.94899999999999995}]},{"osd":7,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":1.0089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":9,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.85699999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.024}]}]},{"osd":10,"up_from":71,"seq":304942678044,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28696,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716456,"statfs":{"total":95995035648,"available":95965650944,"internally_reserved":0,"allocated":1720320,"data_stored":1157780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.72899999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65700000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78200000000000003}]},{"osd":6,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55300000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.86199999999999999}]},{"osd":9,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.80600000000000005}]},{"osd":11,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.879}]}]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":516017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.499,"15min":0.496},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.4279999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":1.27,"5min":0.65600000000000003,"15min":0.55400000000000005},"min":{"1min":0.31900000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":16.077000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.46999999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.6029999999999998,"5min":4.6029999999999998,"15min":4.6029999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.47399999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.42799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.442,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.501}]},{"osd":3,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.57499999999999996,"15min":0.54100000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.9969999999999999,"5min":6.9969999999999999,"15min":6.9969999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.35199999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.7589999999999999,"5min":3.7589999999999999,"15min":3.7589999999999999},"last":0.40799999999999997}]},{"osd":4,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.748,"15min":0.71999999999999997},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.81000000000000005,"15min":0.78600000000000003},"min":{"1min":0.47799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":5.3390000000000004,"5min":5.3390000000000004,"15min":5.3390000000000004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73799999999999999,"15min":0.72199999999999998},"min":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.0410000000000004,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.59299999999999997}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.9570000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.345}]},{"osd":7,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":0.83599999999999997,"15min":0.80100000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":7.3650000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.87,"15min":0.86299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":4.3680000000000003,"5min":4.3680000000000003,"15min":4.3680000000000003},"last":0.56699999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.88200000000000001,"15min":0.86799999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":5.2930000000000001,"5min":5.2930000000000001,"15min":5.2930000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.86499999999999999,"15min":0.82899999999999996},"min":{"1min":0.498,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":0.61699999999999999}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.65000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":8,"up_from":58,"seq":249108103204,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":507599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.57399999999999995,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":5.5860000000000003,"5min":5.5860000000000003,"15min":5.5860000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.55700000000000005,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.56000000000000005}]},{"osd":1,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.628,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.64600000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.52000000000000002}]},{"osd":3,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56799999999999995,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84199999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73199999999999998}]},{"osd":4,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.88500000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.78000000000000003}]},{"osd":6,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91500000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.69399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":0.84599999999999997}]},{"osd":9,"last update":"Sun Oct 8 13:50:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.441,"5min":0.432,"15min":0.432},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.72999999999999998,"15min":0.70299999999999996},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":0.95299999999999996}]},{"osd":10,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":1.0209999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":7,"up_from":50,"seq":214748364841,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1104699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.439}]},{"osd":1,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.27000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.495,"5min":0.438,"15min":0.42799999999999999},"min":{"1min":0.28299999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.60499999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.88400000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.751}]},{"osd":4,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64100000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.627,"15min":0.62},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.85599999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67400000000000004,"15min":0.66900000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.80900000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.93500000000000005}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71999999999999997,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.625}]},{"osd":10,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528365,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":507425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.37},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.56599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.371},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.41299999999999998,"15min":0.40600000000000003},"min":{"1min":0.29099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61199999999999999,"5min":0.879,"15min":0.879},"last":0.309}]},{"osd":1,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.54000000000000004},"min":{"1min":0.42199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.752,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.49299999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85599999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81699999999999995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.44800000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69099999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69499999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.40899999999999997}]},{"osd":4,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59199999999999997,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84399999999999997,"5min":1.073,"15min":1.073},"last":0.73299999999999998}]},{"osd":5,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92800000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.629},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66900000000000004,"15min":0.65800000000000003},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79500000000000004,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.42499999999999999}]},{"osd":7,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.69699999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:50:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1080000000000001,"5min":1.147,"15min":1.147},"last":0.54200000000000004}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.73299999999999998,"15min":0.747},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.96799999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":1.042,"15min":1.042},"last":0.52500000000000002}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.748},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.66400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542210,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":571756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.52400000000000002},"min":{"1min":0.38700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.90400000000000003,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.48399999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52400000000000002,"15min":0.50700000000000001},"min":{"1min":0.441,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80300000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.88800000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58399999999999996,"15min":0.53500000000000003},"min":{"1min":0.42199999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80100000000000005,"5min":3.7959999999999998,"15min":3.7959999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.63600000000000001}]},{"osd":4,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.316,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.378,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66400000000000003}]},{"osd":5,"last update":"Sun Oct 8 13:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94599999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.91800000000000004}]},{"osd":6,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.72699999999999998,"15min":0.77400000000000002},"min":{"1min":0.46300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.84199999999999997,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":0.623},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.76100000000000001},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0109999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.77800000000000002}]},{"osd":7,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.35499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.93500000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89500000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.71699999999999997}]},{"osd":8,"last update":"Sun Oct 8 13:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.97099999999999997}]},{"osd":9,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92600000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.61099999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.97699999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.80100000000000005}]},{"osd":10,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705735,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":508881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 13:50:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54100000000000004,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72199999999999998,"5min":1.01,"15min":1.01},"last":0.46000000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56299999999999994,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85299999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Sun Oct 8 13:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.37},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.47099999999999997,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.311}]},{"osd":4,"last update":"Sun Oct 8 13:50:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.81899999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71399999999999997,"5min":1.444,"15min":1.444},"last":0.63700000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85999999999999999,"5min":1.012,"15min":1.012},"last":0.625},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.60099999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.069,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.38400000000000001}]},{"osd":7,"last update":"Sun Oct 8 13:49:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.50700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.67000000000000004}]},{"osd":8,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76600000000000001,"15min":0.77800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1419999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.746}]},{"osd":9,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.44600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90600000000000003,"5min":1.774,"15min":1.774},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.018,"5min":1.839,"15min":1.839},"last":0.79200000000000004}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.47899999999999998}]},{"osd":11,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.76300000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378685,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":570732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.55200000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.60199999999999998}]},{"osd":1,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80600000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.46800000000000003}]},{"osd":3,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.61599999999999999,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.68999999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73699999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.434}]},{"osd":5,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44700000000000001,"15min":0.42899999999999999},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.626},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.38200000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.66200000000000003}]},{"osd":6,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.80300000000000005}]},{"osd":7,"last update":"Sun Oct 8 13:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.70999999999999996}]},{"osd":8,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.72599999999999998}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.56000000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.88300000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.82299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":507133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.375,"15min":0.34100000000000003},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46100000000000002,"15min":0.41499999999999998},"min":{"1min":0.372,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.72799999999999998,"5min":0.87,"15min":0.87},"last":0.64600000000000002}]},{"osd":1,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.52900000000000003},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73499999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.53000000000000003},"min":{"1min":0.439,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.749,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.46800000000000003}]},{"osd":2,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.56100000000000005},"min":{"1min":0.5,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78600000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.56200000000000006},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.91100000000000003,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.66500000000000004}]},{"osd":4,"last update":"Sun Oct 8 13:50:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":2.077,"15min":2.077},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60299999999999998,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88300000000000001,"5min":2.129,"15min":2.129},"last":0.56499999999999995}]},{"osd":5,"last update":"Sun Oct 8 13:49:59 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.60599999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96999999999999997,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.32400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80200000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.625},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68400000000000005}]},{"osd":7,"last update":"Sun Oct 8 13:49:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.55300000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99399999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.69599999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:48:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.46800000000000003}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.069,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.625}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.60199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":511889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.51800000000000002,"15min":0.47999999999999998},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.35599999999999998},"min":{"1min":0.30599999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.36199999999999999},"min":{"1min":0.30399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.54900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.65800000000000003}]},{"osd":3,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.51500000000000001},"min":{"1min":0.40200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.45000000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.57999999999999996},"min":{"1min":0.501,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.876,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.64800000000000002,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.66300000000000003,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.377,"15min":0.377},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.53100000000000003}]},{"osd":7,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.38800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77100000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.59299999999999997,"15min":0.57299999999999995},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84599999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68899999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.54000000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93100000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93000000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71399999999999997}]},{"osd":9,"last update":"Sun Oct 8 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.75},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.70099999999999996}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1103656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48599999999999999,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.49299999999999999,"15min":0.46600000000000003},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.871,"15min":0.871},"last":0.53500000000000003}]},{"osd":1,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.55000000000000004,"15min":0.53000000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84999999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61899999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.38600000000000001,"15min":0.36899999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58699999999999997,"5min":0.629,"15min":0.629},"last":0.441},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.40300000000000002,"15min":0.38300000000000001},"min":{"1min":0.29599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.313,"5min":0.309,"15min":0.309},"max":{"1min":0.71499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.40699999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.77400000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.58499999999999996}]},{"osd":4,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56499999999999995}]},{"osd":6,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.54700000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84799999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65700000000000003,"15min":0.65500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68700000000000006}]},{"osd":8,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73399999999999999}]},{"osd":9,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.66400000000000003,"15min":0.64300000000000002},"min":{"1min":0.57599999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73599999999999999,"15min":0.72099999999999997},"min":{"1min":0.629,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.752}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.79200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":59933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T13:50:45.388 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-08T13:50:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:45 smithi115 ceph-mon[104408]: pgmap v273: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T13:50:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:45 smithi115 ceph-mon[104408]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:46.193 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:45 smithi057 ceph-mon[99476]: pgmap v273: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T13:50:46.193 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:45 smithi057 ceph-mon[99476]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:45 smithi099 ceph-mon[103849]: pgmap v273: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T13:50:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:45 smithi099 ceph-mon[103849]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:47.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.040+0000 7f9125b76700 1 -- 172.21.15.57:0/4107735724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91201008a0 msgr2=0x7f9120100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:47.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.040+0000 7f9125b76700 1 --2- 172.21.15.57:0/4107735724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91201008a0 0x7f9120100d20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9110009990 tx=0x7f9110009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:47.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.040+0000 7f9125b76700 1 -- 172.21.15.57:0/4107735724 shutdown_connections 2023-10-08T13:50:47.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.040+0000 7f9125b76700 1 --2- 172.21.15.57:0/4107735724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9120101260 0x7f912010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:47.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.040+0000 7f9125b76700 1 --2- 172.21.15.57:0/4107735724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91201057c0 0x7f9120105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:47.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.040+0000 7f9125b76700 1 --2- 172.21.15.57:0/4107735724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91201008a0 0x7f9120100d20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:47.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.040+0000 7f9125b76700 1 -- 172.21.15.57:0/4107735724 >> 172.21.15.57:0/4107735724 conn(0x7f91200779e0 msgr2=0x7f9120077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:47.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.040+0000 7f9125b76700 1 -- 172.21.15.57:0/4107735724 shutdown_connections 2023-10-08T13:50:47.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.040+0000 7f9125b76700 1 -- 172.21.15.57:0/4107735724 wait complete. 2023-10-08T13:50:47.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f9125b76700 1 Processor -- start 2023-10-08T13:50:47.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f9125b76700 1 -- start start 2023-10-08T13:50:47.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f9125b76700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91201008a0 0x7f9120072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:47.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f9125b76700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9120101260 0x7f9120073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:47.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f9125b76700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91201057c0 0x7f9120073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f9125b76700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91201a4f00 con 0x7f91201008a0 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f9125b76700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91201a5070 con 0x7f91201057c0 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f9125b76700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91201a5230 con 0x7f9120101260 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f911f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91201008a0 0x7f9120072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f911f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91201008a0 0x7f9120072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55000/0 (socket says 172.21.15.57:55000) 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.041+0000 7f911f7fe700 1 -- 172.21.15.57:0/1510214149 learned_addr learned my addr 172.21.15.57:0/1510214149 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911ffff700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91201057c0 0x7f9120073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911effd700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9120101260 0x7f9120073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911f7fe700 1 -- 172.21.15.57:0/1510214149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9120101260 msgr2=0x7f9120073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911f7fe700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9120101260 0x7f9120073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911f7fe700 1 -- 172.21.15.57:0/1510214149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91201057c0 msgr2=0x7f9120073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:47.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911f7fe700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91201057c0 0x7f9120073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:47.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911f7fe700 1 -- 172.21.15.57:0/1510214149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100095f0 con 0x7f91201008a0 2023-10-08T13:50:47.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911f7fe700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91201008a0 0x7f9120072af0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f911400e950 tx=0x7f911400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:47.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911cff9700 1 -- 172.21.15.57:0/1510214149 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114009770 con 0x7f91201008a0 2023-10-08T13:50:47.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911cff9700 1 -- 172.21.15.57:0/1510214149 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f911400eea0 con 0x7f91201008a0 2023-10-08T13:50:47.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f9125b76700 1 -- 172.21.15.57:0/1510214149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91201a5510 con 0x7f91201008a0 2023-10-08T13:50:47.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f911cff9700 1 -- 172.21.15.57:0/1510214149 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114003cf0 con 0x7f91201008a0 2023-10-08T13:50:47.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.042+0000 7f9125b76700 1 -- 172.21.15.57:0/1510214149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91201a59c0 con 0x7f91201008a0 2023-10-08T13:50:47.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.043+0000 7f9125b76700 1 -- 172.21.15.57:0/1510214149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9120108da0 con 0x7f91201008a0 2023-10-08T13:50:47.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.045+0000 7f911cff9700 1 -- 172.21.15.57:0/1510214149 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f911400ccd0 con 0x7f91201008a0 2023-10-08T13:50:47.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.045+0000 7f911cff9700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91080a1560 0x7f91080a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:47.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.046+0000 7f911effd700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91080a1560 0x7f91080a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:47.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.046+0000 7f911cff9700 1 -- 172.21.15.57:0/1510214149 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f9114014070 con 0x7f91201008a0 2023-10-08T13:50:47.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.046+0000 7f911effd700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91080a1560 0x7f91080a3a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9110006010 tx=0x7f911000b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:47.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.048+0000 7f911cff9700 1 -- 172.21.15.57:0/1510214149 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9114051100 con 0x7f91201008a0 2023-10-08T13:50:47.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:46 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:50:47.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:46 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:50:47.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.257+0000 7f9125b76700 1 -- 172.21.15.57:0/1510214149 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9120073fd0 con 0x7f91080a1560 2023-10-08T13:50:47.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:46 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:50:47.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.272+0000 7f911cff9700 1 -- 172.21.15.57:0/1510214149 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423200 (secure 0 0 0) 0x7f9120073fd0 con 0x7f91080a1560 2023-10-08T13:50:47.278 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:47.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 -- 172.21.15.57:0/1510214149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91080a1560 msgr2=0x7f91080a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:47.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91080a1560 0x7f91080a3a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9110006010 tx=0x7f911000b260 comp rx=0 tx=0).stop 2023-10-08T13:50:47.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 -- 172.21.15.57:0/1510214149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91201008a0 msgr2=0x7f9120072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:47.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91201008a0 0x7f9120072af0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f911400e950 tx=0x7f911400ec60 comp rx=0 tx=0).stop 2023-10-08T13:50:47.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 -- 172.21.15.57:0/1510214149 shutdown_connections 2023-10-08T13:50:47.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91080a1560 0x7f91080a3a20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:47.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91201057c0 0x7f9120073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:47.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91201008a0 0x7f9120072af0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:47.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 --2- 172.21.15.57:0/1510214149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9120101260 0x7f9120073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:47.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 -- 172.21.15.57:0/1510214149 >> 172.21.15.57:0/1510214149 conn(0x7f91200779e0 msgr2=0x7f91200fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:47.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 -- 172.21.15.57:0/1510214149 shutdown_connections 2023-10-08T13:50:47.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:47.277+0000 7f9125b76700 1 -- 172.21.15.57:0/1510214149 wait complete. 2023-10-08T13:50:47.283 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-08T13:50:47.730 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-10-08T13:50:46.118623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656009,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":527,"num_read_kb":388,"num_write":416,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338420,"kb_used_data":14172,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603404,"statfs":{"total":1151940427776,"available":1151593885696,"internally_reserved":0,"allocated":14512128,"data_stored":8079456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":60,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004170"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.781367+0000","last_change":"2023-10-08T13:49:44.786130+0000","last_active":"2023-10-08T13:50:14.781367+0000","last_peered":"2023-10-08T13:50:14.781367+0000","last_clean":"2023-10-08T13:50:14.781367+0000","last_became_active":"2023-10-08T13:49:44.785792+0000","last_became_peered":"2023-10-08T13:49:44.785792+0000","last_unstale":"2023-10-08T13:50:14.781367+0000","last_undegraded":"2023-10-08T13:50:14.781367+0000","last_fullsized":"2023-10-08T13:50:14.781367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:17.046800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780058+0000","last_change":"2023-10-08T13:49:44.777601+0000","last_active":"2023-10-08T13:50:14.780058+0000","last_peered":"2023-10-08T13:50:14.780058+0000","last_clean":"2023-10-08T13:50:14.780058+0000","last_became_active":"2023-10-08T13:49:44.777287+0000","last_became_peered":"2023-10-08T13:49:44.777287+0000","last_unstale":"2023-10-08T13:50:14.780058+0000","last_undegraded":"2023-10-08T13:50:14.780058+0000","last_fullsized":"2023-10-08T13:50:14.780058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:09:17.317629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779588+0000","last_change":"2023-10-08T13:49:44.787035+0000","last_active":"2023-10-08T13:50:14.779588+0000","last_peered":"2023-10-08T13:50:14.779588+0000","last_clean":"2023-10-08T13:50:14.779588+0000","last_became_active":"2023-10-08T13:49:44.785834+0000","last_became_peered":"2023-10-08T13:49:44.785834+0000","last_unstale":"2023-10-08T13:50:14.779588+0000","last_undegraded":"2023-10-08T13:50:14.779588+0000","last_fullsized":"2023-10-08T13:50:14.779588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:11.611404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773779+0000","last_change":"2023-10-08T13:49:44.781937+0000","last_active":"2023-10-08T13:50:14.773779+0000","last_peered":"2023-10-08T13:50:14.773779+0000","last_clean":"2023-10-08T13:50:14.773779+0000","last_became_active":"2023-10-08T13:49:44.780712+0000","last_became_peered":"2023-10-08T13:49:44.780712+0000","last_unstale":"2023-10-08T13:50:14.773779+0000","last_undegraded":"2023-10-08T13:50:14.773779+0000","last_fullsized":"2023-10-08T13:50:14.773779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:02.514007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.772899+0000","last_change":"2023-10-08T13:49:44.788661+0000","last_active":"2023-10-08T13:50:14.772899+0000","last_peered":"2023-10-08T13:50:14.772899+0000","last_clean":"2023-10-08T13:50:14.772899+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:50:14.772899+0000","last_undegraded":"2023-10-08T13:50:14.772899+0000","last_fullsized":"2023-10-08T13:50:14.772899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:35.770287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776011+0000","last_change":"2023-10-08T13:49:44.778519+0000","last_active":"2023-10-08T13:50:14.776011+0000","last_peered":"2023-10-08T13:50:14.776011+0000","last_clean":"2023-10-08T13:50:14.776011+0000","last_became_active":"2023-10-08T13:49:44.778193+0000","last_became_peered":"2023-10-08T13:49:44.778193+0000","last_unstale":"2023-10-08T13:50:14.776011+0000","last_undegraded":"2023-10-08T13:50:14.776011+0000","last_fullsized":"2023-10-08T13:50:14.776011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:12.916019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777538+0000","last_change":"2023-10-08T13:49:44.791759+0000","last_active":"2023-10-08T13:50:14.777538+0000","last_peered":"2023-10-08T13:50:14.777538+0000","last_clean":"2023-10-08T13:50:14.777538+0000","last_became_active":"2023-10-08T13:49:44.791494+0000","last_became_peered":"2023-10-08T13:49:44.791494+0000","last_unstale":"2023-10-08T13:50:14.777538+0000","last_undegraded":"2023-10-08T13:50:14.777538+0000","last_fullsized":"2023-10-08T13:50:14.777538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:55.859609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775547+0000","last_change":"2023-10-08T13:49:44.786831+0000","last_active":"2023-10-08T13:50:14.775547+0000","last_peered":"2023-10-08T13:50:14.775547+0000","last_clean":"2023-10-08T13:50:14.775547+0000","last_became_active":"2023-10-08T13:49:44.786516+0000","last_became_peered":"2023-10-08T13:49:44.786516+0000","last_unstale":"2023-10-08T13:50:14.775547+0000","last_undegraded":"2023-10-08T13:50:14.775547+0000","last_fullsized":"2023-10-08T13:50:14.775547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:07.616821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780365+0000","last_change":"2023-10-08T13:49:44.789827+0000","last_active":"2023-10-08T13:50:14.780365+0000","last_peered":"2023-10-08T13:50:14.780365+0000","last_clean":"2023-10-08T13:50:14.780365+0000","last_became_active":"2023-10-08T13:49:44.786635+0000","last_became_peered":"2023-10-08T13:49:44.786635+0000","last_unstale":"2023-10-08T13:50:14.780365+0000","last_undegraded":"2023-10-08T13:50:14.780365+0000","last_fullsized":"2023-10-08T13:50:14.780365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:35.492437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776603+0000","last_change":"2023-10-08T13:49:44.784240+0000","last_active":"2023-10-08T13:50:14.776603+0000","last_peered":"2023-10-08T13:50:14.776603+0000","last_clean":"2023-10-08T13:50:14.776603+0000","last_became_active":"2023-10-08T13:49:44.783475+0000","last_became_peered":"2023-10-08T13:49:44.783475+0000","last_unstale":"2023-10-08T13:50:14.776603+0000","last_undegraded":"2023-10-08T13:50:14.776603+0000","last_fullsized":"2023-10-08T13:50:14.776603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:39.730679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777907+0000","last_change":"2023-10-08T13:49:44.790901+0000","last_active":"2023-10-08T13:50:14.777907+0000","last_peered":"2023-10-08T13:50:14.777907+0000","last_clean":"2023-10-08T13:50:14.777907+0000","last_became_active":"2023-10-08T13:49:44.790611+0000","last_became_peered":"2023-10-08T13:49:44.790611+0000","last_unstale":"2023-10-08T13:50:14.777907+0000","last_undegraded":"2023-10-08T13:50:14.777907+0000","last_fullsized":"2023-10-08T13:50:14.777907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:19:21.764105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773901+0000","last_change":"2023-10-08T13:49:44.775226+0000","last_active":"2023-10-08T13:50:14.773901+0000","last_peered":"2023-10-08T13:50:14.773901+0000","last_clean":"2023-10-08T13:50:14.773901+0000","last_became_active":"2023-10-08T13:49:44.774756+0000","last_became_peered":"2023-10-08T13:49:44.774756+0000","last_unstale":"2023-10-08T13:50:14.773901+0000","last_undegraded":"2023-10-08T13:50:14.773901+0000","last_fullsized":"2023-10-08T13:50:14.773901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:57:14.749066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778685+0000","last_change":"2023-10-08T13:49:44.781610+0000","last_active":"2023-10-08T13:50:14.778685+0000","last_peered":"2023-10-08T13:50:14.778685+0000","last_clean":"2023-10-08T13:50:14.778685+0000","last_became_active":"2023-10-08T13:49:44.781281+0000","last_became_peered":"2023-10-08T13:49:44.781281+0000","last_unstale":"2023-10-08T13:50:14.778685+0000","last_undegraded":"2023-10-08T13:50:14.778685+0000","last_fullsized":"2023-10-08T13:50:14.778685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:50.823909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779251+0000","last_change":"2023-10-08T13:49:44.775836+0000","last_active":"2023-10-08T13:50:14.779251+0000","last_peered":"2023-10-08T13:50:14.779251+0000","last_clean":"2023-10-08T13:50:14.779251+0000","last_became_active":"2023-10-08T13:49:44.775206+0000","last_became_peered":"2023-10-08T13:49:44.775206+0000","last_unstale":"2023-10-08T13:50:14.779251+0000","last_undegraded":"2023-10-08T13:50:14.779251+0000","last_fullsized":"2023-10-08T13:50:14.779251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:51.566800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779976+0000","last_change":"2023-10-08T13:49:44.778495+0000","last_active":"2023-10-08T13:50:14.779976+0000","last_peered":"2023-10-08T13:50:14.779976+0000","last_clean":"2023-10-08T13:50:14.779976+0000","last_became_active":"2023-10-08T13:49:44.778196+0000","last_became_peered":"2023-10-08T13:49:44.778196+0000","last_unstale":"2023-10-08T13:50:14.779976+0000","last_undegraded":"2023-10-08T13:50:14.779976+0000","last_fullsized":"2023-10-08T13:50:14.779976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:57:38.531853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779796+0000","last_change":"2023-10-08T13:49:44.792960+0000","last_active":"2023-10-08T13:50:14.779796+0000","last_peered":"2023-10-08T13:50:14.779796+0000","last_clean":"2023-10-08T13:50:14.779796+0000","last_became_active":"2023-10-08T13:49:44.792674+0000","last_became_peered":"2023-10-08T13:49:44.792674+0000","last_unstale":"2023-10-08T13:50:14.779796+0000","last_undegraded":"2023-10-08T13:50:14.779796+0000","last_fullsized":"2023-10-08T13:50:14.779796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:13.934954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.781270+0000","last_change":"2023-10-08T13:49:44.785971+0000","last_active":"2023-10-08T13:50:14.781270+0000","last_peered":"2023-10-08T13:50:14.781270+0000","last_clean":"2023-10-08T13:50:14.781270+0000","last_became_active":"2023-10-08T13:49:44.785680+0000","last_became_peered":"2023-10-08T13:49:44.785680+0000","last_unstale":"2023-10-08T13:50:14.781270+0000","last_undegraded":"2023-10-08T13:50:14.781270+0000","last_fullsized":"2023-10-08T13:50:14.781270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:35.447519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773879+0000","last_change":"2023-10-08T13:49:44.787553+0000","last_active":"2023-10-08T13:50:14.773879+0000","last_peered":"2023-10-08T13:50:14.773879+0000","last_clean":"2023-10-08T13:50:14.773879+0000","last_became_active":"2023-10-08T13:49:44.787288+0000","last_became_peered":"2023-10-08T13:49:44.787288+0000","last_unstale":"2023-10-08T13:50:14.773879+0000","last_undegraded":"2023-10-08T13:50:14.773879+0000","last_fullsized":"2023-10-08T13:50:14.773879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:48.386120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780408+0000","last_change":"2023-10-08T13:49:44.789252+0000","last_active":"2023-10-08T13:50:14.780408+0000","last_peered":"2023-10-08T13:50:14.780408+0000","last_clean":"2023-10-08T13:50:14.780408+0000","last_became_active":"2023-10-08T13:49:44.788947+0000","last_became_peered":"2023-10-08T13:49:44.788947+0000","last_unstale":"2023-10-08T13:50:14.780408+0000","last_undegraded":"2023-10-08T13:50:14.780408+0000","last_fullsized":"2023-10-08T13:50:14.780408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:30.266913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779610+0000","last_change":"2023-10-08T13:49:49.163619+0000","last_active":"2023-10-08T13:50:14.779610+0000","last_peered":"2023-10-08T13:50:14.779610+0000","last_clean":"2023-10-08T13:50:14.779610+0000","last_became_active":"2023-10-08T13:49:49.163020+0000","last_became_peered":"2023-10-08T13:49:49.163020+0000","last_unstale":"2023-10-08T13:50:14.779610+0000","last_undegraded":"2023-10-08T13:50:14.779610+0000","last_fullsized":"2023-10-08T13:50:14.779610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:56.092404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":70,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:43.801268+0000","last_change":"2023-10-08T13:49:43.782863+0000","last_active":"2023-10-08T13:50:43.801268+0000","last_peered":"2023-10-08T13:50:43.801268+0000","last_clean":"2023-10-08T13:50:43.801268+0000","last_became_active":"2023-10-08T13:49:43.782477+0000","last_became_peered":"2023-10-08T13:49:43.782477+0000","last_unstale":"2023-10-08T13:50:43.801268+0000","last_undegraded":"2023-10-08T13:50:43.801268+0000","last_fullsized":"2023-10-08T13:50:43.801268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:10.134907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775780+0000","last_change":"2023-10-08T13:49:44.788452+0000","last_active":"2023-10-08T13:50:14.775780+0000","last_peered":"2023-10-08T13:50:14.775780+0000","last_clean":"2023-10-08T13:50:14.775780+0000","last_became_active":"2023-10-08T13:49:44.788122+0000","last_became_peered":"2023-10-08T13:49:44.788122+0000","last_unstale":"2023-10-08T13:50:14.775780+0000","last_undegraded":"2023-10-08T13:50:14.775780+0000","last_fullsized":"2023-10-08T13:50:14.775780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:25:45.766329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773267+0000","last_change":"2023-10-08T13:49:49.164248+0000","last_active":"2023-10-08T13:50:14.773267+0000","last_peered":"2023-10-08T13:50:14.773267+0000","last_clean":"2023-10-08T13:50:14.773267+0000","last_became_active":"2023-10-08T13:49:49.163624+0000","last_became_peered":"2023-10-08T13:49:49.163624+0000","last_unstale":"2023-10-08T13:50:14.773267+0000","last_undegraded":"2023-10-08T13:50:14.773267+0000","last_fullsized":"2023-10-08T13:50:14.773267+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:04.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776273+0000","last_change":"2023-10-08T13:49:43.788026+0000","last_active":"2023-10-08T13:50:14.776273+0000","last_peered":"2023-10-08T13:50:14.776273+0000","last_clean":"2023-10-08T13:50:14.776273+0000","last_became_active":"2023-10-08T13:49:43.787639+0000","last_became_peered":"2023-10-08T13:49:43.787639+0000","last_unstale":"2023-10-08T13:50:14.776273+0000","last_undegraded":"2023-10-08T13:50:14.776273+0000","last_fullsized":"2023-10-08T13:50:14.776273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:25:48.367765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779569+0000","last_change":"2023-10-08T13:49:44.786322+0000","last_active":"2023-10-08T13:50:14.779569+0000","last_peered":"2023-10-08T13:50:14.779569+0000","last_clean":"2023-10-08T13:50:14.779569+0000","last_became_active":"2023-10-08T13:49:44.785953+0000","last_became_peered":"2023-10-08T13:49:44.785953+0000","last_unstale":"2023-10-08T13:50:14.779569+0000","last_undegraded":"2023-10-08T13:50:14.779569+0000","last_fullsized":"2023-10-08T13:50:14.779569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:39.866785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.976961+0000","last_change":"2023-10-08T13:50:15.976961+0000","last_active":"2023-10-08T13:50:15.976961+0000","last_peered":"2023-10-08T13:50:15.976961+0000","last_clean":"2023-10-08T13:50:15.976961+0000","last_became_active":"2023-10-08T13:50:15.976492+0000","last_became_peered":"2023-10-08T13:50:15.976492+0000","last_unstale":"2023-10-08T13:50:15.976961+0000","last_undegraded":"2023-10-08T13:50:15.976961+0000","last_fullsized":"2023-10-08T13:50:15.976961+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:05:48.947259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.774448+0000","last_change":"2023-10-08T13:49:43.776239+0000","last_active":"2023-10-08T13:50:14.774448+0000","last_peered":"2023-10-08T13:50:14.774448+0000","last_clean":"2023-10-08T13:50:14.774448+0000","last_became_active":"2023-10-08T13:49:43.775969+0000","last_became_peered":"2023-10-08T13:49:43.775969+0000","last_unstale":"2023-10-08T13:50:14.774448+0000","last_undegraded":"2023-10-08T13:50:14.774448+0000","last_fullsized":"2023-10-08T13:50:14.774448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:24.011655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780166+0000","last_change":"2023-10-08T13:49:44.794912+0000","last_active":"2023-10-08T13:50:14.780166+0000","last_peered":"2023-10-08T13:50:14.780166+0000","last_clean":"2023-10-08T13:50:14.780166+0000","last_became_active":"2023-10-08T13:49:44.794590+0000","last_became_peered":"2023-10-08T13:49:44.794590+0000","last_unstale":"2023-10-08T13:50:14.780166+0000","last_undegraded":"2023-10-08T13:50:14.780166+0000","last_fullsized":"2023-10-08T13:50:14.780166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:51.631386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773315+0000","last_change":"2023-10-08T13:49:49.152149+0000","last_active":"2023-10-08T13:50:14.773315+0000","last_peered":"2023-10-08T13:50:14.773315+0000","last_clean":"2023-10-08T13:50:14.773315+0000","last_became_active":"2023-10-08T13:49:49.151764+0000","last_became_peered":"2023-10-08T13:49:49.151764+0000","last_unstale":"2023-10-08T13:50:14.773315+0000","last_undegraded":"2023-10-08T13:50:14.773315+0000","last_fullsized":"2023-10-08T13:50:14.773315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:25:08.232556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.800831+0000","last_change":"2023-10-08T13:50:15.800467+0000","last_active":"2023-10-08T13:50:15.800831+0000","last_peered":"2023-10-08T13:50:15.800831+0000","last_clean":"2023-10-08T13:50:15.800831+0000","last_became_active":"2023-10-08T13:50:15.788865+0000","last_became_peered":"2023-10-08T13:50:15.788865+0000","last_unstale":"2023-10-08T13:50:15.800831+0000","last_undegraded":"2023-10-08T13:50:15.800831+0000","last_fullsized":"2023-10-08T13:50:15.800831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:11:01.887072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776118+0000","last_change":"2023-10-08T13:49:44.783915+0000","last_active":"2023-10-08T13:50:14.776118+0000","last_peered":"2023-10-08T13:50:14.776118+0000","last_clean":"2023-10-08T13:50:14.776118+0000","last_became_active":"2023-10-08T13:49:44.783628+0000","last_became_peered":"2023-10-08T13:49:44.783628+0000","last_unstale":"2023-10-08T13:50:14.776118+0000","last_undegraded":"2023-10-08T13:50:14.776118+0000","last_fullsized":"2023-10-08T13:50:14.776118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:43.759681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780342+0000","last_change":"2023-10-08T13:49:49.156241+0000","last_active":"2023-10-08T13:50:14.780342+0000","last_peered":"2023-10-08T13:50:14.780342+0000","last_clean":"2023-10-08T13:50:14.780342+0000","last_became_active":"2023-10-08T13:49:49.155652+0000","last_became_peered":"2023-10-08T13:49:49.155652+0000","last_unstale":"2023-10-08T13:50:14.780342+0000","last_undegraded":"2023-10-08T13:50:14.780342+0000","last_fullsized":"2023-10-08T13:50:14.780342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":77,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:43.829782+0000","last_change":"2023-10-08T13:50:15.795816+0000","last_active":"2023-10-08T13:50:43.829782+0000","last_peered":"2023-10-08T13:50:43.829782+0000","last_clean":"2023-10-08T13:50:43.829782+0000","last_became_active":"2023-10-08T13:50:15.789936+0000","last_became_peered":"2023-10-08T13:50:15.789936+0000","last_unstale":"2023-10-08T13:50:43.829782+0000","last_undegraded":"2023-10-08T13:50:43.829782+0000","last_fullsized":"2023-10-08T13:50:43.829782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:21.496490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773190+0000","last_change":"2023-10-08T13:49:44.780935+0000","last_active":"2023-10-08T13:50:14.773190+0000","last_peered":"2023-10-08T13:50:14.773190+0000","last_clean":"2023-10-08T13:50:14.773190+0000","last_became_active":"2023-10-08T13:49:44.780613+0000","last_became_peered":"2023-10-08T13:49:44.780613+0000","last_unstale":"2023-10-08T13:50:14.773190+0000","last_undegraded":"2023-10-08T13:50:14.773190+0000","last_fullsized":"2023-10-08T13:50:14.773190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:57.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779985+0000","last_change":"2023-10-08T13:49:49.163652+0000","last_active":"2023-10-08T13:50:14.779985+0000","last_peered":"2023-10-08T13:50:14.779985+0000","last_clean":"2023-10-08T13:50:14.779985+0000","last_became_active":"2023-10-08T13:49:49.163140+0000","last_became_peered":"2023-10-08T13:49:49.163140+0000","last_unstale":"2023-10-08T13:50:14.779985+0000","last_undegraded":"2023-10-08T13:50:14.779985+0000","last_fullsized":"2023-10-08T13:50:14.779985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:48.733675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775983+0000","last_change":"2023-10-08T13:49:43.775799+0000","last_active":"2023-10-08T13:50:14.775983+0000","last_peered":"2023-10-08T13:50:14.775983+0000","last_clean":"2023-10-08T13:50:14.775983+0000","last_became_active":"2023-10-08T13:49:43.775470+0000","last_became_peered":"2023-10-08T13:49:43.775470+0000","last_unstale":"2023-10-08T13:50:14.775983+0000","last_undegraded":"2023-10-08T13:50:14.775983+0000","last_fullsized":"2023-10-08T13:50:14.775983+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:50.404554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.785798+0000","last_change":"2023-10-08T13:50:15.785798+0000","last_active":"2023-10-08T13:50:15.785798+0000","last_peered":"2023-10-08T13:50:15.785798+0000","last_clean":"2023-10-08T13:50:15.785798+0000","last_became_active":"2023-10-08T13:50:15.785343+0000","last_became_peered":"2023-10-08T13:50:15.785343+0000","last_unstale":"2023-10-08T13:50:15.785798+0000","last_undegraded":"2023-10-08T13:50:15.785798+0000","last_fullsized":"2023-10-08T13:50:15.785798+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:36.717707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.781065+0000","last_change":"2023-10-08T13:49:49.166702+0000","last_active":"2023-10-08T13:50:14.781065+0000","last_peered":"2023-10-08T13:50:14.781065+0000","last_clean":"2023-10-08T13:50:14.781065+0000","last_became_active":"2023-10-08T13:49:49.166256+0000","last_became_peered":"2023-10-08T13:49:49.166256+0000","last_unstale":"2023-10-08T13:50:14.781065+0000","last_undegraded":"2023-10-08T13:50:14.781065+0000","last_fullsized":"2023-10-08T13:50:14.781065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:18:57.565040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778550+0000","last_change":"2023-10-08T13:49:43.785085+0000","last_active":"2023-10-08T13:50:14.778550+0000","last_peered":"2023-10-08T13:50:14.778550+0000","last_clean":"2023-10-08T13:50:14.778550+0000","last_became_active":"2023-10-08T13:49:43.784746+0000","last_became_peered":"2023-10-08T13:49:43.784746+0000","last_unstale":"2023-10-08T13:50:14.778550+0000","last_undegraded":"2023-10-08T13:50:14.778550+0000","last_fullsized":"2023-10-08T13:50:14.778550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:13.471834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775477+0000","last_change":"2023-10-08T13:49:44.785670+0000","last_active":"2023-10-08T13:50:14.775477+0000","last_peered":"2023-10-08T13:50:14.775477+0000","last_clean":"2023-10-08T13:50:14.775477+0000","last_became_active":"2023-10-08T13:49:44.785221+0000","last_became_peered":"2023-10-08T13:49:44.785221+0000","last_unstale":"2023-10-08T13:50:14.775477+0000","last_undegraded":"2023-10-08T13:50:14.775477+0000","last_fullsized":"2023-10-08T13:50:14.775477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:25.415168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780015+0000","last_change":"2023-10-08T13:49:49.156294+0000","last_active":"2023-10-08T13:50:14.780015+0000","last_peered":"2023-10-08T13:50:14.780015+0000","last_clean":"2023-10-08T13:50:14.780015+0000","last_became_active":"2023-10-08T13:49:49.155821+0000","last_became_peered":"2023-10-08T13:49:49.155821+0000","last_unstale":"2023-10-08T13:50:14.780015+0000","last_undegraded":"2023-10-08T13:50:14.780015+0000","last_fullsized":"2023-10-08T13:50:14.780015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:59:43.831634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775524+0000","last_change":"2023-10-08T13:49:43.780052+0000","last_active":"2023-10-08T13:50:14.775524+0000","last_peered":"2023-10-08T13:50:14.775524+0000","last_clean":"2023-10-08T13:50:14.775524+0000","last_became_active":"2023-10-08T13:49:43.779731+0000","last_became_peered":"2023-10-08T13:49:43.779731+0000","last_unstale":"2023-10-08T13:50:14.775524+0000","last_undegraded":"2023-10-08T13:50:14.775524+0000","last_fullsized":"2023-10-08T13:50:14.775524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:38.800783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773039+0000","last_change":"2023-10-08T13:49:44.787880+0000","last_active":"2023-10-08T13:50:14.773039+0000","last_peered":"2023-10-08T13:50:14.773039+0000","last_clean":"2023-10-08T13:50:14.773039+0000","last_became_active":"2023-10-08T13:49:44.787591+0000","last_became_peered":"2023-10-08T13:49:44.787591+0000","last_unstale":"2023-10-08T13:50:14.773039+0000","last_undegraded":"2023-10-08T13:50:14.773039+0000","last_fullsized":"2023-10-08T13:50:14.773039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:53.078903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775715+0000","last_change":"2023-10-08T13:49:49.155004+0000","last_active":"2023-10-08T13:50:14.775715+0000","last_peered":"2023-10-08T13:50:14.775715+0000","last_clean":"2023-10-08T13:50:14.775715+0000","last_became_active":"2023-10-08T13:49:49.154503+0000","last_became_peered":"2023-10-08T13:49:49.154503+0000","last_unstale":"2023-10-08T13:50:14.775715+0000","last_undegraded":"2023-10-08T13:50:14.775715+0000","last_fullsized":"2023-10-08T13:50:14.775715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:21.565816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:41.785193+0000","last_change":"2023-10-08T13:49:43.775819+0000","last_active":"2023-10-08T13:50:41.785193+0000","last_peered":"2023-10-08T13:50:41.785193+0000","last_clean":"2023-10-08T13:50:41.785193+0000","last_became_active":"2023-10-08T13:49:43.775578+0000","last_became_peered":"2023-10-08T13:49:43.775578+0000","last_unstale":"2023-10-08T13:50:41.785193+0000","last_undegraded":"2023-10-08T13:50:41.785193+0000","last_fullsized":"2023-10-08T13:50:41.785193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:31.518191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780366+0000","last_change":"2023-10-08T13:49:44.785869+0000","last_active":"2023-10-08T13:50:14.780366+0000","last_peered":"2023-10-08T13:50:14.780366+0000","last_clean":"2023-10-08T13:50:14.780366+0000","last_became_active":"2023-10-08T13:49:44.785566+0000","last_became_peered":"2023-10-08T13:49:44.785566+0000","last_unstale":"2023-10-08T13:50:14.780366+0000","last_undegraded":"2023-10-08T13:50:14.780366+0000","last_fullsized":"2023-10-08T13:50:14.780366+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:29.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.772921+0000","last_change":"2023-10-08T13:49:49.162075+0000","last_active":"2023-10-08T13:50:14.772921+0000","last_peered":"2023-10-08T13:50:14.772921+0000","last_clean":"2023-10-08T13:50:14.772921+0000","last_became_active":"2023-10-08T13:49:49.161485+0000","last_became_peered":"2023-10-08T13:49:49.161485+0000","last_unstale":"2023-10-08T13:50:14.772921+0000","last_undegraded":"2023-10-08T13:50:14.772921+0000","last_fullsized":"2023-10-08T13:50:14.772921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:16.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778043+0000","last_change":"2023-10-08T13:49:43.776555+0000","last_active":"2023-10-08T13:50:14.778043+0000","last_peered":"2023-10-08T13:50:14.778043+0000","last_clean":"2023-10-08T13:50:14.778043+0000","last_became_active":"2023-10-08T13:49:43.776259+0000","last_became_peered":"2023-10-08T13:49:43.776259+0000","last_unstale":"2023-10-08T13:50:14.778043+0000","last_undegraded":"2023-10-08T13:50:14.778043+0000","last_fullsized":"2023-10-08T13:50:14.778043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:34.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779744+0000","last_change":"2023-10-08T13:49:44.779253+0000","last_active":"2023-10-08T13:50:14.779744+0000","last_peered":"2023-10-08T13:50:14.779744+0000","last_clean":"2023-10-08T13:50:14.779744+0000","last_became_active":"2023-10-08T13:49:44.778902+0000","last_became_peered":"2023-10-08T13:49:44.778902+0000","last_unstale":"2023-10-08T13:50:14.779744+0000","last_undegraded":"2023-10-08T13:50:14.779744+0000","last_fullsized":"2023-10-08T13:50:14.779744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:31:01.091424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773632+0000","last_change":"2023-10-08T13:49:49.164859+0000","last_active":"2023-10-08T13:50:14.773632+0000","last_peered":"2023-10-08T13:50:14.773632+0000","last_clean":"2023-10-08T13:50:14.773632+0000","last_became_active":"2023-10-08T13:49:49.164221+0000","last_became_peered":"2023-10-08T13:49:49.164221+0000","last_unstale":"2023-10-08T13:50:14.773632+0000","last_undegraded":"2023-10-08T13:50:14.773632+0000","last_fullsized":"2023-10-08T13:50:14.773632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:22.484934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775988+0000","last_change":"2023-10-08T13:49:43.788136+0000","last_active":"2023-10-08T13:50:14.775988+0000","last_peered":"2023-10-08T13:50:14.775988+0000","last_clean":"2023-10-08T13:50:14.775988+0000","last_became_active":"2023-10-08T13:49:43.787910+0000","last_became_peered":"2023-10-08T13:49:43.787910+0000","last_unstale":"2023-10-08T13:50:14.775988+0000","last_undegraded":"2023-10-08T13:50:14.775988+0000","last_fullsized":"2023-10-08T13:50:14.775988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:00:32.897972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780262+0000","last_change":"2023-10-08T13:49:44.786833+0000","last_active":"2023-10-08T13:50:14.780262+0000","last_peered":"2023-10-08T13:50:14.780262+0000","last_clean":"2023-10-08T13:50:14.780262+0000","last_became_active":"2023-10-08T13:49:44.786523+0000","last_became_peered":"2023-10-08T13:49:44.786523+0000","last_unstale":"2023-10-08T13:50:14.780262+0000","last_undegraded":"2023-10-08T13:50:14.780262+0000","last_fullsized":"2023-10-08T13:50:14.780262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:36:22.156252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.977115+0000","last_change":"2023-10-08T13:50:15.977115+0000","last_active":"2023-10-08T13:50:15.977115+0000","last_peered":"2023-10-08T13:50:15.977115+0000","last_clean":"2023-10-08T13:50:15.977115+0000","last_became_active":"2023-10-08T13:50:15.976617+0000","last_became_peered":"2023-10-08T13:50:15.976617+0000","last_unstale":"2023-10-08T13:50:15.977115+0000","last_undegraded":"2023-10-08T13:50:15.977115+0000","last_fullsized":"2023-10-08T13:50:15.977115+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:57.124054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":128,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:41.826537+0000","last_change":"2023-10-08T13:49:43.777736+0000","last_active":"2023-10-08T13:50:41.826537+0000","last_peered":"2023-10-08T13:50:41.826537+0000","last_clean":"2023-10-08T13:50:41.826537+0000","last_became_active":"2023-10-08T13:49:43.777431+0000","last_became_peered":"2023-10-08T13:49:43.777431+0000","last_unstale":"2023-10-08T13:50:41.826537+0000","last_undegraded":"2023-10-08T13:50:41.826537+0000","last_fullsized":"2023-10-08T13:50:41.826537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:12.907196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58583,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":95,"num_write":35,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.774165+0000","last_change":"2023-10-08T13:49:44.789122+0000","last_active":"2023-10-08T13:50:14.774165+0000","last_peered":"2023-10-08T13:50:14.774165+0000","last_clean":"2023-10-08T13:50:14.774165+0000","last_became_active":"2023-10-08T13:49:44.788831+0000","last_became_peered":"2023-10-08T13:49:44.788831+0000","last_unstale":"2023-10-08T13:50:14.774165+0000","last_undegraded":"2023-10-08T13:50:14.774165+0000","last_fullsized":"2023-10-08T13:50:14.774165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:45.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776462+0000","last_change":"2023-10-08T13:49:49.157319+0000","last_active":"2023-10-08T13:50:14.776462+0000","last_peered":"2023-10-08T13:50:14.776462+0000","last_clean":"2023-10-08T13:50:14.776462+0000","last_became_active":"2023-10-08T13:49:49.156897+0000","last_became_peered":"2023-10-08T13:49:49.156897+0000","last_unstale":"2023-10-08T13:50:14.776462+0000","last_undegraded":"2023-10-08T13:50:14.776462+0000","last_fullsized":"2023-10-08T13:50:14.776462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:25.460707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780243+0000","last_change":"2023-10-08T13:49:43.776699+0000","last_active":"2023-10-08T13:50:14.780243+0000","last_peered":"2023-10-08T13:50:14.780243+0000","last_clean":"2023-10-08T13:50:14.780243+0000","last_became_active":"2023-10-08T13:49:43.776293+0000","last_became_peered":"2023-10-08T13:49:43.776293+0000","last_unstale":"2023-10-08T13:50:14.780243+0000","last_undegraded":"2023-10-08T13:50:14.780243+0000","last_fullsized":"2023-10-08T13:50:14.780243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:19.056804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778762+0000","last_change":"2023-10-08T13:49:44.789915+0000","last_active":"2023-10-08T13:50:14.778762+0000","last_peered":"2023-10-08T13:50:14.778762+0000","last_clean":"2023-10-08T13:50:14.778762+0000","last_became_active":"2023-10-08T13:49:44.789617+0000","last_became_peered":"2023-10-08T13:49:44.789617+0000","last_unstale":"2023-10-08T13:50:14.778762+0000","last_undegraded":"2023-10-08T13:50:14.778762+0000","last_fullsized":"2023-10-08T13:50:14.778762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:06.286155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777143+0000","last_change":"2023-10-08T13:49:49.165946+0000","last_active":"2023-10-08T13:50:14.777143+0000","last_peered":"2023-10-08T13:50:14.777143+0000","last_clean":"2023-10-08T13:50:14.777143+0000","last_became_active":"2023-10-08T13:49:49.165581+0000","last_became_peered":"2023-10-08T13:49:49.165581+0000","last_unstale":"2023-10-08T13:50:14.777143+0000","last_undegraded":"2023-10-08T13:50:14.777143+0000","last_fullsized":"2023-10-08T13:50:14.777143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:13.772198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777873+0000","last_change":"2023-10-08T13:49:43.779161+0000","last_active":"2023-10-08T13:50:14.777873+0000","last_peered":"2023-10-08T13:50:14.777873+0000","last_clean":"2023-10-08T13:50:14.777873+0000","last_became_active":"2023-10-08T13:49:43.778862+0000","last_became_peered":"2023-10-08T13:49:43.778862+0000","last_unstale":"2023-10-08T13:50:14.777873+0000","last_undegraded":"2023-10-08T13:50:14.777873+0000","last_fullsized":"2023-10-08T13:50:14.777873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:57:44.075322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778206+0000","last_change":"2023-10-08T13:49:44.782072+0000","last_active":"2023-10-08T13:50:14.778206+0000","last_peered":"2023-10-08T13:50:14.778206+0000","last_clean":"2023-10-08T13:50:14.778206+0000","last_became_active":"2023-10-08T13:49:44.781756+0000","last_became_peered":"2023-10-08T13:49:44.781756+0000","last_unstale":"2023-10-08T13:50:14.778206+0000","last_undegraded":"2023-10-08T13:50:14.778206+0000","last_fullsized":"2023-10-08T13:50:14.778206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:32.877818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:16.100169+0000","last_change":"2023-10-08T13:50:16.099755+0000","last_active":"2023-10-08T13:50:16.100169+0000","last_peered":"2023-10-08T13:50:16.100169+0000","last_clean":"2023-10-08T13:50:16.100169+0000","last_became_active":"2023-10-08T13:50:16.099267+0000","last_became_peered":"2023-10-08T13:50:16.099267+0000","last_unstale":"2023-10-08T13:50:16.100169+0000","last_undegraded":"2023-10-08T13:50:16.100169+0000","last_fullsized":"2023-10-08T13:50:16.100169+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:19.949246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:41.799215+0000","last_change":"2023-10-08T13:49:43.782822+0000","last_active":"2023-10-08T13:50:41.799215+0000","last_peered":"2023-10-08T13:50:41.799215+0000","last_clean":"2023-10-08T13:50:41.799215+0000","last_became_active":"2023-10-08T13:49:43.782524+0000","last_became_peered":"2023-10-08T13:49:43.782524+0000","last_unstale":"2023-10-08T13:50:41.799215+0000","last_undegraded":"2023-10-08T13:50:41.799215+0000","last_fullsized":"2023-10-08T13:50:41.799215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:11.847851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777500+0000","last_change":"2023-10-08T13:49:44.794296+0000","last_active":"2023-10-08T13:50:14.777500+0000","last_peered":"2023-10-08T13:50:14.777500+0000","last_clean":"2023-10-08T13:50:14.777500+0000","last_became_active":"2023-10-08T13:49:44.793987+0000","last_became_peered":"2023-10-08T13:49:44.793987+0000","last_unstale":"2023-10-08T13:50:14.777500+0000","last_undegraded":"2023-10-08T13:50:14.777500+0000","last_fullsized":"2023-10-08T13:50:14.777500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:41.562569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780499+0000","last_change":"2023-10-08T13:49:49.165016+0000","last_active":"2023-10-08T13:50:14.780499+0000","last_peered":"2023-10-08T13:50:14.780499+0000","last_clean":"2023-10-08T13:50:14.780499+0000","last_became_active":"2023-10-08T13:49:49.164531+0000","last_became_peered":"2023-10-08T13:49:49.164531+0000","last_unstale":"2023-10-08T13:50:14.780499+0000","last_undegraded":"2023-10-08T13:50:14.780499+0000","last_fullsized":"2023-10-08T13:50:14.780499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:45.014512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.772750+0000","last_change":"2023-10-08T13:49:43.786929+0000","last_active":"2023-10-08T13:50:14.772750+0000","last_peered":"2023-10-08T13:50:14.772750+0000","last_clean":"2023-10-08T13:50:14.772750+0000","last_became_active":"2023-10-08T13:49:43.786641+0000","last_became_peered":"2023-10-08T13:49:43.786641+0000","last_unstale":"2023-10-08T13:50:14.772750+0000","last_undegraded":"2023-10-08T13:50:14.772750+0000","last_fullsized":"2023-10-08T13:50:14.772750+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:30.054067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775577+0000","last_change":"2023-10-08T13:49:44.786718+0000","last_active":"2023-10-08T13:50:14.775577+0000","last_peered":"2023-10-08T13:50:14.775577+0000","last_clean":"2023-10-08T13:50:14.775577+0000","last_became_active":"2023-10-08T13:49:44.786413+0000","last_became_peered":"2023-10-08T13:49:44.786413+0000","last_unstale":"2023-10-08T13:50:14.775577+0000","last_undegraded":"2023-10-08T13:50:14.775577+0000","last_fullsized":"2023-10-08T13:50:14.775577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.302426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775695+0000","last_change":"2023-10-08T13:49:49.152226+0000","last_active":"2023-10-08T13:50:14.775695+0000","last_peered":"2023-10-08T13:50:14.775695+0000","last_clean":"2023-10-08T13:50:14.775695+0000","last_became_active":"2023-10-08T13:49:49.151772+0000","last_became_peered":"2023-10-08T13:49:49.151772+0000","last_unstale":"2023-10-08T13:50:14.775695+0000","last_undegraded":"2023-10-08T13:50:14.775695+0000","last_fullsized":"2023-10-08T13:50:14.775695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:53.856496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.793964+0000","last_change":"2023-10-08T13:50:15.793598+0000","last_active":"2023-10-08T13:50:15.793964+0000","last_peered":"2023-10-08T13:50:15.793964+0000","last_clean":"2023-10-08T13:50:15.793964+0000","last_became_active":"2023-10-08T13:50:15.786421+0000","last_became_peered":"2023-10-08T13:50:15.786421+0000","last_unstale":"2023-10-08T13:50:15.793964+0000","last_undegraded":"2023-10-08T13:50:15.793964+0000","last_fullsized":"2023-10-08T13:50:15.793964+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:09:37.586245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776958+0000","last_change":"2023-10-08T13:49:44.782805+0000","last_active":"2023-10-08T13:50:14.776958+0000","last_peered":"2023-10-08T13:50:14.776958+0000","last_clean":"2023-10-08T13:50:14.776958+0000","last_became_active":"2023-10-08T13:49:44.782485+0000","last_became_peered":"2023-10-08T13:49:44.782485+0000","last_unstale":"2023-10-08T13:50:14.776958+0000","last_undegraded":"2023-10-08T13:50:14.776958+0000","last_fullsized":"2023-10-08T13:50:14.776958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:19.805734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773088+0000","last_change":"2023-10-08T13:49:49.546226+0000","last_active":"2023-10-08T13:50:14.773088+0000","last_peered":"2023-10-08T13:50:14.773088+0000","last_clean":"2023-10-08T13:50:14.773088+0000","last_became_active":"2023-10-08T13:49:49.545626+0000","last_became_peered":"2023-10-08T13:49:49.545626+0000","last_unstale":"2023-10-08T13:50:14.773088+0000","last_undegraded":"2023-10-08T13:50:14.773088+0000","last_fullsized":"2023-10-08T13:50:14.773088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:37.184944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773503+0000","last_change":"2023-10-08T13:49:43.784705+0000","last_active":"2023-10-08T13:50:14.773503+0000","last_peered":"2023-10-08T13:50:14.773503+0000","last_clean":"2023-10-08T13:50:14.773503+0000","last_became_active":"2023-10-08T13:49:43.784391+0000","last_became_peered":"2023-10-08T13:49:43.784391+0000","last_unstale":"2023-10-08T13:50:14.773503+0000","last_undegraded":"2023-10-08T13:50:14.773503+0000","last_fullsized":"2023-10-08T13:50:14.773503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:45.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776355+0000","last_change":"2023-10-08T13:49:44.789845+0000","last_active":"2023-10-08T13:50:14.776355+0000","last_peered":"2023-10-08T13:50:14.776355+0000","last_clean":"2023-10-08T13:50:14.776355+0000","last_became_active":"2023-10-08T13:49:44.789580+0000","last_became_peered":"2023-10-08T13:49:44.789580+0000","last_unstale":"2023-10-08T13:50:14.776355+0000","last_undegraded":"2023-10-08T13:50:14.776355+0000","last_fullsized":"2023-10-08T13:50:14.776355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:58.098757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778757+0000","last_change":"2023-10-08T13:49:49.153277+0000","last_active":"2023-10-08T13:50:14.778757+0000","last_peered":"2023-10-08T13:50:14.778757+0000","last_clean":"2023-10-08T13:50:14.778757+0000","last_became_active":"2023-10-08T13:49:49.152575+0000","last_became_peered":"2023-10-08T13:49:49.152575+0000","last_unstale":"2023-10-08T13:50:14.778757+0000","last_undegraded":"2023-10-08T13:50:14.778757+0000","last_fullsized":"2023-10-08T13:50:14.778757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:35.677823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:41.799278+0000","last_change":"2023-10-08T13:49:43.778442+0000","last_active":"2023-10-08T13:50:41.799278+0000","last_peered":"2023-10-08T13:50:41.799278+0000","last_clean":"2023-10-08T13:50:41.799278+0000","last_became_active":"2023-10-08T13:49:43.774799+0000","last_became_peered":"2023-10-08T13:49:43.774799+0000","last_unstale":"2023-10-08T13:50:41.799278+0000","last_undegraded":"2023-10-08T13:50:41.799278+0000","last_fullsized":"2023-10-08T13:50:41.799278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:15.464736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776195+0000","last_change":"2023-10-08T13:49:44.780097+0000","last_active":"2023-10-08T13:50:14.776195+0000","last_peered":"2023-10-08T13:50:14.776195+0000","last_clean":"2023-10-08T13:50:14.776195+0000","last_became_active":"2023-10-08T13:49:44.779774+0000","last_became_peered":"2023-10-08T13:49:44.779774+0000","last_unstale":"2023-10-08T13:50:14.776195+0000","last_undegraded":"2023-10-08T13:50:14.776195+0000","last_fullsized":"2023-10-08T13:50:14.776195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:19.132295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779203+0000","last_change":"2023-10-08T13:49:49.547129+0000","last_active":"2023-10-08T13:50:14.779203+0000","last_peered":"2023-10-08T13:50:14.779203+0000","last_clean":"2023-10-08T13:50:14.779203+0000","last_became_active":"2023-10-08T13:49:49.546694+0000","last_became_peered":"2023-10-08T13:49:49.546694+0000","last_unstale":"2023-10-08T13:50:14.779203+0000","last_undegraded":"2023-10-08T13:50:14.779203+0000","last_fullsized":"2023-10-08T13:50:14.779203+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:45.524650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775627+0000","last_change":"2023-10-08T13:49:44.779875+0000","last_active":"2023-10-08T13:50:14.775627+0000","last_peered":"2023-10-08T13:50:14.775627+0000","last_clean":"2023-10-08T13:50:14.775627+0000","last_became_active":"2023-10-08T13:49:44.779570+0000","last_became_peered":"2023-10-08T13:49:44.779570+0000","last_unstale":"2023-10-08T13:50:14.775627+0000","last_undegraded":"2023-10-08T13:50:14.775627+0000","last_fullsized":"2023-10-08T13:50:14.775627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:02.981998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:43.801267+0000","last_change":"2023-10-08T13:49:43.787574+0000","last_active":"2023-10-08T13:50:43.801267+0000","last_peered":"2023-10-08T13:50:43.801267+0000","last_clean":"2023-10-08T13:50:43.801267+0000","last_became_active":"2023-10-08T13:49:43.787261+0000","last_became_peered":"2023-10-08T13:49:43.787261+0000","last_unstale":"2023-10-08T13:50:43.801267+0000","last_undegraded":"2023-10-08T13:50:43.801267+0000","last_fullsized":"2023-10-08T13:50:43.801267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:28.676135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777768+0000","last_change":"2023-10-08T13:49:49.163856+0000","last_active":"2023-10-08T13:50:14.777768+0000","last_peered":"2023-10-08T13:50:14.777768+0000","last_clean":"2023-10-08T13:50:14.777768+0000","last_became_active":"2023-10-08T13:49:49.163456+0000","last_became_peered":"2023-10-08T13:49:49.163456+0000","last_unstale":"2023-10-08T13:50:14.777768+0000","last_undegraded":"2023-10-08T13:50:14.777768+0000","last_fullsized":"2023-10-08T13:50:14.777768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:31.654309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775665+0000","last_change":"2023-10-08T13:49:44.792935+0000","last_active":"2023-10-08T13:50:14.775665+0000","last_peered":"2023-10-08T13:50:14.775665+0000","last_clean":"2023-10-08T13:50:14.775665+0000","last_became_active":"2023-10-08T13:49:44.792667+0000","last_became_peered":"2023-10-08T13:49:44.792667+0000","last_unstale":"2023-10-08T13:50:14.775665+0000","last_undegraded":"2023-10-08T13:50:14.775665+0000","last_fullsized":"2023-10-08T13:50:14.775665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:40.964452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773491+0000","last_change":"2023-10-08T13:49:43.781424+0000","last_active":"2023-10-08T13:50:14.773491+0000","last_peered":"2023-10-08T13:50:14.773491+0000","last_clean":"2023-10-08T13:50:14.773491+0000","last_became_active":"2023-10-08T13:49:43.780930+0000","last_became_peered":"2023-10-08T13:49:43.780930+0000","last_unstale":"2023-10-08T13:50:14.773491+0000","last_undegraded":"2023-10-08T13:50:14.773491+0000","last_fullsized":"2023-10-08T13:50:14.773491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:56:50.397289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":325,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777396+0000","last_change":"2023-10-08T13:48:45.192219+0000","last_active":"2023-10-08T13:50:14.777396+0000","last_peered":"2023-10-08T13:50:14.777396+0000","last_clean":"2023-10-08T13:50:14.777396+0000","last_became_active":"2023-10-08T13:48:45.191813+0000","last_became_peered":"2023-10-08T13:48:45.191813+0000","last_unstale":"2023-10-08T13:50:14.777396+0000","last_undegraded":"2023-10-08T13:50:14.777396+0000","last_fullsized":"2023-10-08T13:50:14.777396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_clean_scrub_stamp":"2023-10-08T13:45:51.040695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.303937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776642+0000","last_change":"2023-10-08T13:49:49.546181+0000","last_active":"2023-10-08T13:50:14.776642+0000","last_peered":"2023-10-08T13:50:14.776642+0000","last_clean":"2023-10-08T13:50:14.776642+0000","last_became_active":"2023-10-08T13:49:49.545726+0000","last_became_peered":"2023-10-08T13:49:49.545726+0000","last_unstale":"2023-10-08T13:50:14.776642+0000","last_undegraded":"2023-10-08T13:50:14.776642+0000","last_fullsized":"2023-10-08T13:50:14.776642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:45.370909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778348+0000","last_change":"2023-10-08T13:49:44.785544+0000","last_active":"2023-10-08T13:50:14.778348+0000","last_peered":"2023-10-08T13:50:14.778348+0000","last_clean":"2023-10-08T13:50:14.778348+0000","last_became_active":"2023-10-08T13:49:44.785007+0000","last_became_peered":"2023-10-08T13:49:44.785007+0000","last_unstale":"2023-10-08T13:50:14.778348+0000","last_undegraded":"2023-10-08T13:50:14.778348+0000","last_fullsized":"2023-10-08T13:50:14.778348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:16:55.058693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777478+0000","last_change":"2023-10-08T13:49:43.779207+0000","last_active":"2023-10-08T13:50:14.777478+0000","last_peered":"2023-10-08T13:50:14.777478+0000","last_clean":"2023-10-08T13:50:14.777478+0000","last_became_active":"2023-10-08T13:49:43.778846+0000","last_became_peered":"2023-10-08T13:49:43.778846+0000","last_unstale":"2023-10-08T13:50:14.777478+0000","last_undegraded":"2023-10-08T13:50:14.777478+0000","last_fullsized":"2023-10-08T13:50:14.777478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:45.868187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776400+0000","last_change":"2023-10-08T13:49:49.162156+0000","last_active":"2023-10-08T13:50:14.776400+0000","last_peered":"2023-10-08T13:50:14.776400+0000","last_clean":"2023-10-08T13:50:14.776400+0000","last_became_active":"2023-10-08T13:49:49.161720+0000","last_became_peered":"2023-10-08T13:49:49.161720+0000","last_unstale":"2023-10-08T13:50:14.776400+0000","last_undegraded":"2023-10-08T13:50:14.776400+0000","last_fullsized":"2023-10-08T13:50:14.776400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:13.499915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.774269+0000","last_change":"2023-10-08T13:49:43.782322+0000","last_active":"2023-10-08T13:50:14.774269+0000","last_peered":"2023-10-08T13:50:14.774269+0000","last_clean":"2023-10-08T13:50:14.774269+0000","last_became_active":"2023-10-08T13:49:43.781938+0000","last_became_peered":"2023-10-08T13:49:43.781938+0000","last_unstale":"2023-10-08T13:50:14.774269+0000","last_undegraded":"2023-10-08T13:50:14.774269+0000","last_fullsized":"2023-10-08T13:50:14.774269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:57.627507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778880+0000","last_change":"2023-10-08T13:49:44.778340+0000","last_active":"2023-10-08T13:50:14.778880+0000","last_peered":"2023-10-08T13:50:14.778880+0000","last_clean":"2023-10-08T13:50:14.778880+0000","last_became_active":"2023-10-08T13:49:44.778000+0000","last_became_peered":"2023-10-08T13:49:44.778000+0000","last_unstale":"2023-10-08T13:50:14.778880+0000","last_undegraded":"2023-10-08T13:50:14.778880+0000","last_fullsized":"2023-10-08T13:50:14.778880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:23.350449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775457+0000","last_change":"2023-10-08T13:49:49.545915+0000","last_active":"2023-10-08T13:50:14.775457+0000","last_peered":"2023-10-08T13:50:14.775457+0000","last_clean":"2023-10-08T13:50:14.775457+0000","last_became_active":"2023-10-08T13:49:49.545396+0000","last_became_peered":"2023-10-08T13:49:49.545396+0000","last_unstale":"2023-10-08T13:50:14.775457+0000","last_undegraded":"2023-10-08T13:50:14.775457+0000","last_fullsized":"2023-10-08T13:50:14.775457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:07:03.850707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773842+0000","last_change":"2023-10-08T13:49:43.776479+0000","last_active":"2023-10-08T13:50:14.773842+0000","last_peered":"2023-10-08T13:50:14.773842+0000","last_clean":"2023-10-08T13:50:14.773842+0000","last_became_active":"2023-10-08T13:49:43.776130+0000","last_became_peered":"2023-10-08T13:49:43.776130+0000","last_unstale":"2023-10-08T13:50:14.773842+0000","last_undegraded":"2023-10-08T13:50:14.773842+0000","last_fullsized":"2023-10-08T13:50:14.773842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.760660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776442+0000","last_change":"2023-10-08T13:49:44.793864+0000","last_active":"2023-10-08T13:50:14.776442+0000","last_peered":"2023-10-08T13:50:14.776442+0000","last_clean":"2023-10-08T13:50:14.776442+0000","last_became_active":"2023-10-08T13:49:44.793469+0000","last_became_peered":"2023-10-08T13:49:44.793469+0000","last_unstale":"2023-10-08T13:50:14.776442+0000","last_undegraded":"2023-10-08T13:50:14.776442+0000","last_fullsized":"2023-10-08T13:50:14.776442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:28.669711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779427+0000","last_change":"2023-10-08T13:49:49.150696+0000","last_active":"2023-10-08T13:50:14.779427+0000","last_peered":"2023-10-08T13:50:14.779427+0000","last_clean":"2023-10-08T13:50:14.779427+0000","last_became_active":"2023-10-08T13:49:49.150188+0000","last_became_peered":"2023-10-08T13:49:49.150188+0000","last_unstale":"2023-10-08T13:50:14.779427+0000","last_undegraded":"2023-10-08T13:50:14.779427+0000","last_fullsized":"2023-10-08T13:50:14.779427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:51:35.775240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.774208+0000","last_change":"2023-10-08T13:49:43.786245+0000","last_active":"2023-10-08T13:50:14.774208+0000","last_peered":"2023-10-08T13:50:14.774208+0000","last_clean":"2023-10-08T13:50:14.774208+0000","last_became_active":"2023-10-08T13:49:43.785986+0000","last_became_peered":"2023-10-08T13:49:43.785986+0000","last_unstale":"2023-10-08T13:50:14.774208+0000","last_undegraded":"2023-10-08T13:50:14.774208+0000","last_fullsized":"2023-10-08T13:50:14.774208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:53:59.643447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780099+0000","last_change":"2023-10-08T13:49:44.790494+0000","last_active":"2023-10-08T13:50:14.780099+0000","last_peered":"2023-10-08T13:50:14.780099+0000","last_clean":"2023-10-08T13:50:14.780099+0000","last_became_active":"2023-10-08T13:49:44.790091+0000","last_became_peered":"2023-10-08T13:49:44.790091+0000","last_unstale":"2023-10-08T13:50:14.780099+0000","last_undegraded":"2023-10-08T13:50:14.780099+0000","last_fullsized":"2023-10-08T13:50:14.780099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:49.309597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773799+0000","last_change":"2023-10-08T13:49:49.547271+0000","last_active":"2023-10-08T13:50:14.773799+0000","last_peered":"2023-10-08T13:50:14.773799+0000","last_clean":"2023-10-08T13:50:14.773799+0000","last_became_active":"2023-10-08T13:49:49.546603+0000","last_became_peered":"2023-10-08T13:49:49.546603+0000","last_unstale":"2023-10-08T13:50:14.773799+0000","last_undegraded":"2023-10-08T13:50:14.773799+0000","last_fullsized":"2023-10-08T13:50:14.773799+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:29.908321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779809+0000","last_change":"2023-10-08T13:49:43.788638+0000","last_active":"2023-10-08T13:50:14.779809+0000","last_peered":"2023-10-08T13:50:14.779809+0000","last_clean":"2023-10-08T13:50:14.779809+0000","last_became_active":"2023-10-08T13:49:43.788303+0000","last_became_peered":"2023-10-08T13:49:43.788303+0000","last_unstale":"2023-10-08T13:50:14.779809+0000","last_undegraded":"2023-10-08T13:50:14.779809+0000","last_fullsized":"2023-10-08T13:50:14.779809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:52.637775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.772743+0000","last_change":"2023-10-08T13:49:44.777546+0000","last_active":"2023-10-08T13:50:14.772743+0000","last_peered":"2023-10-08T13:50:14.772743+0000","last_clean":"2023-10-08T13:50:14.772743+0000","last_became_active":"2023-10-08T13:49:44.777261+0000","last_became_peered":"2023-10-08T13:49:44.777261+0000","last_unstale":"2023-10-08T13:50:14.772743+0000","last_undegraded":"2023-10-08T13:50:14.772743+0000","last_fullsized":"2023-10-08T13:50:14.772743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:26:36.368188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.790468+0000","last_change":"2023-10-08T13:50:15.787317+0000","last_active":"2023-10-08T13:50:15.790468+0000","last_peered":"2023-10-08T13:50:15.790468+0000","last_clean":"2023-10-08T13:50:15.790468+0000","last_became_active":"2023-10-08T13:50:15.786816+0000","last_became_peered":"2023-10-08T13:50:15.786816+0000","last_unstale":"2023-10-08T13:50:15.790468+0000","last_undegraded":"2023-10-08T13:50:15.790468+0000","last_fullsized":"2023-10-08T13:50:15.790468+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:53.766803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775828+0000","last_change":"2023-10-08T13:49:43.776693+0000","last_active":"2023-10-08T13:50:14.775828+0000","last_peered":"2023-10-08T13:50:14.775828+0000","last_clean":"2023-10-08T13:50:14.775828+0000","last_became_active":"2023-10-08T13:49:43.776367+0000","last_became_peered":"2023-10-08T13:49:43.776367+0000","last_unstale":"2023-10-08T13:50:14.775828+0000","last_undegraded":"2023-10-08T13:50:14.775828+0000","last_fullsized":"2023-10-08T13:50:14.775828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:17.415011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776044+0000","last_change":"2023-10-08T13:49:44.779472+0000","last_active":"2023-10-08T13:50:14.776044+0000","last_peered":"2023-10-08T13:50:14.776044+0000","last_clean":"2023-10-08T13:50:14.776044+0000","last_became_active":"2023-10-08T13:49:44.779172+0000","last_became_peered":"2023-10-08T13:49:44.779172+0000","last_unstale":"2023-10-08T13:50:14.776044+0000","last_undegraded":"2023-10-08T13:50:14.776044+0000","last_fullsized":"2023-10-08T13:50:14.776044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:17.314243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773591+0000","last_change":"2023-10-08T13:49:49.164535+0000","last_active":"2023-10-08T13:50:14.773591+0000","last_peered":"2023-10-08T13:50:14.773591+0000","last_clean":"2023-10-08T13:50:14.773591+0000","last_became_active":"2023-10-08T13:49:49.163467+0000","last_became_peered":"2023-10-08T13:49:49.163467+0000","last_unstale":"2023-10-08T13:50:14.773591+0000","last_undegraded":"2023-10-08T13:50:14.773591+0000","last_fullsized":"2023-10-08T13:50:14.773591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:55.058656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780711+0000","last_change":"2023-10-08T13:49:43.775311+0000","last_active":"2023-10-08T13:50:14.780711+0000","last_peered":"2023-10-08T13:50:14.780711+0000","last_clean":"2023-10-08T13:50:14.780711+0000","last_became_active":"2023-10-08T13:49:43.775005+0000","last_became_peered":"2023-10-08T13:49:43.775005+0000","last_unstale":"2023-10-08T13:50:14.780711+0000","last_undegraded":"2023-10-08T13:50:14.780711+0000","last_fullsized":"2023-10-08T13:50:14.780711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:33.713395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776317+0000","last_change":"2023-10-08T13:49:44.792051+0000","last_active":"2023-10-08T13:50:14.776317+0000","last_peered":"2023-10-08T13:50:14.776317+0000","last_clean":"2023-10-08T13:50:14.776317+0000","last_became_active":"2023-10-08T13:49:44.790045+0000","last_became_peered":"2023-10-08T13:49:44.790045+0000","last_unstale":"2023-10-08T13:50:14.776317+0000","last_undegraded":"2023-10-08T13:50:14.776317+0000","last_fullsized":"2023-10-08T13:50:14.776317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:28.699799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.773740+0000","last_change":"2023-10-08T13:49:49.546219+0000","last_active":"2023-10-08T13:50:14.773740+0000","last_peered":"2023-10-08T13:50:14.773740+0000","last_clean":"2023-10-08T13:50:14.773740+0000","last_became_active":"2023-10-08T13:49:49.545783+0000","last_became_peered":"2023-10-08T13:49:49.545783+0000","last_unstale":"2023-10-08T13:50:14.773740+0000","last_undegraded":"2023-10-08T13:50:14.773740+0000","last_fullsized":"2023-10-08T13:50:14.773740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:00.497908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.772903+0000","last_change":"2023-10-08T13:49:43.776252+0000","last_active":"2023-10-08T13:50:14.772903+0000","last_peered":"2023-10-08T13:50:14.772903+0000","last_clean":"2023-10-08T13:50:14.772903+0000","last_became_active":"2023-10-08T13:49:43.775935+0000","last_became_peered":"2023-10-08T13:49:43.775935+0000","last_unstale":"2023-10-08T13:50:14.772903+0000","last_undegraded":"2023-10-08T13:50:14.772903+0000","last_fullsized":"2023-10-08T13:50:14.772903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:58.949365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778127+0000","last_change":"2023-10-08T13:49:44.788628+0000","last_active":"2023-10-08T13:50:14.778127+0000","last_peered":"2023-10-08T13:50:14.778127+0000","last_clean":"2023-10-08T13:50:14.778127+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:50:14.778127+0000","last_undegraded":"2023-10-08T13:50:14.778127+0000","last_fullsized":"2023-10-08T13:50:14.778127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:46:42.311296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779858+0000","last_change":"2023-10-08T13:49:49.153641+0000","last_active":"2023-10-08T13:50:14.779858+0000","last_peered":"2023-10-08T13:50:14.779858+0000","last_clean":"2023-10-08T13:50:14.779858+0000","last_became_active":"2023-10-08T13:49:49.153172+0000","last_became_peered":"2023-10-08T13:49:49.153172+0000","last_unstale":"2023-10-08T13:50:14.779858+0000","last_undegraded":"2023-10-08T13:50:14.779858+0000","last_fullsized":"2023-10-08T13:50:14.779858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:40:51.726343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:42.787442+0000","last_change":"2023-10-08T13:49:43.782683+0000","last_active":"2023-10-08T13:50:42.787442+0000","last_peered":"2023-10-08T13:50:42.787442+0000","last_clean":"2023-10-08T13:50:42.787442+0000","last_became_active":"2023-10-08T13:49:43.782269+0000","last_became_peered":"2023-10-08T13:49:43.782269+0000","last_unstale":"2023-10-08T13:50:42.787442+0000","last_undegraded":"2023-10-08T13:50:42.787442+0000","last_fullsized":"2023-10-08T13:50:42.787442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:50.850581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777949+0000","last_change":"2023-10-08T13:49:44.787894+0000","last_active":"2023-10-08T13:50:14.777949+0000","last_peered":"2023-10-08T13:50:14.777949+0000","last_clean":"2023-10-08T13:50:14.777949+0000","last_became_active":"2023-10-08T13:49:44.787599+0000","last_became_peered":"2023-10-08T13:49:44.787599+0000","last_unstale":"2023-10-08T13:50:14.777949+0000","last_undegraded":"2023-10-08T13:50:14.777949+0000","last_fullsized":"2023-10-08T13:50:14.777949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:17:15.131051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779473+0000","last_change":"2023-10-08T13:49:49.164256+0000","last_active":"2023-10-08T13:50:14.779473+0000","last_peered":"2023-10-08T13:50:14.779473+0000","last_clean":"2023-10-08T13:50:14.779473+0000","last_became_active":"2023-10-08T13:49:49.163793+0000","last_became_peered":"2023-10-08T13:49:49.163793+0000","last_unstale":"2023-10-08T13:50:14.779473+0000","last_undegraded":"2023-10-08T13:50:14.779473+0000","last_fullsized":"2023-10-08T13:50:14.779473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:19.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776219+0000","last_change":"2023-10-08T13:49:43.778311+0000","last_active":"2023-10-08T13:50:14.776219+0000","last_peered":"2023-10-08T13:50:14.776219+0000","last_clean":"2023-10-08T13:50:14.776219+0000","last_became_active":"2023-10-08T13:49:43.777984+0000","last_became_peered":"2023-10-08T13:49:43.777984+0000","last_unstale":"2023-10-08T13:50:14.776219+0000","last_undegraded":"2023-10-08T13:50:14.776219+0000","last_fullsized":"2023-10-08T13:50:14.776219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:56.890729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.777769+0000","last_change":"2023-10-08T13:49:44.793424+0000","last_active":"2023-10-08T13:50:14.777769+0000","last_peered":"2023-10-08T13:50:14.777769+0000","last_clean":"2023-10-08T13:50:14.777769+0000","last_became_active":"2023-10-08T13:49:44.792825+0000","last_became_peered":"2023-10-08T13:49:44.792825+0000","last_unstale":"2023-10-08T13:50:14.777769+0000","last_undegraded":"2023-10-08T13:50:14.777769+0000","last_fullsized":"2023-10-08T13:50:14.777769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:39.463676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775289+0000","last_change":"2023-10-08T13:49:49.151915+0000","last_active":"2023-10-08T13:50:14.775289+0000","last_peered":"2023-10-08T13:50:14.775289+0000","last_clean":"2023-10-08T13:50:14.775289+0000","last_became_active":"2023-10-08T13:49:49.151434+0000","last_became_peered":"2023-10-08T13:49:49.151434+0000","last_unstale":"2023-10-08T13:50:14.775289+0000","last_undegraded":"2023-10-08T13:50:14.775289+0000","last_fullsized":"2023-10-08T13:50:14.775289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:54:07.631498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.787243+0000","last_change":"2023-10-08T13:50:15.787243+0000","last_active":"2023-10-08T13:50:15.787243+0000","last_peered":"2023-10-08T13:50:15.787243+0000","last_clean":"2023-10-08T13:50:15.787243+0000","last_became_active":"2023-10-08T13:50:15.782305+0000","last_became_peered":"2023-10-08T13:50:15.782305+0000","last_unstale":"2023-10-08T13:50:15.787243+0000","last_undegraded":"2023-10-08T13:50:15.787243+0000","last_fullsized":"2023-10-08T13:50:15.787243+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:08.520010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778420+0000","last_change":"2023-10-08T13:49:44.790949+0000","last_active":"2023-10-08T13:50:14.778420+0000","last_peered":"2023-10-08T13:50:14.778420+0000","last_clean":"2023-10-08T13:50:14.778420+0000","last_became_active":"2023-10-08T13:49:44.790645+0000","last_became_peered":"2023-10-08T13:49:44.790645+0000","last_unstale":"2023-10-08T13:50:14.778420+0000","last_undegraded":"2023-10-08T13:50:14.778420+0000","last_fullsized":"2023-10-08T13:50:14.778420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:48.890730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776140+0000","last_change":"2023-10-08T13:49:44.781165+0000","last_active":"2023-10-08T13:50:14.776140+0000","last_peered":"2023-10-08T13:50:14.776140+0000","last_clean":"2023-10-08T13:50:14.776140+0000","last_became_active":"2023-10-08T13:49:44.780828+0000","last_became_peered":"2023-10-08T13:49:44.780828+0000","last_unstale":"2023-10-08T13:50:14.776140+0000","last_undegraded":"2023-10-08T13:50:14.776140+0000","last_fullsized":"2023-10-08T13:50:14.776140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:04.259482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779799+0000","last_change":"2023-10-08T13:49:44.781271+0000","last_active":"2023-10-08T13:50:14.779799+0000","last_peered":"2023-10-08T13:50:14.779799+0000","last_clean":"2023-10-08T13:50:14.779799+0000","last_became_active":"2023-10-08T13:49:44.780912+0000","last_became_peered":"2023-10-08T13:49:44.780912+0000","last_unstale":"2023-10-08T13:50:14.779799+0000","last_undegraded":"2023-10-08T13:50:14.779799+0000","last_fullsized":"2023-10-08T13:50:14.779799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:17.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780636+0000","last_change":"2023-10-08T13:49:44.777613+0000","last_active":"2023-10-08T13:50:14.780636+0000","last_peered":"2023-10-08T13:50:14.780636+0000","last_clean":"2023-10-08T13:50:14.780636+0000","last_became_active":"2023-10-08T13:49:44.777304+0000","last_became_peered":"2023-10-08T13:49:44.777304+0000","last_unstale":"2023-10-08T13:50:14.780636+0000","last_undegraded":"2023-10-08T13:50:14.780636+0000","last_fullsized":"2023-10-08T13:50:14.780636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:45.482935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779823+0000","last_change":"2023-10-08T13:49:44.782685+0000","last_active":"2023-10-08T13:50:14.779823+0000","last_peered":"2023-10-08T13:50:14.779823+0000","last_clean":"2023-10-08T13:50:14.779823+0000","last_became_active":"2023-10-08T13:49:44.781816+0000","last_became_peered":"2023-10-08T13:49:44.781816+0000","last_unstale":"2023-10-08T13:50:14.779823+0000","last_undegraded":"2023-10-08T13:50:14.779823+0000","last_fullsized":"2023-10-08T13:50:14.779823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:53.781237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778962+0000","last_change":"2023-10-08T13:49:44.783973+0000","last_active":"2023-10-08T13:50:14.778962+0000","last_peered":"2023-10-08T13:50:14.778962+0000","last_clean":"2023-10-08T13:50:14.778962+0000","last_became_active":"2023-10-08T13:49:44.783648+0000","last_became_peered":"2023-10-08T13:49:44.783648+0000","last_unstale":"2023-10-08T13:50:14.778962+0000","last_undegraded":"2023-10-08T13:50:14.778962+0000","last_fullsized":"2023-10-08T13:50:14.778962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:01.253610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776705+0000","last_change":"2023-10-08T13:49:44.781848+0000","last_active":"2023-10-08T13:50:14.776705+0000","last_peered":"2023-10-08T13:50:14.776705+0000","last_clean":"2023-10-08T13:50:14.776705+0000","last_became_active":"2023-10-08T13:49:44.781542+0000","last_became_peered":"2023-10-08T13:49:44.781542+0000","last_unstale":"2023-10-08T13:50:14.776705+0000","last_undegraded":"2023-10-08T13:50:14.776705+0000","last_fullsized":"2023-10-08T13:50:14.776705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:26:38.726298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.780659+0000","last_change":"2023-10-08T13:49:44.786541+0000","last_active":"2023-10-08T13:50:14.780659+0000","last_peered":"2023-10-08T13:50:14.780659+0000","last_clean":"2023-10-08T13:50:14.780659+0000","last_became_active":"2023-10-08T13:49:44.786214+0000","last_became_peered":"2023-10-08T13:49:44.786214+0000","last_unstale":"2023-10-08T13:50:14.780659+0000","last_undegraded":"2023-10-08T13:50:14.780659+0000","last_fullsized":"2023-10-08T13:50:14.780659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:30:18.342774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.776997+0000","last_change":"2023-10-08T13:49:44.788445+0000","last_active":"2023-10-08T13:50:14.776997+0000","last_peered":"2023-10-08T13:50:14.776997+0000","last_clean":"2023-10-08T13:50:14.776997+0000","last_became_active":"2023-10-08T13:49:44.788075+0000","last_became_peered":"2023-10-08T13:49:44.788075+0000","last_unstale":"2023-10-08T13:50:14.776997+0000","last_undegraded":"2023-10-08T13:50:14.776997+0000","last_fullsized":"2023-10-08T13:50:14.776997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:08:20.701231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775451+0000","last_change":"2023-10-08T13:49:44.783755+0000","last_active":"2023-10-08T13:50:14.775451+0000","last_peered":"2023-10-08T13:50:14.775451+0000","last_clean":"2023-10-08T13:50:14.775451+0000","last_became_active":"2023-10-08T13:49:44.783467+0000","last_became_peered":"2023-10-08T13:49:44.783467+0000","last_unstale":"2023-10-08T13:50:14.775451+0000","last_undegraded":"2023-10-08T13:50:14.775451+0000","last_fullsized":"2023-10-08T13:50:14.775451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:39.838940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.775876+0000","last_change":"2023-10-08T13:49:44.779342+0000","last_active":"2023-10-08T13:50:14.775876+0000","last_peered":"2023-10-08T13:50:14.775876+0000","last_clean":"2023-10-08T13:50:14.775876+0000","last_became_active":"2023-10-08T13:49:44.778518+0000","last_became_peered":"2023-10-08T13:49:44.778518+0000","last_unstale":"2023-10-08T13:50:14.775876+0000","last_undegraded":"2023-10-08T13:50:14.775876+0000","last_fullsized":"2023-10-08T13:50:14.775876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:33.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.779999+0000","last_change":"2023-10-08T13:49:44.792955+0000","last_active":"2023-10-08T13:50:14.779999+0000","last_peered":"2023-10-08T13:50:14.779999+0000","last_clean":"2023-10-08T13:50:14.779999+0000","last_became_active":"2023-10-08T13:49:44.792637+0000","last_became_peered":"2023-10-08T13:49:44.792637+0000","last_unstale":"2023-10-08T13:50:14.779999+0000","last_undegraded":"2023-10-08T13:50:14.779999+0000","last_fullsized":"2023-10-08T13:50:14.779999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:40.635913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T13:50:14.778825+0000","last_change":"2023-10-08T13:49:44.786817+0000","last_active":"2023-10-08T13:50:14.778825+0000","last_peered":"2023-10-08T13:50:14.778825+0000","last_clean":"2023-10-08T13:50:14.778825+0000","last_became_active":"2023-10-08T13:49:44.786439+0000","last_became_peered":"2023-10-08T13:49:44.786439+0000","last_unstale":"2023-10-08T13:50:14.778825+0000","last_undegraded":"2023-10-08T13:50:14.778825+0000","last_fullsized":"2023-10-08T13:50:14.778825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:45.308600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T13:50:15.788414+0000","last_change":"2023-10-08T13:50:15.788414+0000","last_active":"2023-10-08T13:50:15.788414+0000","last_peered":"2023-10-08T13:50:15.788414+0000","last_clean":"2023-10-08T13:50:15.788414+0000","last_became_active":"2023-10-08T13:50:15.788079+0000","last_became_peered":"2023-10-08T13:50:15.788079+0000","last_unstale":"2023-10-08T13:50:15.788414+0000","last_undegraded":"2023-10-08T13:50:15.788414+0000","last_fullsized":"2023-10-08T13:50:15.788414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:43.723339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65395,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":391,"num_read_kb":269,"num_write":233,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":196185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416408,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":511889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77700000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.754},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.88100000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65400000000000003}]},{"osd":6,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.94899999999999995}]},{"osd":7,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":1.0089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":9,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.85699999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:50:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.024}]}]},{"osd":10,"up_from":71,"seq":304942678044,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28696,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716456,"statfs":{"total":95995035648,"available":95965650944,"internally_reserved":0,"allocated":1720320,"data_stored":1157780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.72899999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65700000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78200000000000003}]},{"osd":6,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55300000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.86199999999999999}]},{"osd":9,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.80600000000000005}]},{"osd":11,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.879}]}]},{"osd":9,"up_from":66,"seq":283467841568,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":516017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.499,"15min":0.496},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.4279999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.27,"5min":0.65600000000000003,"15min":0.55400000000000005},"min":{"1min":0.31900000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":16.077000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.39400000000000002}]},{"osd":1,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.46999999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.6029999999999998,"5min":4.6029999999999998,"15min":4.6029999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.51100000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.42799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.442,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.52200000000000002}]},{"osd":3,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.57499999999999996,"15min":0.54100000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.9969999999999999,"5min":6.9969999999999999,"15min":6.9969999999999999},"last":0.372},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.35199999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.7589999999999999,"5min":3.7589999999999999,"15min":3.7589999999999999},"last":0.42499999999999999}]},{"osd":4,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.748,"15min":0.71999999999999997},"min":{"1min":0.46500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.5}]},{"osd":5,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.81000000000000005,"15min":0.78600000000000003},"min":{"1min":0.47799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":5.3390000000000004,"5min":5.3390000000000004,"15min":5.3390000000000004},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73799999999999999,"15min":0.72199999999999998},"min":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.0410000000000004,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.44800000000000001}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.9570000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.345}]},{"osd":7,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":0.83599999999999997,"15min":0.80100000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":7.3650000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.87,"15min":0.86299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":4.3680000000000003,"5min":4.3680000000000003,"15min":4.3680000000000003},"last":0.65200000000000002}]},{"osd":8,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.88200000000000001,"15min":0.86799999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":5.2930000000000001,"5min":5.2930000000000001,"15min":5.2930000000000001},"last":0.745},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.86499999999999999,"15min":0.82899999999999996},"min":{"1min":0.498,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":0.64300000000000002}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103204,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":507599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.57399999999999995,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":5.5860000000000003,"5min":5.5860000000000003,"15min":5.5860000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.55700000000000005,"15min":0.52900000000000003},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.56000000000000005}]},{"osd":1,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.628,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.64600000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.52000000000000002}]},{"osd":3,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56799999999999995,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84199999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73199999999999998}]},{"osd":4,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.88500000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.78000000000000003}]},{"osd":6,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91500000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.69399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":0.84599999999999997}]},{"osd":9,"last update":"Sun Oct 8 13:50:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.441,"5min":0.432,"15min":0.432},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.72999999999999998,"15min":0.70299999999999996},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":0.95299999999999996}]},{"osd":10,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":1.0209999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":7,"up_from":50,"seq":214748364841,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1104699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.439}]},{"osd":1,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.27000000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.495,"5min":0.438,"15min":0.42799999999999999},"min":{"1min":0.28299999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.60499999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.88400000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.751}]},{"osd":4,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64100000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.627,"15min":0.62},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.85599999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67400000000000004,"15min":0.66900000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.80900000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.93500000000000005}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71999999999999997,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.625}]},{"osd":10,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528366,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":507425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.37},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.56599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.41299999999999998,"15min":0.40600000000000003},"min":{"1min":0.29099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61199999999999999,"5min":0.879,"15min":0.879},"last":0.38300000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.54000000000000004},"min":{"1min":0.42199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.752,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66100000000000003}]},{"osd":2,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85599999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81699999999999995,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.50600000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69099999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.432},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69499999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.41299999999999998}]},{"osd":4,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59199999999999997,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84399999999999997,"5min":1.073,"15min":1.073},"last":0.74199999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92800000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.625},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66900000000000004,"15min":0.65800000000000003},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79500000000000004,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":1.075,"15min":1.075},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.749,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71699999999999997}]},{"osd":8,"last update":"Sun Oct 8 13:50:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1080000000000001,"5min":1.147,"15min":1.147},"last":0.70599999999999996}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.73299999999999998,"15min":0.747},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.96799999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":1.042,"15min":1.042},"last":0.53300000000000003}]},{"osd":10,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86899999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0469999999999999,"5min":1.159,"15min":1.159},"last":0.79500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":1,"up_from":14,"seq":60129542211,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":571756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.497},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.52400000000000002},"min":{"1min":0.38700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.90400000000000003,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Sun Oct 8 13:49:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.40200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52400000000000002,"15min":0.50700000000000001},"min":{"1min":0.441,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80300000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.69499999999999995}]},{"osd":3,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58399999999999996,"15min":0.53500000000000003},"min":{"1min":0.42199999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80100000000000005,"5min":3.7959999999999998,"15min":3.7959999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.37,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.41899999999999998},"min":{"1min":0.40300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.75,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.44700000000000001},"min":{"1min":0.38100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.628}]},{"osd":5,"last update":"Sun Oct 8 13:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94599999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.80900000000000005}]},{"osd":6,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.72699999999999998,"15min":0.77400000000000002},"min":{"1min":0.46300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.84199999999999997,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.76100000000000001},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0109999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.48199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60699999999999998,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.67200000000000004}]},{"osd":8,"last update":"Sun Oct 8 13:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83199999999999996}]},{"osd":9,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92600000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.61099999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.97699999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.872}]},{"osd":10,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.54000000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79700000000000004,"15min":0.78800000000000003},"min":{"1min":0.61299999999999999,"5min":0.498,"15min":0.498},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.77300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":0,"up_from":9,"seq":38654705735,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":508881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 13:50:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54100000000000004,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72199999999999998,"5min":1.01,"15min":1.01},"last":0.46000000000000002}]},{"osd":2,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.56299999999999994,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85299999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Sun Oct 8 13:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.37},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.47099999999999997,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.311}]},{"osd":4,"last update":"Sun Oct 8 13:50:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.81899999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71399999999999997,"5min":1.444,"15min":1.444},"last":0.63700000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85999999999999999,"5min":1.012,"15min":1.012},"last":0.625},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.60099999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.069,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.38400000000000001}]},{"osd":7,"last update":"Sun Oct 8 13:49:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.50700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.67000000000000004}]},{"osd":8,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76600000000000001,"15min":0.77800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1419999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.746}]},{"osd":9,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.44600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90600000000000003,"5min":1.774,"15min":1.774},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.018,"5min":1.839,"15min":1.839},"last":0.79200000000000004}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.47899999999999998}]},{"osd":11,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.76300000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378686,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":570732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.55200000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.63800000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80600000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.85799999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.61599999999999999,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.68899999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44700000000000001,"15min":0.42899999999999999},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.38200000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.58899999999999997}]},{"osd":6,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Sun Oct 8 13:49:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.98899999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:49:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.76600000000000001}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.877},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.56000000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.95899999999999996}]},{"osd":10,"last update":"Sun Oct 8 13:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.93500000000000005}]},{"osd":11,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.97999999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":507133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.375,"15min":0.34100000000000003},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46100000000000002,"15min":0.41499999999999998},"min":{"1min":0.372,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.72799999999999998,"5min":0.87,"15min":0.87},"last":0.64600000000000002}]},{"osd":1,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.52900000000000003},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73499999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.53000000000000003},"min":{"1min":0.439,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.749,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.46800000000000003}]},{"osd":2,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.56100000000000005},"min":{"1min":0.5,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78600000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.56200000000000006},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.91100000000000003,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.66500000000000004}]},{"osd":4,"last update":"Sun Oct 8 13:50:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":2.077,"15min":2.077},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60299999999999998,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88300000000000001,"5min":2.129,"15min":2.129},"last":0.56499999999999995}]},{"osd":5,"last update":"Sun Oct 8 13:49:59 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.60599999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64000000000000001,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96999999999999997,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.32400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80200000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.625},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68400000000000005}]},{"osd":7,"last update":"Sun Oct 8 13:49:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.55300000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99399999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.69599999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:48:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.46800000000000003}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.069,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.625}]},{"osd":10,"last update":"Sun Oct 8 13:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.60199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":511889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.51800000000000002,"15min":0.47999999999999998},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.35599999999999998},"min":{"1min":0.30599999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.36199999999999999},"min":{"1min":0.30399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.54900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.65800000000000003}]},{"osd":3,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.51500000000000001},"min":{"1min":0.40200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.45000000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.57999999999999996},"min":{"1min":0.501,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.876,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.64800000000000002,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.66300000000000003,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.377,"15min":0.377},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.53100000000000003}]},{"osd":7,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.38800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77100000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.59299999999999997,"15min":0.57299999999999995},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84599999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68899999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:50:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.54000000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93100000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93000000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71399999999999997}]},{"osd":9,"last update":"Sun Oct 8 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.75},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.70099999999999996}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1103656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48599999999999999,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.49299999999999999,"15min":0.46600000000000003},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.871,"15min":0.871},"last":0.74399999999999999}]},{"osd":1,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.55000000000000004,"15min":0.53000000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84999999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54400000000000004}]},{"osd":2,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.38600000000000001,"15min":0.36899999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58699999999999997,"5min":0.629,"15min":0.629},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.40300000000000002,"15min":0.38300000000000001},"min":{"1min":0.29599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.36099999999999999}]},{"osd":3,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.313,"5min":0.309,"15min":0.309},"max":{"1min":0.71499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.40699999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.77400000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.84999999999999998}]},{"osd":4,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.58899999999999997}]},{"osd":6,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.54700000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84799999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65700000000000003,"15min":0.65500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61799999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:50:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65000000000000002}]},{"osd":9,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.66400000000000003,"15min":0.64300000000000002},"min":{"1min":0.57599999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73599999999999999,"15min":0.72099999999999997},"min":{"1min":0.629,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.81299999999999994}]},{"osd":10,"last update":"Sun Oct 8 13:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.77800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":59933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T13:50:47.740 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-08T13:50:47.741 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-08T13:50:47.741 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-08T13:50:47.741 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph health --format=json 2023-10-08T13:50:47.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:47 smithi057 ceph-mon[99476]: pgmap v274: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T13:50:48.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:47 smithi115 ceph-mon[104408]: pgmap v274: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T13:50:48.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:47 smithi099 ceph-mon[103849]: pgmap v274: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T13:50:48.321 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:49.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:48 smithi057 ceph-mon[99476]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:49.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:48 smithi099 ceph-mon[103849]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:49.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:48 smithi115 ceph-mon[104408]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:50:49.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.601+0000 7fe210fe1700 1 -- 172.21.15.57:0/4076088754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20c1056b0 msgr2=0x7fe20c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:49.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.601+0000 7fe210fe1700 1 --2- 172.21.15.57:0/4076088754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20c1056b0 0x7fe20c105a90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00b0a0 tx=0x7fe1fc00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:49.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.601+0000 7fe210fe1700 1 -- 172.21.15.57:0/4076088754 shutdown_connections 2023-10-08T13:50:49.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.601+0000 7fe210fe1700 1 --2- 172.21.15.57:0/4076088754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe20c100110 0x7fe20c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:49.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.601+0000 7fe210fe1700 1 --2- 172.21.15.57:0/4076088754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20c1056b0 0x7fe20c105a90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:49.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.601+0000 7fe210fe1700 1 --2- 172.21.15.57:0/4076088754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe20c0ff750 0x7fe20c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:49.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.601+0000 7fe210fe1700 1 -- 172.21.15.57:0/4076088754 >> 172.21.15.57:0/4076088754 conn(0x7fe20c075790 msgr2=0x7fe20c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:49.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.601+0000 7fe210fe1700 1 -- 172.21.15.57:0/4076088754 shutdown_connections 2023-10-08T13:50:49.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.601+0000 7fe210fe1700 1 -- 172.21.15.57:0/4076088754 wait complete. 2023-10-08T13:50:49.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.602+0000 7fe210fe1700 1 Processor -- start 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.602+0000 7fe210fe1700 1 -- start start 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.602+0000 7fe210fe1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20c0ff750 0x7fe20c196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.602+0000 7fe210fe1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe20c100110 0x7fe20c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.602+0000 7fe210fe1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe20c1056b0 0x7fe20c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.602+0000 7fe210fe1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c19ab60 con 0x7fe20c0ff750 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.602+0000 7fe210fe1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c19acd0 con 0x7fe20c100110 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.602+0000 7fe210fe1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c19ae90 con 0x7fe20c1056b0 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.602+0000 7fe20a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20c0ff750 0x7fe20c196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.602+0000 7fe209d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe20c100110 0x7fe20c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe20a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20c0ff750 0x7fe20c196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55012/0 (socket says 172.21.15.57:55012) 2023-10-08T13:50:49.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe20a59c700 1 -- 172.21.15.57:0/2661377017 learned_addr learned my addr 172.21.15.57:0/2661377017 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe20ad9d700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe20c1056b0 0x7fe20c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe20a59c700 1 -- 172.21.15.57:0/2661377017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe20c1056b0 msgr2=0x7fe20c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe20a59c700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe20c1056b0 0x7fe20c19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe20a59c700 1 -- 172.21.15.57:0/2661377017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe20c100110 msgr2=0x7fe20c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe20a59c700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe20c100110 0x7fe20c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe20a59c700 1 -- 172.21.15.57:0/2661377017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc009d00 con 0x7fe20c0ff750 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe20ad9d700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe20c1056b0 0x7fe20c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe209d9b700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe20c100110 0x7fe20c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe20a59c700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20c0ff750 0x7fe20c196d70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00b780 tx=0x7fe1fc007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe2037fe700 1 -- 172.21.15.57:0/2661377017 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00e040 con 0x7fe20c0ff750 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe210fe1700 1 -- 172.21.15.57:0/2661377017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20c19b110 con 0x7fe20c0ff750 2023-10-08T13:50:49.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe2037fe700 1 -- 172.21.15.57:0/2661377017 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1fc01d470 con 0x7fe20c0ff750 2023-10-08T13:50:49.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe2037fe700 1 -- 172.21.15.57:0/2661377017 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc0052a0 con 0x7fe20c0ff750 2023-10-08T13:50:49.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.603+0000 7fe210fe1700 1 -- 172.21.15.57:0/2661377017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c1a3ac0 con 0x7fe20c0ff750 2023-10-08T13:50:49.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.604+0000 7fe210fe1700 1 -- 172.21.15.57:0/2661377017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20c100df0 con 0x7fe20c0ff750 2023-10-08T13:50:49.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.605+0000 7fe2037fe700 1 -- 172.21.15.57:0/2661377017 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fe1fc01d5e0 con 0x7fe20c0ff750 2023-10-08T13:50:49.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.606+0000 7fe2037fe700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1f80a15e0 0x7fe1f80a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:49.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.606+0000 7fe209d9b700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1f80a15e0 0x7fe1f80a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:49.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.606+0000 7fe2037fe700 1 -- 172.21.15.57:0/2661377017 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fe1fc0ce260 con 0x7fe20c0ff750 2023-10-08T13:50:49.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.606+0000 7fe209d9b700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1f80a15e0 0x7fe1f80a3aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe20c198240 tx=0x7fe1f4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:49.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.610+0000 7fe2037fe700 1 -- 172.21.15.57:0/2661377017 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe1fc097400 con 0x7fe20c0ff750 2023-10-08T13:50:49.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.869+0000 7fe210fe1700 1 -- 172.21.15.57:0/2661377017 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe20c04e4c0 con 0x7fe20c0ff750 2023-10-08T13:50:49.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.869+0000 7fe2037fe700 1 -- 172.21.15.57:0/2661377017 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe1fc09aa20 con 0x7fe20c0ff750 2023-10-08T13:50:49.870 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:49.870 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-08T13:50:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.871+0000 7fe210fe1700 1 -- 172.21.15.57:0/2661377017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1f80a15e0 msgr2=0x7fe1f80a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.871+0000 7fe210fe1700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1f80a15e0 0x7fe1f80a3aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe20c198240 tx=0x7fe1f4009260 comp rx=0 tx=0).stop 2023-10-08T13:50:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.871+0000 7fe210fe1700 1 -- 172.21.15.57:0/2661377017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20c0ff750 msgr2=0x7fe20c196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.871+0000 7fe210fe1700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20c0ff750 0x7fe20c196d70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00b780 tx=0x7fe1fc007940 comp rx=0 tx=0).stop 2023-10-08T13:50:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.872+0000 7fe210fe1700 1 -- 172.21.15.57:0/2661377017 shutdown_connections 2023-10-08T13:50:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.872+0000 7fe210fe1700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe20c1056b0 0x7fe20c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.872+0000 7fe210fe1700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe20c0ff750 0x7fe20c196d70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.872+0000 7fe210fe1700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1f80a15e0 0x7fe1f80a3aa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:49.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.872+0000 7fe210fe1700 1 --2- 172.21.15.57:0/2661377017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe20c100110 0x7fe20c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:49.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.872+0000 7fe210fe1700 1 -- 172.21.15.57:0/2661377017 >> 172.21.15.57:0/2661377017 conn(0x7fe20c075790 msgr2=0x7fe20c1035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:49.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.872+0000 7fe210fe1700 1 -- 172.21.15.57:0/2661377017 shutdown_connections 2023-10-08T13:50:49.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:49.872+0000 7fe210fe1700 1 -- 172.21.15.57:0/2661377017 wait complete. 2023-10-08T13:50:50.201 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:49 smithi057 ceph-mon[99476]: pgmap v275: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T13:50:50.201 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:49 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:49 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2661377017' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-08T13:50:50.239 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-08T13:50:50.239 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:50:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:49 smithi099 ceph-mon[103849]: pgmap v275: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T13:50:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:49 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:49 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2661377017' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-08T13:50:50.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.369+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/3328735070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bc40fff30 msgr2=0x7f1bc410c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:50.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.369+0000 7f1bcbbb5700 1 --2- 172.21.15.57:0/3328735070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bc40fff30 0x7f1bc410c3f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00b0d0 tx=0x7f1bbc00b3e0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.369+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/3328735070 shutdown_connections 2023-10-08T13:50:50.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.369+0000 7f1bcbbb5700 1 --2- 172.21.15.57:0/3328735070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bc40fff30 0x7f1bc410c3f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.369+0000 7f1bcbbb5700 1 --2- 172.21.15.57:0/3328735070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bc4105630 0x7f1bc4105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.369+0000 7f1bcbbb5700 1 --2- 172.21.15.57:0/3328735070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc40ff570 0x7f1bc40ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.369+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/3328735070 >> 172.21.15.57:0/3328735070 conn(0x7f1bc40fb040 msgr2=0x7f1bc40fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:50.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.369+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/3328735070 shutdown_connections 2023-10-08T13:50:50.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.369+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/3328735070 wait complete. 2023-10-08T13:50:50.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bcbbb5700 1 Processor -- start 2023-10-08T13:50:50.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bcbbb5700 1 -- start start 2023-10-08T13:50:50.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bcbbb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bc40ff570 0x7f1bc4195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:50.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bcbbb5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bc40fff30 0x7f1bc4195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:50.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bcbbb5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc4105630 0x7f1bc41962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:50.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bcbbb5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc41a4ff0 con 0x7f1bc40ff570 2023-10-08T13:50:50.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bcbbb5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc41a5160 con 0x7f1bc4105630 2023-10-08T13:50:50.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bcbbb5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc41a5320 con 0x7f1bc40fff30 2023-10-08T13:50:50.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bc9951700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bc40ff570 0x7f1bc4195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:50.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bc9951700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bc40ff570 0x7f1bc4195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55026/0 (socket says 172.21.15.57:55026) 2023-10-08T13:50:50.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bc9951700 1 -- 172.21.15.57:0/4251097066 learned_addr learned my addr 172.21.15.57:0/4251097066 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.370+0000 7f1bc9150700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bc40fff30 0x7f1bc4195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bc9951700 1 -- 172.21.15.57:0/4251097066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bc40fff30 msgr2=0x7f1bc4195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bc9951700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bc40fff30 0x7f1bc4195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bc9951700 1 -- 172.21.15.57:0/4251097066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc4105630 msgr2=0x7f1bc41962d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bc9951700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc4105630 0x7f1bc41962d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bc9951700 1 -- 172.21.15.57:0/4251097066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc009d00 con 0x7f1bc40ff570 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bc9951700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bc40ff570 0x7f1bc4195850 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000d950 tx=0x7f1bc000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bbaffd700 1 -- 172.21.15.57:0/4251097066 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0009940 con 0x7f1bc40ff570 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bbaffd700 1 -- 172.21.15.57:0/4251097066 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bc0010460 con 0x7f1bc40ff570 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/4251097066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc41a5600 con 0x7f1bc40ff570 2023-10-08T13:50:50.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bbaffd700 1 -- 172.21.15.57:0/4251097066 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000f620 con 0x7f1bc40ff570 2023-10-08T13:50:50.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.371+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/4251097066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc41a5b60 con 0x7f1bc40ff570 2023-10-08T13:50:50.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.372+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/4251097066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc4107ad0 con 0x7f1bc40ff570 2023-10-08T13:50:50.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.373+0000 7f1bbaffd700 1 -- 172.21.15.57:0/4251097066 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1bc00108d0 con 0x7f1bc40ff570 2023-10-08T13:50:50.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.374+0000 7f1bbaffd700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1bb00a15e0 0x7f1bb00a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:50.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.374+0000 7f1bc9150700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1bb00a15e0 0x7f1bb00a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:50.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.374+0000 7f1bbaffd700 1 -- 172.21.15.57:0/4251097066 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f1bc00cd010 con 0x7f1bc40ff570 2023-10-08T13:50:50.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.375+0000 7f1bc9150700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1bb00a15e0 0x7f1bb00a3aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4069290 tx=0x7f1bb4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:50.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.379+0000 7f1bbaffd700 1 -- 172.21.15.57:0/4251097066 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1bc0096120 con 0x7f1bc40ff570 2023-10-08T13:50:50.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:49 smithi115 ceph-mon[104408]: pgmap v275: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T13:50:50.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:49 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:49 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2661377017' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-08T13:50:50.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.584+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/4251097066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bc4100c90 con 0x7f1bc40ff570 2023-10-08T13:50:50.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.585+0000 7f1bbaffd700 1 -- 172.21.15.57:0/4251097066 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24753 (secure 0 0 0) 0x7f1bc0099740 con 0x7f1bc40ff570 2023-10-08T13:50:50.587 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:50.587 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:50:15.770421+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:50:50.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/4251097066 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1bb00a15e0 msgr2=0x7f1bb00a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1bb00a15e0 0x7f1bb00a3aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4069290 tx=0x7f1bb4009260 comp rx=0 tx=0).stop 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/4251097066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bc40ff570 msgr2=0x7f1bc4195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bc40ff570 0x7f1bc4195850 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000d950 tx=0x7f1bc000dc60 comp rx=0 tx=0).stop 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/4251097066 shutdown_connections 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1bb00a15e0 0x7f1bb00a3aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc4105630 0x7f1bc41962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bc40ff570 0x7f1bc4195850 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 --2- 172.21.15.57:0/4251097066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bc40fff30 0x7f1bc4195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/4251097066 >> 172.21.15.57:0/4251097066 conn(0x7f1bc40fb040 msgr2=0x7f1bc40fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.587+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/4251097066 shutdown_connections 2023-10-08T13:50:50.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.588+0000 7f1bcbbb5700 1 -- 172.21.15.57:0/4251097066 wait complete. 2023-10-08T13:50:50.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T13:50:50.603 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:50:50.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.732+0000 7f5d63e93700 1 -- 172.21.15.57:0/2956846506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c100000 msgr2=0x7f5d5c10c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:50.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.732+0000 7f5d63e93700 1 --2- 172.21.15.57:0/2956846506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c100000 0x7f5d5c10c4f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400b0a0 tx=0x7f5d5400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.732+0000 7f5d63e93700 1 -- 172.21.15.57:0/2956846506 shutdown_connections 2023-10-08T13:50:50.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.732+0000 7f5d63e93700 1 --2- 172.21.15.57:0/2956846506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c100000 0x7f5d5c10c4f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.732+0000 7f5d63e93700 1 --2- 172.21.15.57:0/2956846506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c105630 0x7f5d5c105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.732+0000 7f5d63e93700 1 --2- 172.21.15.57:0/2956846506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d5c0ff640 0x7f5d5c0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.732+0000 7f5d63e93700 1 -- 172.21.15.57:0/2956846506 >> 172.21.15.57:0/2956846506 conn(0x7f5d5c076790 msgr2=0x7f5d5c076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:50.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.732+0000 7f5d63e93700 1 -- 172.21.15.57:0/2956846506 shutdown_connections 2023-10-08T13:50:50.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.732+0000 7f5d63e93700 1 -- 172.21.15.57:0/2956846506 wait complete. 2023-10-08T13:50:50.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.733+0000 7f5d63e93700 1 Processor -- start 2023-10-08T13:50:50.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.733+0000 7f5d63e93700 1 -- start start 2023-10-08T13:50:50.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.733+0000 7f5d63e93700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c0ff640 0x7f5d5c195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:50.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.733+0000 7f5d63e93700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d5c100000 0x7f5d5c195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:50.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.733+0000 7f5d63e93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c105630 0x7f5d5c1961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:50.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.733+0000 7f5d63e93700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1a4ea0 con 0x7f5d5c0ff640 2023-10-08T13:50:50.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.733+0000 7f5d63e93700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1a5010 con 0x7f5d5c105630 2023-10-08T13:50:50.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.733+0000 7f5d63e93700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1a51d0 con 0x7f5d5c100000 2023-10-08T13:50:50.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d61c2f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c0ff640 0x7f5d5c195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:50.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d61c2f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c0ff640 0x7f5d5c195720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55040/0 (socket says 172.21.15.57:55040) 2023-10-08T13:50:50.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d61c2f700 1 -- 172.21.15.57:0/2773120854 learned_addr learned my addr 172.21.15.57:0/2773120854 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:50.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d62430700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c105630 0x7f5d5c1961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:50.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d6142e700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d5c100000 0x7f5d5c195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:50.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d61c2f700 1 -- 172.21.15.57:0/2773120854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d5c100000 msgr2=0x7f5d5c195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:50.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d61c2f700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d5c100000 0x7f5d5c195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d61c2f700 1 -- 172.21.15.57:0/2773120854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c105630 msgr2=0x7f5d5c1961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:50.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d61c2f700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c105630 0x7f5d5c1961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d61c2f700 1 -- 172.21.15.57:0/2773120854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d54009d00 con 0x7f5d5c0ff640 2023-10-08T13:50:50.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d6142e700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d5c100000 0x7f5d5c195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:50.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d62430700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c105630 0x7f5d5c1961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:50.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d61c2f700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c0ff640 0x7f5d5c195720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00d950 tx=0x7f5d4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:50.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d52ffd700 1 -- 172.21.15.57:0/2773120854 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c009940 con 0x7f5d5c0ff640 2023-10-08T13:50:50.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d52ffd700 1 -- 172.21.15.57:0/2773120854 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d4c010460 con 0x7f5d5c0ff640 2023-10-08T13:50:50.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d63e93700 1 -- 172.21.15.57:0/2773120854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c1a54b0 con 0x7f5d5c0ff640 2023-10-08T13:50:50.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d52ffd700 1 -- 172.21.15.57:0/2773120854 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00f620 con 0x7f5d5c0ff640 2023-10-08T13:50:50.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.734+0000 7f5d63e93700 1 -- 172.21.15.57:0/2773120854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1a59c0 con 0x7f5d5c0ff640 2023-10-08T13:50:50.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.735+0000 7f5d63e93700 1 -- 172.21.15.57:0/2773120854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c107c40 con 0x7f5d5c0ff640 2023-10-08T13:50:50.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.736+0000 7f5d52ffd700 1 -- 172.21.15.57:0/2773120854 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f5d4c0108d0 con 0x7f5d5c0ff640 2023-10-08T13:50:50.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.737+0000 7f5d52ffd700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5d480a1560 0x7f5d480a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:50.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.737+0000 7f5d52ffd700 1 -- 172.21.15.57:0/2773120854 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f5d4c0cd0d0 con 0x7f5d5c0ff640 2023-10-08T13:50:50.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.737+0000 7f5d6142e700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5d480a1560 0x7f5d480a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:50.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.737+0000 7f5d6142e700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5d480a1560 0x7f5d480a3a20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5d58005950 tx=0x7f5d580058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:50.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.741+0000 7f5d52ffd700 1 -- 172.21.15.57:0/2773120854 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5d4c0961e0 con 0x7f5d5c0ff640 2023-10-08T13:50:50.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.942+0000 7f5d63e93700 1 -- 172.21.15.57:0/2773120854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d5c069230 con 0x7f5d5c0ff640 2023-10-08T13:50:50.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.943+0000 7f5d52ffd700 1 -- 172.21.15.57:0/2773120854 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d4c0160a0 con 0x7f5d5c0ff640 2023-10-08T13:50:50.944 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:50:50.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 -- 172.21.15.57:0/2773120854 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5d480a1560 msgr2=0x7f5d480a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:50.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5d480a1560 0x7f5d480a3a20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5d58005950 tx=0x7f5d580058e0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 -- 172.21.15.57:0/2773120854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c0ff640 msgr2=0x7f5d5c195720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:50.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c0ff640 0x7f5d5c195720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00d950 tx=0x7f5d4c00dc60 comp rx=0 tx=0).stop 2023-10-08T13:50:50.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 -- 172.21.15.57:0/2773120854 shutdown_connections 2023-10-08T13:50:50.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5d480a1560 0x7f5d480a3a20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d5c105630 0x7f5d5c1961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c0ff640 0x7f5d5c195720 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 --2- 172.21.15.57:0/2773120854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d5c100000 0x7f5d5c195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:50.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 -- 172.21.15.57:0/2773120854 >> 172.21.15.57:0/2773120854 conn(0x7f5d5c076790 msgr2=0x7f5d5c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:50.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 -- 172.21.15.57:0/2773120854 shutdown_connections 2023-10-08T13:50:50.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:50.945+0000 7f5d63e93700 1 -- 172.21.15.57:0/2773120854 wait complete. 2023-10-08T13:50:50.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:50 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:50:50.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:50 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:50:50.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:50 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:50.949 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:50 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4251097066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:50.960 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T13:50:51.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.087+0000 7f0e32886700 1 -- 172.21.15.57:0/4275477971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e2c0ffd90 msgr2=0x7f0e2c100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.087+0000 7f0e32886700 1 --2- 172.21.15.57:0/4275477971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e2c0ffd90 0x7f0e2c100210 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009990 tx=0x7f0e20009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.087+0000 7f0e32886700 1 -- 172.21.15.57:0/4275477971 shutdown_connections 2023-10-08T13:50:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.087+0000 7f0e32886700 1 --2- 172.21.15.57:0/4275477971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e2c100750 0x7f0e2c10c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.087+0000 7f0e32886700 1 --2- 172.21.15.57:0/4275477971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e2c105630 0x7f0e2c105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.087+0000 7f0e32886700 1 --2- 172.21.15.57:0/4275477971 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e2c0ffd90 0x7f0e2c100210 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.087+0000 7f0e32886700 1 -- 172.21.15.57:0/4275477971 >> 172.21.15.57:0/4275477971 conn(0x7f0e2c075c20 msgr2=0x7f0e2c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.087+0000 7f0e32886700 1 -- 172.21.15.57:0/4275477971 shutdown_connections 2023-10-08T13:50:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.087+0000 7f0e32886700 1 -- 172.21.15.57:0/4275477971 wait complete. 2023-10-08T13:50:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e32886700 1 Processor -- start 2023-10-08T13:50:51.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e32886700 1 -- start start 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e32886700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e2c0ffd90 0x7f0e2c101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e32886700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e2c100750 0x7f0e2c101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e32886700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e2c105630 0x7f0e2c101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e32886700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c1a4fb0 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e32886700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c1a5120 con 0x7f0e2c105630 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e32886700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c1a52e0 con 0x7f0e2c100750 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e2bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e2c0ffd90 0x7f0e2c101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e2bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e2c0ffd90 0x7f0e2c101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55054/0 (socket says 172.21.15.57:55054) 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e2bfff700 1 -- 172.21.15.57:0/1654377065 learned_addr learned my addr 172.21.15.57:0/1654377065 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.088+0000 7f0e30e23700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e2c105630 0x7f0e2c101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e2b7fe700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e2c100750 0x7f0e2c101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e2bfff700 1 -- 172.21.15.57:0/1654377065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e2c100750 msgr2=0x7f0e2c101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e2bfff700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e2c100750 0x7f0e2c101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e2bfff700 1 -- 172.21.15.57:0/1654377065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e2c105630 msgr2=0x7f0e2c101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e2bfff700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e2c105630 0x7f0e2c101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e2bfff700 1 -- 172.21.15.57:0/1654377065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200095f0 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e2bfff700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e2c0ffd90 0x7f0e2c101230 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00d950 tx=0x7f0e1c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e297fa700 1 -- 172.21.15.57:0/1654377065 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c009b40 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e297fa700 1 -- 172.21.15.57:0/1654377065 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e1c009ca0 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e32886700 1 -- 172.21.15.57:0/1654377065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c1a55c0 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e297fa700 1 -- 172.21.15.57:0/1654377065 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c00f620 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.089+0000 7f0e32886700 1 -- 172.21.15.57:0/1654377065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c1a5ad0 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.090+0000 7f0e32886700 1 -- 172.21.15.57:0/1654377065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c100da0 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.091+0000 7f0e297fa700 1 -- 172.21.15.57:0/1654377065 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f0e1c010460 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.092+0000 7f0e297fa700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a1560 0x7f0e140a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.092+0000 7f0e2b7fe700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a1560 0x7f0e140a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.092+0000 7f0e297fa700 1 -- 172.21.15.57:0/1654377065 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f0e1c0cd610 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.092+0000 7f0e2b7fe700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a1560 0x7f0e140a3a20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0e20006010 tx=0x7f0e2001a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:51.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.095+0000 7f0e297fa700 1 -- 172.21.15.57:0/1654377065 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0e1c096720 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:50 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:50:51.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:50 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:50:51.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:50 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:51.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:50 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4251097066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.297+0000 7f0e32886700 1 -- 172.21.15.57:0/1654377065 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e2c10f510 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.298+0000 7f0e297fa700 1 -- 172.21.15.57:0/1654377065 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e1c099d40 con 0x7f0e2c0ffd90 2023-10-08T13:50:51.299 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:50:51.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 -- 172.21.15.57:0/1654377065 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a1560 msgr2=0x7f0e140a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a1560 0x7f0e140a3a20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0e20006010 tx=0x7f0e2001a040 comp rx=0 tx=0).stop 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 -- 172.21.15.57:0/1654377065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e2c0ffd90 msgr2=0x7f0e2c101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e2c0ffd90 0x7f0e2c101230 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00d950 tx=0x7f0e1c00dd10 comp rx=0 tx=0).stop 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 -- 172.21.15.57:0/1654377065 shutdown_connections 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a1560 0x7f0e140a3a20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e2c105630 0x7f0e2c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e2c0ffd90 0x7f0e2c101230 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 --2- 172.21.15.57:0/1654377065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e2c100750 0x7f0e2c101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 -- 172.21.15.57:0/1654377065 >> 172.21.15.57:0/1654377065 conn(0x7f0e2c075c20 msgr2=0x7f0e2c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 -- 172.21.15.57:0/1654377065 shutdown_connections 2023-10-08T13:50:51.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.300+0000 7f0e32886700 1 -- 172.21.15.57:0/1654377065 wait complete. 2023-10-08T13:50:51.314 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T13:50:51.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:50 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:50:51.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:50 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:50:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:50 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:50:51.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:50 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4251097066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.441+0000 7ffa42ecf700 1 -- 172.21.15.57:0/1489832983 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c0ff620 msgr2=0x7ffa3c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.441+0000 7ffa42ecf700 1 --2- 172.21.15.57:0/1489832983 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c0ff620 0x7ffa3c0ffaa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009990 tx=0x7ffa28009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.441+0000 7ffa42ecf700 1 -- 172.21.15.57:0/1489832983 shutdown_connections 2023-10-08T13:50:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.441+0000 7ffa42ecf700 1 --2- 172.21.15.57:0/1489832983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c0fffe0 0x7ffa3c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.441+0000 7ffa42ecf700 1 --2- 172.21.15.57:0/1489832983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa3c1056b0 0x7ffa3c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.441+0000 7ffa42ecf700 1 --2- 172.21.15.57:0/1489832983 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c0ff620 0x7ffa3c0ffaa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.441+0000 7ffa42ecf700 1 -- 172.21.15.57:0/1489832983 >> 172.21.15.57:0/1489832983 conn(0x7ffa3c076780 msgr2=0x7ffa3c076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:51.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.442+0000 7ffa42ecf700 1 -- 172.21.15.57:0/1489832983 shutdown_connections 2023-10-08T13:50:51.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.442+0000 7ffa42ecf700 1 -- 172.21.15.57:0/1489832983 wait complete. 2023-10-08T13:50:51.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.442+0000 7ffa42ecf700 1 Processor -- start 2023-10-08T13:50:51.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.442+0000 7ffa42ecf700 1 -- start start 2023-10-08T13:50:51.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa42ecf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa3c0ff620 0x7ffa3c195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa42ecf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c0fffe0 0x7ffa3c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa42ecf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c1056b0 0x7ffa3c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa42ecf700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c19a910 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa42ecf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c19aa80 con 0x7ffa3c1056b0 2023-10-08T13:50:51.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa42ecf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c19ac40 con 0x7ffa3c0ff620 2023-10-08T13:50:51.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa3bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c0fffe0 0x7ffa3c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa3bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c0fffe0 0x7ffa3c195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55068/0 (socket says 172.21.15.57:55068) 2023-10-08T13:50:51.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa3bfff700 1 -- 172.21.15.57:0/3490287612 learned_addr learned my addr 172.21.15.57:0/3490287612 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:51.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa40c6b700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa3c0ff620 0x7ffa3c195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa4146c700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c1056b0 0x7ffa3c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa3bfff700 1 -- 172.21.15.57:0/3490287612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa3c0ff620 msgr2=0x7ffa3c195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa3bfff700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa3c0ff620 0x7ffa3c195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa3bfff700 1 -- 172.21.15.57:0/3490287612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c1056b0 msgr2=0x7ffa3c19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa3bfff700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c1056b0 0x7ffa3c19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa3bfff700 1 -- 172.21.15.57:0/3490287612 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa280095f0 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa3bfff700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c0fffe0 0x7ffa3c195f70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa280098e0 tx=0x7ffa2800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:51.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.443+0000 7ffa39ffb700 1 -- 172.21.15.57:0/3490287612 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2801d070 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.444+0000 7ffa39ffb700 1 -- 172.21.15.57:0/3490287612 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa28022470 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.444+0000 7ffa42ecf700 1 -- 172.21.15.57:0/3490287612 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3c19aec0 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.444+0000 7ffa39ffb700 1 -- 172.21.15.57:0/3490287612 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28017650 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.444+0000 7ffa42ecf700 1 -- 172.21.15.57:0/3490287612 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3c19b1c0 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.445+0000 7ffa42ecf700 1 -- 172.21.15.57:0/3490287612 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3c107c30 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.446+0000 7ffa39ffb700 1 -- 172.21.15.57:0/3490287612 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7ffa280225e0 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.446+0000 7ffa39ffb700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa2c0a1560 0x7ffa2c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.447+0000 7ffa39ffb700 1 -- 172.21.15.57:0/3490287612 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7ffa280ce310 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.447+0000 7ffa40c6b700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa2c0a1560 0x7ffa2c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.447+0000 7ffa40c6b700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa2c0a1560 0x7ffa2c0a3a20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c077ab0 tx=0x7ffa3000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:51.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.451+0000 7ffa39ffb700 1 -- 172.21.15.57:0/3490287612 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffa280974b0 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.654+0000 7ffa42ecf700 1 -- 172.21.15.57:0/3490287612 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffa3c196d30 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.654+0000 7ffa39ffb700 1 -- 172.21.15.57:0/3490287612 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffa2809aad0 con 0x7ffa3c0fffe0 2023-10-08T13:50:51.656 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T13:50:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.656+0000 7ffa42ecf700 1 -- 172.21.15.57:0/3490287612 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa2c0a1560 msgr2=0x7ffa2c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.656+0000 7ffa42ecf700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa2c0a1560 0x7ffa2c0a3a20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c077ab0 tx=0x7ffa3000b410 comp rx=0 tx=0).stop 2023-10-08T13:50:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.656+0000 7ffa42ecf700 1 -- 172.21.15.57:0/3490287612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c0fffe0 msgr2=0x7ffa3c195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.656+0000 7ffa42ecf700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c0fffe0 0x7ffa3c195f70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa280098e0 tx=0x7ffa2800f670 comp rx=0 tx=0).stop 2023-10-08T13:50:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.657+0000 7ffa42ecf700 1 -- 172.21.15.57:0/3490287612 shutdown_connections 2023-10-08T13:50:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.657+0000 7ffa42ecf700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa2c0a1560 0x7ffa2c0a3a20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.657+0000 7ffa42ecf700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c1056b0 0x7ffa3c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.657+0000 7ffa42ecf700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa3c0ff620 0x7ffa3c195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.657+0000 7ffa42ecf700 1 --2- 172.21.15.57:0/3490287612 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c0fffe0 0x7ffa3c195f70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.657+0000 7ffa42ecf700 1 -- 172.21.15.57:0/3490287612 >> 172.21.15.57:0/3490287612 conn(0x7ffa3c076780 msgr2=0x7ffa3c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.657+0000 7ffa42ecf700 1 -- 172.21.15.57:0/3490287612 shutdown_connections 2023-10-08T13:50:51.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.657+0000 7ffa42ecf700 1 -- 172.21.15.57:0/3490287612 wait complete. 2023-10-08T13:50:51.670 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T13:50:51.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.798+0000 7f7bfc872700 1 -- 172.21.15.57:0/1426265575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bf80ff620 msgr2=0x7f7bf80ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.798+0000 7f7bfc872700 1 --2- 172.21.15.57:0/1426265575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bf80ff620 0x7f7bf80ffaa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009990 tx=0x7f7be0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.798+0000 7f7bfc872700 1 -- 172.21.15.57:0/1426265575 shutdown_connections 2023-10-08T13:50:51.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.798+0000 7f7bfc872700 1 --2- 172.21.15.57:0/1426265575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf80fffe0 0x7f7bf810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.798+0000 7f7bfc872700 1 --2- 172.21.15.57:0/1426265575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf81056b0 0x7f7bf8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.798+0000 7f7bfc872700 1 --2- 172.21.15.57:0/1426265575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bf80ff620 0x7f7bf80ffaa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.798+0000 7f7bfc872700 1 -- 172.21.15.57:0/1426265575 >> 172.21.15.57:0/1426265575 conn(0x7f7bf8076780 msgr2=0x7f7bf8076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:51.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.798+0000 7f7bfc872700 1 -- 172.21.15.57:0/1426265575 shutdown_connections 2023-10-08T13:50:51.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.798+0000 7f7bfc872700 1 -- 172.21.15.57:0/1426265575 wait complete. 2023-10-08T13:50:51.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7bfc872700 1 Processor -- start 2023-10-08T13:50:51.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7bfc872700 1 -- start start 2023-10-08T13:50:51.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7bfc872700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf80ff620 0x7f7bf8197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7bfc872700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bf80fffe0 0x7f7bf8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7bfc872700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf81056b0 0x7f7bf81a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7bfc872700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf8198720 con 0x7f7bf80fffe0 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7bfc872700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81a2f30 con 0x7f7bf81056b0 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7bfc872700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81a3070 con 0x7f7bf80ff620 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7beffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bf80fffe0 0x7f7bf8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7beffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bf80fffe0 0x7f7bf8197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55082/0 (socket says 172.21.15.57:55082) 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7beffff700 1 -- 172.21.15.57:0/2146365882 learned_addr learned my addr 172.21.15.57:0/2146365882 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7bf6d9d700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf81056b0 0x7f7bf81a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7bf659c700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf80ff620 0x7f7bf8197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7beffff700 1 -- 172.21.15.57:0/2146365882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf80ff620 msgr2=0x7f7bf8197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7beffff700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf80ff620 0x7f7bf8197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7beffff700 1 -- 172.21.15.57:0/2146365882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf81056b0 msgr2=0x7f7bf81a29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:51.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7beffff700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf81056b0 0x7f7bf81a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:51.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.799+0000 7f7beffff700 1 -- 172.21.15.57:0/2146365882 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be00095f0 con 0x7f7bf80fffe0 2023-10-08T13:50:51.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.800+0000 7f7beffff700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bf80fffe0 0x7f7bf8197eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7be0000c00 tx=0x7f7be0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:51.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.800+0000 7f7bef7fe700 1 -- 172.21.15.57:0/2146365882 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be001d070 con 0x7f7bf80fffe0 2023-10-08T13:50:51.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.800+0000 7f7bfc872700 1 -- 172.21.15.57:0/2146365882 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf81a32a0 con 0x7f7bf80fffe0 2023-10-08T13:50:51.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.800+0000 7f7bef7fe700 1 -- 172.21.15.57:0/2146365882 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7be0022470 con 0x7f7bf80fffe0 2023-10-08T13:50:51.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.800+0000 7f7bef7fe700 1 -- 172.21.15.57:0/2146365882 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0017650 con 0x7f7bf80fffe0 2023-10-08T13:50:51.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.800+0000 7f7bfc872700 1 -- 172.21.15.57:0/2146365882 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf81a3780 con 0x7f7bf80fffe0 2023-10-08T13:50:51.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.801+0000 7f7bfc872700 1 -- 172.21.15.57:0/2146365882 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf8107c30 con 0x7f7bf80fffe0 2023-10-08T13:50:51.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.802+0000 7f7bef7fe700 1 -- 172.21.15.57:0/2146365882 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f7be00225e0 con 0x7f7bf80fffe0 2023-10-08T13:50:51.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.803+0000 7f7bef7fe700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7be40a1560 0x7f7be40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:51.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.803+0000 7f7bf659c700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7be40a1560 0x7f7be40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:51.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.803+0000 7f7bef7fe700 1 -- 172.21.15.57:0/2146365882 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f7be00ce4a0 con 0x7f7bf80fffe0 2023-10-08T13:50:51.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.803+0000 7f7bf659c700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7be40a1560 0x7f7be40a3a20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8077ab0 tx=0x7f7be800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:51.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:51.807+0000 7f7bef7fe700 1 -- 172.21.15.57:0/2146365882 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7be0097640 con 0x7f7bf80fffe0 2023-10-08T13:50:52.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.008+0000 7f7bfc872700 1 -- 172.21.15.57:0/2146365882 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7bf8067df0 con 0x7f7bf80fffe0 2023-10-08T13:50:52.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.008+0000 7f7bef7fe700 1 -- 172.21.15.57:0/2146365882 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7be009ac60 con 0x7f7bf80fffe0 2023-10-08T13:50:52.009 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:50:52.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.010+0000 7f7bfc872700 1 -- 172.21.15.57:0/2146365882 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7be40a1560 msgr2=0x7f7be40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.010+0000 7f7bfc872700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7be40a1560 0x7f7be40a3a20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8077ab0 tx=0x7f7be800b410 comp rx=0 tx=0).stop 2023-10-08T13:50:52.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.010+0000 7f7bfc872700 1 -- 172.21.15.57:0/2146365882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bf80fffe0 msgr2=0x7f7bf8197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.010+0000 7f7bfc872700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bf80fffe0 0x7f7bf8197eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7be0000c00 tx=0x7f7be0004730 comp rx=0 tx=0).stop 2023-10-08T13:50:52.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.010+0000 7f7bfc872700 1 -- 172.21.15.57:0/2146365882 shutdown_connections 2023-10-08T13:50:52.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.011+0000 7f7bfc872700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7be40a1560 0x7f7be40a3a20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.011+0000 7f7bfc872700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf81056b0 0x7f7bf81a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.011+0000 7f7bfc872700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf80ff620 0x7f7bf8197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.011+0000 7f7bfc872700 1 --2- 172.21.15.57:0/2146365882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bf80fffe0 0x7f7bf8197eb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.011+0000 7f7bfc872700 1 -- 172.21.15.57:0/2146365882 >> 172.21.15.57:0/2146365882 conn(0x7f7bf8076780 msgr2=0x7f7bf80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:52.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.011+0000 7f7bfc872700 1 -- 172.21.15.57:0/2146365882 shutdown_connections 2023-10-08T13:50:52.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.011+0000 7f7bfc872700 1 -- 172.21.15.57:0/2146365882 wait complete. 2023-10-08T13:50:52.024 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:50:52.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.149+0000 7f5b954ab700 1 -- 172.21.15.57:0/461900498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b900ff570 msgr2=0x7f5b900ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.149+0000 7f5b954ab700 1 --2- 172.21.15.57:0/461900498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b900ff570 0x7f5b900ff9f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5b780099c0 tx=0x7f5b78009cd0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.149+0000 7f5b954ab700 1 -- 172.21.15.57:0/461900498 shutdown_connections 2023-10-08T13:50:52.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.149+0000 7f5b954ab700 1 --2- 172.21.15.57:0/461900498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b900fff30 0x7f5b9010c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.149+0000 7f5b954ab700 1 --2- 172.21.15.57:0/461900498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b90105630 0x7f5b90105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.149+0000 7f5b954ab700 1 --2- 172.21.15.57:0/461900498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b900ff570 0x7f5b900ff9f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.149+0000 7f5b954ab700 1 -- 172.21.15.57:0/461900498 >> 172.21.15.57:0/461900498 conn(0x7f5b900fb060 msgr2=0x7f5b900fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:52.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.149+0000 7f5b954ab700 1 -- 172.21.15.57:0/461900498 shutdown_connections 2023-10-08T13:50:52.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.149+0000 7f5b954ab700 1 -- 172.21.15.57:0/461900498 wait complete. 2023-10-08T13:50:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.150+0000 7f5b954ab700 1 Processor -- start 2023-10-08T13:50:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.150+0000 7f5b954ab700 1 -- start start 2023-10-08T13:50:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b954ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b900ff570 0x7f5b9019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b954ab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b900fff30 0x7f5b9019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b954ab700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b90105630 0x7f5b90195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b954ab700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b90195e80 con 0x7f5b900fff30 2023-10-08T13:50:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b954ab700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b90195ff0 con 0x7f5b90105630 2023-10-08T13:50:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b954ab700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b901961b0 con 0x7f5b900ff570 2023-10-08T13:50:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b900fff30 0x7f5b9019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b900fff30 0x7f5b9019bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55096/0 (socket says 172.21.15.57:55096) 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8e7fc700 1 -- 172.21.15.57:0/74865137 learned_addr learned my addr 172.21.15.57:0/74865137 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8effd700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b900ff570 0x7f5b9019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8f7fe700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b90105630 0x7f5b90195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8e7fc700 1 -- 172.21.15.57:0/74865137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b900ff570 msgr2=0x7f5b9019b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8e7fc700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b900ff570 0x7f5b9019b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8e7fc700 1 -- 172.21.15.57:0/74865137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b90105630 msgr2=0x7f5b90195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8e7fc700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b90105630 0x7f5b90195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8e7fc700 1 -- 172.21.15.57:0/74865137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b780095f0 con 0x7f5b900fff30 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.151+0000 7f5b8e7fc700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b900fff30 0x7f5b9019bc20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5b78000c00 tx=0x7f5b7800b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.152+0000 7f5b87fff700 1 -- 172.21.15.57:0/74865137 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7801d070 con 0x7f5b900fff30 2023-10-08T13:50:52.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.152+0000 7f5b87fff700 1 -- 172.21.15.57:0/74865137 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b7800bdf0 con 0x7f5b900fff30 2023-10-08T13:50:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.152+0000 7f5b954ab700 1 -- 172.21.15.57:0/74865137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b90196430 con 0x7f5b900fff30 2023-10-08T13:50:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.152+0000 7f5b87fff700 1 -- 172.21.15.57:0/74865137 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78003720 con 0x7f5b900fff30 2023-10-08T13:50:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.152+0000 7f5b954ab700 1 -- 172.21.15.57:0/74865137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b901a26d0 con 0x7f5b900fff30 2023-10-08T13:50:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.152+0000 7f5b954ab700 1 -- 172.21.15.57:0/74865137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b90107ad0 con 0x7f5b900fff30 2023-10-08T13:50:52.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.154+0000 7f5b87fff700 1 -- 172.21.15.57:0/74865137 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f5b78004830 con 0x7f5b900fff30 2023-10-08T13:50:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.154+0000 7f5b87fff700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b7c0a1560 0x7f5b7c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.154+0000 7f5b8effd700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b7c0a1560 0x7f5b7c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.154+0000 7f5b87fff700 1 -- 172.21.15.57:0/74865137 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f5b780cdf30 con 0x7f5b900fff30 2023-10-08T13:50:52.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.155+0000 7f5b8effd700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b7c0a1560 0x7f5b7c0a3a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b80005950 tx=0x7f5b8000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:52.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.158+0000 7f5b87fff700 1 -- 172.21.15.57:0/74865137 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5b780970d0 con 0x7f5b900fff30 2023-10-08T13:50:52.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:51 smithi057 ceph-mon[99476]: pgmap v276: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T13:50:52.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:51 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2773120854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:50:52.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:51 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1654377065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:50:52.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:51 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3490287612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:50:52.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:51 smithi099 ceph-mon[103849]: pgmap v276: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T13:50:52.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:51 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2773120854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:50:52.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:51 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1654377065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:50:52.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:51 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3490287612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:50:52.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.406+0000 7f5b954ab700 1 -- 172.21.15.57:0/74865137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b90196cd0 con 0x7f5b900fff30 2023-10-08T13:50:52.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.407+0000 7f5b87fff700 1 -- 172.21.15.57:0/74865137 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f5b7809a6f0 con 0x7f5b900fff30 2023-10-08T13:50:52.408 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:52.408 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:50:07.719573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:50:52.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 -- 172.21.15.57:0/74865137 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b7c0a1560 msgr2=0x7f5b7c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b7c0a1560 0x7f5b7c0a3a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b80005950 tx=0x7f5b8000b410 comp rx=0 tx=0).stop 2023-10-08T13:50:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 -- 172.21.15.57:0/74865137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b900fff30 msgr2=0x7f5b9019bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b900fff30 0x7f5b9019bc20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5b78000c00 tx=0x7f5b7800b9c0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 -- 172.21.15.57:0/74865137 shutdown_connections 2023-10-08T13:50:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5b7c0a1560 0x7f5b7c0a3a20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b90105630 0x7f5b90195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b900ff570 0x7f5b9019b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 --2- 172.21.15.57:0/74865137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b900fff30 0x7f5b9019bc20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 -- 172.21.15.57:0/74865137 >> 172.21.15.57:0/74865137 conn(0x7f5b900fb060 msgr2=0x7f5b900fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 -- 172.21.15.57:0/74865137 shutdown_connections 2023-10-08T13:50:52.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.409+0000 7f5b954ab700 1 -- 172.21.15.57:0/74865137 wait complete. 2023-10-08T13:50:52.412 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:50:52.423 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:50:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:51 smithi115 ceph-mon[104408]: pgmap v276: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T13:50:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:51 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2773120854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:50:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:51 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1654377065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:50:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:51 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3490287612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:50:52.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.548+0000 7fdeae124700 1 -- 172.21.15.57:0/133339601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdea80ff560 msgr2=0x7fdea80ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.548+0000 7fdeae124700 1 --2- 172.21.15.57:0/133339601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdea80ff560 0x7fdea80ff9e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0099c0 tx=0x7fde9c009cd0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.548+0000 7fdeae124700 1 -- 172.21.15.57:0/133339601 shutdown_connections 2023-10-08T13:50:52.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.548+0000 7fdeae124700 1 --2- 172.21.15.57:0/133339601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea80fff20 0x7fdea810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.548+0000 7fdeae124700 1 --2- 172.21.15.57:0/133339601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea8105620 0x7fdea8105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.548+0000 7fdeae124700 1 --2- 172.21.15.57:0/133339601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdea80ff560 0x7fdea80ff9e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.548+0000 7fdeae124700 1 -- 172.21.15.57:0/133339601 >> 172.21.15.57:0/133339601 conn(0x7fdea80fb070 msgr2=0x7fdea80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.548+0000 7fdeae124700 1 -- 172.21.15.57:0/133339601 shutdown_connections 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.549+0000 7fdeae124700 1 -- 172.21.15.57:0/133339601 wait complete. 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.549+0000 7fdeae124700 1 Processor -- start 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.549+0000 7fdeae124700 1 -- start start 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.549+0000 7fdeae124700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdea80ff560 0x7fdea819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.549+0000 7fdeae124700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea80fff20 0x7fdea819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.549+0000 7fdeae124700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea8105620 0x7fdea8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.549+0000 7fdeae124700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8199260 con 0x7fdea80ff560 2023-10-08T13:50:52.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.549+0000 7fdeae124700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea81993d0 con 0x7fdea80fff20 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.549+0000 7fdeae124700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8199590 con 0x7fdea8105620 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdea80ff560 0x7fdea819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdea80ff560 0x7fdea819e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55108/0 (socket says 172.21.15.57:55108) 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea77fe700 1 -- 172.21.15.57:0/900561992 learned_addr learned my addr 172.21.15.57:0/900561992 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea7fff700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea8105620 0x7fdea8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea6ffd700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea80fff20 0x7fdea819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea77fe700 1 -- 172.21.15.57:0/900561992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea8105620 msgr2=0x7fdea8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea77fe700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea8105620 0x7fdea8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea77fe700 1 -- 172.21.15.57:0/900561992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea80fff20 msgr2=0x7fdea819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea77fe700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea80fff20 0x7fdea819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea77fe700 1 -- 172.21.15.57:0/900561992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c0095f0 con 0x7fdea80ff560 2023-10-08T13:50:52.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea77fe700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdea80ff560 0x7fdea819e9f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fde9800d950 tx=0x7fde9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:52.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea4ff9700 1 -- 172.21.15.57:0/900561992 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9800f890 con 0x7fdea80ff560 2023-10-08T13:50:52.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea4ff9700 1 -- 172.21.15.57:0/900561992 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde98010460 con 0x7fdea80ff560 2023-10-08T13:50:52.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdeae124700 1 -- 172.21.15.57:0/900561992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea8199870 con 0x7fdea80ff560 2023-10-08T13:50:52.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdea4ff9700 1 -- 172.21.15.57:0/900561992 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9800e610 con 0x7fdea80ff560 2023-10-08T13:50:52.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.550+0000 7fdeae124700 1 -- 172.21.15.57:0/900561992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea81a5970 con 0x7fdea80ff560 2023-10-08T13:50:52.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.551+0000 7fdeae124700 1 -- 172.21.15.57:0/900561992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea8100c80 con 0x7fdea80ff560 2023-10-08T13:50:52.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.552+0000 7fdea4ff9700 1 -- 172.21.15.57:0/900561992 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fde980108d0 con 0x7fdea80ff560 2023-10-08T13:50:52.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.553+0000 7fdea4ff9700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde900a1560 0x7fde900a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.553+0000 7fdea4ff9700 1 -- 172.21.15.57:0/900561992 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fde980cd0b0 con 0x7fdea80ff560 2023-10-08T13:50:52.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.553+0000 7fdea6ffd700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde900a1560 0x7fde900a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.553+0000 7fdea6ffd700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde900a1560 0x7fde900a3a20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009990 tx=0x7fde9c003680 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:52.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.556+0000 7fdea4ff9700 1 -- 172.21.15.57:0/900561992 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fde980961c0 con 0x7fdea80ff560 2023-10-08T13:50:52.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.761+0000 7fdeae124700 1 -- 172.21.15.57:0/900561992 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdea810f480 con 0x7fdea80ff560 2023-10-08T13:50:52.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.763+0000 7fdea4ff9700 1 -- 172.21.15.57:0/900561992 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24753 (secure 0 0 0) 0x7fde98016090 con 0x7fdea80ff560 2023-10-08T13:50:52.764 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:52.764 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:50:15.770421+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:50:52.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 -- 172.21.15.57:0/900561992 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde900a1560 msgr2=0x7fde900a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde900a1560 0x7fde900a3a20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009990 tx=0x7fde9c003680 comp rx=0 tx=0).stop 2023-10-08T13:50:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 -- 172.21.15.57:0/900561992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdea80ff560 msgr2=0x7fdea819e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdea80ff560 0x7fdea819e9f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fde9800d950 tx=0x7fde9800dc60 comp rx=0 tx=0).stop 2023-10-08T13:50:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 -- 172.21.15.57:0/900561992 shutdown_connections 2023-10-08T13:50:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea8105620 0x7fdea8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdea80ff560 0x7fdea819e9f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde900a1560 0x7fde900a3a20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 --2- 172.21.15.57:0/900561992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea80fff20 0x7fdea819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 -- 172.21.15.57:0/900561992 >> 172.21.15.57:0/900561992 conn(0x7fdea80fb070 msgr2=0x7fdea8068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 -- 172.21.15.57:0/900561992 shutdown_connections 2023-10-08T13:50:52.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.765+0000 7fdeae124700 1 -- 172.21.15.57:0/900561992 wait complete. 2023-10-08T13:50:52.780 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T13:50:52.780 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:50:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.908+0000 7f850c35f700 1 -- 172.21.15.57:0/1988692847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85040ff620 msgr2=0x7f85040ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.908+0000 7f850c35f700 1 --2- 172.21.15.57:0/1988692847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85040ff620 0x7f85040ffaa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8500009990 tx=0x7f8500009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.908+0000 7f850c35f700 1 -- 172.21.15.57:0/1988692847 shutdown_connections 2023-10-08T13:50:52.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.908+0000 7f850c35f700 1 --2- 172.21.15.57:0/1988692847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85040fffe0 0x7f850410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.908+0000 7f850c35f700 1 --2- 172.21.15.57:0/1988692847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85041056b0 0x7f8504105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.908+0000 7f850c35f700 1 --2- 172.21.15.57:0/1988692847 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85040ff620 0x7f85040ffaa0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.908+0000 7f850c35f700 1 -- 172.21.15.57:0/1988692847 >> 172.21.15.57:0/1988692847 conn(0x7f8504076780 msgr2=0x7f8504076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:52.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.908+0000 7f850c35f700 1 -- 172.21.15.57:0/1988692847 shutdown_connections 2023-10-08T13:50:52.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.908+0000 7f850c35f700 1 -- 172.21.15.57:0/1988692847 wait complete. 2023-10-08T13:50:52.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850c35f700 1 Processor -- start 2023-10-08T13:50:52.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850c35f700 1 -- start start 2023-10-08T13:50:52.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850c35f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85040ff620 0x7f850419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850c35f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85040fffe0 0x7f850419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850c35f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85041056b0 0x7f8504195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850c35f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8504195f60 con 0x7f85040fffe0 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850c35f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85041960d0 con 0x7f85041056b0 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850c35f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8504196290 con 0x7f85040ff620 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f85098fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85040fffe0 0x7f850419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f85098fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85040fffe0 0x7f850419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55124/0 (socket says 172.21.15.57:55124) 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f85098fa700 1 -- 172.21.15.57:0/4017428626 learned_addr learned my addr 172.21.15.57:0/4017428626 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850a0fb700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85040ff620 0x7f850419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850a8fc700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85041056b0 0x7f8504195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f85098fa700 1 -- 172.21.15.57:0/4017428626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85040ff620 msgr2=0x7f850419b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f85098fa700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85040ff620 0x7f850419b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f85098fa700 1 -- 172.21.15.57:0/4017428626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85041056b0 msgr2=0x7f8504195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:52.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f85098fa700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85041056b0 0x7f8504195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f85098fa700 1 -- 172.21.15.57:0/4017428626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85000095f0 con 0x7f85040fffe0 2023-10-08T13:50:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f850a8fc700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85041056b0 0x7f8504195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.909+0000 7f85098fa700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85040fffe0 0x7f850419bc70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f85000095c0 tx=0x7f850000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.910+0000 7f84fb7fe700 1 -- 172.21.15.57:0/4017428626 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850001d070 con 0x7f85040fffe0 2023-10-08T13:50:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.910+0000 7f84fb7fe700 1 -- 172.21.15.57:0/4017428626 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85000054f0 con 0x7f85040fffe0 2023-10-08T13:50:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.910+0000 7f850c35f700 1 -- 172.21.15.57:0/4017428626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8504196510 con 0x7f85040fffe0 2023-10-08T13:50:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.910+0000 7f84fb7fe700 1 -- 172.21.15.57:0/4017428626 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500004bd0 con 0x7f85040fffe0 2023-10-08T13:50:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.910+0000 7f850c35f700 1 -- 172.21.15.57:0/4017428626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85041a2720 con 0x7f85040fffe0 2023-10-08T13:50:52.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.911+0000 7f850c35f700 1 -- 172.21.15.57:0/4017428626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8504107c30 con 0x7f85040fffe0 2023-10-08T13:50:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.913+0000 7f84fb7fe700 1 -- 172.21.15.57:0/4017428626 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f850000fb80 con 0x7f85040fffe0 2023-10-08T13:50:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.913+0000 7f84fb7fe700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84f00a1560 0x7f84f00a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.914+0000 7f850a0fb700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84f00a1560 0x7f84f00a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.914+0000 7f84fb7fe700 1 -- 172.21.15.57:0/4017428626 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f85000ce510 con 0x7f85040fffe0 2023-10-08T13:50:52.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.914+0000 7f850a0fb700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84f00a1560 0x7f84f00a3a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8504077ab0 tx=0x7f84f400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:52.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:52.917+0000 7f84fb7fe700 1 -- 172.21.15.57:0/4017428626 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8500097650 con 0x7f85040fffe0 2023-10-08T13:50:53.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.124+0000 7f850c35f700 1 -- 172.21.15.57:0/4017428626 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8504067590 con 0x7f85040fffe0 2023-10-08T13:50:53.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.124+0000 7f84fb7fe700 1 -- 172.21.15.57:0/4017428626 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f850009ac70 con 0x7f85040fffe0 2023-10-08T13:50:53.125 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:50:53.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 -- 172.21.15.57:0/4017428626 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84f00a1560 msgr2=0x7f84f00a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84f00a1560 0x7f84f00a3a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8504077ab0 tx=0x7f84f400b410 comp rx=0 tx=0).stop 2023-10-08T13:50:53.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 -- 172.21.15.57:0/4017428626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85040fffe0 msgr2=0x7f850419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85040fffe0 0x7f850419bc70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f85000095c0 tx=0x7f850000f690 comp rx=0 tx=0).stop 2023-10-08T13:50:53.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 -- 172.21.15.57:0/4017428626 shutdown_connections 2023-10-08T13:50:53.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84f00a1560 0x7f84f00a3a20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85041056b0 0x7f8504195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85040ff620 0x7f850419b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 --2- 172.21.15.57:0/4017428626 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85040fffe0 0x7f850419bc70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 -- 172.21.15.57:0/4017428626 >> 172.21.15.57:0/4017428626 conn(0x7f8504076780 msgr2=0x7f85040fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:53.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 -- 172.21.15.57:0/4017428626 shutdown_connections 2023-10-08T13:50:53.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.126+0000 7f850c35f700 1 -- 172.21.15.57:0/4017428626 wait complete. 2023-10-08T13:50:53.139 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:52 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2146365882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:50:53.139 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:52 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/74865137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:53.139 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:52 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/900561992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:53.140 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T13:50:53.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.269+0000 7f938a679700 1 -- 172.21.15.57:0/2755612034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93840ffd90 msgr2=0x7f9384100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.269+0000 7f938a679700 1 --2- 172.21.15.57:0/2755612034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93840ffd90 0x7f9384100210 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9378009990 tx=0x7f9378009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.269+0000 7f938a679700 1 -- 172.21.15.57:0/2755612034 shutdown_connections 2023-10-08T13:50:53.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.269+0000 7f938a679700 1 --2- 172.21.15.57:0/2755612034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9384100750 0x7f938410c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.269+0000 7f938a679700 1 --2- 172.21.15.57:0/2755612034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9384105630 0x7f9384105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.269+0000 7f938a679700 1 --2- 172.21.15.57:0/2755612034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93840ffd90 0x7f9384100210 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.269+0000 7f938a679700 1 -- 172.21.15.57:0/2755612034 >> 172.21.15.57:0/2755612034 conn(0x7f9384075c20 msgr2=0x7f93840fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.269+0000 7f938a679700 1 -- 172.21.15.57:0/2755612034 shutdown_connections 2023-10-08T13:50:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.269+0000 7f938a679700 1 -- 172.21.15.57:0/2755612034 wait complete. 2023-10-08T13:50:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f938a679700 1 Processor -- start 2023-10-08T13:50:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f938a679700 1 -- start start 2023-10-08T13:50:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f938a679700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93840ffd90 0x7f93841979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f938a679700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9384100750 0x7f9384197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f938a679700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9384105630 0x7f93841a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f938a679700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93841987f0 con 0x7f9384100750 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f938a679700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93841a2fb0 con 0x7f9384105630 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f938a679700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93841a30f0 con 0x7f93840ffd90 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f93837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9384100750 0x7f9384197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f93837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9384100750 0x7f9384197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55138/0 (socket says 172.21.15.57:55138) 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.270+0000 7f93837fe700 1 -- 172.21.15.57:0/2660791597 learned_addr learned my addr 172.21.15.57:0/2660791597 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f9383fff700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93840ffd90 0x7f93841979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f9388c16700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9384105630 0x7f93841a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f93837fe700 1 -- 172.21.15.57:0/2660791597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93840ffd90 msgr2=0x7f93841979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f93837fe700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93840ffd90 0x7f93841979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f93837fe700 1 -- 172.21.15.57:0/2660791597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9384105630 msgr2=0x7f93841a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f93837fe700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9384105630 0x7f93841a2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f93837fe700 1 -- 172.21.15.57:0/2660791597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93780095f0 con 0x7f9384100750 2023-10-08T13:50:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f9383fff700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93840ffd90 0x7f93841979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f93837fe700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9384100750 0x7f9384197f30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f93780095c0 tx=0x7f937800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f93817fa700 1 -- 172.21.15.57:0/2660791597 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937801d070 con 0x7f9384100750 2023-10-08T13:50:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f93817fa700 1 -- 172.21.15.57:0/2660791597 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93780054f0 con 0x7f9384100750 2023-10-08T13:50:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f938a679700 1 -- 172.21.15.57:0/2660791597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93841a3370 con 0x7f9384100750 2023-10-08T13:50:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f93817fa700 1 -- 172.21.15.57:0/2660791597 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378004c40 con 0x7f9384100750 2023-10-08T13:50:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.271+0000 7f938a679700 1 -- 172.21.15.57:0/2660791597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93841a3850 con 0x7f9384100750 2023-10-08T13:50:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.272+0000 7f938a679700 1 -- 172.21.15.57:0/2660791597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9384107c40 con 0x7f9384100750 2023-10-08T13:50:53.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:52 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2146365882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:50:53.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:52 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/74865137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:53.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:52 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/900561992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.273+0000 7f93817fa700 1 -- 172.21.15.57:0/2660791597 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f937800fc30 con 0x7f9384100750 2023-10-08T13:50:53.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.274+0000 7f93817fa700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f936c0a1560 0x7f936c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.274+0000 7f9383fff700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f936c0a1560 0x7f936c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.274+0000 7f93817fa700 1 -- 172.21.15.57:0/2660791597 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f93780ce520 con 0x7f9384100750 2023-10-08T13:50:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.274+0000 7f9383fff700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f936c0a1560 0x7f936c0a3a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9374009b50 tx=0x7f9374009550 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.277+0000 7f93817fa700 1 -- 172.21.15.57:0/2660791597 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f93780976c0 con 0x7f9384100750 2023-10-08T13:50:53.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:52 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2146365882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:50:53.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:52 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/74865137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:53.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:52 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/900561992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:53.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.478+0000 7f938a679700 1 -- 172.21.15.57:0/2660791597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9384066cd0 con 0x7f9384100750 2023-10-08T13:50:53.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.479+0000 7f93817fa700 1 -- 172.21.15.57:0/2660791597 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f937809ace0 con 0x7f9384100750 2023-10-08T13:50:53.480 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:50:53.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 -- 172.21.15.57:0/2660791597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f936c0a1560 msgr2=0x7f936c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f936c0a1560 0x7f936c0a3a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9374009b50 tx=0x7f9374009550 comp rx=0 tx=0).stop 2023-10-08T13:50:53.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 -- 172.21.15.57:0/2660791597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9384100750 msgr2=0x7f9384197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9384100750 0x7f9384197f30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f93780095c0 tx=0x7f937800f740 comp rx=0 tx=0).stop 2023-10-08T13:50:53.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 -- 172.21.15.57:0/2660791597 shutdown_connections 2023-10-08T13:50:53.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f936c0a1560 0x7f936c0a3a20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9384105630 0x7f93841a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93840ffd90 0x7f93841979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 --2- 172.21.15.57:0/2660791597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9384100750 0x7f9384197f30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 -- 172.21.15.57:0/2660791597 >> 172.21.15.57:0/2660791597 conn(0x7f9384075c20 msgr2=0x7f93840fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:53.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 -- 172.21.15.57:0/2660791597 shutdown_connections 2023-10-08T13:50:53.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.481+0000 7f938a679700 1 -- 172.21.15.57:0/2660791597 wait complete. 2023-10-08T13:50:53.495 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T13:50:53.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.624+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3232709256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68f40ff640 msgr2=0x7f68f40ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.624+0000 7f68fb2d0700 1 --2- 172.21.15.57:0/3232709256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68f40ff640 0x7f68f40ffac0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009990 tx=0x7f68e4009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.624+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3232709256 shutdown_connections 2023-10-08T13:50:53.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.624+0000 7f68fb2d0700 1 --2- 172.21.15.57:0/3232709256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4100000 0x7f68f410c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.624+0000 7f68fb2d0700 1 --2- 172.21.15.57:0/3232709256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68f4105630 0x7f68f4105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.624+0000 7f68fb2d0700 1 --2- 172.21.15.57:0/3232709256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68f40ff640 0x7f68f40ffac0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.624+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3232709256 >> 172.21.15.57:0/3232709256 conn(0x7f68f4076790 msgr2=0x7f68f4076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:53.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.624+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3232709256 shutdown_connections 2023-10-08T13:50:53.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.624+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3232709256 wait complete. 2023-10-08T13:50:53.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68fb2d0700 1 Processor -- start 2023-10-08T13:50:53.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68fb2d0700 1 -- start start 2023-10-08T13:50:53.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68fb2d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68f40ff640 0x7f68f4106590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68fb2d0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68f4100000 0x7f68f4106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68fb2d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4105630 0x7f68f4107010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68fb2d0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f41a4f20 con 0x7f68f40ff640 2023-10-08T13:50:53.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68fb2d0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f41a5090 con 0x7f68f4105630 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68fb2d0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f41a5250 con 0x7f68f4100000 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f906c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68f40ff640 0x7f68f4106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f906c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68f40ff640 0x7f68f4106590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55150/0 (socket says 172.21.15.57:55150) 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f906c700 1 -- 172.21.15.57:0/3458166916 learned_addr learned my addr 172.21.15.57:0/3458166916 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f886b700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68f4100000 0x7f68f4106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f986d700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4105630 0x7f68f4107010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f906c700 1 -- 172.21.15.57:0/3458166916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68f4100000 msgr2=0x7f68f4106ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f906c700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68f4100000 0x7f68f4106ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f906c700 1 -- 172.21.15.57:0/3458166916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4105630 msgr2=0x7f68f4107010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f906c700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4105630 0x7f68f4107010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f906c700 1 -- 172.21.15.57:0/3458166916 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e40095f0 con 0x7f68f40ff640 2023-10-08T13:50:53.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.625+0000 7f68f906c700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68f40ff640 0x7f68f4106590 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f68f000e9a0 tx=0x7f68f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:53.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.626+0000 7f68ea7fc700 1 -- 172.21.15.57:0/3458166916 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0009790 con 0x7f68f40ff640 2023-10-08T13:50:53.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.626+0000 7f68ea7fc700 1 -- 172.21.15.57:0/3458166916 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68f0010430 con 0x7f68f40ff640 2023-10-08T13:50:53.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.626+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3458166916 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f41a5530 con 0x7f68f40ff640 2023-10-08T13:50:53.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.626+0000 7f68ea7fc700 1 -- 172.21.15.57:0/3458166916 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0018680 con 0x7f68f40ff640 2023-10-08T13:50:53.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.626+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3458166916 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f41a5a60 con 0x7f68f40ff640 2023-10-08T13:50:53.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.627+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3458166916 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f4100d60 con 0x7f68f40ff640 2023-10-08T13:50:53.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.628+0000 7f68ea7fc700 1 -- 172.21.15.57:0/3458166916 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f68f00105a0 con 0x7f68f40ff640 2023-10-08T13:50:53.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.629+0000 7f68ea7fc700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68e00a15e0 0x7f68e00a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.629+0000 7f68ea7fc700 1 -- 172.21.15.57:0/3458166916 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f68f0014070 con 0x7f68f40ff640 2023-10-08T13:50:53.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.629+0000 7f68f886b700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68e00a15e0 0x7f68e00a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.630+0000 7f68f886b700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68e00a15e0 0x7f68e00a3aa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f68e400b2e0 tx=0x7f68e4005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:53.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.633+0000 7f68ea7fc700 1 -- 172.21.15.57:0/3458166916 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f68f0096730 con 0x7f68f40ff640 2023-10-08T13:50:53.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.836+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3458166916 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68f4107ac0 con 0x7f68f40ff640 2023-10-08T13:50:53.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.836+0000 7f68ea7fc700 1 -- 172.21.15.57:0/3458166916 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68f00260a0 con 0x7f68f40ff640 2023-10-08T13:50:53.838 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T13:50:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3458166916 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68e00a15e0 msgr2=0x7f68e00a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68e00a15e0 0x7f68e00a3aa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f68e400b2e0 tx=0x7f68e4005960 comp rx=0 tx=0).stop 2023-10-08T13:50:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3458166916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68f40ff640 msgr2=0x7f68f4106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68f40ff640 0x7f68f4106590 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f68f000e9a0 tx=0x7f68f000ecb0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3458166916 shutdown_connections 2023-10-08T13:50:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68e00a15e0 0x7f68e00a3aa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4105630 0x7f68f4107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68f40ff640 0x7f68f4106590 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 --2- 172.21.15.57:0/3458166916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68f4100000 0x7f68f4106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3458166916 >> 172.21.15.57:0/3458166916 conn(0x7f68f4076790 msgr2=0x7f68f40fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:53.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3458166916 shutdown_connections 2023-10-08T13:50:53.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.839+0000 7f68fb2d0700 1 -- 172.21.15.57:0/3458166916 wait complete. 2023-10-08T13:50:53.852 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T13:50:53.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.981+0000 7f089345c700 1 -- 172.21.15.57:0/3758291384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f088c100740 msgr2=0x7f088c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.981+0000 7f089345c700 1 --2- 172.21.15.57:0/3758291384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f088c100740 0x7f088c100bc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0888009990 tx=0x7f0888009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.981+0000 7f089345c700 1 -- 172.21.15.57:0/3758291384 shutdown_connections 2023-10-08T13:50:53.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.981+0000 7f089345c700 1 --2- 172.21.15.57:0/3758291384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f088c101100 0x7f088c10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.981+0000 7f089345c700 1 --2- 172.21.15.57:0/3758291384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f088c106750 0x7f088c106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.981+0000 7f089345c700 1 --2- 172.21.15.57:0/3758291384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f088c100740 0x7f088c100bc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.981+0000 7f089345c700 1 -- 172.21.15.57:0/3758291384 >> 172.21.15.57:0/3758291384 conn(0x7f088c0fc230 msgr2=0x7f088c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:53.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.981+0000 7f089345c700 1 -- 172.21.15.57:0/3758291384 shutdown_connections 2023-10-08T13:50:53.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.981+0000 7f089345c700 1 -- 172.21.15.57:0/3758291384 wait complete. 2023-10-08T13:50:53.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f089345c700 1 Processor -- start 2023-10-08T13:50:53.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f089345c700 1 -- start start 2023-10-08T13:50:53.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f089345c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f088c100740 0x7f088c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f089345c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f088c101100 0x7f088c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f089345c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f088c106750 0x7f088c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f089345c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c1a4f60 con 0x7f088c106750 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f089345c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c1a50d0 con 0x7f088c101100 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f089345c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c1a5290 con 0x7f088c100740 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f08919f9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f088c106750 0x7f088c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f08911f8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f088c100740 0x7f088c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f08919f9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f088c106750 0x7f088c073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55168/0 (socket says 172.21.15.57:55168) 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f08919f9700 1 -- 172.21.15.57:0/140219447 learned_addr learned my addr 172.21.15.57:0/140219447 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f08909f7700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f088c101100 0x7f088c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f08919f9700 1 -- 172.21.15.57:0/140219447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f088c100740 msgr2=0x7f088c072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f08919f9700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f088c100740 0x7f088c072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f08919f9700 1 -- 172.21.15.57:0/140219447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f088c101100 msgr2=0x7f088c073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:53.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f08919f9700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f088c101100 0x7f088c073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.982+0000 7f08919f9700 1 -- 172.21.15.57:0/140219447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880095f0 con 0x7f088c106750 2023-10-08T13:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.983+0000 7f08911f8700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f088c100740 0x7f088c072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.983+0000 7f08919f9700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f088c106750 0x7f088c073570 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0884007ee0 tx=0x7f088400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.983+0000 7f087e7fc700 1 -- 172.21.15.57:0/140219447 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884017070 con 0x7f088c106750 2023-10-08T13:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.983+0000 7f087e7fc700 1 -- 172.21.15.57:0/140219447 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f088400f040 con 0x7f088c106750 2023-10-08T13:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.983+0000 7f087e7fc700 1 -- 172.21.15.57:0/140219447 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884005580 con 0x7f088c106750 2023-10-08T13:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.983+0000 7f089345c700 1 -- 172.21.15.57:0/140219447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c1a5570 con 0x7f088c106750 2023-10-08T13:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.983+0000 7f089345c700 1 -- 172.21.15.57:0/140219447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c1a5a30 con 0x7f088c106750 2023-10-08T13:50:53.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.984+0000 7f089345c700 1 -- 172.21.15.57:0/140219447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c108d60 con 0x7f088c106750 2023-10-08T13:50:53.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.985+0000 7f087e7fc700 1 -- 172.21.15.57:0/140219447 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f088400db20 con 0x7f088c106750 2023-10-08T13:50:53.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.985+0000 7f087e7fc700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08780a15e0 0x7f08780a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:53.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.986+0000 7f087e7fc700 1 -- 172.21.15.57:0/140219447 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f08840cc990 con 0x7f088c106750 2023-10-08T13:50:53.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.986+0000 7f08911f8700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08780a15e0 0x7f08780a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:53.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.986+0000 7f08911f8700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08780a15e0 0x7f08780a3aa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0880005950 tx=0x7f088000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:53.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:53.990+0000 7f087e7fc700 1 -- 172.21.15.57:0/140219447 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0884095aa0 con 0x7f088c106750 2023-10-08T13:50:54.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.192+0000 7f089345c700 1 -- 172.21.15.57:0/140219447 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f088c073f80 con 0x7f088c106750 2023-10-08T13:50:54.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.193+0000 7f087e7fc700 1 -- 172.21.15.57:0/140219447 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f08840990c0 con 0x7f088c106750 2023-10-08T13:50:54.194 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:50:54.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 -- 172.21.15.57:0/140219447 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08780a15e0 msgr2=0x7f08780a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08780a15e0 0x7f08780a3aa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0880005950 tx=0x7f088000b410 comp rx=0 tx=0).stop 2023-10-08T13:50:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 -- 172.21.15.57:0/140219447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f088c106750 msgr2=0x7f088c073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f088c106750 0x7f088c073570 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0884007ee0 tx=0x7f088400d3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 -- 172.21.15.57:0/140219447 shutdown_connections 2023-10-08T13:50:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f088c106750 0x7f088c073570 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f088c100740 0x7f088c072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08780a15e0 0x7f08780a3aa0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 --2- 172.21.15.57:0/140219447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f088c101100 0x7f088c073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 -- 172.21.15.57:0/140219447 >> 172.21.15.57:0/140219447 conn(0x7f088c0fc230 msgr2=0x7f088c0fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 -- 172.21.15.57:0/140219447 shutdown_connections 2023-10-08T13:50:54.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.195+0000 7f089345c700 1 -- 172.21.15.57:0/140219447 wait complete. 2023-10-08T13:50:54.208 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:53 smithi057 ceph-mon[99476]: pgmap v277: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-08T13:50:54.208 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:53 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4017428626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:50:54.208 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:53 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2660791597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:50:54.208 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:53 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3458166916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:50:54.209 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:50:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:53 smithi099 ceph-mon[103849]: pgmap v277: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-08T13:50:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:53 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4017428626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:50:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:53 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2660791597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:50:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:53 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3458166916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:50:54.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.334+0000 7f6decef8700 1 -- 172.21.15.57:0/4057251682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de80fff10 msgr2=0x7f6de810c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.334+0000 7f6decef8700 1 --2- 172.21.15.57:0/4057251682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de80fff10 0x7f6de810c3d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b0d0 tx=0x7f6de000b3e0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.335+0000 7f6decef8700 1 -- 172.21.15.57:0/4057251682 shutdown_connections 2023-10-08T13:50:54.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.335+0000 7f6decef8700 1 --2- 172.21.15.57:0/4057251682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de80fff10 0x7f6de810c3d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.335+0000 7f6decef8700 1 --2- 172.21.15.57:0/4057251682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de8105610 0x7f6de81059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.335+0000 7f6decef8700 1 --2- 172.21.15.57:0/4057251682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de80ff550 0x7f6de80ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.335+0000 7f6decef8700 1 -- 172.21.15.57:0/4057251682 >> 172.21.15.57:0/4057251682 conn(0x7f6de80fb060 msgr2=0x7f6de80fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:54.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.335+0000 7f6decef8700 1 -- 172.21.15.57:0/4057251682 shutdown_connections 2023-10-08T13:50:54.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.335+0000 7f6decef8700 1 -- 172.21.15.57:0/4057251682 wait complete. 2023-10-08T13:50:54.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.335+0000 7f6decef8700 1 Processor -- start 2023-10-08T13:50:54.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.335+0000 7f6decef8700 1 -- start start 2023-10-08T13:50:54.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6decef8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de80ff550 0x7f6de8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:54.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6decef8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de80fff10 0x7f6de8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:54.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6decef8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de8105610 0x7f6de81a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:54.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6decef8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de8198740 con 0x7f6de80ff550 2023-10-08T13:50:54.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6decef8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de81a2f50 con 0x7f6de80fff10 2023-10-08T13:50:54.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6decef8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de81a3090 con 0x7f6de8105610 2023-10-08T13:50:54.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de80ff550 0x7f6de8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:54.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de80ff550 0x7f6de8197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55178/0 (socket says 172.21.15.57:55178) 2023-10-08T13:50:54.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de659c700 1 -- 172.21.15.57:0/723029030 learned_addr learned my addr 172.21.15.57:0/723029030 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:54.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de6d9d700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de8105610 0x7f6de81a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de5d9b700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de80fff10 0x7f6de8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de659c700 1 -- 172.21.15.57:0/723029030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de8105610 msgr2=0x7f6de81a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de659c700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de8105610 0x7f6de81a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de659c700 1 -- 172.21.15.57:0/723029030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de80fff10 msgr2=0x7f6de8197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de659c700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de80fff10 0x7f6de8197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de659c700 1 -- 172.21.15.57:0/723029030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0009d00 con 0x7f6de80ff550 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6de659c700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de80ff550 0x7f6de8197990 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000d950 tx=0x7f6dd000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6ddf7fe700 1 -- 172.21.15.57:0/723029030 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd000f890 con 0x7f6de80ff550 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.336+0000 7f6decef8700 1 -- 172.21.15.57:0/723029030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de81a3320 con 0x7f6de80ff550 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.337+0000 7f6ddf7fe700 1 -- 172.21.15.57:0/723029030 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dd0010460 con 0x7f6de80ff550 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.337+0000 7f6decef8700 1 -- 172.21.15.57:0/723029030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de81a3880 con 0x7f6de80ff550 2023-10-08T13:50:54.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.337+0000 7f6ddf7fe700 1 -- 172.21.15.57:0/723029030 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd001b650 con 0x7f6de80ff550 2023-10-08T13:50:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.337+0000 7f6decef8700 1 -- 172.21.15.57:0/723029030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de8107ab0 con 0x7f6de80ff550 2023-10-08T13:50:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.339+0000 7f6ddf7fe700 1 -- 172.21.15.57:0/723029030 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f6dd00108d0 con 0x7f6de80ff550 2023-10-08T13:50:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.339+0000 7f6ddf7fe700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6dd40a1560 0x7f6dd40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:54.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.339+0000 7f6de5d9b700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6dd40a1560 0x7f6dd40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:54.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.339+0000 7f6ddf7fe700 1 -- 172.21.15.57:0/723029030 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f6dd00cd4d0 con 0x7f6de80ff550 2023-10-08T13:50:54.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.340+0000 7f6de5d9b700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6dd40a1560 0x7f6dd40a3a20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6de8069280 tx=0x7f6dd800a300 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:54.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.343+0000 7f6ddf7fe700 1 -- 172.21.15.57:0/723029030 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6dd00965e0 con 0x7f6de80ff550 2023-10-08T13:50:54.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:53 smithi115 ceph-mon[104408]: pgmap v277: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-08T13:50:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:53 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4017428626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:50:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:53 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2660791597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:50:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:53 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3458166916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:50:54.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.590+0000 7f6decef8700 1 -- 172.21.15.57:0/723029030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6de8100c70 con 0x7f6de80ff550 2023-10-08T13:50:54.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.591+0000 7f6ddf7fe700 1 -- 172.21.15.57:0/723029030 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f6dd0020090 con 0x7f6de80ff550 2023-10-08T13:50:54.592 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:54.592 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:50:07.719573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:50:54.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 -- 172.21.15.57:0/723029030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6dd40a1560 msgr2=0x7f6dd40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6dd40a1560 0x7f6dd40a3a20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6de8069280 tx=0x7f6dd800a300 comp rx=0 tx=0).stop 2023-10-08T13:50:54.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 -- 172.21.15.57:0/723029030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de80ff550 msgr2=0x7f6de8197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de80ff550 0x7f6de8197990 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000d950 tx=0x7f6dd000dc60 comp rx=0 tx=0).stop 2023-10-08T13:50:54.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 -- 172.21.15.57:0/723029030 shutdown_connections 2023-10-08T13:50:54.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6de8105610 0x7f6de81a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6de80ff550 0x7f6de8197990 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6dd40a1560 0x7f6dd40a3a20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 --2- 172.21.15.57:0/723029030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6de80fff10 0x7f6de8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 -- 172.21.15.57:0/723029030 >> 172.21.15.57:0/723029030 conn(0x7f6de80fb060 msgr2=0x7f6de80fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:54.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 -- 172.21.15.57:0/723029030 shutdown_connections 2023-10-08T13:50:54.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.593+0000 7f6decef8700 1 -- 172.21.15.57:0/723029030 wait complete. 2023-10-08T13:50:54.595 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:50:54.607 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:50:54.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.731+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1617123144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9c81002b0 msgr2=0x7fd9c8100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.731+0000 7fd9d01d3700 1 --2- 172.21.15.57:0/1617123144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9c81002b0 0x7fd9c8100690 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009990 tx=0x7fd9c4009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.731+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1617123144 shutdown_connections 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.731+0000 7fd9d01d3700 1 --2- 172.21.15.57:0/1617123144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c81048e0 0x7fd9c810c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.731+0000 7fd9d01d3700 1 --2- 172.21.15.57:0/1617123144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9c81002b0 0x7fd9c8100690 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.731+0000 7fd9d01d3700 1 --2- 172.21.15.57:0/1617123144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c8100c60 0x7fd9c8104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.731+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1617123144 >> 172.21.15.57:0/1617123144 conn(0x7fd9c8074820 msgr2=0x7fd9c8074c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.731+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1617123144 shutdown_connections 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.731+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1617123144 wait complete. 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9d01d3700 1 Processor -- start 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9d01d3700 1 -- start start 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9d01d3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9c81002b0 0x7fd9c819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9d01d3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c81048e0 0x7fd9c819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9d01d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8198b00 0x7fd9c8198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9d01d3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8199580 con 0x7fd9c81002b0 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9d01d3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c81996f0 con 0x7fd9c81048e0 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9d01d3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c81998b0 con 0x7fd9c8198b00 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9cdf6f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9c81002b0 0x7fd9c819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9cdf6f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9c81002b0 0x7fd9c819e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55192/0 (socket says 172.21.15.57:55192) 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9cdf6f700 1 -- 172.21.15.57:0/1260474019 learned_addr learned my addr 172.21.15.57:0/1260474019 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.732+0000 7fd9cd76e700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c81048e0 0x7fd9c819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9ce770700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8198b00 0x7fd9c8198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9cdf6f700 1 -- 172.21.15.57:0/1260474019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8198b00 msgr2=0x7fd9c8198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9cdf6f700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8198b00 0x7fd9c8198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9cdf6f700 1 -- 172.21.15.57:0/1260474019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c81048e0 msgr2=0x7fd9c819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9cdf6f700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c81048e0 0x7fd9c819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9cdf6f700 1 -- 172.21.15.57:0/1260474019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c40095f0 con 0x7fd9c81002b0 2023-10-08T13:50:54.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9cdf6f700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9c81002b0 0x7fd9c819e9d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fd0 tx=0x7fd9c400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:54.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9baffd700 1 -- 172.21.15.57:0/1260474019 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c401d070 con 0x7fd9c81002b0 2023-10-08T13:50:54.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1260474019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c81a5840 con 0x7fd9c81002b0 2023-10-08T13:50:54.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9baffd700 1 -- 172.21.15.57:0/1260474019 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9c4005510 con 0x7fd9c81002b0 2023-10-08T13:50:54.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9baffd700 1 -- 172.21.15.57:0/1260474019 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4004a90 con 0x7fd9c81002b0 2023-10-08T13:50:54.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.733+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1260474019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c81a5cc0 con 0x7fd9c81002b0 2023-10-08T13:50:54.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.734+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1260474019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c8068880 con 0x7fd9c81002b0 2023-10-08T13:50:54.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.735+0000 7fd9baffd700 1 -- 172.21.15.57:0/1260474019 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fd9c4004c30 con 0x7fd9c81002b0 2023-10-08T13:50:54.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.736+0000 7fd9baffd700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9b40a1560 0x7fd9b40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:54.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.736+0000 7fd9baffd700 1 -- 172.21.15.57:0/1260474019 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fd9c40ce410 con 0x7fd9c81002b0 2023-10-08T13:50:54.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.736+0000 7fd9cd76e700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9b40a1560 0x7fd9b40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:54.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.736+0000 7fd9cd76e700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9b40a1560 0x7fd9b40a3a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8199cc0 tx=0x7fd9bc00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:54.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.739+0000 7fd9baffd700 1 -- 172.21.15.57:0/1260474019 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd9c4097550 con 0x7fd9c81002b0 2023-10-08T13:50:54.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.946+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1260474019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9c80678a0 con 0x7fd9c81002b0 2023-10-08T13:50:54.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.947+0000 7fd9baffd700 1 -- 172.21.15.57:0/1260474019 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24753 (secure 0 0 0) 0x7fd9c409ab70 con 0x7fd9c81002b0 2023-10-08T13:50:54.949 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:54.949 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:50:15.770421+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:50:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.949+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1260474019 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9b40a1560 msgr2=0x7fd9b40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.949+0000 7fd9d01d3700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9b40a1560 0x7fd9b40a3a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8199cc0 tx=0x7fd9bc00b410 comp rx=0 tx=0).stop 2023-10-08T13:50:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.949+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1260474019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9c81002b0 msgr2=0x7fd9c819e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.949+0000 7fd9d01d3700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9c81002b0 0x7fd9c819e9d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fd0 tx=0x7fd9c400f670 comp rx=0 tx=0).stop 2023-10-08T13:50:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.949+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1260474019 shutdown_connections 2023-10-08T13:50:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.949+0000 7fd9d01d3700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9c8198b00 0x7fd9c8198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.949+0000 7fd9d01d3700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9c81002b0 0x7fd9c819e9d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.949+0000 7fd9d01d3700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9b40a1560 0x7fd9b40a3a20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.949+0000 7fd9d01d3700 1 --2- 172.21.15.57:0/1260474019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c81048e0 0x7fd9c819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:54.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.949+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1260474019 >> 172.21.15.57:0/1260474019 conn(0x7fd9c8074820 msgr2=0x7fd9c8105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:54.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.950+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1260474019 shutdown_connections 2023-10-08T13:50:54.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:54.950+0000 7fd9d01d3700 1 -- 172.21.15.57:0/1260474019 wait complete. 2023-10-08T13:50:54.964 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:50:55.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.088+0000 7f3988341700 1 -- 172.21.15.57:0/667173720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39800ff540 msgr2=0x7f39800ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.088+0000 7f3988341700 1 --2- 172.21.15.57:0/667173720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39800ff540 0x7f39800ff9c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f39740099c0 tx=0x7f3974009cd0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.089+0000 7f3988341700 1 -- 172.21.15.57:0/667173720 shutdown_connections 2023-10-08T13:50:55.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.089+0000 7f3988341700 1 --2- 172.21.15.57:0/667173720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39800fff00 0x7f398010c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.089+0000 7f3988341700 1 --2- 172.21.15.57:0/667173720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3980105600 0x7f39801059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.089+0000 7f3988341700 1 --2- 172.21.15.57:0/667173720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39800ff540 0x7f39800ff9c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.089+0000 7f3988341700 1 -- 172.21.15.57:0/667173720 >> 172.21.15.57:0/667173720 conn(0x7f39800fb030 msgr2=0x7f39800fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.089+0000 7f3988341700 1 -- 172.21.15.57:0/667173720 shutdown_connections 2023-10-08T13:50:55.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.089+0000 7f3988341700 1 -- 172.21.15.57:0/667173720 wait complete. 2023-10-08T13:50:55.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.089+0000 7f3988341700 1 Processor -- start 2023-10-08T13:50:55.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.089+0000 7f3988341700 1 -- start start 2023-10-08T13:50:55.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f3988341700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39800ff540 0x7f3980197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f3988341700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39800fff00 0x7f3980197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f3988341700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3980105600 0x7f39801a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f3988341700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3980198630 con 0x7f3980105600 2023-10-08T13:50:55.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f3988341700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39801a2e40 con 0x7f39800ff540 2023-10-08T13:50:55.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f3988341700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39801a2f80 con 0x7f39800fff00 2023-10-08T13:50:55.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39868de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3980105600 0x7f39801a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39868de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3980105600 0x7f39801a2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55210/0 (socket says 172.21.15.57:55210) 2023-10-08T13:50:55.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39868de700 1 -- 172.21.15.57:0/2038025232 learned_addr learned my addr 172.21.15.57:0/2038025232 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39858dc700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39800fff00 0x7f3980197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39860dd700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39800ff540 0x7f3980197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39868de700 1 -- 172.21.15.57:0/2038025232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39800fff00 msgr2=0x7f3980197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39868de700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39800fff00 0x7f3980197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39868de700 1 -- 172.21.15.57:0/2038025232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39800ff540 msgr2=0x7f3980197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39868de700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39800ff540 0x7f3980197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39868de700 1 -- 172.21.15.57:0/2038025232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39740095f0 con 0x7f3980105600 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39868de700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3980105600 0x7f39801a2900 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f397800ef30 tx=0x7f397800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39737fe700 1 -- 172.21.15.57:0/2038025232 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978018070 con 0x7f3980105600 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39737fe700 1 -- 172.21.15.57:0/2038025232 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f397800f040 con 0x7f3980105600 2023-10-08T13:50:55.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f3988341700 1 -- 172.21.15.57:0/2038025232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39801a31b0 con 0x7f3980105600 2023-10-08T13:50:55.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.090+0000 7f39737fe700 1 -- 172.21.15.57:0/2038025232 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397800cd00 con 0x7f3980105600 2023-10-08T13:50:55.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.091+0000 7f3988341700 1 -- 172.21.15.57:0/2038025232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39801a3690 con 0x7f3980105600 2023-10-08T13:50:55.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.091+0000 7f3988341700 1 -- 172.21.15.57:0/2038025232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3980107aa0 con 0x7f3980105600 2023-10-08T13:50:55.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.092+0000 7f39737fe700 1 -- 172.21.15.57:0/2038025232 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f3978003a00 con 0x7f3980105600 2023-10-08T13:50:55.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.093+0000 7f39737fe700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f396c0a15e0 0x7f396c0a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.093+0000 7f39737fe700 1 -- 172.21.15.57:0/2038025232 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f39780cd780 con 0x7f3980105600 2023-10-08T13:50:55.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.093+0000 7f39860dd700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f396c0a15e0 0x7f396c0a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.094+0000 7f39860dd700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f396c0a15e0 0x7f396c0a3aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f397c005950 tx=0x7f397c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.097+0000 7f39737fe700 1 -- 172.21.15.57:0/2038025232 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3978096920 con 0x7f3980105600 2023-10-08T13:50:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:54 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/140219447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:50:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:54 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/723029030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:54 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1260474019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:55.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:54 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/140219447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:50:55.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:54 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/723029030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:55.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:54 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1260474019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:55.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.342+0000 7f3988341700 1 -- 172.21.15.57:0/2038025232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3980100c60 con 0x7f3980105600 2023-10-08T13:50:55.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.343+0000 7f39737fe700 1 -- 172.21.15.57:0/2038025232 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f3978099f40 con 0x7f3980105600 2023-10-08T13:50:55.344 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:50:55.344 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:50:07.719573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:50:55.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 -- 172.21.15.57:0/2038025232 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f396c0a15e0 msgr2=0x7f396c0a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f396c0a15e0 0x7f396c0a3aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f397c005950 tx=0x7f397c00b410 comp rx=0 tx=0).stop 2023-10-08T13:50:55.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 -- 172.21.15.57:0/2038025232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3980105600 msgr2=0x7f39801a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3980105600 0x7f39801a2900 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f397800ef30 tx=0x7f397800c3a0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 -- 172.21.15.57:0/2038025232 shutdown_connections 2023-10-08T13:50:55.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3980105600 0x7f39801a2900 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f396c0a15e0 0x7f396c0a3aa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39800ff540 0x7f3980197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 --2- 172.21.15.57:0/2038025232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39800fff00 0x7f3980197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 -- 172.21.15.57:0/2038025232 >> 172.21.15.57:0/2038025232 conn(0x7f39800fb030 msgr2=0x7f39800fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 -- 172.21.15.57:0/2038025232 shutdown_connections 2023-10-08T13:50:55.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.345+0000 7f3988341700 1 -- 172.21.15.57:0/2038025232 wait complete. 2023-10-08T13:50:55.348 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:50:55.359 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-08T13:49:44.186896+0000', 'modified': '2023-10-08T13:50:07.719573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24662, 'mds_1': 24652, 'mds_2': 24634, 'mds_3': 24622, 'mds_4': 24617}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14790': {'gid': 14790, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.57:6841/1847373212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6840', 'nonce': 1847373212}, {'type': 'v1', 'addr': '172.21.15.57:6841', 'nonce': 1847373212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24557': {'gid': 24557, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6833/2949059139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 2949059139}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 2949059139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24572': {'gid': 24572, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.57:6837/4079715365', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6836', 'nonce': 4079715365}, {'type': 'v1', 'addr': '172.21.15.57:6837', 'nonce': 4079715365}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24581': {'gid': 24581, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/2732351731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 2732351731}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 2732351731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24589': {'gid': 24589, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6833/3794414355', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 3794414355}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 3794414355}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24617': {'gid': 24617, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.99:6837/822922696', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 822922696}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 822922696}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24622': {'gid': 24622, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.115:6835/389121345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 389121345}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 389121345}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24634': {'gid': 24634, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.115:6837/383062444', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 383062444}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 383062444}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24652': {'gid': 24652, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.115:6839/2892511272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 2892511272}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 2892511272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24662': {'gid': 24662, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6839/687823376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 687823376}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 687823376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T13:50:55.359 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-08T13:50:55.359 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-08T13:50:55.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:54 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/140219447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:50:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:54 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/723029030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:54 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1260474019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:50:55.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.485+0000 7f1aa16bc700 1 -- 172.21.15.57:0/1730719764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0fffe0 msgr2=0x7f1a9c10c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.485+0000 7f1aa16bc700 1 --2- 172.21.15.57:0/1730719764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0fffe0 0x7f1a9c10c4e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400b0a0 tx=0x7f1a9400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.485+0000 7f1aa16bc700 1 -- 172.21.15.57:0/1730719764 shutdown_connections 2023-10-08T13:50:55.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.485+0000 7f1aa16bc700 1 --2- 172.21.15.57:0/1730719764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0fffe0 0x7f1a9c10c4e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.485+0000 7f1aa16bc700 1 --2- 172.21.15.57:0/1730719764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a9c1056b0 0x7f1a9c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.485+0000 7f1aa16bc700 1 --2- 172.21.15.57:0/1730719764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c0ff620 0x7f1a9c0ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.485+0000 7f1aa16bc700 1 -- 172.21.15.57:0/1730719764 >> 172.21.15.57:0/1730719764 conn(0x7f1a9c076780 msgr2=0x7f1a9c076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.485+0000 7f1aa16bc700 1 -- 172.21.15.57:0/1730719764 shutdown_connections 2023-10-08T13:50:55.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.485+0000 7f1aa16bc700 1 -- 172.21.15.57:0/1730719764 wait complete. 2023-10-08T13:50:55.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1aa16bc700 1 Processor -- start 2023-10-08T13:50:55.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1aa16bc700 1 -- start start 2023-10-08T13:50:55.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1aa16bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0ff620 0x7f1a9c197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1aa16bc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a9c0fffe0 0x7f1a9c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1aa16bc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c1056b0 0x7f1a9c1a2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1aa16bc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c198680 con 0x7f1a9c0ff620 2023-10-08T13:50:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1aa16bc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c1a2ed0 con 0x7f1a9c0fffe0 2023-10-08T13:50:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1aa16bc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c1a3010 con 0x7f1a9c1056b0 2023-10-08T13:50:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1a9affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0ff620 0x7f1a9c197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1a9affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0ff620 0x7f1a9c197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55220/0 (socket says 172.21.15.57:55220) 2023-10-08T13:50:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1a9affd700 1 -- 172.21.15.57:0/3521263101 learned_addr learned my addr 172.21.15.57:0/3521263101 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1a927fc700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a9c0fffe0 0x7f1a9c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.486+0000 7f1a9b7fe700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c1056b0 0x7f1a9c1a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1a9affd700 1 -- 172.21.15.57:0/3521263101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c1056b0 msgr2=0x7f1a9c1a2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1a9affd700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c1056b0 0x7f1a9c1a2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1a9affd700 1 -- 172.21.15.57:0/3521263101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a9c0fffe0 msgr2=0x7f1a9c197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1a9affd700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a9c0fffe0 0x7f1a9c197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1a9affd700 1 -- 172.21.15.57:0/3521263101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94009d00 con 0x7f1a9c0ff620 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1a927fc700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a9c0fffe0 0x7f1a9c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1a9affd700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0ff620 0x7f1a9c197910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00d980 tx=0x7f1a8c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1a98ff9700 1 -- 172.21.15.57:0/3521263101 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00de60 con 0x7f1a9c0ff620 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1a98ff9700 1 -- 172.21.15.57:0/3521263101 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a8c0044e0 con 0x7f1a9c0ff620 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1a98ff9700 1 -- 172.21.15.57:0/3521263101 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c003be0 con 0x7f1a9c0ff620 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1aa16bc700 1 -- 172.21.15.57:0/3521263101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c1a3270 con 0x7f1a9c0ff620 2023-10-08T13:50:55.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.487+0000 7f1aa16bc700 1 -- 172.21.15.57:0/3521263101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c1a3750 con 0x7f1a9c0ff620 2023-10-08T13:50:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.489+0000 7f1a98ff9700 1 -- 172.21.15.57:0/3521263101 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1a8c009bb0 con 0x7f1a9c0ff620 2023-10-08T13:50:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.490+0000 7f1a98ff9700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1a880a15e0 0x7f1a880a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.490+0000 7f1a98ff9700 1 -- 172.21.15.57:0/3521263101 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f1a8c0cd080 con 0x7f1a9c0ff620 2023-10-08T13:50:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.490+0000 7f1a927fc700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1a880a15e0 0x7f1a880a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.490+0000 7f1a927fc700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1a880a15e0 0x7f1a880a3aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c067ff0 tx=0x7f1a84005f50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.490+0000 7f1aad8a5b80 1 Processor -- start 2023-10-08T13:50:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1aad8a5b80 1 -- start start 2023-10-08T13:50:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073eb9e30 0x55f073eba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073ec5600 0x55f073ec79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073e23410 con 0x55f073eba750 2023-10-08T13:50:55.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073e14bd0 con 0x55f073ec5600 2023-10-08T13:50:55.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073e124f0 con 0x55f073eb9e30 2023-10-08T13:50:55.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1a927fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1a927fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55230/0 (socket says 172.21.15.57:55230) 2023-10-08T13:50:55.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1a927fc700 1 -- 172.21.15.57:0/268470337 learned_addr learned my addr 172.21.15.57:0/268470337 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1a9affd700 1 --2- 172.21.15.57:0/268470337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073eb9e30 0x55f073eba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1a9b7fe700 1 --2- 172.21.15.57:0/268470337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073ec5600 0x55f073ec79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1a927fc700 1 -- 172.21.15.57:0/268470337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073eb9e30 msgr2=0x55f073eba210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1a927fc700 1 --2- 172.21.15.57:0/268470337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073eb9e30 0x55f073eba210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1a927fc700 1 -- 172.21.15.57:0/268470337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073ec5600 msgr2=0x55f073ec79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1a927fc700 1 --2- 172.21.15.57:0/268470337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073ec5600 0x55f073ec79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.491+0000 7f1a927fc700 1 -- 172.21.15.57:0/268470337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f073c38d90 con 0x55f073eba750 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.492+0000 7f1a927fc700 1 --2- 172.21.15.57:0/268470337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec50c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400ed00 tx=0x7f1a84011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed7c50a14372c14 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.492+0000 7f1a917fa700 1 -- 172.21.15.57:0/268470337 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84010ca0 con 0x55f073eba750 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.492+0000 7f1a917fa700 1 -- 172.21.15.57:0/268470337 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a84010e00 con 0x55f073eba750 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.492+0000 7f1a917fa700 1 -- 172.21.15.57:0/268470337 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8401d450 con 0x55f073eba750 2023-10-08T13:50:55.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.492+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/268470337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 msgr2=0x55f073ec50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.492+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/268470337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec50c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400ed00 tx=0x7f1a84011000 comp rx=0 tx=0).stop 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/268470337 shutdown_connections 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/268470337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073ec5600 0x55f073ec79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/268470337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073eb9e30 0x55f073eba210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/268470337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec50c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/268470337 >> 172.21.15.57:0/268470337 conn(0x55f073e2f210 msgr2=0x55f073eb6280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/268470337 shutdown_connections 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/268470337 wait complete. 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 -- start start 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 Processor -- start 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eb9e30 0x55f073ec5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073ec2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073e0dde0 con 0x55f073eba750 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073daa6c0 con 0x55f073eb9e30 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073daa360 con 0x55f073ec5600 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1aad8a5b80 10 client.0 _ll_register_callbacks cb 0x55f073c077b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1a927fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1a927fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55236/0 (socket says 172.21.15.57:55236) 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.493+0000 7f1a927fc700 1 -- 172.21.15.57:0/3721779925 learned_addr learned my addr 172.21.15.57:0/3721779925 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a9b7fe700 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073ec2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a9affd700 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eb9e30 0x55f073ec5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a927fc700 1 -- 172.21.15.57:0/3721779925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 msgr2=0x55f073ec2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a927fc700 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073ec2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a927fc700 1 -- 172.21.15.57:0/3721779925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eb9e30 msgr2=0x55f073ec5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a927fc700 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eb9e30 0x55f073ec5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a927fc700 1 -- 172.21.15.57:0/3721779925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8400e9e0 con 0x55f073eba750 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a927fc700 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec23b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400aaf0 tx=0x7f1a84005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8e6d47c31af1f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a827fc700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a827fc700 1 -- 172.21.15.57:0/3721779925 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84004b00 con 0x55f073eba750 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a827fc700 1 -- 172.21.15.57:0/3721779925 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f1a84004c60 con 0x55f073eba750 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/3721779925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f073dd5e60 con 0x55f073eba750 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1a827fc700 1 -- 172.21.15.57:0/3721779925 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a84004e20 con 0x55f073eba750 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1aad8a5b80 10 client.15372 fetch_fsmap learned FSMap version 27 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.494+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/3721779925 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f073c21c10 con 0x55f073eba750 2023-10-08T13:50:55.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1a827fc700 1 -- 172.21.15.57:0/3721779925 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f073dd5e60 con 0x55f073eba750 2023-10-08T13:50:55.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1a827fc700 1 -- 172.21.15.57:0/3721779925 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1a84025400 con 0x55f073eba750 2023-10-08T13:50:55.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1aad8a5b80 10 client.15372 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:50:55.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1aad8a5b80 10 client.15372 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-08T13:50:55.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1aad8a5b80 20 client.15372 populate_metadata read hostname 'smithi057' 2023-10-08T13:50:55.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55f073ed5110 0x55f073ed3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1aad8a5b80 4 client.15372 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:50:55.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/3721779925 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f073e06dc0 con 0x55f073ed5110 2023-10-08T13:50:55.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1a9affd700 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55f073ed5110 0x55f073ed3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1a9affd700 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55f073ed5110 0x55f073ed3c30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6db5878e9ad8d378 server_cookie=2d1d40af028ef129 in_seq=0 out_seq=0 2023-10-08T13:50:55.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.495+0000 7f1a827fc700 10 client.15372 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:50:55.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1a827fc700 1 -- 172.21.15.57:0/3721779925 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f073e06dc0 con 0x55f073ed5110 2023-10-08T13:50:55.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1a827fc700 10 client.15372 handle_command_reply: tid=0 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 client.15372 shutdown 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/3721779925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 msgr2=0x55f073ec23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec23b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400aaf0 tx=0x7f1a84005620 comp rx=0 tx=0).stop 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/3721779925 shutdown_connections 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55f073ed5110 0x55f073ed3c30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073ec2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1a827fc700 0 client.15372 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eb9e30 0x55f073ec5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/3721779925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eba750 0x55f073ec23b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/3721779925 >> 172.21.15.57:0/3721779925 conn(0x55f073e2f210 msgr2=0x55f073eb5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/3721779925 shutdown_connections 2023-10-08T13:50:55.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.496+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/3721779925 wait complete. 2023-10-08T13:50:55.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.497+0000 7f1aad8a5b80 20 client.15372 trim_cache size 0 max 16384 2023-10-08T13:50:55.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.514+0000 7f1aad8a5b80 1 Processor -- start 2023-10-08T13:50:55.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.514+0000 7f1aad8a5b80 1 -- start start 2023-10-08T13:50:55.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.514+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ed4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.514+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 0x55f073ec79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.514+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073ec4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073c8e8d0 con 0x55f073eb9e30 2023-10-08T13:50:55.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073db44d0 con 0x55f073eba750 2023-10-08T13:50:55.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073c9ade0 con 0x55f073ec5600 2023-10-08T13:50:55.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a9affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ed4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a9affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ed4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55242/0 (socket says 172.21.15.57:55242) 2023-10-08T13:50:55.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a9affd700 1 -- 172.21.15.57:0/2842815284 learned_addr learned my addr 172.21.15.57:0/2842815284 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a927fc700 1 --2- 172.21.15.57:0/2842815284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 0x55f073ec79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a9affd700 1 -- 172.21.15.57:0/2842815284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 msgr2=0x55f073ec4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a9affd700 1 --2- 172.21.15.57:0/2842815284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073ec4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a9affd700 1 -- 172.21.15.57:0/2842815284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 msgr2=0x55f073ec79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a9affd700 1 --2- 172.21.15.57:0/2842815284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 0x55f073ec79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a9affd700 1 -- 172.21.15.57:0/2842815284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f073c5f200 con 0x55f073eb9e30 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a9affd700 1 --2- 172.21.15.57:0/2842815284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ed4340 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0962f0 tx=0x7f1a8c0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaef8ef73fa48a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a91ffb700 1 -- 172.21.15.57:0/2842815284 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c09df40 con 0x55f073eb9e30 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a91ffb700 1 -- 172.21.15.57:0/2842815284 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a8c0a2180 con 0x55f073eb9e30 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.515+0000 7f1a91ffb700 1 -- 172.21.15.57:0/2842815284 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c099540 con 0x55f073eb9e30 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/2842815284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 msgr2=0x55f073ed4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/2842815284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ed4340 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0962f0 tx=0x7f1a8c0cc040 comp rx=0 tx=0).stop 2023-10-08T13:50:55.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/2842815284 shutdown_connections 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/2842815284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073ec4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/2842815284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ed4340 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/2842815284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 0x55f073ec79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/2842815284 >> 172.21.15.57:0/2842815284 conn(0x55f073e2f210 msgr2=0x55f073ed3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/2842815284 shutdown_connections 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/2842815284 wait complete. 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 -- start start 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.516+0000 7f1aad8a5b80 1 Processor -- start 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ebef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 0x55f073ebf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1aad8a5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073f26b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073f27050 con 0x55f073eb9e30 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073f271c0 con 0x55f073eba750 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1aad8a5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f073f27380 con 0x55f073ec5600 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1aad8a5b80 10 client.0 _ll_register_callbacks cb 0x7f1a8c095ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a9affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ebef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a9affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ebef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55248/0 (socket says 172.21.15.57:55248) 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a9affd700 1 -- 172.21.15.57:0/684075835 learned_addr learned my addr 172.21.15.57:0/684075835 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a927fc700 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 0x55f073ebf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a9b7fe700 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073f26b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a9affd700 1 -- 172.21.15.57:0/684075835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 msgr2=0x55f073f26b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a9affd700 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073f26b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a9affd700 1 -- 172.21.15.57:0/684075835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 msgr2=0x55f073ebf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a9affd700 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 0x55f073ebf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a9affd700 1 -- 172.21.15.57:0/684075835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c1a3750 con 0x55f073eb9e30 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a927fc700 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 0x55f073ebf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a9affd700 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ebef30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0a1c50 tx=0x7f1a8c0997c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de31b2d828e5f07d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a827fc700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a827fc700 1 -- 172.21.15.57:0/684075835 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0994e0 con 0x55f073eb9e30 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/684075835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a84010e00 con 0x55f073eb9e30 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.517+0000 7f1a827fc700 1 -- 172.21.15.57:0/684075835 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f1a8c09d040 con 0x55f073eb9e30 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1a827fc700 1 -- 172.21.15.57:0/684075835 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a8c09a050 con 0x55f073eb9e30 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1aad8a5b80 10 client.15378 fetch_fsmap learned FSMap version 27 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/684075835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f073c37260 con 0x55f073eb9e30 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1a827fc700 1 -- 172.21.15.57:0/684075835 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a84010e00 con 0x55f073eb9e30 2023-10-08T13:50:55.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1a827fc700 1 -- 172.21.15.57:0/684075835 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1a8c0a1cf0 con 0x55f073eb9e30 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1aad8a5b80 10 client.15378 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1aad8a5b80 10 client.15378 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1aad8a5b80 20 client.15378 populate_metadata read hostname 'smithi057' 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55f073f32490 0x55f073f2e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1aad8a5b80 4 client.15378 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/684075835 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f073e06ae0 con 0x55f073f32490 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.518+0000 7f1a927fc700 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55f073f32490 0x55f073f2e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1a927fc700 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55f073f32490 0x55f073f2e930 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=41cce60adef328a0 server_cookie=c528c26c5948d1f1 in_seq=0 out_seq=0 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1a827fc700 10 client.15378 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1a827fc700 1 -- 172.21.15.57:0/684075835 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f073e06ae0 con 0x55f073f32490 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1a827fc700 10 client.15378 handle_command_reply: tid=0 2023-10-08T13:50:55.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 client.15378 shutdown 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/684075835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 msgr2=0x55f073ebef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ebef30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0a1c50 tx=0x7f1a8c0997c0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/684075835 shutdown_connections 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55f073f32490 0x55f073f2e930 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f073ec5600 0x55f073f26b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1a827fc700 0 client.15378 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f073eb9e30 0x55f073ebef30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 --2- 172.21.15.57:0/684075835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f073eba750 0x55f073ebf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/684075835 >> 172.21.15.57:0/684075835 conn(0x55f073e2f210 msgr2=0x55f073eb5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/684075835 shutdown_connections 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.519+0000 7f1aad8a5b80 1 -- 172.21.15.57:0/684075835 wait complete. 2023-10-08T13:50:55.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aad8a5b80 20 client.15378 trim_cache size 0 max 16384 2023-10-08T13:50:55.528 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:50:55.528 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:50:55.528 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 4, 2023-10-08T13:50:55.528 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24617, 2023-10-08T13:50:55.528 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:50:55.528 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:50:55.528 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:50:55.528 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 48.206186299000002, 2023-10-08T13:50:55.528 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 27, 2023-10-08T13:50:55.529 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 90, 2023-10-08T13:50:55.529 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 88, 2023-10-08T13:50:55.529 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 85.632263929999993 2023-10-08T13:50:55.529 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:50:55.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 -- 172.21.15.57:0/3521263101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1a880a15e0 msgr2=0x7f1a880a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1a880a15e0 0x7f1a880a3aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c067ff0 tx=0x7f1a84005f50 comp rx=0 tx=0).stop 2023-10-08T13:50:55.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 -- 172.21.15.57:0/3521263101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0ff620 msgr2=0x7f1a9c197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0ff620 0x7f1a9c197910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00d980 tx=0x7f1a8c00dd40 comp rx=0 tx=0).stop 2023-10-08T13:50:55.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 -- 172.21.15.57:0/3521263101 shutdown_connections 2023-10-08T13:50:55.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a9c1056b0 0x7f1a9c1a2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a9c0ff620 0x7f1a9c197910 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1a880a15e0 0x7f1a880a3aa0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 --2- 172.21.15.57:0/3521263101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a9c0fffe0 0x7f1a9c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 -- 172.21.15.57:0/3521263101 >> 172.21.15.57:0/3521263101 conn(0x7f1a9c076780 msgr2=0x7f1a9c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 -- 172.21.15.57:0/3521263101 shutdown_connections 2023-10-08T13:50:55.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.520+0000 7f1aa16bc700 1 -- 172.21.15.57:0/3521263101 wait complete. 2023-10-08T13:50:55.533 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-08T13:50:55.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.660+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/939739992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2eb40ffd90 msgr2=0x7f2eb4100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.660+0000 7f2ebc2e5700 1 --2- 172.21.15.57:0/939739992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2eb40ffd90 0x7f2eb4100210 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009990 tx=0x7f2eb0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.660+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/939739992 shutdown_connections 2023-10-08T13:50:55.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.660+0000 7f2ebc2e5700 1 --2- 172.21.15.57:0/939739992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb4100750 0x7f2eb410c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.660+0000 7f2ebc2e5700 1 --2- 172.21.15.57:0/939739992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4105680 0x7f2eb4105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.660+0000 7f2ebc2e5700 1 --2- 172.21.15.57:0/939739992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2eb40ffd90 0x7f2eb4100210 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.660+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/939739992 >> 172.21.15.57:0/939739992 conn(0x7f2eb4075c20 msgr2=0x7f2eb40fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.660+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/939739992 shutdown_connections 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.660+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/939739992 wait complete. 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2ebc2e5700 1 Processor -- start 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2ebc2e5700 1 -- start start 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2ebc2e5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2eb40ffd90 0x7f2eb4101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2ebc2e5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb4100750 0x7f2eb4101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2ebc2e5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4105680 0x7f2eb4101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2ebc2e5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb41a4f40 con 0x7f2eb40ffd90 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2ebc2e5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb41a50b0 con 0x7f2eb4100750 2023-10-08T13:50:55.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2ebc2e5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb41a5270 con 0x7f2eb4105680 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2eba081700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2eb40ffd90 0x7f2eb4101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2eba081700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2eb40ffd90 0x7f2eb4101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55262/0 (socket says 172.21.15.57:55262) 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2eba081700 1 -- 172.21.15.57:0/3605866911 learned_addr learned my addr 172.21.15.57:0/3605866911 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2eb9880700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb4100750 0x7f2eb4101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2eba882700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4105680 0x7f2eb4101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.661+0000 7f2eba081700 1 -- 172.21.15.57:0/3605866911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4105680 msgr2=0x7f2eb4101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2eba081700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4105680 0x7f2eb4101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2eba081700 1 -- 172.21.15.57:0/3605866911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb4100750 msgr2=0x7f2eb4101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2eba081700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb4100750 0x7f2eb4101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2eba081700 1 -- 172.21.15.57:0/3605866911 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb00095f0 con 0x7f2eb40ffd90 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2eba882700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4105680 0x7f2eb4101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2eba081700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2eb40ffd90 0x7f2eb4101230 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400c9b0 tx=0x7f2ea400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2eab7fe700 1 -- 172.21.15.57:0/3605866911 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4007b00 con 0x7f2eb40ffd90 2023-10-08T13:50:55.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/3605866911 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb41a5550 con 0x7f2eb40ffd90 2023-10-08T13:50:55.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2eab7fe700 1 -- 172.21.15.57:0/3605866911 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ea4007c60 con 0x7f2eb40ffd90 2023-10-08T13:50:55.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/3605866911 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb41a5a30 con 0x7f2eb40ffd90 2023-10-08T13:50:55.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.662+0000 7f2eab7fe700 1 -- 172.21.15.57:0/3605866911 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4004a30 con 0x7f2eb40ffd90 2023-10-08T13:50:55.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.664+0000 7f2eab7fe700 1 -- 172.21.15.57:0/3605866911 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f2ea4004b90 con 0x7f2eb40ffd90 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.665+0000 7f2eab7fe700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ea00a1560 0x7f2ea00a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.665+0000 7f2eb9880700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ea00a1560 0x7f2ea00a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.665+0000 7f2eab7fe700 1 -- 172.21.15.57:0/3605866911 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f2ea40cd3a0 con 0x7f2eb40ffd90 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.665+0000 7f2eb9880700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ea00a1560 0x7f2ea00a3a20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000b5c0 tx=0x7f2eb0005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2ec84ceb80 1 Processor -- start 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2ec84ceb80 1 -- start start 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4842210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 0x561eb484d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb484d600 0x561eb484f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb47ab410 con 0x561eb4841e30 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb479cbd0 con 0x561eb484d600 2023-10-08T13:50:55.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb479a4f0 con 0x561eb4842750 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2eba081700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4842210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2eba081700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4842210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55270/0 (socket says 172.21.15.57:55270) 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2eba081700 1 -- 172.21.15.57:0/1864118518 learned_addr learned my addr 172.21.15.57:0/1864118518 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2eba882700 1 --2- 172.21.15.57:0/1864118518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb484d600 0x561eb484f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2eb9880700 1 --2- 172.21.15.57:0/1864118518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 0x561eb484d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2eba081700 1 -- 172.21.15.57:0/1864118518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 msgr2=0x561eb484d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2eba081700 1 --2- 172.21.15.57:0/1864118518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 0x561eb484d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.666+0000 7f2eba081700 1 -- 172.21.15.57:0/1864118518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb484d600 msgr2=0x561eb484f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.667+0000 7f2eba081700 1 --2- 172.21.15.57:0/1864118518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb484d600 0x561eb484f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.667+0000 7f2eba081700 1 -- 172.21.15.57:0/1864118518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eb45c0d90 con 0x561eb4841e30 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.667+0000 7f2eb9880700 1 --2- 172.21.15.57:0/1864118518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 0x561eb484d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:55.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.667+0000 7f2eba081700 1 --2- 172.21.15.57:0/1864118518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4842210 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4096610 tx=0x7f2ea4099af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8f24b7b72b231c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.667+0000 7f2ea8d82700 1 -- 172.21.15.57:0/1864118518 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea409a810 con 0x561eb4841e30 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.667+0000 7f2ea8d82700 1 -- 172.21.15.57:0/1864118518 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ea409a110 con 0x561eb4841e30 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.667+0000 7f2ea8d82700 1 -- 172.21.15.57:0/1864118518 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40cac70 con 0x561eb4841e30 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.667+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1864118518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 msgr2=0x561eb4842210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/1864118518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4842210 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4096610 tx=0x7f2ea4099af0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1864118518 shutdown_connections 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/1864118518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb484d600 0x561eb484f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/1864118518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4842210 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/1864118518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 0x561eb484d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1864118518 >> 172.21.15.57:0/1864118518 conn(0x561eb47b7210 msgr2=0x561eb483e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1864118518 shutdown_connections 2023-10-08T13:50:55.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1864118518 wait complete. 2023-10-08T13:50:55.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 -- start start 2023-10-08T13:50:55.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 Processor -- start 2023-10-08T13:50:55.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4841e30 0x561eb484cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4842750 0x561eb484a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb484d600 0x561eb484a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb4795de0 con 0x561eb4842750 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb47326c0 con 0x561eb4841e30 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb4732360 con 0x561eb484d600 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.668+0000 7f2ec84ceb80 10 client.0 _ll_register_callbacks cb 0x561eb458f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eb9880700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4842750 0x561eb484a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eb9880700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4842750 0x561eb484a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55278/0 (socket says 172.21.15.57:55278) 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eb9880700 1 -- 172.21.15.57:0/112138349 learned_addr learned my addr 172.21.15.57:0/112138349 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eba882700 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb484d600 0x561eb484a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eba081700 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4841e30 0x561eb484cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eb9880700 1 -- 172.21.15.57:0/112138349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb484d600 msgr2=0x561eb484a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eb9880700 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb484d600 0x561eb484a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eb9880700 1 -- 172.21.15.57:0/112138349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4841e30 msgr2=0x561eb484cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eb9880700 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4841e30 0x561eb484cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eb9880700 1 -- 172.21.15.57:0/112138349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb41a5a30 con 0x561eb4842750 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eba882700 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb484d600 0x561eb484a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2eb9880700 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4842750 0x561eb484a240 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009990 tx=0x7f2eb0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de58640f80c44f8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2e9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2e9dffb700 1 -- 172.21.15.57:0/112138349 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0005700 con 0x561eb4842750 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/112138349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561eb475de60 con 0x561eb4842750 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.669+0000 7f2e9dffb700 1 -- 172.21.15.57:0/112138349 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f2eb000f460 con 0x561eb4842750 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2e9dffb700 1 -- 172.21.15.57:0/112138349 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2eb0034630 con 0x561eb4842750 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2ec84ceb80 10 client.15390 fetch_fsmap learned FSMap version 27 2023-10-08T13:50:55.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/112138349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561eb45a9c10 con 0x561eb4842750 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2e9dffb700 1 -- 172.21.15.57:0/112138349 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561eb475de60 con 0x561eb4842750 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2e9dffb700 1 -- 172.21.15.57:0/112138349 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2eb000f6b0 con 0x561eb4842750 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2ec84ceb80 10 client.15390 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2ec84ceb80 10 client.15390 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2ec84ceb80 20 client.15390 populate_metadata read hostname 'smithi057' 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x561eb485cf50 0x561eb485ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2ec84ceb80 4 client.15390 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/112138349 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561eb478edc0 con 0x561eb485cf50 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.670+0000 7f2eba081700 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x561eb485cf50 0x561eb485ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.671+0000 7f2e9dffb700 10 client.15390 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.671+0000 7f2eba081700 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x561eb485cf50 0x561eb485ba20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd7596f570411996 server_cookie=b2de9d201ec38fea in_seq=0 out_seq=0 2023-10-08T13:50:55.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.671+0000 7f2e9dffb700 1 -- 172.21.15.57:0/112138349 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561eb478edc0 con 0x561eb485cf50 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.671+0000 7f2e9dffb700 10 client.15390 handle_command_reply: tid=0 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.671+0000 7f2ec84ceb80 1 client.15390 shutdown 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.671+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/112138349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4842750 msgr2=0x561eb484a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.671+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4842750 0x561eb484a240 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009990 tx=0x7f2eb0009310 comp rx=0 tx=0).stop 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.671+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/112138349 shutdown_connections 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.672+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x561eb485cf50 0x561eb485ba20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.672+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb484d600 0x561eb484a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.672+0000 7f2e9dffb700 0 client.15390 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.672+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4841e30 0x561eb484cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.672+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/112138349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4842750 0x561eb484a240 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.672+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/112138349 >> 172.21.15.57:0/112138349 conn(0x561eb47b7210 msgr2=0x561eb483de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.672+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/112138349 shutdown_connections 2023-10-08T13:50:55.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.672+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/112138349 wait complete. 2023-10-08T13:50:55.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.672+0000 7f2ec84ceb80 20 client.15390 trim_cache size 0 max 16384 2023-10-08T13:50:55.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.689+0000 7f2ec84ceb80 1 Processor -- start 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.689+0000 7f2ec84ceb80 1 -- start start 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.689+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4844d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.689+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 0x561eb48658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb484d600 0x561eb48455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb46168d0 con 0x561eb4841e30 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb4791ed0 con 0x561eb484d600 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb4622de0 con 0x561eb4842750 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eba081700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4844d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eba081700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4844d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55284/0 (socket says 172.21.15.57:55284) 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eba081700 1 -- 172.21.15.57:0/3753993771 learned_addr learned my addr 172.21.15.57:0/3753993771 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eb9880700 1 --2- 172.21.15.57:0/3753993771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 0x561eb48658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eba882700 1 --2- 172.21.15.57:0/3753993771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb484d600 0x561eb48455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eba081700 1 -- 172.21.15.57:0/3753993771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 msgr2=0x561eb48658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eba081700 1 --2- 172.21.15.57:0/3753993771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 0x561eb48658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eba081700 1 -- 172.21.15.57:0/3753993771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb484d600 msgr2=0x561eb48455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eba081700 1 --2- 172.21.15.57:0/3753993771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb484d600 0x561eb48455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eba081700 1 -- 172.21.15.57:0/3753993771 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eb45e7200 con 0x561eb4841e30 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eb9880700 1 --2- 172.21.15.57:0/3753993771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 0x561eb48658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2eba081700 1 --2- 172.21.15.57:0/3753993771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4844d00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2ea409e420 tx=0x7f2ea40964f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd0fdd5fc504d3b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2ea9583700 1 -- 172.21.15.57:0/3753993771 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4061ea0 con 0x561eb4841e30 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2ea9583700 1 -- 172.21.15.57:0/3753993771 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ea405fe90 con 0x561eb4841e30 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.690+0000 7f2ea9583700 1 -- 172.21.15.57:0/3753993771 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40cab50 con 0x561eb4841e30 2023-10-08T13:50:55.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/3753993771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 msgr2=0x561eb4844d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/3753993771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4844d00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2ea409e420 tx=0x7f2ea40964f0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/3753993771 shutdown_connections 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/3753993771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb484d600 0x561eb48455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/3753993771 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb4841e30 0x561eb4844d00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/3753993771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4842750 0x561eb48658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/3753993771 >> 172.21.15.57:0/3753993771 conn(0x561eb47b7210 msgr2=0x561eb48497e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/3753993771 shutdown_connections 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/3753993771 wait complete. 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 -- start start 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 Processor -- start 2023-10-08T13:50:55.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4841e30 0x561eb484ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4842750 0x561eb484f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb484d600 0x561eb4847520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb4865db0 con 0x561eb484d600 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb4847a60 con 0x561eb4842750 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.691+0000 7f2ec84ceb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eb4847c20 con 0x561eb4841e30 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2ec84ceb80 10 client.0 _ll_register_callbacks cb 0x7f2eb000bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba882700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb484d600 0x561eb4847520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba882700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb484d600 0x561eb4847520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55294/0 (socket says 172.21.15.57:55294) 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba882700 1 -- 172.21.15.57:0/1262262575 learned_addr learned my addr 172.21.15.57:0/1262262575 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba081700 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4841e30 0x561eb484ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eb9880700 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4842750 0x561eb484f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba882700 1 -- 172.21.15.57:0/1262262575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4841e30 msgr2=0x561eb484ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba882700 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4841e30 0x561eb484ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba882700 1 -- 172.21.15.57:0/1262262575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4842750 msgr2=0x561eb484f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba882700 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4842750 0x561eb484f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba882700 1 -- 172.21.15.57:0/1262262575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561eb45c0d90 con 0x561eb484d600 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eb9880700 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4842750 0x561eb484f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba081700 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4841e30 0x561eb484ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2eba882700 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb484d600 0x561eb4847520 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2eac007f00 tx=0x7f2eac016620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34ee155b21ea854 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2e9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2e9dffb700 1 -- 172.21.15.57:0/1262262575 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac021660 con 0x561eb484d600 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2e9dffb700 1 -- 172.21.15.57:0/1262262575 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f2eac016e80 con 0x561eb484d600 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1262262575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ea409a110 con 0x561eb484d600 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2e9dffb700 1 -- 172.21.15.57:0/1262262575 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2eac0217c0 con 0x561eb484d600 2023-10-08T13:50:55.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.692+0000 7f2ec84ceb80 10 client.15396 fetch_fsmap learned FSMap version 27 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1262262575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561eb45bf260 con 0x561eb484d600 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2e9dffb700 1 -- 172.21.15.57:0/1262262575 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ea409a110 con 0x561eb484d600 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2e9dffb700 1 -- 172.21.15.57:0/1262262575 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2eac014050 con 0x561eb484d600 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2ec84ceb80 10 client.15396 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2ec84ceb80 10 client.15396 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2ec84ceb80 20 client.15396 populate_metadata read hostname 'smithi057' 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x561eb48b53a0 0x561eb48b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2ec84ceb80 4 client.15396 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1262262575 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561eb478eae0 con 0x561eb48b53a0 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2eba081700 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x561eb48b53a0 0x561eb48b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2e9dffb700 10 client.15396 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:50:55.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.693+0000 7f2eba081700 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x561eb48b53a0 0x561eb48b2950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d4db56adc4dea4b server_cookie=c2b052e88169913 in_seq=0 out_seq=0 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2e9dffb700 1 -- 172.21.15.57:0/1262262575 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x561eb478eae0 con 0x561eb48b53a0 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2e9dffb700 10 client.15396 handle_command_reply: tid=0 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 client.15396 shutdown 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1262262575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb484d600 msgr2=0x561eb4847520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb484d600 0x561eb4847520 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2eac007f00 tx=0x7f2eac016620 comp rx=0 tx=0).stop 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1262262575 shutdown_connections 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x561eb48b53a0 0x561eb48b2950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561eb484d600 0x561eb4847520 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561eb4841e30 0x561eb484ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2e9dffb700 0 client.15396 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 --2- 172.21.15.57:0/1262262575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eb4842750 0x561eb484f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1262262575 >> 172.21.15.57:0/1262262575 conn(0x561eb47b7210 msgr2=0x561eb48455c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1262262575 shutdown_connections 2023-10-08T13:50:55.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 1 -- 172.21.15.57:0/1262262575 wait complete. 2023-10-08T13:50:55.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ec84ceb80 20 client.15396 trim_cache size 0 max 16384 2023-10-08T13:50:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:50:55.703 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:50:55.703 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 3, 2023-10-08T13:50:55.703 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24622, 2023-10-08T13:50:55.703 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:50:55.703 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:50:55.704 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:50:55.704 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 49.391110935, 2023-10-08T13:50:55.704 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 27, 2023-10-08T13:50:55.704 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 90, 2023-10-08T13:50:55.704 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 88, 2023-10-08T13:50:55.704 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 90.367659478999997 2023-10-08T13:50:55.704 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:50:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/3605866911 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ea00a1560 msgr2=0x7f2ea00a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ebc2e5700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ea00a1560 0x7f2ea00a3a20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000b5c0 tx=0x7f2eb0005960 comp rx=0 tx=0).stop 2023-10-08T13:50:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/3605866911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2eb40ffd90 msgr2=0x7f2eb4101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.694+0000 7f2ebc2e5700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2eb40ffd90 0x7f2eb4101230 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400c9b0 tx=0x7f2ea400ccc0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.695+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/3605866911 shutdown_connections 2023-10-08T13:50:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.695+0000 7f2ebc2e5700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eb4105680 0x7f2eb4101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.695+0000 7f2ebc2e5700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2eb40ffd90 0x7f2eb4101230 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.695+0000 7f2ebc2e5700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ea00a1560 0x7f2ea00a3a20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.695+0000 7f2ebc2e5700 1 --2- 172.21.15.57:0/3605866911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb4100750 0x7f2eb4101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.695+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/3605866911 >> 172.21.15.57:0/3605866911 conn(0x7f2eb4075c20 msgr2=0x7f2eb40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.695+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/3605866911 shutdown_connections 2023-10-08T13:50:55.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.695+0000 7f2ebc2e5700 1 -- 172.21.15.57:0/3605866911 wait complete. 2023-10-08T13:50:55.708 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T13:50:55.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.836+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/3699182711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff8067ce0 msgr2=0x7f2ff80680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.836+0000 7f2ffcb5b700 1 --2- 172.21.15.57:0/3699182711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff8067ce0 0x7f2ff80680c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00099c0 tx=0x7f2fe0009cd0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.836+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/3699182711 shutdown_connections 2023-10-08T13:50:55.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.836+0000 7f2ffcb5b700 1 --2- 172.21.15.57:0/3699182711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ff8105b80 0x7f2ff810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.836+0000 7f2ffcb5b700 1 --2- 172.21.15.57:0/3699182711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff8067ce0 0x7f2ff80680c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.836+0000 7f2ffcb5b700 1 --2- 172.21.15.57:0/3699182711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff81051c0 0x7f2ff8105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.836+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/3699182711 >> 172.21.15.57:0/3699182711 conn(0x7f2ff8076780 msgr2=0x7f2ff8076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.836+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/3699182711 shutdown_connections 2023-10-08T13:50:55.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.836+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/3699182711 wait complete. 2023-10-08T13:50:55.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.837+0000 7f2ffcb5b700 1 Processor -- start 2023-10-08T13:50:55.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.837+0000 7f2ffcb5b700 1 -- start start 2023-10-08T13:50:55.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.837+0000 7f2ffcb5b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ff8067ce0 0x7f2ff819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.837+0000 7f2ffcb5b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff81051c0 0x7f2ff819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.837+0000 7f2ffcb5b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff8105b80 0x7f2ff8196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.837+0000 7f2ffcb5b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff8197060 con 0x7f2ff8105b80 2023-10-08T13:50:55.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.837+0000 7f2ffcb5b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff81971d0 con 0x7f2ff81051c0 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.837+0000 7f2ffcb5b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff8197390 con 0x7f2ff8067ce0 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff8105b80 0x7f2ff8196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff8105b80 0x7f2ff8196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55308/0 (socket says 172.21.15.57:55308) 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff6d9d700 1 -- 172.21.15.57:0/244311336 learned_addr learned my addr 172.21.15.57:0/244311336 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff659c700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ff8067ce0 0x7f2ff819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff81051c0 0x7f2ff819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff6d9d700 1 -- 172.21.15.57:0/244311336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ff8067ce0 msgr2=0x7f2ff819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff6d9d700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ff8067ce0 0x7f2ff819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff6d9d700 1 -- 172.21.15.57:0/244311336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff81051c0 msgr2=0x7f2ff819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff6d9d700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff81051c0 0x7f2ff819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff6d9d700 1 -- 172.21.15.57:0/244311336 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00095f0 con 0x7f2ff8105b80 2023-10-08T13:50:55.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff659c700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ff8067ce0 0x7f2ff819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:55.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff81051c0 0x7f2ff819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:55.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ff6d9d700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff8105b80 0x7f2ff8196a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000ada0 tx=0x7f2ff000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2fef7fe700 1 -- 172.21.15.57:0/244311336 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff000f070 con 0x7f2ff8105b80 2023-10-08T13:50:55.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/244311336 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff8197670 con 0x7f2ff8105b80 2023-10-08T13:50:55.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2fef7fe700 1 -- 172.21.15.57:0/244311336 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ff0007b30 con 0x7f2ff8105b80 2023-10-08T13:50:55.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2fef7fe700 1 -- 172.21.15.57:0/244311336 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0005440 con 0x7f2ff8105b80 2023-10-08T13:50:55.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.838+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/244311336 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff81a38c0 con 0x7f2ff8105b80 2023-10-08T13:50:55.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.840+0000 7f2fef7fe700 1 -- 172.21.15.57:0/244311336 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f2ff0004ad0 con 0x7f2ff8105b80 2023-10-08T13:50:55.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.841+0000 7f2fef7fe700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2fe40a1560 0x7f2fe40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.841+0000 7f2ff659c700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2fe40a1560 0x7f2fe40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.841+0000 7f2fef7fe700 1 -- 172.21.15.57:0/244311336 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f2ff00cc910 con 0x7f2ff8105b80 2023-10-08T13:50:55.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.842+0000 7f2ff659c700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2fe40a1560 0x7f2fe40a3a20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009fd0 tx=0x7f2fe0005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.842+0000 7f3008d44b80 1 Processor -- start 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.842+0000 7f3008d44b80 1 -- start start 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.842+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bc98210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.842+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc98750 0x55806bca74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.842+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.842+0000 7f3008d44b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bc01410 con 0x55806bc97e30 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.842+0000 7f3008d44b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bbf2bd0 con 0x55806bca7a10 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.842+0000 7f3008d44b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bbf04f0 con 0x55806bc98750 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bc98210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bc98210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55312/0 (socket says 172.21.15.57:55312) 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff659c700 1 -- 172.21.15.57:0/2476137650 learned_addr learned my addr 172.21.15.57:0/2476137650 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/2476137650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc98750 0x55806bca74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff6d9d700 1 --2- 172.21.15.57:0/2476137650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff659c700 1 -- 172.21.15.57:0/2476137650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc98750 msgr2=0x55806bca74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff659c700 1 --2- 172.21.15.57:0/2476137650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc98750 0x55806bca74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff659c700 1 -- 172.21.15.57:0/2476137650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 msgr2=0x55806bca9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff659c700 1 --2- 172.21.15.57:0/2476137650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff659c700 1 -- 172.21.15.57:0/2476137650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806ba16d90 con 0x55806bc97e30 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2ff659c700 1 --2- 172.21.15.57:0/2476137650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bc98210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0015000 tx=0x7f2fe001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7bf129e86912 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2fecff9700 1 -- 172.21.15.57:0/2476137650 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe002b070 con 0x55806bc97e30 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2fecff9700 1 -- 172.21.15.57:0/2476137650 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fe001e040 con 0x55806bc97e30 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.843+0000 7f2fecff9700 1 -- 172.21.15.57:0/2476137650 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00039c0 con 0x55806bc97e30 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.844+0000 7f3008d44b80 1 -- 172.21.15.57:0/2476137650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 msgr2=0x55806bc98210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.844+0000 7f3008d44b80 1 --2- 172.21.15.57:0/2476137650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bc98210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0015000 tx=0x7f2fe001f000 comp rx=0 tx=0).stop 2023-10-08T13:50:55.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.844+0000 7f3008d44b80 1 -- 172.21.15.57:0/2476137650 shutdown_connections 2023-10-08T13:50:55.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.844+0000 7f3008d44b80 1 --2- 172.21.15.57:0/2476137650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.844+0000 7f3008d44b80 1 --2- 172.21.15.57:0/2476137650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bc98210 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.844+0000 7f3008d44b80 1 --2- 172.21.15.57:0/2476137650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc98750 0x55806bca74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.844+0000 7f3008d44b80 1 -- 172.21.15.57:0/2476137650 >> 172.21.15.57:0/2476137650 conn(0x55806bc0d210 msgr2=0x55806bc94280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.844+0000 7f3008d44b80 1 -- 172.21.15.57:0/2476137650 shutdown_connections 2023-10-08T13:50:55.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.844+0000 7f3008d44b80 1 -- 172.21.15.57:0/2476137650 wait complete. 2023-10-08T13:50:55.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.844+0000 7f3008d44b80 1 -- start start 2023-10-08T13:50:55.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f3008d44b80 1 Processor -- start 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc97e30 0x55806bca7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bca4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f3008d44b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bbebde0 con 0x55806bc98750 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f3008d44b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bb886c0 con 0x55806bca7a10 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f3008d44b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bb88360 con 0x55806bc97e30 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f3008d44b80 10 client.0 _ll_register_callbacks cb 0x55806b9e57b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bca4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bca4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55320/0 (socket says 172.21.15.57:55320) 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff5d9b700 1 -- 172.21.15.57:0/1155915645 learned_addr learned my addr 172.21.15.57:0/1155915645 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff659c700 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc97e30 0x55806bca7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff6d9d700 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff5d9b700 1 -- 172.21.15.57:0/1155915645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc97e30 msgr2=0x55806bca7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc97e30 0x55806bca7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff5d9b700 1 -- 172.21.15.57:0/1155915645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 msgr2=0x55806bca4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff5d9b700 1 -- 172.21.15.57:0/1155915645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff810dec0 con 0x55806bc98750 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bca4810 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800baa0 tx=0x7f2fe800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434ba56ba9851f33 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.845+0000 7f2fddffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f2fddffb700 1 -- 172.21.15.57:0/1155915645 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80041f0 con 0x55806bc98750 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f2fddffb700 1 -- 172.21.15.57:0/1155915645 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f2fe8011440 con 0x55806bc98750 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f3008d44b80 1 -- 172.21.15.57:0/1155915645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55806bbb3e60 con 0x55806bc98750 2023-10-08T13:50:55.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f2fddffb700 1 -- 172.21.15.57:0/1155915645 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fe8011600 con 0x55806bc98750 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f3008d44b80 10 client.15408 fetch_fsmap learned FSMap version 27 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f3008d44b80 1 -- 172.21.15.57:0/1155915645 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55806b9ffc10 con 0x55806bc98750 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f2fddffb700 1 -- 172.21.15.57:0/1155915645 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55806bbb3e60 con 0x55806bc98750 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f2fddffb700 1 -- 172.21.15.57:0/1155915645 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2fe801b880 con 0x55806bc98750 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f3008d44b80 10 client.15408 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f3008d44b80 10 client.15408 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f3008d44b80 20 client.15408 populate_metadata read hostname 'smithi057' 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f3008d44b80 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55806bcb75c0 0x55806bcb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f3008d44b80 4 client.15408 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f3008d44b80 1 -- 172.21.15.57:0/1155915645 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55806bbe4dc0 con 0x55806bcb75c0 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.846+0000 7f2ff659c700 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55806bcb75c0 0x55806bcb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.847+0000 7f2ff659c700 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55806bcb75c0 0x55806bcb6090 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1be5eba6b80cf18b server_cookie=a48a14e2389c654b in_seq=0 out_seq=0 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.847+0000 7f2fddffb700 10 client.15408 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.847+0000 7f2fddffb700 1 -- 172.21.15.57:0/1155915645 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55806bbe4dc0 con 0x55806bcb75c0 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.847+0000 7f2fddffb700 10 client.15408 handle_command_reply: tid=0 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.847+0000 7f3008d44b80 1 client.15408 shutdown 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 1 -- 172.21.15.57:0/1155915645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 msgr2=0x55806bca4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bca4810 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800baa0 tx=0x7f2fe800bdb0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 1 -- 172.21.15.57:0/1155915645 shutdown_connections 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55806bcb75c0 0x55806bcb6090 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f2fddffb700 0 client.15408 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:50:55.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc97e30 0x55806bca7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 1 --2- 172.21.15.57:0/1155915645 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bca4810 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 1 -- 172.21.15.57:0/1155915645 >> 172.21.15.57:0/1155915645 conn(0x55806bc0d210 msgr2=0x55806bc93e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 1 -- 172.21.15.57:0/1155915645 shutdown_connections 2023-10-08T13:50:55.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 1 -- 172.21.15.57:0/1155915645 wait complete. 2023-10-08T13:50:55.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.848+0000 7f3008d44b80 20 client.15408 trim_cache size 0 max 16384 2023-10-08T13:50:55.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.865+0000 7f3008d44b80 1 Processor -- start 2023-10-08T13:50:55.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.865+0000 7f3008d44b80 1 -- start start 2023-10-08T13:50:55.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc97e30 0x55806bc94430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bcbff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f3008d44b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806ba6c8d0 con 0x55806bc98750 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f3008d44b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bbe7ed0 con 0x55806bca7a10 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f3008d44b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bca5d70 con 0x55806bc97e30 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bcbff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bcbff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55328/0 (socket says 172.21.15.57:55328) 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff5d9b700 1 -- 172.21.15.57:0/1176038205 learned_addr learned my addr 172.21.15.57:0/1176038205 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff6d9d700 1 --2- 172.21.15.57:0/1176038205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff5d9b700 1 -- 172.21.15.57:0/1176038205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc97e30 msgr2=0x55806bc94430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/1176038205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc97e30 0x55806bc94430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff5d9b700 1 -- 172.21.15.57:0/1176038205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 msgr2=0x55806bca6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/1176038205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff5d9b700 1 -- 172.21.15.57:0/1176038205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806ba3d200 con 0x55806bc98750 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff6d9d700 1 --2- 172.21.15.57:0/1176038205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/1176038205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bcbff30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8005a40 tx=0x7f2fe8005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a745a3487cb2053a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2fed7fa700 1 -- 172.21.15.57:0/1176038205 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8003f80 con 0x55806bc98750 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2fed7fa700 1 -- 172.21.15.57:0/1176038205 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fe8003960 con 0x55806bc98750 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.866+0000 7f2fed7fa700 1 -- 172.21.15.57:0/1176038205 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800fbe0 con 0x55806bc98750 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 -- 172.21.15.57:0/1176038205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 msgr2=0x55806bcbff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 --2- 172.21.15.57:0/1176038205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bcbff30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8005a40 tx=0x7f2fe8005720 comp rx=0 tx=0).stop 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 -- 172.21.15.57:0/1176038205 shutdown_connections 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 --2- 172.21.15.57:0/1176038205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bca7a10 0x55806bca6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 --2- 172.21.15.57:0/1176038205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bc97e30 0x55806bc94430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 --2- 172.21.15.57:0/1176038205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc98750 0x55806bcbff30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 -- 172.21.15.57:0/1176038205 >> 172.21.15.57:0/1176038205 conn(0x55806bc0d210 msgr2=0x55806bc93d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 -- 172.21.15.57:0/1176038205 shutdown_connections 2023-10-08T13:50:55.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 -- 172.21.15.57:0/1176038205 wait complete. 2023-10-08T13:50:55.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 -- start start 2023-10-08T13:50:55.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.867+0000 7f3008d44b80 1 Processor -- start 2023-10-08T13:50:55.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bcba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bc98750 0x55806bcbaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f3008d44b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bca7a10 0x55806bcbb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f3008d44b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bcc04d0 con 0x55806bc97e30 2023-10-08T13:50:55.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f3008d44b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bd0afb0 con 0x55806bc98750 2023-10-08T13:50:55.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f3008d44b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806bd0b140 con 0x55806bca7a10 2023-10-08T13:50:55.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f3008d44b80 10 client.0 _ll_register_callbacks cb 0x7f2fe80043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bcba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bcba590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55332/0 (socket says 172.21.15.57:55332) 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff659c700 1 -- 172.21.15.57:0/529656055 learned_addr learned my addr 172.21.15.57:0/529656055 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bc98750 0x55806bcbaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff6d9d700 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bca7a10 0x55806bcbb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff659c700 1 -- 172.21.15.57:0/529656055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bca7a10 msgr2=0x55806bcbb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff659c700 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bca7a10 0x55806bcbb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff659c700 1 -- 172.21.15.57:0/529656055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bc98750 msgr2=0x55806bcbaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff659c700 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bc98750 0x55806bcbaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff659c700 1 -- 172.21.15.57:0/529656055 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe8020040 con 0x55806bc97e30 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bc98750 0x55806bcbaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:55.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2ff659c700 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bcba590 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000fd60 tx=0x7f2fe0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45587d76540578e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2fddffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2fddffb700 1 -- 172.21.15.57:0/529656055 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe002b040 con 0x55806bc97e30 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f2fddffb700 1 -- 172.21.15.57:0/529656055 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f2fe003ac00 con 0x55806bc97e30 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.868+0000 7f3008d44b80 1 -- 172.21.15.57:0/529656055 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fe001e040 con 0x55806bc97e30 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f2fddffb700 1 -- 172.21.15.57:0/529656055 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fe003adc0 con 0x55806bc97e30 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f3008d44b80 10 client.15414 fetch_fsmap learned FSMap version 27 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f3008d44b80 1 -- 172.21.15.57:0/529656055 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55806ba15260 con 0x55806bc97e30 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f2fddffb700 1 -- 172.21.15.57:0/529656055 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fe001e040 con 0x55806bc97e30 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f2fddffb700 1 -- 172.21.15.57:0/529656055 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2fe0005030 con 0x55806bc97e30 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f3008d44b80 10 client.15414 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f3008d44b80 10 client.15414 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f3008d44b80 20 client.15414 populate_metadata read hostname 'smithi057' 2023-10-08T13:50:55.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f3008d44b80 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55806bd0ff50 0x55806bd0c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f3008d44b80 4 client.15414 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f3008d44b80 1 -- 172.21.15.57:0/529656055 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55806bbe4ae0 con 0x55806bd0ff50 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.869+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55806bd0ff50 0x55806bd0c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f2fddffb700 10 client.15414 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f2ff5d9b700 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55806bd0ff50 0x55806bd0c400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bdd293bda07bbed3 server_cookie=abd3ea389c1f5b46 in_seq=0 out_seq=0 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f2fddffb700 1 -- 172.21.15.57:0/529656055 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55806bbe4ae0 con 0x55806bd0ff50 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f2fddffb700 10 client.15414 handle_command_reply: tid=0 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 client.15414 shutdown 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 -- 172.21.15.57:0/529656055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 msgr2=0x55806bcba590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bcba590 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000fd60 tx=0x7f2fe0009310 comp rx=0 tx=0).stop 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 -- 172.21.15.57:0/529656055 shutdown_connections 2023-10-08T13:50:55.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55806bd0ff50 0x55806bd0c400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55806bca7a10 0x55806bcbb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f2fddffb700 0 client.15414 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:50:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55806bc97e30 0x55806bcba590 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 --2- 172.21.15.57:0/529656055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55806bc98750 0x55806bcbaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 -- 172.21.15.57:0/529656055 >> 172.21.15.57:0/529656055 conn(0x55806bc0d210 msgr2=0x55806bca40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 -- 172.21.15.57:0/529656055 shutdown_connections 2023-10-08T13:50:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 1 -- 172.21.15.57:0/529656055 wait complete. 2023-10-08T13:50:55.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.870+0000 7f3008d44b80 20 client.15414 trim_cache size 0 max 16384 2023-10-08T13:50:55.878 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 2, 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24634, 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 50.583698022999997, 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 27, 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 90, 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 88, 2023-10-08T13:50:55.879 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 88.396381758999993 2023-10-08T13:50:55.880 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:50:55.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/244311336 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2fe40a1560 msgr2=0x7f2fe40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2fe40a1560 0x7f2fe40a3a20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009fd0 tx=0x7f2fe0005fd0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/244311336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff8105b80 msgr2=0x7f2ff8196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:55.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff8105b80 0x7f2ff8196a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000ada0 tx=0x7f2ff000c330 comp rx=0 tx=0).stop 2023-10-08T13:50:55.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/244311336 shutdown_connections 2023-10-08T13:50:55.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ff8105b80 0x7f2ff8196a00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ff8067ce0 0x7f2ff819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2fe40a1560 0x7f2fe40a3a20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 --2- 172.21.15.57:0/244311336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff81051c0 0x7f2ff819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:55.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/244311336 >> 172.21.15.57:0/244311336 conn(0x7f2ff8076780 msgr2=0x7f2ff80fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:55.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/244311336 shutdown_connections 2023-10-08T13:50:55.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:55.871+0000 7f2ffcb5b700 1 -- 172.21.15.57:0/244311336 wait complete. 2023-10-08T13:50:55.884 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-08T13:50:55.973 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:55 smithi057 ceph-mon[99476]: pgmap v278: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:50:55.973 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:55 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2038025232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.010+0000 7f223951e700 1 -- 172.21.15.57:0/3749340171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22341010e0 msgr2=0x7f223410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.010+0000 7f223951e700 1 --2- 172.21.15.57:0/3749340171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22341010e0 0x7f223410d5f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b0a0 tx=0x7f222c00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.011+0000 7f223951e700 1 -- 172.21.15.57:0/3749340171 shutdown_connections 2023-10-08T13:50:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.011+0000 7f223951e700 1 --2- 172.21.15.57:0/3749340171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22341010e0 0x7f223410d5f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.011+0000 7f223951e700 1 --2- 172.21.15.57:0/3749340171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2234106730 0x7f2234106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.011+0000 7f223951e700 1 --2- 172.21.15.57:0/3749340171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2234100720 0x7f2234100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.011+0000 7f223951e700 1 -- 172.21.15.57:0/3749340171 >> 172.21.15.57:0/3749340171 conn(0x7f22340fc230 msgr2=0x7f22340fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.011+0000 7f223951e700 1 -- 172.21.15.57:0/3749340171 shutdown_connections 2023-10-08T13:50:56.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.011+0000 7f223951e700 1 -- 172.21.15.57:0/3749340171 wait complete. 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.011+0000 7f223951e700 1 Processor -- start 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f223951e700 1 -- start start 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f223951e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2234100720 0x7f223419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f223951e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22341010e0 0x7f223419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f223951e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2234106730 0x7f22341969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f223951e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22341971c0 con 0x7f2234100720 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f223951e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2234197330 con 0x7f2234106730 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f223951e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22341974f0 con 0x7f22341010e0 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f2232ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2234100720 0x7f223419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f2232ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2234100720 0x7f223419c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55344/0 (socket says 172.21.15.57:55344) 2023-10-08T13:50:56.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f2232ffd700 1 -- 172.21.15.57:0/4042409042 learned_addr learned my addr 172.21.15.57:0/4042409042 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f22327fc700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22341010e0 0x7f223419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f22337fe700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2234106730 0x7f22341969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f2232ffd700 1 -- 172.21.15.57:0/4042409042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22341010e0 msgr2=0x7f223419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f2232ffd700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22341010e0 0x7f223419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f2232ffd700 1 -- 172.21.15.57:0/4042409042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2234106730 msgr2=0x7f22341969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.012+0000 7f2232ffd700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2234106730 0x7f22341969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.013+0000 7f2232ffd700 1 -- 172.21.15.57:0/4042409042 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c009d00 con 0x7f2234100720 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.013+0000 7f22337fe700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2234106730 0x7f22341969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.013+0000 7f2232ffd700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2234100720 0x7f223419c950 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f222400d950 tx=0x7f222400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.013+0000 7f221bfff700 1 -- 172.21.15.57:0/4042409042 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224009940 con 0x7f2234100720 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.013+0000 7f221bfff700 1 -- 172.21.15.57:0/4042409042 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2224010460 con 0x7f2234100720 2023-10-08T13:50:56.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.013+0000 7f223951e700 1 -- 172.21.15.57:0/4042409042 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22341977d0 con 0x7f2234100720 2023-10-08T13:50:56.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.013+0000 7f221bfff700 1 -- 172.21.15.57:0/4042409042 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222400f620 con 0x7f2234100720 2023-10-08T13:50:56.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.013+0000 7f223951e700 1 -- 172.21.15.57:0/4042409042 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22341a38d0 con 0x7f2234100720 2023-10-08T13:50:56.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.015+0000 7f221bfff700 1 -- 172.21.15.57:0/4042409042 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f22240108d0 con 0x7f2234100720 2023-10-08T13:50:56.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.016+0000 7f221bfff700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f221c0a1560 0x7f221c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.016+0000 7f22327fc700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f221c0a1560 0x7f221c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.016+0000 7f221bfff700 1 -- 172.21.15.57:0/4042409042 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f22240cd140 con 0x7f2234100720 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.016+0000 7f22327fc700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f221c0a1560 0x7f221c0a3a20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2234198160 tx=0x7f2228009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.016+0000 7f2245707b80 1 Processor -- start 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.016+0000 7f2245707b80 1 -- start start 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2245707b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6244e30 0x5580a6245210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2245707b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 0x5580a62500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2245707b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6250600 0x5580a62529f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2245707b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a61ae410 con 0x5580a6244e30 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2245707b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a619fbd0 con 0x5580a6245750 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2245707b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a619d4f0 con 0x5580a6250600 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2232ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6244e30 0x5580a6245210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2232ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6244e30 0x5580a6245210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55352/0 (socket says 172.21.15.57:55352) 2023-10-08T13:50:56.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2232ffd700 1 -- 172.21.15.57:0/3697164127 learned_addr learned my addr 172.21.15.57:0/3697164127 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f22337fe700 1 --2- 172.21.15.57:0/3697164127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6250600 0x5580a62529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f22327fc700 1 --2- 172.21.15.57:0/3697164127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 0x5580a62500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2232ffd700 1 -- 172.21.15.57:0/3697164127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6250600 msgr2=0x5580a62529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2232ffd700 1 --2- 172.21.15.57:0/3697164127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6250600 0x5580a62529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2232ffd700 1 -- 172.21.15.57:0/3697164127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 msgr2=0x5580a62500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2232ffd700 1 --2- 172.21.15.57:0/3697164127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 0x5580a62500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2232ffd700 1 -- 172.21.15.57:0/3697164127 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a5fc3d90 con 0x5580a6244e30 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f22327fc700 1 --2- 172.21.15.57:0/3697164127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 0x5580a62500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f2232ffd700 1 --2- 172.21.15.57:0/3697164127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6244e30 0x5580a6245210 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f22240a1d10 tx=0x7f2224099910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb7a0fc1a7940c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f22197fa700 1 -- 172.21.15.57:0/3697164127 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22240a2240 con 0x5580a6244e30 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f22197fa700 1 -- 172.21.15.57:0/3697164127 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22240ca8f0 con 0x5580a6244e30 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.017+0000 7f22197fa700 1 -- 172.21.15.57:0/3697164127 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22240995e0 con 0x5580a6244e30 2023-10-08T13:50:56.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.018+0000 7f2245707b80 1 -- 172.21.15.57:0/3697164127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6244e30 msgr2=0x5580a6245210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.018+0000 7f2245707b80 1 --2- 172.21.15.57:0/3697164127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6244e30 0x5580a6245210 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f22240a1d10 tx=0x7f2224099910 comp rx=0 tx=0).stop 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.018+0000 7f2245707b80 1 -- 172.21.15.57:0/3697164127 shutdown_connections 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.018+0000 7f2245707b80 1 --2- 172.21.15.57:0/3697164127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6250600 0x5580a62529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.018+0000 7f2245707b80 1 --2- 172.21.15.57:0/3697164127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6244e30 0x5580a6245210 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.018+0000 7f2245707b80 1 --2- 172.21.15.57:0/3697164127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 0x5580a62500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.018+0000 7f2245707b80 1 -- 172.21.15.57:0/3697164127 >> 172.21.15.57:0/3697164127 conn(0x5580a61ba210 msgr2=0x5580a6241280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.018+0000 7f2245707b80 1 -- 172.21.15.57:0/3697164127 shutdown_connections 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.018+0000 7f2245707b80 1 -- 172.21.15.57:0/3697164127 wait complete. 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f2245707b80 1 -- start start 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f2245707b80 1 Processor -- start 2023-10-08T13:50:56.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f2245707b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6244e30 0x5580a624ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f2245707b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6245750 0x5580a624d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f2245707b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6250600 0x5580a624d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f2245707b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a6198de0 con 0x5580a6245750 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f2245707b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a61356c0 con 0x5580a6250600 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f2245707b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a6135360 con 0x5580a6244e30 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f2245707b80 10 client.0 _ll_register_callbacks cb 0x5580a5f927b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f22327fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6245750 0x5580a624d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f22327fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6245750 0x5580a624d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55360/0 (socket says 172.21.15.57:55360) 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f22327fc700 1 -- 172.21.15.57:0/2936324101 learned_addr learned my addr 172.21.15.57:0/2936324101 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f2232ffd700 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6244e30 0x5580a624ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f22337fe700 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6250600 0x5580a624d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f22327fc700 1 -- 172.21.15.57:0/2936324101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6244e30 msgr2=0x5580a624ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f22327fc700 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6244e30 0x5580a624ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f22327fc700 1 -- 172.21.15.57:0/2936324101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6250600 msgr2=0x5580a624d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f22327fc700 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6250600 0x5580a624d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f22327fc700 1 -- 172.21.15.57:0/2936324101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22341a38d0 con 0x5580a6245750 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.019+0000 7f22337fe700 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6250600 0x5580a624d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f22327fc700 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6245750 0x5580a624d210 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f222801d7e0 tx=0x7f22280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b25f0a2232e3f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2216ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2216ff5700 1 -- 172.21.15.57:0/2936324101 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228004220 con 0x5580a6245750 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2216ff5700 1 -- 172.21.15.57:0/2936324101 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f2228004380 con 0x5580a6245750 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2245707b80 1 -- 172.21.15.57:0/2936324101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580a6160e60 con 0x5580a6245750 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2216ff5700 1 -- 172.21.15.57:0/2936324101 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2228004540 con 0x5580a6245750 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2245707b80 10 client.15426 fetch_fsmap learned FSMap version 27 2023-10-08T13:50:56.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2245707b80 1 -- 172.21.15.57:0/2936324101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5580a5facc10 con 0x5580a6245750 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2216ff5700 1 -- 172.21.15.57:0/2936324101 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580a6160e60 con 0x5580a6245750 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2216ff5700 1 -- 172.21.15.57:0/2936324101 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f222800eac0 con 0x5580a6245750 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2245707b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2245707b80 10 client.15426 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2245707b80 20 client.15426 populate_metadata read hostname 'smithi057' 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2245707b80 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5580a6260150 0x5580a625ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2245707b80 4 client.15426 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.020+0000 7f2245707b80 1 -- 172.21.15.57:0/2936324101 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580a6191dc0 con 0x5580a6260150 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.021+0000 7f2232ffd700 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5580a6260150 0x5580a625ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.021+0000 7f2232ffd700 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5580a6260150 0x5580a625ebd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea36dade45dea08d server_cookie=c2ed54d2cb0594c8 in_seq=0 out_seq=0 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.021+0000 7f2216ff5700 10 client.15426 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:50:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2216ff5700 1 -- 172.21.15.57:0/2936324101 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5580a6191dc0 con 0x5580a6260150 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2216ff5700 10 client.15426 handle_command_reply: tid=0 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 client.15426 shutdown 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 -- 172.21.15.57:0/2936324101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6245750 msgr2=0x5580a624d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6245750 0x5580a624d210 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f222801d7e0 tx=0x7f22280047c0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 -- 172.21.15.57:0/2936324101 shutdown_connections 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5580a6260150 0x5580a625ebd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6250600 0x5580a624d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2216ff5700 0 client.15426 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6244e30 0x5580a624ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 --2- 172.21.15.57:0/2936324101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6245750 0x5580a624d210 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 -- 172.21.15.57:0/2936324101 >> 172.21.15.57:0/2936324101 conn(0x5580a61ba210 msgr2=0x5580a6240e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 -- 172.21.15.57:0/2936324101 shutdown_connections 2023-10-08T13:50:56.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 1 -- 172.21.15.57:0/2936324101 wait complete. 2023-10-08T13:50:56.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.022+0000 7f2245707b80 20 client.15426 trim_cache size 0 max 16384 2023-10-08T13:50:56.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f2245707b80 1 Processor -- start 2023-10-08T13:50:56.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f2245707b80 1 -- start start 2023-10-08T13:50:56.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f2245707b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6244e30 0x5580a6241430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f2245707b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6245750 0x5580a6247d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f2245707b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a624fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f2245707b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a60198d0 con 0x5580a6250600 2023-10-08T13:50:56.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f2245707b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a619a070 con 0x5580a6244e30 2023-10-08T13:50:56.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f2245707b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a613f4d0 con 0x5580a6245750 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f22337fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a624fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f22337fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a624fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55370/0 (socket says 172.21.15.57:55370) 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f22337fe700 1 -- 172.21.15.57:0/3384652726 learned_addr learned my addr 172.21.15.57:0/3384652726 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f22327fc700 1 --2- 172.21.15.57:0/3384652726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6245750 0x5580a6247d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f2232ffd700 1 --2- 172.21.15.57:0/3384652726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6244e30 0x5580a6241430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f22337fe700 1 -- 172.21.15.57:0/3384652726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6245750 msgr2=0x5580a6247d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f22337fe700 1 --2- 172.21.15.57:0/3384652726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6245750 0x5580a6247d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f22337fe700 1 -- 172.21.15.57:0/3384652726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6244e30 msgr2=0x5580a6241430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f22337fe700 1 --2- 172.21.15.57:0/3384652726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6244e30 0x5580a6241430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.040+0000 7f22337fe700 1 -- 172.21.15.57:0/3384652726 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580a5fea200 con 0x5580a6250600 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f22337fe700 1 --2- 172.21.15.57:0/3384652726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a624fac0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f222c015000 tx=0x7f222c0094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de32c3cc34e0ee2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2219ffb700 1 -- 172.21.15.57:0/3384652726 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00e040 con 0x5580a6250600 2023-10-08T13:50:56.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2219ffb700 1 -- 172.21.15.57:0/3384652726 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f222c004a50 con 0x5580a6250600 2023-10-08T13:50:56.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2219ffb700 1 -- 172.21.15.57:0/3384652726 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c005040 con 0x5580a6250600 2023-10-08T13:50:56.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2245707b80 1 -- 172.21.15.57:0/3384652726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 msgr2=0x5580a624fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2245707b80 1 --2- 172.21.15.57:0/3384652726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a624fac0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f222c015000 tx=0x7f222c0094d0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2245707b80 1 -- 172.21.15.57:0/3384652726 shutdown_connections 2023-10-08T13:50:56.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2245707b80 1 --2- 172.21.15.57:0/3384652726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a624fac0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2245707b80 1 --2- 172.21.15.57:0/3384652726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6244e30 0x5580a6241430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2245707b80 1 --2- 172.21.15.57:0/3384652726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6245750 0x5580a6247d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2245707b80 1 -- 172.21.15.57:0/3384652726 >> 172.21.15.57:0/3384652726 conn(0x5580a61ba210 msgr2=0x5580a6240d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2245707b80 1 -- 172.21.15.57:0/3384652726 shutdown_connections 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.041+0000 7f2245707b80 1 -- 172.21.15.57:0/3384652726 wait complete. 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f2245707b80 1 -- start start 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f2245707b80 1 Processor -- start 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f2245707b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6244e30 0x5580a624a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f2245707b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 0x5580a624a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f2245707b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a62b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f2245707b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a624af50 con 0x5580a6250600 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f2245707b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a62b40b0 con 0x5580a6245750 2023-10-08T13:50:56.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f2245707b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580a62b4270 con 0x5580a6244e30 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f2245707b80 10 client.0 _ll_register_callbacks cb 0x7f2228011f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f22337fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a62b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f22337fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a62b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55376/0 (socket says 172.21.15.57:55376) 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f22337fe700 1 -- 172.21.15.57:0/2129174839 learned_addr learned my addr 172.21.15.57:0/2129174839 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f22327fc700 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 0x5580a624a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.042+0000 7f2232ffd700 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6244e30 0x5580a624a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f22337fe700 1 -- 172.21.15.57:0/2129174839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6244e30 msgr2=0x5580a624a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f22337fe700 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6244e30 0x5580a624a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f22337fe700 1 -- 172.21.15.57:0/2129174839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 msgr2=0x5580a624a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f22337fe700 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 0x5580a624a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f22337fe700 1 -- 172.21.15.57:0/2129174839 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223410def0 con 0x5580a6250600 2023-10-08T13:50:56.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f22337fe700 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a62b3b40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f222c009a20 tx=0x7f222c00bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859ef1b375a30cfe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f2216ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f2216ff5700 1 -- 172.21.15.57:0/2129174839 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00e070 con 0x5580a6250600 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f2216ff5700 1 -- 172.21.15.57:0/2129174839 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f222c026820 con 0x5580a6250600 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f2245707b80 1 -- 172.21.15.57:0/2129174839 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22240ca8f0 con 0x5580a6250600 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f2216ff5700 1 -- 172.21.15.57:0/2129174839 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f222c0074a0 con 0x5580a6250600 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f2245707b80 10 client.15432 fetch_fsmap learned FSMap version 27 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f2245707b80 1 -- 172.21.15.57:0/2129174839 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5580a5fc2260 con 0x5580a6250600 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f2216ff5700 1 -- 172.21.15.57:0/2129174839 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22240ca8f0 con 0x5580a6250600 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.043+0000 7f2216ff5700 1 -- 172.21.15.57:0/2129174839 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f222c00b0d0 con 0x5580a6250600 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2245707b80 10 client.15432 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:50:56.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2245707b80 10 client.15432 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2245707b80 20 client.15432 populate_metadata read hostname 'smithi057' 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2245707b80 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5580a62b9470 0x5580a62b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2245707b80 4 client.15432 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2245707b80 1 -- 172.21.15.57:0/2129174839 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5580a6191ae0 con 0x5580a62b9470 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2232ffd700 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5580a62b9470 0x5580a62b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2232ffd700 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5580a62b9470 0x5580a62b58e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e314820ea27ebec1 server_cookie=3983ead5214de083 in_seq=0 out_seq=0 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2216ff5700 10 client.15432 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2216ff5700 1 -- 172.21.15.57:0/2129174839 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5580a6191ae0 con 0x5580a62b9470 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2216ff5700 10 client.15432 handle_command_reply: tid=0 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2245707b80 1 client.15432 shutdown 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.044+0000 7f2245707b80 1 -- 172.21.15.57:0/2129174839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 msgr2=0x5580a62b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2245707b80 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a62b3b40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f222c009a20 tx=0x7f222c00bcc0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2245707b80 1 -- 172.21.15.57:0/2129174839 shutdown_connections 2023-10-08T13:50:56.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2245707b80 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5580a62b9470 0x5580a62b58e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2245707b80 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5580a6250600 0x5580a62b3b40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2245707b80 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5580a6244e30 0x5580a624a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2245707b80 1 --2- 172.21.15.57:0/2129174839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580a6245750 0x5580a624a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2245707b80 1 -- 172.21.15.57:0/2129174839 >> 172.21.15.57:0/2129174839 conn(0x5580a61ba210 msgr2=0x5580a625ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2216ff5700 0 client.15432 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:50:56.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2245707b80 1 -- 172.21.15.57:0/2129174839 shutdown_connections 2023-10-08T13:50:56.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2245707b80 1 -- 172.21.15.57:0/2129174839 wait complete. 2023-10-08T13:50:56.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f2245707b80 20 client.15432 trim_cache size 0 max 16384 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 1, 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24652, 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 51.772530451999998, 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 27, 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 90, 2023-10-08T13:50:56.054 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 88, 2023-10-08T13:50:56.055 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 83.980233303000006 2023-10-08T13:50:56.055 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:50:56.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 -- 172.21.15.57:0/4042409042 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f221c0a1560 msgr2=0x7f221c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f221c0a1560 0x7f221c0a3a20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2234198160 tx=0x7f2228009260 comp rx=0 tx=0).stop 2023-10-08T13:50:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 -- 172.21.15.57:0/4042409042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2234100720 msgr2=0x7f223419c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2234100720 0x7f223419c950 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f222400d950 tx=0x7f222400dc60 comp rx=0 tx=0).stop 2023-10-08T13:50:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 -- 172.21.15.57:0/4042409042 shutdown_connections 2023-10-08T13:50:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f221c0a1560 0x7f221c0a3a20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2234106730 0x7f22341969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2234100720 0x7f223419c950 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 --2- 172.21.15.57:0/4042409042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22341010e0 0x7f223419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 -- 172.21.15.57:0/4042409042 >> 172.21.15.57:0/4042409042 conn(0x7f22340fc230 msgr2=0x7f22340fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 -- 172.21.15.57:0/4042409042 shutdown_connections 2023-10-08T13:50:56.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.045+0000 7f223951e700 1 -- 172.21.15.57:0/4042409042 wait complete. 2023-10-08T13:50:56.058 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T13:50:56.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.185+0000 7f9c548e0700 1 -- 172.21.15.57:0/2160471282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c50106730 msgr2=0x7f9c50106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.185+0000 7f9c548e0700 1 --2- 172.21.15.57:0/2160471282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c50106730 0x7f9c50106b10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009990 tx=0x7f9c44009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.185+0000 7f9c548e0700 1 -- 172.21.15.57:0/2160471282 shutdown_connections 2023-10-08T13:50:56.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.185+0000 7f9c548e0700 1 --2- 172.21.15.57:0/2160471282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c501010e0 0x7f9c5010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.185+0000 7f9c548e0700 1 --2- 172.21.15.57:0/2160471282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c50106730 0x7f9c50106b10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.185+0000 7f9c548e0700 1 --2- 172.21.15.57:0/2160471282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c50100720 0x7f9c50100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.185+0000 7f9c548e0700 1 -- 172.21.15.57:0/2160471282 >> 172.21.15.57:0/2160471282 conn(0x7f9c500fc230 msgr2=0x7f9c500fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.185+0000 7f9c548e0700 1 -- 172.21.15.57:0/2160471282 shutdown_connections 2023-10-08T13:50:56.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.185+0000 7f9c548e0700 1 -- 172.21.15.57:0/2160471282 wait complete. 2023-10-08T13:50:56.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c548e0700 1 Processor -- start 2023-10-08T13:50:56.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c548e0700 1 -- start start 2023-10-08T13:50:56.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c548e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c50100720 0x7f9c50072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c548e0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c501010e0 0x7f9c50073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c548e0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c50106730 0x7f9c50073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c548e0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c501a4f70 con 0x7f9c501010e0 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c548e0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c501a50e0 con 0x7f9c50100720 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c548e0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c501a52a0 con 0x7f9c50106730 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c501010e0 0x7f9c50073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c501010e0 0x7f9c50073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55388/0 (socket says 172.21.15.57:55388) 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c4dd9b700 1 -- 172.21.15.57:0/2194387131 learned_addr learned my addr 172.21.15.57:0/2194387131 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c50106730 0x7f9c50073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c50100720 0x7f9c50072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c4dd9b700 1 -- 172.21.15.57:0/2194387131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c50106730 msgr2=0x7f9c50073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c4dd9b700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c50106730 0x7f9c50073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c4dd9b700 1 -- 172.21.15.57:0/2194387131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c50100720 msgr2=0x7f9c50072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c4dd9b700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c50100720 0x7f9c50072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.186+0000 7f9c4dd9b700 1 -- 172.21.15.57:0/2194387131 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c440095f0 con 0x7f9c501010e0 2023-10-08T13:50:56.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.187+0000 7f9c4dd9b700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c501010e0 0x7f9c50073030 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000d980 tx=0x7f9c4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.187+0000 7f9c3f7fe700 1 -- 172.21.15.57:0/2194387131 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40004220 con 0x7f9c501010e0 2023-10-08T13:50:56.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.187+0000 7f9c548e0700 1 -- 172.21.15.57:0/2194387131 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c501a5580 con 0x7f9c501010e0 2023-10-08T13:50:56.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.187+0000 7f9c3f7fe700 1 -- 172.21.15.57:0/2194387131 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c40007640 con 0x7f9c501010e0 2023-10-08T13:50:56.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.187+0000 7f9c3f7fe700 1 -- 172.21.15.57:0/2194387131 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40005610 con 0x7f9c501010e0 2023-10-08T13:50:56.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.187+0000 7f9c548e0700 1 -- 172.21.15.57:0/2194387131 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c501a5a90 con 0x7f9c501010e0 2023-10-08T13:50:56.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.190+0000 7f9c3f7fe700 1 -- 172.21.15.57:0/2194387131 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f9c40004750 con 0x7f9c501010e0 2023-10-08T13:50:56.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.190+0000 7f9c60ac9b80 1 Processor -- start 2023-10-08T13:50:56.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c60ac9b80 1 -- start start 2023-10-08T13:50:56.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeed3750 0x563ddeede0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeede600 0x563ddeee09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddee3c410 con 0x563ddeed2e30 2023-10-08T13:50:56.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddee2dbd0 con 0x563ddeede600 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddee2b4f0 con 0x563ddeed3750 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c4ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c4ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55392/0 (socket says 172.21.15.57:55392) 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c4ed9d700 1 -- 172.21.15.57:0/2717391128 learned_addr learned my addr 172.21.15.57:0/2717391128 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c3f7fe700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c380a1560 0x7f9c380a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/2717391128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeed3750 0x563ddeede0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.191+0000 7f9c4dd9b700 1 --2- 172.21.15.57:0/2717391128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeede600 0x563ddeee09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c4ed9d700 1 -- 172.21.15.57:0/2717391128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeed3750 msgr2=0x563ddeede0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/2717391128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeed3750 0x563ddeede0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c4ed9d700 1 -- 172.21.15.57:0/2717391128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeede600 msgr2=0x563ddeee09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/2717391128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeede600 0x563ddeee09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c4ed9d700 1 -- 172.21.15.57:0/2717391128 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ddec51d90 con 0x563ddeed2e30 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c4dd9b700 1 --2- 172.21.15.57:0/2717391128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeede600 0x563ddeee09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c3f7fe700 1 -- 172.21.15.57:0/2194387131 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f9c400cd150 con 0x7f9c501010e0 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c380a1560 0x7f9c380a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/2717391128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9c480099c0 tx=0x7f9c48009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99bd816c6a2b52b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c3cff9700 1 -- 172.21.15.57:0/2717391128 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48018440 con 0x563ddeed2e30 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c3cff9700 1 -- 172.21.15.57:0/2717391128 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4800f040 con 0x563ddeed2e30 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c3cff9700 1 -- 172.21.15.57:0/2717391128 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48022750 con 0x563ddeed2e30 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.192+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c380a1560 0x7f9c380a3a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9c48021000 tx=0x7f9c4801e000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/2717391128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 msgr2=0x563ddeed3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/2717391128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9c480099c0 tx=0x7f9c48009cd0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/2717391128 shutdown_connections 2023-10-08T13:50:56.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/2717391128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeede600 0x563ddeee09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/2717391128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/2717391128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeed3750 0x563ddeede0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/2717391128 >> 172.21.15.57:0/2717391128 conn(0x563ddee48210 msgr2=0x563ddeecf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/2717391128 shutdown_connections 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/2717391128 wait complete. 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 -- start start 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.193+0000 7f9c60ac9b80 1 Processor -- start 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed2e30 0x563ddeede010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed3750 0x563ddeedb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeede600 0x563ddeedb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddee26de0 con 0x563ddeed3750 2023-10-08T13:50:56.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddedc36c0 con 0x563ddeed2e30 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddedc3360 con 0x563ddeede600 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c60ac9b80 10 client.0 _ll_register_callbacks cb 0x563ddec207b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed3750 0x563ddeedb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed3750 0x563ddeedb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55402/0 (socket says 172.21.15.57:55402) 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4dd9b700 1 -- 172.21.15.57:0/3297565963 learned_addr learned my addr 172.21.15.57:0/3297565963 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed2e30 0x563ddeede010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeede600 0x563ddeedb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4dd9b700 1 -- 172.21.15.57:0/3297565963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeede600 msgr2=0x563ddeedb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4dd9b700 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeede600 0x563ddeedb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4dd9b700 1 -- 172.21.15.57:0/3297565963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed2e30 msgr2=0x563ddeede010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4dd9b700 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed2e30 0x563ddeede010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4dd9b700 1 -- 172.21.15.57:0/3297565963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c48003c20 con 0x563ddeed3750 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeede600 0x563ddeedb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c4dd9b700 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed3750 0x563ddeedb390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9c400a1d80 tx=0x7f9c400693d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7c9c9fb62c8ab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3297565963 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4009a620 con 0x563ddeed3750 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3297565963 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f9c40099f20 con 0x563ddeed3750 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.194+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3297565963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ddedeee60 con 0x563ddeed3750 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3297565963 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c4009e070 con 0x563ddeed3750 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c60ac9b80 10 client.15444 fetch_fsmap learned FSMap version 27 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3297565963 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563ddec3ac10 con 0x563ddeed3750 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3297565963 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ddedeee60 con 0x563ddeed3750 2023-10-08T13:50:56.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3297565963 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9c400cad20 con 0x563ddeed3750 2023-10-08T13:50:56.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c60ac9b80 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:50:56.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c60ac9b80 10 client.15444 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T13:50:56.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c60ac9b80 20 client.15444 populate_metadata read hostname 'smithi057' 2023-10-08T13:50:56.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563ddeeee080 0x563ddeeecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c60ac9b80 4 client.15444 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:50:56.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3297565963 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ddee1fdc0 con 0x563ddeeee080 2023-10-08T13:50:56.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.195+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563ddeeee080 0x563ddeeecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.196+0000 7f9c35ffb700 10 client.15444 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:50:56.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.196+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563ddeeee080 0x563ddeeecbe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6690e4254aa29325 server_cookie=e9965464f2a7d5ef in_seq=0 out_seq=0 2023-10-08T13:50:56.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.196+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3297565963 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563ddee1fdc0 con 0x563ddeeee080 2023-10-08T13:50:56.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.196+0000 7f9c35ffb700 10 client.15444 handle_command_reply: tid=0 2023-10-08T13:50:56.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.196+0000 7f9c60ac9b80 1 client.15444 shutdown 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.196+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3297565963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed3750 msgr2=0x563ddeedb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.196+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed3750 0x563ddeedb390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9c400a1d80 tx=0x7f9c400693d0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.197+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3297565963 shutdown_connections 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.197+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563ddeeee080 0x563ddeeecbe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.197+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeede600 0x563ddeedb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.197+0000 7f9c35ffb700 0 client.15444 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.197+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed2e30 0x563ddeede010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.197+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3297565963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed3750 0x563ddeedb390 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.197+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3297565963 >> 172.21.15.57:0/3297565963 conn(0x563ddee48210 msgr2=0x563ddeecee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.197+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3297565963 shutdown_connections 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.197+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3297565963 wait complete. 2023-10-08T13:50:56.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.197+0000 7f9c60ac9b80 20 client.15444 trim_cache size 0 max 16384 2023-10-08T13:50:56.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c60ac9b80 1 Processor -- start 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c60ac9b80 1 -- start start 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed3750 0x563ddeeea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeede600 0x563ddeedb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddeca78d0 con 0x563ddeed2e30 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddecb3de0 con 0x563ddeed3750 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddede5590 con 0x563ddeede600 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c4e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c4e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55408/0 (socket says 172.21.15.57:55408) 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c4e59c700 1 -- 172.21.15.57:0/3575934048 learned_addr learned my addr 172.21.15.57:0/3575934048 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:56.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c4dd9b700 1 --2- 172.21.15.57:0/3575934048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed3750 0x563ddeeea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c4e59c700 1 -- 172.21.15.57:0/3575934048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeede600 msgr2=0x563ddeedb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/3575934048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeede600 0x563ddeedb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c4e59c700 1 -- 172.21.15.57:0/3575934048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed3750 msgr2=0x563ddeeea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/3575934048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed3750 0x563ddeeea170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.215+0000 7f9c4e59c700 1 -- 172.21.15.57:0/3575934048 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ddec78200 con 0x563ddeed2e30 2023-10-08T13:50:56.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/3575934048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9c440098e0 tx=0x7f9c4401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de11de0b560ad1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c3d7fa700 1 -- 172.21.15.57:0/3575934048 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4401d070 con 0x563ddeed2e30 2023-10-08T13:50:56.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c3d7fa700 1 -- 172.21.15.57:0/3575934048 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4401e040 con 0x563ddeed2e30 2023-10-08T13:50:56.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c3d7fa700 1 -- 172.21.15.57:0/3575934048 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44005310 con 0x563ddeed2e30 2023-10-08T13:50:56.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3575934048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 msgr2=0x563ddeed3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3575934048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9c440098e0 tx=0x7f9c4401a040 comp rx=0 tx=0).stop 2023-10-08T13:50:56.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3575934048 shutdown_connections 2023-10-08T13:50:56.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3575934048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeede600 0x563ddeedb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3575934048 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeed2e30 0x563ddeed3210 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3575934048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed3750 0x563ddeeea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3575934048 >> 172.21.15.57:0/3575934048 conn(0x563ddee48210 msgr2=0x563ddeed66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3575934048 shutdown_connections 2023-10-08T13:50:56.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.216+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3575934048 wait complete. 2023-10-08T13:50:56.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c60ac9b80 1 -- start start 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c60ac9b80 1 Processor -- start 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed2e30 0x563ddeed8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeed3750 0x563ddeed7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c60ac9b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeede600 0x563ddeed7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddeed9300 con 0x563ddeede600 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddeed9470 con 0x563ddeed2e30 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c60ac9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ddeed80d0 con 0x563ddeed3750 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c60ac9b80 10 client.0 _ll_register_callbacks cb 0x7f9c40099cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c4ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeede600 0x563ddeed7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c4ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeede600 0x563ddeed7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55418/0 (socket says 172.21.15.57:55418) 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c4ed9d700 1 -- 172.21.15.57:0/3145651498 learned_addr learned my addr 172.21.15.57:0/3145651498 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:56.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed2e30 0x563ddeed8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c4dd9b700 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeed3750 0x563ddeed7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c4ed9d700 1 -- 172.21.15.57:0/3145651498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeed3750 msgr2=0x563ddeed7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeed3750 0x563ddeed7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c4ed9d700 1 -- 172.21.15.57:0/3145651498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed2e30 msgr2=0x563ddeed8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed2e30 0x563ddeed8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.217+0000 7f9c4ed9d700 1 -- 172.21.15.57:0/3145651498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5010def0 con 0x563ddeede600 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c4ed9d700 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeede600 0x563ddeed7b60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9c480099c0 tx=0x7f9c480226a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1389922e0b9220fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3145651498 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48004e50 con 0x563ddeede600 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3145651498 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f9c4801d030 con 0x563ddeede600 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3145651498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c4800f040 con 0x563ddeede600 2023-10-08T13:50:56.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3145651498 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c48022a30 con 0x563ddeede600 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c60ac9b80 10 client.15450 fetch_fsmap learned FSMap version 27 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3145651498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563ddec50260 con 0x563ddeede600 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3145651498 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4800f040 con 0x563ddeede600 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3145651498 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9c48018ca0 con 0x563ddeede600 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c60ac9b80 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c60ac9b80 10 client.15450 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c60ac9b80 20 client.15450 populate_metadata read hostname 'smithi057' 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.218+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563ddef467c0 0x563ddef42c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c60ac9b80 4 client.15450 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3145651498 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563ddee1fae0 con 0x563ddef467c0 2023-10-08T13:50:56.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563ddef467c0 0x563ddef42c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c35ffb700 10 client.15450 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c4e59c700 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563ddef467c0 0x563ddef42c20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14cf6ec829d1882 server_cookie=79c68beecd01853e in_seq=0 out_seq=0 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c35ffb700 1 -- 172.21.15.57:0/3145651498 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563ddee1fae0 con 0x563ddef467c0 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c35ffb700 10 client.15450 handle_command_reply: tid=0 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c60ac9b80 1 client.15450 shutdown 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3145651498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeede600 msgr2=0x563ddeed7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeede600 0x563ddeed7b60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9c480099c0 tx=0x7f9c480226a0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.219+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3145651498 shutdown_connections 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563ddef467c0 0x563ddef42c20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563ddeede600 0x563ddeed7b60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ddeed2e30 0x563ddeed8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c60ac9b80 1 --2- 172.21.15.57:0/3145651498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563ddeed3750 0x563ddeed7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c35ffb700 0 client.15450 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:50:56.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3145651498 >> 172.21.15.57:0/3145651498 conn(0x563ddee48210 msgr2=0x563ddeeecbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3145651498 shutdown_connections 2023-10-08T13:50:56.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c60ac9b80 1 -- 172.21.15.57:0/3145651498 wait complete. 2023-10-08T13:50:56.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c60ac9b80 20 client.15450 trim_cache size 0 max 16384 2023-10-08T13:50:56.228 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:50:56.228 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:50:56.228 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-10-08T13:50:56.228 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24662, 2023-10-08T13:50:56.228 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:50:56.229 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:50:56.229 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:50:56.229 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 71.105884814999996, 2023-10-08T13:50:56.229 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 27, 2023-10-08T13:50:56.229 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 90, 2023-10-08T13:50:56.229 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 88, 2023-10-08T13:50:56.229 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 77.035280532000002 2023-10-08T13:50:56.229 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:50:56.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 -- 172.21.15.57:0/2194387131 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c380a1560 msgr2=0x7f9c380a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c380a1560 0x7f9c380a3a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9c48021000 tx=0x7f9c4801e000 comp rx=0 tx=0).stop 2023-10-08T13:50:56.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 -- 172.21.15.57:0/2194387131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c501010e0 msgr2=0x7f9c50073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:56.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c501010e0 0x7f9c50073030 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000d980 tx=0x7f9c4000dc90 comp rx=0 tx=0).stop 2023-10-08T13:50:56.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 -- 172.21.15.57:0/2194387131 shutdown_connections 2023-10-08T13:50:56.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c50106730 0x7f9c50073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c380a1560 0x7f9c380a3a20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c50100720 0x7f9c50072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 --2- 172.21.15.57:0/2194387131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c501010e0 0x7f9c50073030 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:56.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 -- 172.21.15.57:0/2194387131 >> 172.21.15.57:0/2194387131 conn(0x7f9c500fc230 msgr2=0x7f9c500fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:56.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 -- 172.21.15.57:0/2194387131 shutdown_connections 2023-10-08T13:50:56.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:56.220+0000 7f9c548e0700 1 -- 172.21.15.57:0/2194387131 wait complete. 2023-10-08T13:50:56.233 INFO:tasks.cephadm:Setup complete, yielding 2023-10-08T13:50:56.234 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-08T13:50:56.243 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T13:50:56.243 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-08T13:50:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:55 smithi099 ceph-mon[103849]: pgmap v278: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:50:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:55 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2038025232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:56.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:55 smithi115 ceph-mon[104408]: pgmap v278: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:50:56.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:55 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2038025232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:50:56.807 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:50:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.901+0000 7f92bcfc0700 1 -- 172.21.15.57:0/162647949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92b8068190 msgr2=0x7f92b80685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.901+0000 7f92bcfc0700 1 --2- 172.21.15.57:0/162647949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92b8068190 0x7f92b80685f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009990 tx=0x7f92a8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:50:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.901+0000 7f92bcfc0700 1 -- 172.21.15.57:0/162647949 shutdown_connections 2023-10-08T13:50:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.901+0000 7f92bcfc0700 1 --2- 172.21.15.57:0/162647949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b8068b30 0x7f92b810d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.901+0000 7f92bcfc0700 1 --2- 172.21.15.57:0/162647949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b81067a0 0x7f92b8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.901+0000 7f92bcfc0700 1 --2- 172.21.15.57:0/162647949 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92b8068190 0x7f92b80685f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.901+0000 7f92bcfc0700 1 -- 172.21.15.57:0/162647949 >> 172.21.15.57:0/162647949 conn(0x7f92b8075790 msgr2=0x7f92b8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.901+0000 7f92bcfc0700 1 -- 172.21.15.57:0/162647949 shutdown_connections 2023-10-08T13:50:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.901+0000 7f92bcfc0700 1 -- 172.21.15.57:0/162647949 wait complete. 2023-10-08T13:50:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92bcfc0700 1 Processor -- start 2023-10-08T13:50:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92bcfc0700 1 -- start start 2023-10-08T13:50:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92bcfc0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92b8068190 0x7f92b819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92bcfc0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8068b30 0x7f92b819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92bcfc0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b81067a0 0x7f92b8196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92bcfc0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b81970b0 con 0x7f92b8068190 2023-10-08T13:50:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92bcfc0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8197220 con 0x7f92b81067a0 2023-10-08T13:50:57.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92bcfc0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b81973e0 con 0x7f92b8068b30 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92b659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92b8068190 0x7f92b819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92b659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92b8068190 0x7f92b819c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55426/0 (socket says 172.21.15.57:55426) 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92b659c700 1 -- 172.21.15.57:0/328352426 learned_addr learned my addr 172.21.15.57:0/328352426 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92b5d9b700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8068b30 0x7f92b819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.902+0000 7f92b6d9d700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b81067a0 0x7f92b8196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92b659c700 1 -- 172.21.15.57:0/328352426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8068b30 msgr2=0x7f92b819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92b659c700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8068b30 0x7f92b819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92b659c700 1 -- 172.21.15.57:0/328352426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b81067a0 msgr2=0x7f92b8196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92b659c700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b81067a0 0x7f92b8196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92b659c700 1 -- 172.21.15.57:0/328352426 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a80095f0 con 0x7f92b8068190 2023-10-08T13:50:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92b6d9d700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b81067a0 0x7f92b8196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:50:57.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92b659c700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92b8068190 0x7f92b819c8c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c9b0 tx=0x7f92ac00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:57.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92a77fe700 1 -- 172.21.15.57:0/328352426 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac007b00 con 0x7f92b8068190 2023-10-08T13:50:57.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92a77fe700 1 -- 172.21.15.57:0/328352426 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92ac007c60 con 0x7f92b8068190 2023-10-08T13:50:57.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92bcfc0700 1 -- 172.21.15.57:0/328352426 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b81976c0 con 0x7f92b8068190 2023-10-08T13:50:57.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92a77fe700 1 -- 172.21.15.57:0/328352426 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac018660 con 0x7f92b8068190 2023-10-08T13:50:57.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.903+0000 7f92bcfc0700 1 -- 172.21.15.57:0/328352426 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b81a38b0 con 0x7f92b8068190 2023-10-08T13:50:57.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.904+0000 7f92bcfc0700 1 -- 172.21.15.57:0/328352426 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b8108d50 con 0x7f92b8068190 2023-10-08T13:50:57.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.905+0000 7f92a77fe700 1 -- 172.21.15.57:0/328352426 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f92ac018880 con 0x7f92b8068190 2023-10-08T13:50:57.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.906+0000 7f92a77fe700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f92a00a15e0 0x7f92a00a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:50:57.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.906+0000 7f92b5d9b700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f92a00a15e0 0x7f92a00a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:50:57.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.906+0000 7f92a77fe700 1 -- 172.21.15.57:0/328352426 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f92ac0cd260 con 0x7f92b8068190 2023-10-08T13:50:57.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.906+0000 7f92b5d9b700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f92a00a15e0 0x7f92a00a3aa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92a8005850 tx=0x7f92a801a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:50:57.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:57.910+0000 7f92a77fe700 1 -- 172.21.15.57:0/328352426 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f92ac096370 con 0x7f92b8068190 2023-10-08T13:50:58.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.128+0000 7f92bcfc0700 1 -- 172.21.15.57:0/328352426 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92b8197ec0 con 0x7f92a00a15e0 2023-10-08T13:50:58.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.129+0000 7f92a77fe700 1 -- 172.21.15.57:0/328352426 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f92b8197ec0 con 0x7f92a00a15e0 2023-10-08T13:50:58.130 INFO:teuthology.orchestra.run.smithi057.stdout:Backend: cephadm 2023-10-08T13:50:58.131 INFO:teuthology.orchestra.run.smithi057.stdout:Available: Yes 2023-10-08T13:50:58.131 INFO:teuthology.orchestra.run.smithi057.stdout:Paused: No 2023-10-08T13:50:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 -- 172.21.15.57:0/328352426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f92a00a15e0 msgr2=0x7f92a00a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f92a00a15e0 0x7f92a00a3aa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92a8005850 tx=0x7f92a801a040 comp rx=0 tx=0).stop 2023-10-08T13:50:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 -- 172.21.15.57:0/328352426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92b8068190 msgr2=0x7f92b819c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:50:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92b8068190 0x7f92b819c8c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c9b0 tx=0x7f92ac00cd70 comp rx=0 tx=0).stop 2023-10-08T13:50:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 -- 172.21.15.57:0/328352426 shutdown_connections 2023-10-08T13:50:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f92a00a15e0 0x7f92a00a3aa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b81067a0 0x7f92b8196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92b8068190 0x7f92b819c8c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 --2- 172.21.15.57:0/328352426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8068b30 0x7f92b819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:50:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 -- 172.21.15.57:0/328352426 >> 172.21.15.57:0/328352426 conn(0x7f92b8075790 msgr2=0x7f92b80fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:50:58.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 -- 172.21.15.57:0/328352426 shutdown_connections 2023-10-08T13:50:58.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:50:58.132+0000 7f92bcfc0700 1 -- 172.21.15.57:0/328352426 wait complete. 2023-10-08T13:50:58.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:57 smithi057 ceph-mon[99476]: pgmap v279: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-08T13:50:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:57 smithi099 ceph-mon[103849]: pgmap v279: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-08T13:50:58.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:57 smithi115 ceph-mon[104408]: pgmap v279: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-08T13:50:58.475 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-08T13:50:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:51:00.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:59 smithi057 ceph-mon[99476]: pgmap v280: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:00.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:50:59 smithi057 ceph-mon[99476]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:00.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.243+0000 7f4c11d39700 1 -- 172.21.15.57:0/2174661055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c101260 msgr2=0x7f4c0c10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:00.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.243+0000 7f4c11d39700 1 --2- 172.21.15.57:0/2174661055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c101260 0x7f4c0c10d650 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400b0a0 tx=0x7f4c0400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.243+0000 7f4c11d39700 1 -- 172.21.15.57:0/2174661055 shutdown_connections 2023-10-08T13:51:00.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.243+0000 7f4c11d39700 1 --2- 172.21.15.57:0/2174661055 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c101260 0x7f4c0c10d650 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.243+0000 7f4c11d39700 1 --2- 172.21.15.57:0/2174661055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c1057c0 0x7f4c0c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.243+0000 7f4c11d39700 1 --2- 172.21.15.57:0/2174661055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c0c1008a0 0x7f4c0c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.243+0000 7f4c11d39700 1 -- 172.21.15.57:0/2174661055 >> 172.21.15.57:0/2174661055 conn(0x7f4c0c0779e0 msgr2=0x7f4c0c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:00.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.243+0000 7f4c11d39700 1 -- 172.21.15.57:0/2174661055 shutdown_connections 2023-10-08T13:51:00.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.243+0000 7f4c11d39700 1 -- 172.21.15.57:0/2174661055 wait complete. 2023-10-08T13:51:00.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.243+0000 7f4c11d39700 1 Processor -- start 2023-10-08T13:51:00.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c11d39700 1 -- start start 2023-10-08T13:51:00.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c11d39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c0c1008a0 0x7f4c0c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c11d39700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c101260 0x7f4c0c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c11d39700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c1057c0 0x7f4c0c196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c11d39700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1972a0 con 0x7f4c0c1057c0 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c11d39700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c197410 con 0x7f4c0c1008a0 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c11d39700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1975d0 con 0x7f4c0c101260 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c0bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c1057c0 0x7f4c0c196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c0bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c1057c0 0x7f4c0c196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55444/0 (socket says 172.21.15.57:55444) 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c0bfff700 1 -- 172.21.15.57:0/858461300 learned_addr learned my addr 172.21.15.57:0/858461300 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c0affd700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c101260 0x7f4c0c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c0b7fe700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c0c1008a0 0x7f4c0c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:00.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c0bfff700 1 -- 172.21.15.57:0/858461300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c101260 msgr2=0x7f4c0c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c0bfff700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c101260 0x7f4c0c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c0bfff700 1 -- 172.21.15.57:0/858461300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c0c1008a0 msgr2=0x7f4c0c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c0bfff700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c0c1008a0 0x7f4c0c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.244+0000 7f4c0bfff700 1 -- 172.21.15.57:0/858461300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c04009d00 con 0x7f4c0c1057c0 2023-10-08T13:51:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.245+0000 7f4c0bfff700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c1057c0 0x7f4c0c196af0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400b070 tx=0x7f4c04009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.245+0000 7f4c08ff9700 1 -- 172.21.15.57:0/858461300 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0400e040 con 0x7f4c0c1057c0 2023-10-08T13:51:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.245+0000 7f4c08ff9700 1 -- 172.21.15.57:0/858461300 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c0401d480 con 0x7f4c0c1057c0 2023-10-08T13:51:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.245+0000 7f4c11d39700 1 -- 172.21.15.57:0/858461300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0c197850 con 0x7f4c0c1057c0 2023-10-08T13:51:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.245+0000 7f4c08ff9700 1 -- 172.21.15.57:0/858461300 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0401b6b0 con 0x7f4c0c1057c0 2023-10-08T13:51:00.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.245+0000 7f4c11d39700 1 -- 172.21.15.57:0/858461300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c1a39c0 con 0x7f4c0c1057c0 2023-10-08T13:51:00.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.246+0000 7f4c11d39700 1 -- 172.21.15.57:0/858461300 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0c108da0 con 0x7f4c0c1057c0 2023-10-08T13:51:00.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.247+0000 7f4c08ff9700 1 -- 172.21.15.57:0/858461300 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f4c0401d5f0 con 0x7f4c0c1057c0 2023-10-08T13:51:00.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.248+0000 7f4c08ff9700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4bf40a15e0 0x7f4bf40a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:00.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.248+0000 7f4c0b7fe700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4bf40a15e0 0x7f4bf40a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:00.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.248+0000 7f4c08ff9700 1 -- 172.21.15.57:0/858461300 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f4c040cf1d0 con 0x7f4c0c1057c0 2023-10-08T13:51:00.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.248+0000 7f4c0b7fe700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4bf40a15e0 0x7f4bf40a3aa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc005950 tx=0x7f4bfc0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:00.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.251+0000 7f4c08ff9700 1 -- 172.21.15.57:0/858461300 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4c04098370 con 0x7f4c0c1057c0 2023-10-08T13:51:00.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:59 smithi099 ceph-mon[103849]: pgmap v280: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:00.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:50:59 smithi099 ceph-mon[103849]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:00.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:59 smithi115 ceph-mon[104408]: pgmap v280: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:00.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:50:59 smithi115 ceph-mon[104408]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.461+0000 7f4c11d39700 1 -- 172.21.15.57:0/858461300 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c0c197fd0 con 0x7f4bf40a15e0 2023-10-08T13:51:00.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.483+0000 7f4c08ff9700 1 -- 172.21.15.57:0/858461300 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f4c0c197fd0 con 0x7f4bf40a15e0 2023-10-08T13:51:00.484 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-08T13:51:00.484 INFO:teuthology.orchestra.run.smithi057.stdout:mds.a smithi057 running (84s) 10s ago 83s 20.0M - 18.0.0-6639-g7b3143ba 09a1f3752b5d a4099f8c3133 2023-10-08T13:51:00.484 INFO:teuthology.orchestra.run.smithi057.stdout:mds.b smithi099 running (81s) 11s ago 81s 30.9M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 4bbe0e163e80 2023-10-08T13:51:00.484 INFO:teuthology.orchestra.run.smithi057.stdout:mds.c smithi115 running (93s) 11s ago 92s 18.4M - 18.0.0-6639-g7b3143ba 09a1f3752b5d e209e02cf4b5 2023-10-08T13:51:00.484 INFO:teuthology.orchestra.run.smithi057.stdout:mds.d smithi057 running (100s) 10s ago 99s 16.7M - 18.0.0-6639-g7b3143ba 09a1f3752b5d fe9a466a34fc 2023-10-08T13:51:00.484 INFO:teuthology.orchestra.run.smithi057.stdout:mds.e smithi099 running (97s) 11s ago 97s 17.6M - 18.0.0-6639-g7b3143ba 09a1f3752b5d c7bab04cf6ce 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mds.f smithi115 running (95s) 11s ago 95s 18.9M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 20dc39932fd9 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mds.g smithi057 running (86s) 10s ago 86s 14.0M - 18.0.0-6639-g7b3143ba 09a1f3752b5d fa90b16e9821 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mds.h smithi099 running (90s) 11s ago 90s 20.3M - 18.0.0-6639-g7b3143ba 09a1f3752b5d af9d37c2eaeb 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mds.i smithi115 running (88s) 11s ago 88s 21.7M - 18.0.0-6639-g7b3143ba 09a1f3752b5d df7671feed7f 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mds.j smithi057 running (102s) 10s ago 101s 15.0M - 18.0.0-6639-g7b3143ba 09a1f3752b5d cc004d39c45d 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mds.k smithi099 running (104s) 11s ago 104s 17.4M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 6dd2a9595815 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mds.l smithi115 running (106s) 11s ago 106s 20.3M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 59c97729b96f 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi057 *:9283,8765 running (8m) 10s ago 8m 517M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 3a499252d3be 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi099 *:8443,8765 running (6m) 11s ago 6m 454M - 18.0.0-6639-g7b3143ba 09a1f3752b5d bd41aa1c9b2f 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.z smithi115 *:8443,8765 running (6m) 11s ago 6m 451M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 172b5b755eeb 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (8m) 10s ago 8m 102M 2048M 18.0.0-6639-g7b3143ba 09a1f3752b5d be6997884e4d 2023-10-08T13:51:00.485 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi099 running (7m) 11s ago 7m 65.7M 2048M 18.0.0-6639-g7b3143ba 09a1f3752b5d 0e260681513e 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi115 running (7m) 11s ago 7m 67.4M 2048M 18.0.0-6639-g7b3143ba 09a1f3752b5d 44b3f6b3c611 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (6m) 10s ago 6m 73.1M 4293M 18.0.0-6639-g7b3143ba 09a1f3752b5d 981f37d8f3b5 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi099 running (5m) 11s ago 5m 77.5M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d d2b789002651 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi115 running (5m) 11s ago 5m 77.7M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d e1c60dcbf94e 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (4m) 10s ago 4m 65.8M 4293M 18.0.0-6639-g7b3143ba 09a1f3752b5d 37ba8815f6e5 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi099 running (4m) 11s ago 4m 66.8M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 58c71a6de723 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi115 running (4m) 11s ago 4m 71.5M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 9b59988af34c 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi057 running (3m) 10s ago 3m 62.2M 4293M 18.0.0-6639-g7b3143ba 09a1f3752b5d 1bbd98812b03 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi099 running (3m) 11s ago 3m 68.5M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d d9fb5ccfd623 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:osd.8 smithi115 running (3m) 11s ago 3m 65.4M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 8a783168265b 2023-10-08T13:51:00.486 INFO:teuthology.orchestra.run.smithi057.stdout:osd.9 smithi057 running (2m) 10s ago 2m 61.2M 4293M 18.0.0-6639-g7b3143ba 09a1f3752b5d cf3d1038d096 2023-10-08T13:51:00.487 INFO:teuthology.orchestra.run.smithi057.stdout:osd.10 smithi099 running (2m) 11s ago 2m 66.2M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 18de2dcc1db6 2023-10-08T13:51:00.487 INFO:teuthology.orchestra.run.smithi057.stdout:osd.11 smithi115 running (2m) 11s ago 2m 56.7M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 4e7fdcdebe5e 2023-10-08T13:51:00.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 -- 172.21.15.57:0/858461300 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4bf40a15e0 msgr2=0x7f4bf40a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:00.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4bf40a15e0 0x7f4bf40a3aa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc005950 tx=0x7f4bfc0058e0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 -- 172.21.15.57:0/858461300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c1057c0 msgr2=0x7f4c0c196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c1057c0 0x7f4c0c196af0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400b070 tx=0x7f4c04009570 comp rx=0 tx=0).stop 2023-10-08T13:51:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 -- 172.21.15.57:0/858461300 shutdown_connections 2023-10-08T13:51:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c0c1057c0 0x7f4c0c196af0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4bf40a15e0 0x7f4bf40a3aa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c0c1008a0 0x7f4c0c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 --2- 172.21.15.57:0/858461300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c101260 0x7f4c0c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 -- 172.21.15.57:0/858461300 >> 172.21.15.57:0/858461300 conn(0x7f4c0c0779e0 msgr2=0x7f4c0c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 -- 172.21.15.57:0/858461300 shutdown_connections 2023-10-08T13:51:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:00.485+0000 7f4c11d39700 1 -- 172.21.15.57:0/858461300 wait complete. 2023-10-08T13:51:00.904 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-08T13:51:01.470 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:51:02.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:01 smithi057 ceph-mon[99476]: pgmap v281: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T13:51:02.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:01 smithi057 ceph-mon[99476]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:01 smithi099 ceph-mon[103849]: pgmap v281: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T13:51:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:01 smithi099 ceph-mon[103849]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:02.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:01 smithi115 ceph-mon[104408]: pgmap v281: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T13:51:02.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:01 smithi115 ceph-mon[104408]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:02.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.647+0000 7fd860b9f700 1 -- 172.21.15.57:0/2629811430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85c106670 msgr2=0x7fd85c106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:02.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.647+0000 7fd860b9f700 1 --2- 172.21.15.57:0/2629811430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85c106670 0x7fd85c106a50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009990 tx=0x7fd84c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:02.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.647+0000 7fd860b9f700 1 -- 172.21.15.57:0/2629811430 shutdown_connections 2023-10-08T13:51:02.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.647+0000 7fd860b9f700 1 --2- 172.21.15.57:0/2629811430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd85c100fa0 0x7fd85c10d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:02.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.647+0000 7fd860b9f700 1 --2- 172.21.15.57:0/2629811430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85c106670 0x7fd85c106a50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:02.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.647+0000 7fd860b9f700 1 --2- 172.21.15.57:0/2629811430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c1005e0 0x7fd85c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:02.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.647+0000 7fd860b9f700 1 -- 172.21.15.57:0/2629811430 >> 172.21.15.57:0/2629811430 conn(0x7fd85c0fc130 msgr2=0x7fd85c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:02.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.647+0000 7fd860b9f700 1 -- 172.21.15.57:0/2629811430 shutdown_connections 2023-10-08T13:51:02.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.647+0000 7fd860b9f700 1 -- 172.21.15.57:0/2629811430 wait complete. 2023-10-08T13:51:02.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.648+0000 7fd860b9f700 1 Processor -- start 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.648+0000 7fd860b9f700 1 -- start start 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.648+0000 7fd860b9f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd85c1005e0 0x7fd85c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.648+0000 7fd860b9f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85c100fa0 0x7fd85c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.648+0000 7fd860b9f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c106670 0x7fd85c196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.648+0000 7fd860b9f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c196ea0 con 0x7fd85c100fa0 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.648+0000 7fd860b9f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c197010 con 0x7fd85c1005e0 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.648+0000 7fd860b9f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c1971d0 con 0x7fd85c106670 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.648+0000 7fd85affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85c100fa0 0x7fd85c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd85affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85c100fa0 0x7fd85c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55456/0 (socket says 172.21.15.57:55456) 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd85affd700 1 -- 172.21.15.57:0/3314106801 learned_addr learned my addr 172.21.15.57:0/3314106801 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd85bfff700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c106670 0x7fd85c196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:02.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd85b7fe700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd85c1005e0 0x7fd85c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd85affd700 1 -- 172.21.15.57:0/3314106801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c106670 msgr2=0x7fd85c196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd85affd700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c106670 0x7fd85c196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd85affd700 1 -- 172.21.15.57:0/3314106801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd85c1005e0 msgr2=0x7fd85c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd85affd700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd85c1005e0 0x7fd85c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd85affd700 1 -- 172.21.15.57:0/3314106801 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c0095f0 con 0x7fd85c100fa0 2023-10-08T13:51:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd85affd700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85c100fa0 0x7fd85c19cc30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd84400d950 tx=0x7fd84400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd858ff9700 1 -- 172.21.15.57:0/3314106801 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400f890 con 0x7fd85c100fa0 2023-10-08T13:51:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd860b9f700 1 -- 172.21.15.57:0/3314106801 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c1974b0 con 0x7fd85c100fa0 2023-10-08T13:51:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd858ff9700 1 -- 172.21.15.57:0/3314106801 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd844010460 con 0x7fd85c100fa0 2023-10-08T13:51:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.649+0000 7fd860b9f700 1 -- 172.21.15.57:0/3314106801 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c1a36e0 con 0x7fd85c100fa0 2023-10-08T13:51:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.650+0000 7fd858ff9700 1 -- 172.21.15.57:0/3314106801 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84401b650 con 0x7fd85c100fa0 2023-10-08T13:51:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.650+0000 7fd860b9f700 1 -- 172.21.15.57:0/3314106801 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c108b70 con 0x7fd85c100fa0 2023-10-08T13:51:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.652+0000 7fd858ff9700 1 -- 172.21.15.57:0/3314106801 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fd8440108d0 con 0x7fd85c100fa0 2023-10-08T13:51:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.652+0000 7fd858ff9700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd8480a1560 0x7fd8480a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.652+0000 7fd85b7fe700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd8480a1560 0x7fd8480a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.652+0000 7fd858ff9700 1 -- 172.21.15.57:0/3314106801 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fd8440cd420 con 0x7fd85c100fa0 2023-10-08T13:51:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.653+0000 7fd85b7fe700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd8480a1560 0x7fd8480a3a20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd84c006010 tx=0x7fd84c005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:02.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.657+0000 7fd858ff9700 1 -- 172.21.15.57:0/3314106801 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd844096530 con 0x7fd85c100fa0 2023-10-08T13:51:02.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.868+0000 7fd860b9f700 1 -- 172.21.15.57:0/3314106801 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd85c197cf0 con 0x7fd8480a1560 2023-10-08T13:51:02.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.876+0000 7fd858ff9700 1 -- 172.21.15.57:0/3314106801 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7fd85c197cf0 con 0x7fd8480a1560 2023-10-08T13:51:02.877 INFO:teuthology.orchestra.run.smithi057.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-08T13:51:02.877 INFO:teuthology.orchestra.run.smithi057.stdout:mds.all 12/12 13s ago 110s smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi115=c;smithi115=f;smithi115=i;smithi115=l;count:12 2023-10-08T13:51:02.877 INFO:teuthology.orchestra.run.smithi057.stdout:mgr 3/3 13s ago 6m smithi057=x;smithi099=y;smithi115=z;count:3 2023-10-08T13:51:02.877 INFO:teuthology.orchestra.run.smithi057.stdout:mon 3/3 13s ago 7m smithi057:172.21.15.57=a;smithi099:172.21.15.99=b;smithi115:172.21.15.115=c;count:3 2023-10-08T13:51:02.877 INFO:teuthology.orchestra.run.smithi057.stdout:osd 12 13s ago - 2023-10-08T13:51:02.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 -- 172.21.15.57:0/3314106801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd8480a1560 msgr2=0x7fd8480a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:02.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd8480a1560 0x7fd8480a3a20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd84c006010 tx=0x7fd84c005960 comp rx=0 tx=0).stop 2023-10-08T13:51:02.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 -- 172.21.15.57:0/3314106801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85c100fa0 msgr2=0x7fd85c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:02.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85c100fa0 0x7fd85c19cc30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd84400d950 tx=0x7fd84400dc60 comp rx=0 tx=0).stop 2023-10-08T13:51:02.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 -- 172.21.15.57:0/3314106801 shutdown_connections 2023-10-08T13:51:02.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd85c106670 0x7fd85c196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:02.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd8480a1560 0x7fd8480a3a20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:02.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd85c1005e0 0x7fd85c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:02.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 --2- 172.21.15.57:0/3314106801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd85c100fa0 0x7fd85c19cc30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:02.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 -- 172.21.15.57:0/3314106801 >> 172.21.15.57:0/3314106801 conn(0x7fd85c0fc130 msgr2=0x7fd85c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:02.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 -- 172.21.15.57:0/3314106801 shutdown_connections 2023-10-08T13:51:02.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:02.878+0000 7fd860b9f700 1 -- 172.21.15.57:0/3314106801 wait complete. 2023-10-08T13:51:03.231 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-08T13:51:03.799 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:51:04.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:04 smithi057 ceph-mon[99476]: pgmap v282: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-08T13:51:04.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:04 smithi057 ceph-mon[99476]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:04 smithi099 ceph-mon[103849]: pgmap v282: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-08T13:51:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:04 smithi099 ceph-mon[103849]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:04.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:04 smithi115 ceph-mon[104408]: pgmap v282: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-08T13:51:04.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:04 smithi115 ceph-mon[104408]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:04.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.979+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/3913871724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1e4101260 msgr2=0x7ff1e410d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:04.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.979+0000 7ff1ec5e9700 1 --2- 172.21.15.57:0/3913871724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1e4101260 0x7ff1e410d650 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009f90 tx=0x7ff1dc00b330 comp rx=0 tx=0).stop 2023-10-08T13:51:04.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.980+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/3913871724 shutdown_connections 2023-10-08T13:51:04.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.980+0000 7ff1ec5e9700 1 --2- 172.21.15.57:0/3913871724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1e4101260 0x7ff1e410d650 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:04.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.980+0000 7ff1ec5e9700 1 --2- 172.21.15.57:0/3913871724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1e41057c0 0x7ff1e4105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:04.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.980+0000 7ff1ec5e9700 1 --2- 172.21.15.57:0/3913871724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e41008a0 0x7ff1e4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:04.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.980+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/3913871724 >> 172.21.15.57:0/3913871724 conn(0x7ff1e40779e0 msgr2=0x7ff1e4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:04.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.980+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/3913871724 shutdown_connections 2023-10-08T13:51:04.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.980+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/3913871724 wait complete. 2023-10-08T13:51:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.980+0000 7ff1ec5e9700 1 Processor -- start 2023-10-08T13:51:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.980+0000 7ff1ec5e9700 1 -- start start 2023-10-08T13:51:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1ec5e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e41008a0 0x7ff1e4198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1ec5e9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1e4101260 0x7ff1e41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1ec5e9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1e41057c0 0x7ff1e419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1ec5e9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419dea0 con 0x7ff1e4101260 2023-10-08T13:51:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1ec5e9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419e010 con 0x7ff1e41008a0 2023-10-08T13:51:04.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1ec5e9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419e1d0 con 0x7ff1e41057c0 2023-10-08T13:51:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1e9b84700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1e4101260 0x7ff1e41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1e9b84700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1e4101260 0x7ff1e41994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55468/0 (socket says 172.21.15.57:55468) 2023-10-08T13:51:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1e9b84700 1 -- 172.21.15.57:0/2969502459 learned_addr learned my addr 172.21.15.57:0/2969502459 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1ea385700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e41008a0 0x7ff1e4198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1eab86700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1e41057c0 0x7ff1e419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1e9b84700 1 -- 172.21.15.57:0/2969502459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1e41057c0 msgr2=0x7ff1e419d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1e9b84700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1e41057c0 0x7ff1e419d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:04.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1e9b84700 1 -- 172.21.15.57:0/2969502459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e41008a0 msgr2=0x7ff1e4198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:04.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1e9b84700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e41008a0 0x7ff1e4198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:04.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1e9b84700 1 -- 172.21.15.57:0/2969502459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d40095f0 con 0x7ff1e4101260 2023-10-08T13:51:04.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1e9b84700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1e4101260 0x7ff1e41994e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000d980 tx=0x7ff1e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:04.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1db7fe700 1 -- 172.21.15.57:0/2969502459 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0004220 con 0x7ff1e4101260 2023-10-08T13:51:04.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1db7fe700 1 -- 172.21.15.57:0/2969502459 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1e0007640 con 0x7ff1e4101260 2023-10-08T13:51:04.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/2969502459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc009c40 con 0x7ff1e4101260 2023-10-08T13:51:04.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1db7fe700 1 -- 172.21.15.57:0/2969502459 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0005610 con 0x7ff1e4101260 2023-10-08T13:51:04.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.981+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/2969502459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e41a5b40 con 0x7ff1e4101260 2023-10-08T13:51:04.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.982+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/2969502459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e4101f30 con 0x7ff1e4101260 2023-10-08T13:51:04.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.983+0000 7ff1db7fe700 1 -- 172.21.15.57:0/2969502459 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7ff1e0004750 con 0x7ff1e4101260 2023-10-08T13:51:04.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.984+0000 7ff1db7fe700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1d00a15e0 0x7ff1d00a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:04.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.984+0000 7ff1ea385700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1d00a15e0 0x7ff1d00a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:04.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.984+0000 7ff1db7fe700 1 -- 172.21.15.57:0/2969502459 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7ff1e00cd650 con 0x7ff1e4101260 2023-10-08T13:51:04.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.985+0000 7ff1ea385700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1d00a15e0 0x7ff1d00a3aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4101930 tx=0x7ff1d4009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:04.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:04.988+0000 7ff1db7fe700 1 -- 172.21.15.57:0/2969502459 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff1e0096700 con 0x7ff1e4101260 2023-10-08T13:51:05.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.196+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/2969502459 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1e4110730 con 0x7ff1d00a15e0 2023-10-08T13:51:05.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.198+0000 7ff1db7fe700 1 -- 172.21.15.57:0/2969502459 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7ff1e4110730 con 0x7ff1d00a15e0 2023-10-08T13:51:05.199 INFO:teuthology.orchestra.run.smithi057.stdout:HOST ADDR LABELS STATUS 2023-10-08T13:51:05.199 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057 172.21.15.57 2023-10-08T13:51:05.199 INFO:teuthology.orchestra.run.smithi057.stdout:smithi099 172.21.15.99 2023-10-08T13:51:05.200 INFO:teuthology.orchestra.run.smithi057.stdout:smithi115 172.21.15.115 2023-10-08T13:51:05.200 INFO:teuthology.orchestra.run.smithi057.stdout:3 hosts in cluster 2023-10-08T13:51:05.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.200+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/2969502459 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1d00a15e0 msgr2=0x7ff1d00a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:05.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.200+0000 7ff1ec5e9700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1d00a15e0 0x7ff1d00a3aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4101930 tx=0x7ff1d4009310 comp rx=0 tx=0).stop 2023-10-08T13:51:05.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.200+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/2969502459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1e4101260 msgr2=0x7ff1e41994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:05.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.200+0000 7ff1ec5e9700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1e4101260 0x7ff1e41994e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000d980 tx=0x7ff1e000dc90 comp rx=0 tx=0).stop 2023-10-08T13:51:05.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.200+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/2969502459 shutdown_connections 2023-10-08T13:51:05.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.200+0000 7ff1ec5e9700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1e41057c0 0x7ff1e419d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:05.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.200+0000 7ff1ec5e9700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1d00a15e0 0x7ff1d00a3aa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:05.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.200+0000 7ff1ec5e9700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1e41008a0 0x7ff1e4198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.200+0000 7ff1ec5e9700 1 --2- 172.21.15.57:0/2969502459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1e4101260 0x7ff1e41994e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.200+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/2969502459 >> 172.21.15.57:0/2969502459 conn(0x7ff1e40779e0 msgr2=0x7ff1e410ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.201+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/2969502459 shutdown_connections 2023-10-08T13:51:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:05.201+0000 7ff1ec5e9700 1 -- 172.21.15.57:0/2969502459 wait complete. 2023-10-08T13:51:05.554 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-08T13:51:06.119 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:51:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:06 smithi099 ceph-mon[103849]: pgmap v283: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:06.309 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:06 smithi057 ceph-mon[99476]: pgmap v283: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:06.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:06 smithi115 ceph-mon[104408]: pgmap v283: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:07.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.200+0000 7f9f78174700 1 -- 172.21.15.57:0/2918359501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f701056b0 msgr2=0x7f9f70105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:07.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.200+0000 7f9f78174700 1 --2- 172.21.15.57:0/2918359501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f701056b0 0x7f9f70105a90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009990 tx=0x7f9f64009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.201+0000 7f9f78174700 1 -- 172.21.15.57:0/2918359501 shutdown_connections 2023-10-08T13:51:07.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.201+0000 7f9f78174700 1 --2- 172.21.15.57:0/2918359501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f70100110 0x7f9f7010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.201+0000 7f9f78174700 1 --2- 172.21.15.57:0/2918359501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f701056b0 0x7f9f70105a90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.201+0000 7f9f78174700 1 --2- 172.21.15.57:0/2918359501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f700ff750 0x7f9f700ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.201+0000 7f9f78174700 1 -- 172.21.15.57:0/2918359501 >> 172.21.15.57:0/2918359501 conn(0x7f9f70075790 msgr2=0x7f9f70075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.201+0000 7f9f78174700 1 -- 172.21.15.57:0/2918359501 shutdown_connections 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.201+0000 7f9f78174700 1 -- 172.21.15.57:0/2918359501 wait complete. 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.201+0000 7f9f78174700 1 Processor -- start 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.201+0000 7f9f78174700 1 -- start start 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f78174700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f700ff750 0x7f9f70197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f78174700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f70100110 0x7f9f70197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f78174700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f701056b0 0x7f9f701a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f78174700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f70198840 con 0x7f9f70100110 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f78174700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f701a3050 con 0x7f9f700ff750 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f78174700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f701a3190 con 0x7f9f701056b0 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f7570f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f70100110 0x7f9f70197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:07.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f7570f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f70100110 0x7f9f70197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55482/0 (socket says 172.21.15.57:55482) 2023-10-08T13:51:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f7570f700 1 -- 172.21.15.57:0/1990123085 learned_addr learned my addr 172.21.15.57:0/1990123085 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f75f10700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f700ff750 0x7f9f70197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f76711700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f701056b0 0x7f9f701a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f7570f700 1 -- 172.21.15.57:0/1990123085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f701056b0 msgr2=0x7f9f701a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f7570f700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f701056b0 0x7f9f701a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f7570f700 1 -- 172.21.15.57:0/1990123085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f700ff750 msgr2=0x7f9f70197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f7570f700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f700ff750 0x7f9f70197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f7570f700 1 -- 172.21.15.57:0/1990123085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640095f0 con 0x7f9f70100110 2023-10-08T13:51:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.202+0000 7f9f7570f700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f70100110 0x7f9f70197fd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c9b0 tx=0x7f9f6c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:07.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.203+0000 7f9f62ffd700 1 -- 172.21.15.57:0/1990123085 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c007b00 con 0x7f9f70100110 2023-10-08T13:51:07.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.203+0000 7f9f62ffd700 1 -- 172.21.15.57:0/1990123085 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f6c00ce50 con 0x7f9f70100110 2023-10-08T13:51:07.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.203+0000 7f9f78174700 1 -- 172.21.15.57:0/1990123085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f701a3420 con 0x7f9f70100110 2023-10-08T13:51:07.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.203+0000 7f9f62ffd700 1 -- 172.21.15.57:0/1990123085 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c018660 con 0x7f9f70100110 2023-10-08T13:51:07.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.203+0000 7f9f78174700 1 -- 172.21.15.57:0/1990123085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f701a3920 con 0x7f9f70100110 2023-10-08T13:51:07.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.204+0000 7f9f78174700 1 -- 172.21.15.57:0/1990123085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f70107cb0 con 0x7f9f70100110 2023-10-08T13:51:07.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.205+0000 7f9f62ffd700 1 -- 172.21.15.57:0/1990123085 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f9f6c00f450 con 0x7f9f70100110 2023-10-08T13:51:07.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.205+0000 7f9f62ffd700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9f5c0a16b0 0x7f9f5c0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:07.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.206+0000 7f9f75f10700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9f5c0a16b0 0x7f9f5c0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:07.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.206+0000 7f9f62ffd700 1 -- 172.21.15.57:0/1990123085 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f9f6c0cd0f0 con 0x7f9f70100110 2023-10-08T13:51:07.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.206+0000 7f9f75f10700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9f5c0a16b0 0x7f9f5c0a3b70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400b5c0 tx=0x7f9f64005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:07.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.210+0000 7f9f62ffd700 1 -- 172.21.15.57:0/1990123085 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9f6c096200 con 0x7f9f70100110 2023-10-08T13:51:07.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:07 smithi099 ceph-mon[103849]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:07.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.418+0000 7f9f78174700 1 -- 172.21.15.57:0/1990123085 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f7004e4c0 con 0x7f9f5c0a16b0 2023-10-08T13:51:07.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.420+0000 7f9f62ffd700 1 -- 172.21.15.57:0/1990123085 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f9f7004e4c0 con 0x7f9f5c0a16b0 2023-10-08T13:51:07.422 INFO:teuthology.orchestra.run.smithi057.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-08T13:51:07.422 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE7361049H4P0KGN 3726G No 2m ago Has a FileSystem, LVM detected 2023-10-08T13:51:07.422 INFO:teuthology.orchestra.run.smithi057.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-08T13:51:07.422 INFO:teuthology.orchestra.run.smithi057.stdout:smithi115 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UU400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-08T13:51:07.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.422+0000 7f9f78174700 1 -- 172.21.15.57:0/1990123085 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9f5c0a16b0 msgr2=0x7f9f5c0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:07.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.422+0000 7f9f78174700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9f5c0a16b0 0x7f9f5c0a3b70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400b5c0 tx=0x7f9f64005960 comp rx=0 tx=0).stop 2023-10-08T13:51:07.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.422+0000 7f9f78174700 1 -- 172.21.15.57:0/1990123085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f70100110 msgr2=0x7f9f70197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:07.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.422+0000 7f9f78174700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f70100110 0x7f9f70197fd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c9b0 tx=0x7f9f6c00ccc0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.422+0000 7f9f78174700 1 -- 172.21.15.57:0/1990123085 shutdown_connections 2023-10-08T13:51:07.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.422+0000 7f9f78174700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f701056b0 0x7f9f701a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.422+0000 7f9f78174700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9f5c0a16b0 0x7f9f5c0a3b70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.423+0000 7f9f78174700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f700ff750 0x7f9f70197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.423+0000 7f9f78174700 1 --2- 172.21.15.57:0/1990123085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f70100110 0x7f9f70197fd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:07.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.423+0000 7f9f78174700 1 -- 172.21.15.57:0/1990123085 >> 172.21.15.57:0/1990123085 conn(0x7f9f70075790 msgr2=0x7f9f700fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:07.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.423+0000 7f9f78174700 1 -- 172.21.15.57:0/1990123085 shutdown_connections 2023-10-08T13:51:07.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:07.423+0000 7f9f78174700 1 -- 172.21.15.57:0/1990123085 wait complete. 2023-10-08T13:51:07.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:07 smithi115 ceph-mon[104408]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:07.446 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:07 smithi057 ceph-mon[99476]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:07.774 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-08T13:51:07.783 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T13:51:07.783 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-08T13:51:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:08 smithi099 ceph-mon[103849]: pgmap v284: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-08T13:51:08.352 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:51:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:08 smithi115 ceph-mon[104408]: pgmap v284: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-08T13:51:08.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:08 smithi057 ceph-mon[99476]: pgmap v284: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-08T13:51:09.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:09 smithi057 ceph-mon[99476]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:09.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:09 smithi115 ceph-mon[104408]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:09.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:09 smithi099 ceph-mon[103849]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:09.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.549+0000 7f1dd2edb700 1 -- 172.21.15.57:0/1994262979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dcc1056b0 msgr2=0x7f1dcc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:09.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.549+0000 7f1dd2edb700 1 --2- 172.21.15.57:0/1994262979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dcc1056b0 0x7f1dcc105a90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009990 tx=0x7f1dc0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.549+0000 7f1dd2edb700 1 -- 172.21.15.57:0/1994262979 shutdown_connections 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.549+0000 7f1dd2edb700 1 --2- 172.21.15.57:0/1994262979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dcc100110 0x7f1dcc10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.549+0000 7f1dd2edb700 1 --2- 172.21.15.57:0/1994262979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dcc1056b0 0x7f1dcc105a90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.549+0000 7f1dd2edb700 1 --2- 172.21.15.57:0/1994262979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dcc0ff750 0x7f1dcc0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.549+0000 7f1dd2edb700 1 -- 172.21.15.57:0/1994262979 >> 172.21.15.57:0/1994262979 conn(0x7f1dcc075790 msgr2=0x7f1dcc075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.549+0000 7f1dd2edb700 1 -- 172.21.15.57:0/1994262979 shutdown_connections 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.549+0000 7f1dd2edb700 1 -- 172.21.15.57:0/1994262979 wait complete. 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd2edb700 1 Processor -- start 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd2edb700 1 -- start start 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd2edb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dcc0ff750 0x7f1dcc101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd2edb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dcc100110 0x7f1dcc1017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd2edb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dcc1056b0 0x7f1dcc101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd2edb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc1a4f90 con 0x7f1dcc1056b0 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd2edb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc1a5100 con 0x7f1dcc100110 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd2edb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc1a52c0 con 0x7f1dcc0ff750 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd1478700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dcc1056b0 0x7f1dcc101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd0c77700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dcc0ff750 0x7f1dcc101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd1478700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dcc1056b0 0x7f1dcc101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55496/0 (socket says 172.21.15.57:55496) 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.550+0000 7f1dd1478700 1 -- 172.21.15.57:0/2743863790 learned_addr learned my addr 172.21.15.57:0/2743863790 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dcbfff700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dcc100110 0x7f1dcc1017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dd1478700 1 -- 172.21.15.57:0/2743863790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dcc0ff750 msgr2=0x7f1dcc101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:09.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dd1478700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dcc0ff750 0x7f1dcc101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dd1478700 1 -- 172.21.15.57:0/2743863790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dcc100110 msgr2=0x7f1dcc1017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:09.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dd1478700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dcc100110 0x7f1dcc1017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dd1478700 1 -- 172.21.15.57:0/2743863790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00095f0 con 0x7f1dcc1056b0 2023-10-08T13:51:09.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dd1478700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dcc1056b0 0x7f1dcc101d00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400ef30 tx=0x7f1dc400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:09.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dc9ffb700 1 -- 172.21.15.57:0/2743863790 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4018070 con 0x7f1dcc1056b0 2023-10-08T13:51:09.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dc9ffb700 1 -- 172.21.15.57:0/2743863790 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc400f040 con 0x7f1dcc1056b0 2023-10-08T13:51:09.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dc9ffb700 1 -- 172.21.15.57:0/2743863790 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400cdb0 con 0x7f1dcc1056b0 2023-10-08T13:51:09.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dd2edb700 1 -- 172.21.15.57:0/2743863790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc1a5540 con 0x7f1dcc1056b0 2023-10-08T13:51:09.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.551+0000 7f1dd2edb700 1 -- 172.21.15.57:0/2743863790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc1a59f0 con 0x7f1dcc1056b0 2023-10-08T13:51:09.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.552+0000 7f1dd2edb700 1 -- 172.21.15.57:0/2743863790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc108cc0 con 0x7f1dcc1056b0 2023-10-08T13:51:09.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.553+0000 7f1dc9ffb700 1 -- 172.21.15.57:0/2743863790 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1dc4003a00 con 0x7f1dcc1056b0 2023-10-08T13:51:09.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.554+0000 7f1dc9ffb700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1dbc0a15e0 0x7f1dbc0a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:09.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.554+0000 7f1dd0c77700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1dbc0a15e0 0x7f1dbc0a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:09.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.554+0000 7f1dc9ffb700 1 -- 172.21.15.57:0/2743863790 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f1dc40cce30 con 0x7f1dcc1056b0 2023-10-08T13:51:09.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.555+0000 7f1dd0c77700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1dbc0a15e0 0x7f1dbc0a3aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000b2e0 tx=0x7f1dc0005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:09.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.559+0000 7f1dc9ffb700 1 -- 172.21.15.57:0/2743863790 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1dc4095f70 con 0x7f1dcc1056b0 2023-10-08T13:51:09.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.803+0000 7f1dd2edb700 1 -- 172.21.15.57:0/2743863790 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1dcc102800 con 0x7f1dcc1056b0 2023-10-08T13:51:09.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.804+0000 7f1dc9ffb700 1 -- 172.21.15.57:0/2743863790 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2967 (secure 0 0 0) 0x7f1dc4099590 con 0x7f1dcc1056b0 2023-10-08T13:51:09.805 INFO:teuthology.orchestra.run.smithi057.stdout:e27 2023-10-08T13:51:09.806 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-08T13:51:09.806 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T13:51:09.806 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2023-10-08T13:51:09.806 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:09.806 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2023-10-08T13:51:09.806 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2023-10-08T13:51:09.806 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 27 2023-10-08T13:51:09.806 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-08T13:51:09.806 INFO:teuthology.orchestra.run.smithi057.stdout:created 2023-10-08T13:49:44.186896+0000 2023-10-08T13:51:09.806 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2023-10-08T13:50:07.719573+0000 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 300 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 0 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T13:51:09.807 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 5 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1,2,3,4 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=24662,1=24652,2=24634,3=24622,4=24617} 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3,4] 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 0 2023-10-08T13:51:09.808 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.b{0:24662} state up:active seq 3 addr [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.l{0:24589} state up:standby-replay seq 1 addr [v2:172.21.15.115:6832/3794414355,v1:172.21.15.115:6833/3794414355] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.i{1:24652} state up:active seq 10 addr [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.e{1:24581} state up:standby-replay seq 1 addr [v2:172.21.15.99:6834/2732351731,v1:172.21.15.99:6835/2732351731] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.c{2:24634} state up:active seq 11 addr [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.d{2:24572} state up:standby-replay seq 1 addr [v2:172.21.15.57:6836/4079715365,v1:172.21.15.57:6837/4079715365] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.f{3:24622} state up:active seq 12 addr [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.k{3:24557} state up:standby-replay seq 1 addr [v2:172.21.15.99:6832/2949059139,v1:172.21.15.99:6833/2949059139] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.h{4:24617} state up:active seq 11 addr [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.a{4:14790} state up:standby-replay seq 1 addr [v2:172.21.15.57:6840/1847373212,v1:172.21.15.57:6841/1847373212] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:09.809 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2023-10-08T13:51:09.810 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:09.810 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.j{-1:14718} state up:standby seq 1 addr [v2:172.21.15.57:6834/2801838933,v1:172.21.15.57:6835/2801838933] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.810 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.g{-1:14778} state up:standby seq 1 addr [v2:172.21.15.57:6838/1513163924,v1:172.21.15.57:6839/1513163924] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T13:51:09.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 -- 172.21.15.57:0/2743863790 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1dbc0a15e0 msgr2=0x7f1dbc0a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1dbc0a15e0 0x7f1dbc0a3aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000b2e0 tx=0x7f1dc0005960 comp rx=0 tx=0).stop 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 -- 172.21.15.57:0/2743863790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dcc1056b0 msgr2=0x7f1dcc101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dcc1056b0 0x7f1dcc101d00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400ef30 tx=0x7f1dc400c3a0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 -- 172.21.15.57:0/2743863790 shutdown_connections 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1dcc1056b0 0x7f1dcc101d00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dcc0ff750 0x7f1dcc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1dbc0a15e0 0x7f1dbc0a3aa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 --2- 172.21.15.57:0/2743863790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dcc100110 0x7f1dcc1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 -- 172.21.15.57:0/2743863790 >> 172.21.15.57:0/2743863790 conn(0x7f1dcc075790 msgr2=0x7f1dcc0684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 -- 172.21.15.57:0/2743863790 shutdown_connections 2023-10-08T13:51:09.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:09.806+0000 7f1dd2edb700 1 -- 172.21.15.57:0/2743863790 wait complete. 2023-10-08T13:51:09.812 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:51:10.167 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:10 smithi057 ceph-mon[99476]: pgmap v285: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:10.167 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:10 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2743863790' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-08T13:51:10.218 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-08T13:51:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:10 smithi115 ceph-mon[104408]: pgmap v285: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:10 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2743863790' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-08T13:51:10.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:10 smithi099 ceph-mon[103849]: pgmap v285: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:10.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:10 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2743863790' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-08T13:51:10.783 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:51:11.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:11 smithi057 ceph-mon[99476]: pgmap v286: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T13:51:11.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:11 smithi115 ceph-mon[104408]: pgmap v286: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T13:51:11.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:11 smithi099 ceph-mon[103849]: pgmap v286: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T13:51:11.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.953+0000 7f330173e700 1 -- 172.21.15.57:0/2016469556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1045b0 msgr2=0x7f32fc104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:11.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.953+0000 7f330173e700 1 --2- 172.21.15.57:0/2016469556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1045b0 0x7f32fc104990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009990 tx=0x7f32ec009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:11.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.953+0000 7f330173e700 1 -- 172.21.15.57:0/2016469556 shutdown_connections 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.953+0000 7f330173e700 1 --2- 172.21.15.57:0/2016469556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32fc1000a0 0x7f32fc10c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.953+0000 7f330173e700 1 --2- 172.21.15.57:0/2016469556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1045b0 0x7f32fc104990 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.953+0000 7f330173e700 1 --2- 172.21.15.57:0/2016469556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32fc0ff6e0 0x7f32fc0ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.953+0000 7f330173e700 1 -- 172.21.15.57:0/2016469556 >> 172.21.15.57:0/2016469556 conn(0x7f32fc0766c0 msgr2=0x7f32fc076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.953+0000 7f330173e700 1 -- 172.21.15.57:0/2016469556 shutdown_connections 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.953+0000 7f330173e700 1 -- 172.21.15.57:0/2016469556 wait complete. 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f330173e700 1 Processor -- start 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f330173e700 1 -- start start 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f330173e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32fc0ff6e0 0x7f32fc197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f330173e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1000a0 0x7f32fc197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:11.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f330173e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32fc1045b0 0x7f32fc1a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f330173e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc198800 con 0x7f32fc1000a0 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f330173e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc1a3030 con 0x7f32fc1045b0 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f330173e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc1a3170 con 0x7f32fc0ff6e0 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f32fa7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1000a0 0x7f32fc197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f32fa7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1000a0 0x7f32fc197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55508/0 (socket says 172.21.15.57:55508) 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.954+0000 7f32fa7fc700 1 -- 172.21.15.57:0/2772486500 learned_addr learned my addr 172.21.15.57:0/2772486500 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32faffd700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32fc0ff6e0 0x7f32fc197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32fb7fe700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32fc1045b0 0x7f32fc1a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32fa7fc700 1 -- 172.21.15.57:0/2772486500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32fc0ff6e0 msgr2=0x7f32fc197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32fa7fc700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32fc0ff6e0 0x7f32fc197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32fa7fc700 1 -- 172.21.15.57:0/2772486500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32fc1045b0 msgr2=0x7f32fc1a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:11.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32fa7fc700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32fc1045b0 0x7f32fc1a2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:11.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32fa7fc700 1 -- 172.21.15.57:0/2772486500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec0095f0 con 0x7f32fc1000a0 2023-10-08T13:51:11.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32fa7fc700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1000a0 0x7f32fc197fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f32e400c950 tx=0x7f32e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:11.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32f3fff700 1 -- 172.21.15.57:0/2772486500 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e400cea0 con 0x7f32fc1000a0 2023-10-08T13:51:11.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f330173e700 1 -- 172.21.15.57:0/2772486500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc1a33d0 con 0x7f32fc1000a0 2023-10-08T13:51:11.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32f3fff700 1 -- 172.21.15.57:0/2772486500 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32e4004830 con 0x7f32fc1000a0 2023-10-08T13:51:11.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f32f3fff700 1 -- 172.21.15.57:0/2772486500 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4005460 con 0x7f32fc1000a0 2023-10-08T13:51:11.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.955+0000 7f330173e700 1 -- 172.21.15.57:0/2772486500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc1a3920 con 0x7f32fc1000a0 2023-10-08T13:51:11.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.956+0000 7f330173e700 1 -- 172.21.15.57:0/2772486500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc107c20 con 0x7f32fc1000a0 2023-10-08T13:51:11.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.957+0000 7f32f3fff700 1 -- 172.21.15.57:0/2772486500 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f32e4007ac0 con 0x7f32fc1000a0 2023-10-08T13:51:11.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.958+0000 7f32f3fff700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32e80a1560 0x7f32e80a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:11.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.958+0000 7f32faffd700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32e80a1560 0x7f32e80a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:11.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.958+0000 7f32f3fff700 1 -- 172.21.15.57:0/2772486500 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f32e40ccfa0 con 0x7f32fc1000a0 2023-10-08T13:51:11.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.958+0000 7f32faffd700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32e80a1560 0x7f32e80a3a20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b5c0 tx=0x7f32ec005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:11.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:11.962+0000 7f32f3fff700 1 -- 172.21.15.57:0/2772486500 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f32e4096050 con 0x7f32fc1000a0 2023-10-08T13:51:12.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.163+0000 7f330173e700 1 -- 172.21.15.57:0/2772486500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f32fc069000 con 0x7f32fc1000a0 2023-10-08T13:51:12.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.164+0000 7f32f3fff700 1 -- 172.21.15.57:0/2772486500 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) v1 ==== 56+0+5969 (secure 0 0 0) 0x7f32e4099670 con 0x7f32fc1000a0 2023-10-08T13:51:12.164 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 90 2023-10-08T13:51:12.164 INFO:teuthology.orchestra.run.smithi057.stdout:fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:created 2023-10-08T13:42:06.689196+0000 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2023-10-08T13:50:15.770421+0000 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:crush_version 27 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:full_ratio 0.95 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:backfillfull_ratio 0.9 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:nearfull_ratio 0.85 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:require_min_compat_client luminous 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:min_compat_client luminous 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:require_osd_release reef 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:stretch_mode_enabled false 2023-10-08T13:51:12.165 INFO:teuthology.orchestra.run.smithi057.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:max_osd 12 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 up in weight 1 up_from 9 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6802/1388588372,v1:172.21.15.57:6803/1388588372] [v2:172.21.15.57:6804/1388588372,v1:172.21.15.57:6805/1388588372] exists,up 82271e50-2757-4c42-a3ca-a105fbaa1391 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 up in weight 1 up_from 14 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/4047402511,v1:172.21.15.99:6801/4047402511] [v2:172.21.15.99:6802/4047402511,v1:172.21.15.99:6803/4047402511] exists,up ef769aca-5124-4f13-931f-2520cbbe2144 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 up in weight 1 up_from 19 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/3640540626,v1:172.21.15.115:6801/3640540626] [v2:172.21.15.115:6802/3640540626,v1:172.21.15.115:6803/3640540626] exists,up d8c72122-dbc2-4a3b-9ee3-6c83f989107a 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6810/966286851,v1:172.21.15.57:6811/966286851] [v2:172.21.15.57:6812/966286851,v1:172.21.15.57:6813/966286851] exists,up c969ed29-1e01-479b-ab6f-00f8b0dde361 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 up in weight 1 up_from 33 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/1502873104,v1:172.21.15.99:6809/1502873104] [v2:172.21.15.99:6810/1502873104,v1:172.21.15.99:6811/1502873104] exists,up 842feff0-faab-4c13-88b1-91355f53e0e2 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 up in weight 1 up_from 38 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/2071180529,v1:172.21.15.115:6809/2071180529] [v2:172.21.15.115:6810/2071180529,v1:172.21.15.115:6811/2071180529] exists,up 97f91a58-677f-4005-8a2d-66113c9cdda6 2023-10-08T13:51:12.166 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 up in weight 1 up_from 45 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6818/3636365210,v1:172.21.15.57:6819/3636365210] [v2:172.21.15.57:6820/3636365210,v1:172.21.15.57:6821/3636365210] exists,up fd2240ad-a8f2-49ed-bf62-8fe8eff60156 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 up in weight 1 up_from 50 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6816/4070534570,v1:172.21.15.99:6817/4070534570] [v2:172.21.15.99:6818/4070534570,v1:172.21.15.99:6819/4070534570] exists,up 8bc31070-8d5a-4a0c-b14f-5fa855e951de 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:osd.8 up in weight 1 up_from 58 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/3107603962,v1:172.21.15.115:6817/3107603962] [v2:172.21.15.115:6818/3107603962,v1:172.21.15.115:6819/3107603962] exists,up 61534853-3ba8-40cc-b62e-7b1c2efdb996 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:osd.9 up in weight 1 up_from 66 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6826/1074674274,v1:172.21.15.57:6827/1074674274] [v2:172.21.15.57:6828/1074674274,v1:172.21.15.57:6829/1074674274] exists,up 28533ac1-c3f9-4587-93d0-752be200eeda 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:osd.10 up in weight 1 up_from 71 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/317517288,v1:172.21.15.99:6825/317517288] [v2:172.21.15.99:6826/317517288,v1:172.21.15.99:6827/317517288] exists,up 15226d7f-8a7d-45da-8750-adaa98217259 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:osd.11 up in weight 1 up_from 79 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/901134235,v1:172.21.15.115:6825/901134235] [v2:172.21.15.115:6826/901134235,v1:172.21.15.115:6827/901134235] exists,up bd75deff-374f-4629-ba93-d7b9e20882a6 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.e [2,4] 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.f [8,11] 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.1b [2,1] 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.1c [8,11] 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.1b [4,8] 2023-10-08T13:51:12.167 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.2c [4,9] 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.f [9,8] 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.12 [5,10] 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.17 [9,8] 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.1b [9,10] 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/466175178 expires 2023-10-09T13:44:14.025065+0000 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/1455537828 expires 2023-10-09T13:44:14.025065+0000 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6801/2534706720 expires 2023-10-09T13:44:14.025065+0000 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6800/2534706720 expires 2023-10-09T13:44:14.025065+0000 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/1379042066 expires 2023-10-09T13:44:14.025065+0000 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/19108071 expires 2023-10-09T13:43:06.093214+0000 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/3114287016 expires 2023-10-09T13:43:06.093214+0000 2023-10-08T13:51:12.168 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/3318653521 expires 2023-10-09T13:43:06.093214+0000 2023-10-08T13:51:12.169 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/2210095932 expires 2023-10-09T13:42:39.884205+0000 2023-10-08T13:51:12.169 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/955690770 expires 2023-10-09T13:42:39.884205+0000 2023-10-08T13:51:12.169 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6801/140726743 expires 2023-10-09T13:42:39.884205+0000 2023-10-08T13:51:12.169 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6801/2406655290 expires 2023-10-09T13:43:06.093214+0000 2023-10-08T13:51:12.169 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/2229924806 expires 2023-10-09T13:42:39.884205+0000 2023-10-08T13:51:12.169 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6800/140726743 expires 2023-10-09T13:42:39.884205+0000 2023-10-08T13:51:12.169 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6800/2406655290 expires 2023-10-09T13:43:06.093214+0000 2023-10-08T13:51:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 -- 172.21.15.57:0/2772486500 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32e80a1560 msgr2=0x7f32e80a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32e80a1560 0x7f32e80a3a20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b5c0 tx=0x7f32ec005960 comp rx=0 tx=0).stop 2023-10-08T13:51:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 -- 172.21.15.57:0/2772486500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1000a0 msgr2=0x7f32fc197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1000a0 0x7f32fc197fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f32e400c950 tx=0x7f32e400cc60 comp rx=0 tx=0).stop 2023-10-08T13:51:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 -- 172.21.15.57:0/2772486500 shutdown_connections 2023-10-08T13:51:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f32e80a1560 0x7f32e80a3a20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32fc1045b0 0x7f32fc1a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32fc0ff6e0 0x7f32fc197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 --2- 172.21.15.57:0/2772486500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32fc1000a0 0x7f32fc197fb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 -- 172.21.15.57:0/2772486500 >> 172.21.15.57:0/2772486500 conn(0x7f32fc0766c0 msgr2=0x7f32fc0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:12.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 -- 172.21.15.57:0/2772486500 shutdown_connections 2023-10-08T13:51:12.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.166+0000 7f330173e700 1 -- 172.21.15.57:0/2772486500 wait complete. 2023-10-08T13:51:12.463 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:12 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2772486500' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-08T13:51:12.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:12 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2772486500' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-08T13:51:12.538 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-08T13:51:12.550 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:51:12.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.680+0000 7f57d33a2700 1 -- 172.21.15.57:0/750954526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57cc0fff60 msgr2=0x7f57cc10c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:12.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.680+0000 7f57d33a2700 1 --2- 172.21.15.57:0/750954526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57cc0fff60 0x7f57cc10c420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f57c400b0d0 tx=0x7f57c400b3e0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.680+0000 7f57d33a2700 1 -- 172.21.15.57:0/750954526 shutdown_connections 2023-10-08T13:51:12.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.680+0000 7f57d33a2700 1 --2- 172.21.15.57:0/750954526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57cc0fff60 0x7f57cc10c420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.680+0000 7f57d33a2700 1 --2- 172.21.15.57:0/750954526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57cc105660 0x7f57cc105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 --2- 172.21.15.57:0/750954526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57cc0ff5a0 0x7f57cc0ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 -- 172.21.15.57:0/750954526 >> 172.21.15.57:0/750954526 conn(0x7f57cc0fb070 msgr2=0x7f57cc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:12.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 -- 172.21.15.57:0/750954526 shutdown_connections 2023-10-08T13:51:12.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 -- 172.21.15.57:0/750954526 wait complete. 2023-10-08T13:51:12.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 Processor -- start 2023-10-08T13:51:12.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 -- start start 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57cc0ff5a0 0x7f57cc101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57cc0fff60 0x7f57cc101750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57cc105660 0x7f57cc101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a4fb0 con 0x7f57cc0fff60 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a5120 con 0x7f57cc105660 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.681+0000 7f57d33a2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a52e0 con 0x7f57cc0ff5a0 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d1b9f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57cc0fff60 0x7f57cc101750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d1b9f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57cc0fff60 0x7f57cc101750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55522/0 (socket says 172.21.15.57:55522) 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d1b9f700 1 -- 172.21.15.57:0/1753298282 learned_addr learned my addr 172.21.15.57:0/1753298282 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d23a0700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57cc0ff5a0 0x7f57cc101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d2ba1700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57cc105660 0x7f57cc101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:12.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d1b9f700 1 -- 172.21.15.57:0/1753298282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57cc0ff5a0 msgr2=0x7f57cc101210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:12.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d1b9f700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57cc0ff5a0 0x7f57cc101210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d1b9f700 1 -- 172.21.15.57:0/1753298282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57cc105660 msgr2=0x7f57cc101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:12.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d1b9f700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57cc105660 0x7f57cc101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d1b9f700 1 -- 172.21.15.57:0/1753298282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4009d00 con 0x7f57cc0fff60 2023-10-08T13:51:12.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d1b9f700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57cc0fff60 0x7f57cc101750 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f57c800e9a0 tx=0x7f57c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:12.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57c37fe700 1 -- 172.21.15.57:0/1753298282 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8009790 con 0x7f57cc0fff60 2023-10-08T13:51:12.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57c37fe700 1 -- 172.21.15.57:0/1753298282 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c8010430 con 0x7f57cc0fff60 2023-10-08T13:51:12.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d33a2700 1 -- 172.21.15.57:0/1753298282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc1a55c0 con 0x7f57cc0fff60 2023-10-08T13:51:12.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57c37fe700 1 -- 172.21.15.57:0/1753298282 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8018680 con 0x7f57cc0fff60 2023-10-08T13:51:12.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.682+0000 7f57d33a2700 1 -- 172.21.15.57:0/1753298282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc1a5b20 con 0x7f57cc0fff60 2023-10-08T13:51:12.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.683+0000 7f57d33a2700 1 -- 172.21.15.57:0/1753298282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc100cc0 con 0x7f57cc0fff60 2023-10-08T13:51:12.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.685+0000 7f57c37fe700 1 -- 172.21.15.57:0/1753298282 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f57c80105a0 con 0x7f57cc0fff60 2023-10-08T13:51:12.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.685+0000 7f57c37fe700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57b80a15e0 0x7f57b80a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:12.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.685+0000 7f57c37fe700 1 -- 172.21.15.57:0/1753298282 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f57c8014070 con 0x7f57cc0fff60 2023-10-08T13:51:12.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.685+0000 7f57d23a0700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57b80a15e0 0x7f57b80a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:12.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.686+0000 7f57d23a0700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57b80a15e0 0x7f57b80a3aa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f57bc005950 tx=0x7f57bc00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:12.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:12 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2772486500' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-08T13:51:12.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.688+0000 7f57c37fe700 1 -- 172.21.15.57:0/1753298282 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f57c8096e00 con 0x7f57cc0fff60 2023-10-08T13:51:12.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.894+0000 7f57d33a2700 1 -- 172.21.15.57:0/1753298282 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57cc066b50 con 0x7f57cc0fff60 2023-10-08T13:51:12.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.895+0000 7f57c37fe700 1 -- 172.21.15.57:0/1753298282 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24753 (secure 0 0 0) 0x7f57c809a420 con 0x7f57cc0fff60 2023-10-08T13:51:12.896 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:12.896 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:50:15.770421+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:51:12.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 -- 172.21.15.57:0/1753298282 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57b80a15e0 msgr2=0x7f57b80a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:12.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57b80a15e0 0x7f57b80a3aa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f57bc005950 tx=0x7f57bc00b410 comp rx=0 tx=0).stop 2023-10-08T13:51:12.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 -- 172.21.15.57:0/1753298282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57cc0fff60 msgr2=0x7f57cc101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57cc0fff60 0x7f57cc101750 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f57c800e9a0 tx=0x7f57c800ecb0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 -- 172.21.15.57:0/1753298282 shutdown_connections 2023-10-08T13:51:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57b80a15e0 0x7f57b80a3aa0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57cc105660 0x7f57cc101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57cc0ff5a0 0x7f57cc101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 --2- 172.21.15.57:0/1753298282 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57cc0fff60 0x7f57cc101750 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 -- 172.21.15.57:0/1753298282 >> 172.21.15.57:0/1753298282 conn(0x7f57cc0fb070 msgr2=0x7f57cc0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 -- 172.21.15.57:0/1753298282 shutdown_connections 2023-10-08T13:51:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:12.898+0000 7f57d33a2700 1 -- 172.21.15.57:0/1753298282 wait complete. 2023-10-08T13:51:12.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T13:51:12.911 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:51:13.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.041+0000 7f8e9e242700 1 -- 172.21.15.57:0/1496153599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e98067ce0 msgr2=0x7f8e980680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.041+0000 7f8e9e242700 1 --2- 172.21.15.57:0/1496153599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e98067ce0 0x7f8e980680c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8e880099c0 tx=0x7f8e88009cd0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.041+0000 7f8e9e242700 1 -- 172.21.15.57:0/1496153599 shutdown_connections 2023-10-08T13:51:13.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.041+0000 7f8e9e242700 1 --2- 172.21.15.57:0/1496153599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e98105b80 0x7f8e9810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.041+0000 7f8e9e242700 1 --2- 172.21.15.57:0/1496153599 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e98067ce0 0x7f8e980680c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.041+0000 7f8e9e242700 1 --2- 172.21.15.57:0/1496153599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e981051c0 0x7f8e98105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.041+0000 7f8e9e242700 1 -- 172.21.15.57:0/1496153599 >> 172.21.15.57:0/1496153599 conn(0x7f8e98076780 msgr2=0x7f8e98076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:13.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.041+0000 7f8e9e242700 1 -- 172.21.15.57:0/1496153599 shutdown_connections 2023-10-08T13:51:13.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.041+0000 7f8e9e242700 1 -- 172.21.15.57:0/1496153599 wait complete. 2023-10-08T13:51:13.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e9e242700 1 Processor -- start 2023-10-08T13:51:13.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e9e242700 1 -- start start 2023-10-08T13:51:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e9e242700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e98067ce0 0x7f8e980ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e9e242700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e981051c0 0x7f8e980ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e9e242700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e98105b80 0x7f8e98100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e9e242700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e981a4fd0 con 0x7f8e98067ce0 2023-10-08T13:51:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e9e242700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e981a5140 con 0x7f8e981051c0 2023-10-08T13:51:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e9e242700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e981a5300 con 0x7f8e98105b80 2023-10-08T13:51:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e977fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e98067ce0 0x7f8e980ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e977fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e98067ce0 0x7f8e980ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55534/0 (socket says 172.21.15.57:55534) 2023-10-08T13:51:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.042+0000 7f8e977fe700 1 -- 172.21.15.57:0/2010938666 learned_addr learned my addr 172.21.15.57:0/2010938666 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:13.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e97fff700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e98105b80 0x7f8e98100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e96ffd700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e981051c0 0x7f8e980ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e977fe700 1 -- 172.21.15.57:0/2010938666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e98105b80 msgr2=0x7f8e98100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e977fe700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e98105b80 0x7f8e98100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e977fe700 1 -- 172.21.15.57:0/2010938666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e981051c0 msgr2=0x7f8e980ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e977fe700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e981051c0 0x7f8e980ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e977fe700 1 -- 172.21.15.57:0/2010938666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e880095f0 con 0x7f8e98067ce0 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e977fe700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e98067ce0 0x7f8e980ff6b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009990 tx=0x7f8e88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e94ff9700 1 -- 172.21.15.57:0/2010938666 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8801d070 con 0x7f8e98067ce0 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e94ff9700 1 -- 172.21.15.57:0/2010938666 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e8800fb30 con 0x7f8e98067ce0 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e9e242700 1 -- 172.21.15.57:0/2010938666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e981a5580 con 0x7f8e98067ce0 2023-10-08T13:51:13.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e94ff9700 1 -- 172.21.15.57:0/2010938666 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88003ee0 con 0x7f8e98067ce0 2023-10-08T13:51:13.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.043+0000 7f8e9e242700 1 -- 172.21.15.57:0/2010938666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e981a59a0 con 0x7f8e98067ce0 2023-10-08T13:51:13.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.044+0000 7f8e9e242700 1 -- 172.21.15.57:0/2010938666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e98108c70 con 0x7f8e98067ce0 2023-10-08T13:51:13.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.045+0000 7f8e94ff9700 1 -- 172.21.15.57:0/2010938666 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f8e8800fca0 con 0x7f8e98067ce0 2023-10-08T13:51:13.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.046+0000 7f8e94ff9700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8e840a1560 0x7f8e840a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.046+0000 7f8e94ff9700 1 -- 172.21.15.57:0/2010938666 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f8e880ce350 con 0x7f8e98067ce0 2023-10-08T13:51:13.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.046+0000 7f8e96ffd700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8e840a1560 0x7f8e840a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.046+0000 7f8e96ffd700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8e840a1560 0x7f8e840a3a20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e80005950 tx=0x7f8e800058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:13.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.049+0000 7f8e94ff9700 1 -- 172.21.15.57:0/2010938666 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8e880974f0 con 0x7f8e98067ce0 2023-10-08T13:51:13.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.248+0000 7f8e9e242700 1 -- 172.21.15.57:0/2010938666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e98100b90 con 0x7f8e98067ce0 2023-10-08T13:51:13.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.249+0000 7f8e94ff9700 1 -- 172.21.15.57:0/2010938666 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e880260a0 con 0x7f8e98067ce0 2023-10-08T13:51:13.249 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:51:13.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.251+0000 7f8e9e242700 1 -- 172.21.15.57:0/2010938666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8e840a1560 msgr2=0x7f8e840a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.251+0000 7f8e9e242700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8e840a1560 0x7f8e840a3a20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e80005950 tx=0x7f8e800058e0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.251+0000 7f8e9e242700 1 -- 172.21.15.57:0/2010938666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e98067ce0 msgr2=0x7f8e980ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.251+0000 7f8e9e242700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e98067ce0 0x7f8e980ff6b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009990 tx=0x7f8e88004730 comp rx=0 tx=0).stop 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.251+0000 7f8e9e242700 1 -- 172.21.15.57:0/2010938666 shutdown_connections 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.251+0000 7f8e9e242700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e98105b80 0x7f8e98100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.251+0000 7f8e9e242700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e98067ce0 0x7f8e980ff6b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.251+0000 7f8e9e242700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8e840a1560 0x7f8e840a3a20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.251+0000 7f8e9e242700 1 --2- 172.21.15.57:0/2010938666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e981051c0 0x7f8e980ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.251+0000 7f8e9e242700 1 -- 172.21.15.57:0/2010938666 >> 172.21.15.57:0/2010938666 conn(0x7f8e98076780 msgr2=0x7f8e980fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.252+0000 7f8e9e242700 1 -- 172.21.15.57:0/2010938666 shutdown_connections 2023-10-08T13:51:13.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.252+0000 7f8e9e242700 1 -- 172.21.15.57:0/2010938666 wait complete. 2023-10-08T13:51:13.264 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T13:51:13.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.394+0000 7fca2973d700 1 -- 172.21.15.57:0/2154269354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca241056b0 msgr2=0x7fca24105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.394+0000 7fca2973d700 1 --2- 172.21.15.57:0/2154269354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca241056b0 0x7fca24105a90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fca14009990 tx=0x7fca14009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.395+0000 7fca2973d700 1 -- 172.21.15.57:0/2154269354 shutdown_connections 2023-10-08T13:51:13.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.395+0000 7fca2973d700 1 --2- 172.21.15.57:0/2154269354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca240fffd0 0x7fca2410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.395+0000 7fca2973d700 1 --2- 172.21.15.57:0/2154269354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca241056b0 0x7fca24105a90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.395+0000 7fca2973d700 1 --2- 172.21.15.57:0/2154269354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca240ff610 0x7fca240ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.395+0000 7fca2973d700 1 -- 172.21.15.57:0/2154269354 >> 172.21.15.57:0/2154269354 conn(0x7fca24076820 msgr2=0x7fca24076c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:13.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.395+0000 7fca2973d700 1 -- 172.21.15.57:0/2154269354 shutdown_connections 2023-10-08T13:51:13.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.395+0000 7fca2973d700 1 -- 172.21.15.57:0/2154269354 wait complete. 2023-10-08T13:51:13.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.395+0000 7fca2973d700 1 Processor -- start 2023-10-08T13:51:13.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.395+0000 7fca2973d700 1 -- start start 2023-10-08T13:51:13.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca2973d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca240ff610 0x7fca241979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca2973d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca240fffd0 0x7fca24197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca2973d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca241056b0 0x7fca241a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca2973d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca24198740 con 0x7fca241056b0 2023-10-08T13:51:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca2973d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca241a2f70 con 0x7fca240fffd0 2023-10-08T13:51:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca2973d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca241a30b0 con 0x7fca240ff610 2023-10-08T13:51:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca237fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca241056b0 0x7fca241a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca237fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca241056b0 0x7fca241a2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55552/0 (socket says 172.21.15.57:55552) 2023-10-08T13:51:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca237fe700 1 -- 172.21.15.57:0/2270405896 learned_addr learned my addr 172.21.15.57:0/2270405896 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca227fc700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca240fffd0 0x7fca24197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca22ffd700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca240ff610 0x7fca241979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca237fe700 1 -- 172.21.15.57:0/2270405896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca240ff610 msgr2=0x7fca241979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca237fe700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca240ff610 0x7fca241979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca237fe700 1 -- 172.21.15.57:0/2270405896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca240fffd0 msgr2=0x7fca24197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca237fe700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca240fffd0 0x7fca24197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca237fe700 1 -- 172.21.15.57:0/2270405896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca140095f0 con 0x7fca241056b0 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca237fe700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca241056b0 0x7fca241a2a30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fca1c007f10 tx=0x7fca1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca0bfff700 1 -- 172.21.15.57:0/2270405896 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c00ddd0 con 0x7fca241056b0 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca2973d700 1 -- 172.21.15.57:0/2270405896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca241a3310 con 0x7fca241056b0 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca0bfff700 1 -- 172.21.15.57:0/2270405896 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca1c00f040 con 0x7fca241056b0 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca0bfff700 1 -- 172.21.15.57:0/2270405896 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c015400 con 0x7fca241056b0 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.396+0000 7fca2973d700 1 -- 172.21.15.57:0/2270405896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca241a37d0 con 0x7fca241056b0 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.397+0000 7fca2973d700 1 -- 172.21.15.57:0/2270405896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca24107c10 con 0x7fca241056b0 2023-10-08T13:51:13.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.399+0000 7fca0bfff700 1 -- 172.21.15.57:0/2270405896 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fca1c004000 con 0x7fca241056b0 2023-10-08T13:51:13.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.399+0000 7fca0bfff700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca0c0a1560 0x7fca0c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.399+0000 7fca0bfff700 1 -- 172.21.15.57:0/2270405896 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fca1c0cd3a0 con 0x7fca241056b0 2023-10-08T13:51:13.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.399+0000 7fca22ffd700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca0c0a1560 0x7fca0c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.400+0000 7fca22ffd700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca0c0a1560 0x7fca0c0a3a20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fca14006010 tx=0x7fca1401a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:13.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.403+0000 7fca0bfff700 1 -- 172.21.15.57:0/2270405896 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fca1c096550 con 0x7fca241056b0 2023-10-08T13:51:13.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.604+0000 7fca2973d700 1 -- 172.21.15.57:0/2270405896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca240692d0 con 0x7fca241056b0 2023-10-08T13:51:13.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.605+0000 7fca0bfff700 1 -- 172.21.15.57:0/2270405896 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca1c099b70 con 0x7fca241056b0 2023-10-08T13:51:13.605 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:51:13.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 -- 172.21.15.57:0/2270405896 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca0c0a1560 msgr2=0x7fca0c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca0c0a1560 0x7fca0c0a3a20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fca14006010 tx=0x7fca1401a040 comp rx=0 tx=0).stop 2023-10-08T13:51:13.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 -- 172.21.15.57:0/2270405896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca241056b0 msgr2=0x7fca241a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca241056b0 0x7fca241a2a30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fca1c007f10 tx=0x7fca1c00d3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 -- 172.21.15.57:0/2270405896 shutdown_connections 2023-10-08T13:51:13.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca241056b0 0x7fca241a2a30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca240ff610 0x7fca241979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca0c0a1560 0x7fca0c0a3a20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 --2- 172.21.15.57:0/2270405896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca240fffd0 0x7fca24197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 -- 172.21.15.57:0/2270405896 >> 172.21.15.57:0/2270405896 conn(0x7fca24076820 msgr2=0x7fca240fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:13.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 -- 172.21.15.57:0/2270405896 shutdown_connections 2023-10-08T13:51:13.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.607+0000 7fca2973d700 1 -- 172.21.15.57:0/2270405896 wait complete. 2023-10-08T13:51:13.620 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T13:51:13.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.749+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2663981594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0ff620 msgr2=0x7f3d6c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.749+0000 7f3d73ccf700 1 --2- 172.21.15.57:0/2663981594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0ff620 0x7f3d6c0ffaa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009990 tx=0x7f3d60009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.749+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2663981594 shutdown_connections 2023-10-08T13:51:13.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.749+0000 7f3d73ccf700 1 --2- 172.21.15.57:0/2663981594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d6c0fffe0 0x7f3d6c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.749+0000 7f3d73ccf700 1 --2- 172.21.15.57:0/2663981594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d6c1056b0 0x7f3d6c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.749+0000 7f3d73ccf700 1 --2- 172.21.15.57:0/2663981594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0ff620 0x7f3d6c0ffaa0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.749+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2663981594 >> 172.21.15.57:0/2663981594 conn(0x7f3d6c076780 msgr2=0x7f3d6c076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.749+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2663981594 shutdown_connections 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.749+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2663981594 wait complete. 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.750+0000 7f3d73ccf700 1 Processor -- start 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.750+0000 7f3d73ccf700 1 -- start start 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.750+0000 7f3d73ccf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d6c0ff620 0x7f3d6c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.750+0000 7f3d73ccf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0fffe0 0x7f3d6c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.750+0000 7f3d73ccf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d6c1056b0 0x7f3d6c1a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.750+0000 7f3d73ccf700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c198740 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.750+0000 7f3d73ccf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c1a2f70 con 0x7f3d6c1056b0 2023-10-08T13:51:13.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.750+0000 7f3d73ccf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c1a30b0 con 0x7f3d6c0ff620 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.750+0000 7f3d7126a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0fffe0 0x7f3d6c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.751+0000 7f3d71a6b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d6c0ff620 0x7f3d6c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.751+0000 7f3d7126a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0fffe0 0x7f3d6c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55564/0 (socket says 172.21.15.57:55564) 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.751+0000 7f3d7126a700 1 -- 172.21.15.57:0/2410284675 learned_addr learned my addr 172.21.15.57:0/2410284675 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.751+0000 7f3d7226c700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d6c1056b0 0x7f3d6c1a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.751+0000 7f3d7126a700 1 -- 172.21.15.57:0/2410284675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d6c0ff620 msgr2=0x7f3d6c1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.751+0000 7f3d7126a700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d6c0ff620 0x7f3d6c1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.751+0000 7f3d7126a700 1 -- 172.21.15.57:0/2410284675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d6c1056b0 msgr2=0x7f3d6c1a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.751+0000 7f3d7126a700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d6c1056b0 0x7f3d6c1a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.751+0000 7f3d7126a700 1 -- 172.21.15.57:0/2410284675 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600095f0 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.751+0000 7f3d7126a700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0fffe0 0x7f3d6c197ef0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3d60000c00 tx=0x7f3d6000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:13.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.758+0000 7f3d5effd700 1 -- 172.21.15.57:0/2410284675 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6001d070 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.758+0000 7f3d5effd700 1 -- 172.21.15.57:0/2410284675 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d60022470 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.758+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2410284675 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c1a32b0 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.758+0000 7f3d5effd700 1 -- 172.21.15.57:0/2410284675 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60017650 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.758+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2410284675 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c1a37e0 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.759+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2410284675 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6c107c30 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.760+0000 7f3d5effd700 1 -- 172.21.15.57:0/2410284675 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f3d600225e0 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.760+0000 7f3d5effd700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3d580a15e0 0x7f3d580a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:13.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.760+0000 7f3d5effd700 1 -- 172.21.15.57:0/2410284675 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f3d600ce590 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.760+0000 7f3d71a6b700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3d580a15e0 0x7f3d580a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:13.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.761+0000 7f3d71a6b700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3d580a15e0 0x7f3d580a3aa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c077ab0 tx=0x7f3d6800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:13.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.764+0000 7f3d5effd700 1 -- 172.21.15.57:0/2410284675 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3d600977b0 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.967+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2410284675 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d6c067560 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.968+0000 7f3d5effd700 1 -- 172.21.15.57:0/2410284675 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d600270a0 con 0x7f3d6c0fffe0 2023-10-08T13:51:13.968 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T13:51:13.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2410284675 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3d580a15e0 msgr2=0x7f3d580a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3d580a15e0 0x7f3d580a3aa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c077ab0 tx=0x7f3d6800b410 comp rx=0 tx=0).stop 2023-10-08T13:51:13.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2410284675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0fffe0 msgr2=0x7f3d6c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:13.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0fffe0 0x7f3d6c197ef0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3d60000c00 tx=0x7f3d6000f670 comp rx=0 tx=0).stop 2023-10-08T13:51:13.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2410284675 shutdown_connections 2023-10-08T13:51:13.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3d580a15e0 0x7f3d580a3aa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d6c1056b0 0x7f3d6c1a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d6c0ff620 0x7f3d6c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 --2- 172.21.15.57:0/2410284675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0fffe0 0x7f3d6c197ef0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:13.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2410284675 >> 172.21.15.57:0/2410284675 conn(0x7f3d6c076780 msgr2=0x7f3d6c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:13.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2410284675 shutdown_connections 2023-10-08T13:51:13.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:13.970+0000 7f3d73ccf700 1 -- 172.21.15.57:0/2410284675 wait complete. 2023-10-08T13:51:13.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:13 smithi057 ceph-mon[99476]: pgmap v287: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-08T13:51:13.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:13 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1753298282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:13.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:13 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2010938666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:13.981 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:13 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2270405896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:13.982 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T13:51:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:13 smithi099 ceph-mon[103849]: pgmap v287: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-08T13:51:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:13 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1753298282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:13 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2010938666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:13 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2270405896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:14.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.112+0000 7fe4bbebc700 1 -- 172.21.15.57:0/4284131417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4b41056b0 msgr2=0x7fe4b4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.112+0000 7fe4bbebc700 1 --2- 172.21.15.57:0/4284131417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4b41056b0 0x7fe4b4105a90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009990 tx=0x7fe4b0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.112+0000 7fe4bbebc700 1 -- 172.21.15.57:0/4284131417 shutdown_connections 2023-10-08T13:51:14.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.112+0000 7fe4bbebc700 1 --2- 172.21.15.57:0/4284131417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4b40fffe0 0x7fe4b410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.112+0000 7fe4bbebc700 1 --2- 172.21.15.57:0/4284131417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4b41056b0 0x7fe4b4105a90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.112+0000 7fe4bbebc700 1 --2- 172.21.15.57:0/4284131417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b40ff620 0x7fe4b40ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.112+0000 7fe4bbebc700 1 -- 172.21.15.57:0/4284131417 >> 172.21.15.57:0/4284131417 conn(0x7fe4b4076780 msgr2=0x7fe4b4076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:14.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.112+0000 7fe4bbebc700 1 -- 172.21.15.57:0/4284131417 shutdown_connections 2023-10-08T13:51:14.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.112+0000 7fe4bbebc700 1 -- 172.21.15.57:0/4284131417 wait complete. 2023-10-08T13:51:14.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.113+0000 7fe4bbebc700 1 Processor -- start 2023-10-08T13:51:14.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.113+0000 7fe4bbebc700 1 -- start start 2023-10-08T13:51:14.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.113+0000 7fe4bbebc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4b40ff620 0x7fe4b4195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.113+0000 7fe4bbebc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b40fffe0 0x7fe4b4195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.113+0000 7fe4bbebc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4b41056b0 0x7fe4b419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.113+0000 7fe4bbebc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b419a900 con 0x7fe4b41056b0 2023-10-08T13:51:14.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.113+0000 7fe4bbebc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b419aa70 con 0x7fe4b40fffe0 2023-10-08T13:51:14.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.113+0000 7fe4bbebc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b419ac30 con 0x7fe4b40ff620 2023-10-08T13:51:14.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.113+0000 7fe4ba459700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4b41056b0 0x7fe4b419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.113+0000 7fe4ba459700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4b41056b0 0x7fe4b419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55580/0 (socket says 172.21.15.57:55580) 2023-10-08T13:51:14.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4ba459700 1 -- 172.21.15.57:0/1338540901 learned_addr learned my addr 172.21.15.57:0/1338540901 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:14.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4b9457700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b40fffe0 0x7fe4b4195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4b9c58700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4b40ff620 0x7fe4b4195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4ba459700 1 -- 172.21.15.57:0/1338540901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4b40ff620 msgr2=0x7fe4b4195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4ba459700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4b40ff620 0x7fe4b4195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4ba459700 1 -- 172.21.15.57:0/1338540901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b40fffe0 msgr2=0x7fe4b4195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4ba459700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b40fffe0 0x7fe4b4195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4ba459700 1 -- 172.21.15.57:0/1338540901 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b00095f0 con 0x7fe4b41056b0 2023-10-08T13:51:14.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4ba459700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4b41056b0 0x7fe4b419a330 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00ef30 tx=0x7fe4ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:14.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4a6ffd700 1 -- 172.21.15.57:0/1338540901 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac018070 con 0x7fe4b41056b0 2023-10-08T13:51:14.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4a6ffd700 1 -- 172.21.15.57:0/1338540901 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4ac00f040 con 0x7fe4b41056b0 2023-10-08T13:51:14.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4a6ffd700 1 -- 172.21.15.57:0/1338540901 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac00cd00 con 0x7fe4b41056b0 2023-10-08T13:51:14.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4bbebc700 1 -- 172.21.15.57:0/1338540901 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b419aeb0 con 0x7fe4b41056b0 2023-10-08T13:51:14.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.114+0000 7fe4bbebc700 1 -- 172.21.15.57:0/1338540901 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b419b190 con 0x7fe4b41056b0 2023-10-08T13:51:14.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.115+0000 7fe4bbebc700 1 -- 172.21.15.57:0/1338540901 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b4107c10 con 0x7fe4b41056b0 2023-10-08T13:51:14.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.116+0000 7fe4a6ffd700 1 -- 172.21.15.57:0/1338540901 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fe4ac003a00 con 0x7fe4b41056b0 2023-10-08T13:51:14.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.117+0000 7fe4a6ffd700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4a00a1560 0x7fe4a00a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.117+0000 7fe4b9c58700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4a00a1560 0x7fe4a00a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.117+0000 7fe4a6ffd700 1 -- 172.21.15.57:0/1338540901 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fe4ac0cd7e0 con 0x7fe4b41056b0 2023-10-08T13:51:14.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.118+0000 7fe4b9c58700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4a00a1560 0x7fe4a00a3a20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0006010 tx=0x7fe4b0005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:14.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.121+0000 7fe4a6ffd700 1 -- 172.21.15.57:0/1338540901 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe4ac096980 con 0x7fe4b41056b0 2023-10-08T13:51:14.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:13 smithi115 ceph-mon[104408]: pgmap v287: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-10-08T13:51:14.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:13 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1753298282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:13 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2010938666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:13 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2270405896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:14.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.329+0000 7fe4bbebc700 1 -- 172.21.15.57:0/1338540901 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe4b4196d20 con 0x7fe4b41056b0 2023-10-08T13:51:14.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.330+0000 7fe4a6ffd700 1 -- 172.21.15.57:0/1338540901 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe4ac099fa0 con 0x7fe4b41056b0 2023-10-08T13:51:14.330 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:51:14.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 -- 172.21.15.57:0/1338540901 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4a00a1560 msgr2=0x7fe4a00a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4a00a1560 0x7fe4a00a3a20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0006010 tx=0x7fe4b0005960 comp rx=0 tx=0).stop 2023-10-08T13:51:14.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 -- 172.21.15.57:0/1338540901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4b41056b0 msgr2=0x7fe4b419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4b41056b0 0x7fe4b419a330 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00ef30 tx=0x7fe4ac00c3a0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 -- 172.21.15.57:0/1338540901 shutdown_connections 2023-10-08T13:51:14.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4b41056b0 0x7fe4b419a330 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4b40ff620 0x7fe4b4195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4a00a1560 0x7fe4a00a3a20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 --2- 172.21.15.57:0/1338540901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b40fffe0 0x7fe4b4195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 -- 172.21.15.57:0/1338540901 >> 172.21.15.57:0/1338540901 conn(0x7fe4b4076780 msgr2=0x7fe4b40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:14.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 -- 172.21.15.57:0/1338540901 shutdown_connections 2023-10-08T13:51:14.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.332+0000 7fe4bbebc700 1 -- 172.21.15.57:0/1338540901 wait complete. 2023-10-08T13:51:14.345 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:51:14.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.470+0000 7fa2608ca700 1 -- 172.21.15.57:0/2631039294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25c100ff0 msgr2=0x7fa25c10d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.470+0000 7fa2608ca700 1 --2- 172.21.15.57:0/2631039294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25c100ff0 0x7fa25c10d500 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa25400b0a0 tx=0x7fa25400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.470+0000 7fa2608ca700 1 -- 172.21.15.57:0/2631039294 shutdown_connections 2023-10-08T13:51:14.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.470+0000 7fa2608ca700 1 --2- 172.21.15.57:0/2631039294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25c100ff0 0x7fa25c10d500 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.470+0000 7fa2608ca700 1 --2- 172.21.15.57:0/2631039294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c106640 0x7fa25c106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.470+0000 7fa2608ca700 1 --2- 172.21.15.57:0/2631039294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa25c100630 0x7fa25c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.470+0000 7fa2608ca700 1 -- 172.21.15.57:0/2631039294 >> 172.21.15.57:0/2631039294 conn(0x7fa25c0fc1a0 msgr2=0x7fa25c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:14.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.470+0000 7fa2608ca700 1 -- 172.21.15.57:0/2631039294 shutdown_connections 2023-10-08T13:51:14.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.470+0000 7fa2608ca700 1 -- 172.21.15.57:0/2631039294 wait complete. 2023-10-08T13:51:14.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.471+0000 7fa2608ca700 1 Processor -- start 2023-10-08T13:51:14.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.471+0000 7fa2608ca700 1 -- start start 2023-10-08T13:51:14.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa2608ca700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25c100630 0x7fa25c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa2608ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c100ff0 0x7fa25c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa2608ca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa25c106640 0x7fa25c1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa2608ca700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c197090 con 0x7fa25c100630 2023-10-08T13:51:14.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa2608ca700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c197200 con 0x7fa25c106640 2023-10-08T13:51:14.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa2608ca700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c1973c0 con 0x7fa25c100ff0 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25c100630 0x7fa25c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25c100630 0x7fa25c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55590/0 (socket says 172.21.15.57:55590) 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25a59c700 1 -- 172.21.15.57:0/871492198 learned_addr learned my addr 172.21.15.57:0/871492198 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa259d9b700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c100ff0 0x7fa25c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25ad9d700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa25c106640 0x7fa25c1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25a59c700 1 -- 172.21.15.57:0/871492198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c100ff0 msgr2=0x7fa25c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25a59c700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c100ff0 0x7fa25c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25a59c700 1 -- 172.21.15.57:0/871492198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa25c106640 msgr2=0x7fa25c1968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25a59c700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa25c106640 0x7fa25c1968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25a59c700 1 -- 172.21.15.57:0/871492198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa254009d00 con 0x7fa25c100630 2023-10-08T13:51:14.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa259d9b700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c100ff0 0x7fa25c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:14.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25ad9d700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa25c106640 0x7fa25c1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:14.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa25a59c700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25c100630 0x7fa25c19c820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa25000e9a0 tx=0x7fa25000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:14.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa24b7fe700 1 -- 172.21.15.57:0/871492198 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250009790 con 0x7fa25c100630 2023-10-08T13:51:14.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa24b7fe700 1 -- 172.21.15.57:0/871492198 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa25000cbd0 con 0x7fa25c100630 2023-10-08T13:51:14.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa2608ca700 1 -- 172.21.15.57:0/871492198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c1976a0 con 0x7fa25c100630 2023-10-08T13:51:14.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa24b7fe700 1 -- 172.21.15.57:0/871492198 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250018680 con 0x7fa25c100630 2023-10-08T13:51:14.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.472+0000 7fa2608ca700 1 -- 172.21.15.57:0/871492198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c1a37f0 con 0x7fa25c100630 2023-10-08T13:51:14.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.473+0000 7fa2608ca700 1 -- 172.21.15.57:0/871492198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c108c50 con 0x7fa25c100630 2023-10-08T13:51:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.475+0000 7fa24b7fe700 1 -- 172.21.15.57:0/871492198 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fa25000cd40 con 0x7fa25c100630 2023-10-08T13:51:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.476+0000 7fa24b7fe700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa2440a15e0 0x7fa2440a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.476+0000 7fa259d9b700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa2440a15e0 0x7fa2440a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.476+0000 7fa24b7fe700 1 -- 172.21.15.57:0/871492198 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fa250014070 con 0x7fa25c100630 2023-10-08T13:51:14.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.476+0000 7fa259d9b700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa2440a15e0 0x7fa2440a3aa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa25c198030 tx=0x7fa24c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:14.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.479+0000 7fa24b7fe700 1 -- 172.21.15.57:0/871492198 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa250096f80 con 0x7fa25c100630 2023-10-08T13:51:14.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.727+0000 7fa2608ca700 1 -- 172.21.15.57:0/871492198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa25c197da0 con 0x7fa25c100630 2023-10-08T13:51:14.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.728+0000 7fa24b7fe700 1 -- 172.21.15.57:0/871492198 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11361 (secure 0 0 0) 0x7fa25009a5a0 con 0x7fa25c100630 2023-10-08T13:51:14.728 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:14.728 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:50:07.719573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:51:14.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 -- 172.21.15.57:0/871492198 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa2440a15e0 msgr2=0x7fa2440a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa2440a15e0 0x7fa2440a3aa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa25c198030 tx=0x7fa24c009260 comp rx=0 tx=0).stop 2023-10-08T13:51:14.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 -- 172.21.15.57:0/871492198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25c100630 msgr2=0x7fa25c19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25c100630 0x7fa25c19c820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa25000e9a0 tx=0x7fa25000ed60 comp rx=0 tx=0).stop 2023-10-08T13:51:14.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 -- 172.21.15.57:0/871492198 shutdown_connections 2023-10-08T13:51:14.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa2440a15e0 0x7fa2440a3aa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa25c106640 0x7fa25c1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa25c100630 0x7fa25c19c820 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 --2- 172.21.15.57:0/871492198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa25c100ff0 0x7fa25c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 -- 172.21.15.57:0/871492198 >> 172.21.15.57:0/871492198 conn(0x7fa25c0fc1a0 msgr2=0x7fa25c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:14.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 -- 172.21.15.57:0/871492198 shutdown_connections 2023-10-08T13:51:14.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.730+0000 7fa2608ca700 1 -- 172.21.15.57:0/871492198 wait complete. 2023-10-08T13:51:14.732 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:51:14.743 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:51:14.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.870+0000 7fccf23c2700 1 -- 172.21.15.57:0/1774876398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccec0fff00 msgr2=0x7fccec10c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.870+0000 7fccf23c2700 1 --2- 172.21.15.57:0/1774876398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccec0fff00 0x7fccec10c3c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcce400b0d0 tx=0x7fcce400b3e0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.870+0000 7fccf23c2700 1 -- 172.21.15.57:0/1774876398 shutdown_connections 2023-10-08T13:51:14.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.870+0000 7fccf23c2700 1 --2- 172.21.15.57:0/1774876398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccec0fff00 0x7fccec10c3c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.870+0000 7fccf23c2700 1 --2- 172.21.15.57:0/1774876398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec105600 0x7fccec1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.870+0000 7fccf23c2700 1 --2- 172.21.15.57:0/1774876398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccec0ff540 0x7fccec0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.870+0000 7fccf23c2700 1 -- 172.21.15.57:0/1774876398 >> 172.21.15.57:0/1774876398 conn(0x7fccec0fb030 msgr2=0x7fccec0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:14.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.870+0000 7fccf23c2700 1 -- 172.21.15.57:0/1774876398 shutdown_connections 2023-10-08T13:51:14.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.870+0000 7fccf23c2700 1 -- 172.21.15.57:0/1774876398 wait complete. 2023-10-08T13:51:14.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf23c2700 1 Processor -- start 2023-10-08T13:51:14.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf23c2700 1 -- start start 2023-10-08T13:51:14.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf23c2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec0ff540 0x7fccec1011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf23c2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccec0fff00 0x7fccec1016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf23c2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccec105600 0x7fccec101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf23c2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a4fe0 con 0x7fccec105600 2023-10-08T13:51:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf23c2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a5150 con 0x7fccec0ff540 2023-10-08T13:51:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf23c2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a5310 con 0x7fccec0fff00 2023-10-08T13:51:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf095f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccec105600 0x7fccec101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec0ff540 0x7fccec1011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf095f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccec105600 0x7fccec101c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55610/0 (socket says 172.21.15.57:55610) 2023-10-08T13:51:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccebfff700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec0ff540 0x7fccec1011b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:57032/0 (socket says 172.21.15.57:57032) 2023-10-08T13:51:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fccf095f700 1 -- 172.21.15.57:0/1742778999 learned_addr learned my addr 172.21.15.57:0/1742778999 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.871+0000 7fcceb7fe700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccec0fff00 0x7fccec1016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fccebfff700 1 -- 172.21.15.57:0/1742778999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccec0fff00 msgr2=0x7fccec1016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fccebfff700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccec0fff00 0x7fccec1016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fccebfff700 1 -- 172.21.15.57:0/1742778999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccec105600 msgr2=0x7fccec101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fccebfff700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccec105600 0x7fccec101c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fccebfff700 1 -- 172.21.15.57:0/1742778999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce4009d00 con 0x7fccec0ff540 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fccf095f700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccec105600 0x7fccec101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fccebfff700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec0ff540 0x7fccec1011b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e9f0 tx=0x7fccdc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fcce97fa700 1 -- 172.21.15.57:0/1742778999 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00c9c0 con 0x7fccec0ff540 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fccf23c2700 1 -- 172.21.15.57:0/1742778999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec1a55f0 con 0x7fccec0ff540 2023-10-08T13:51:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fcce97fa700 1 -- 172.21.15.57:0/1742778999 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccdc010430 con 0x7fccec0ff540 2023-10-08T13:51:14.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fcce97fa700 1 -- 172.21.15.57:0/1742778999 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc018680 con 0x7fccec0ff540 2023-10-08T13:51:14.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.872+0000 7fccf23c2700 1 -- 172.21.15.57:0/1742778999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec1a5b50 con 0x7fccec0ff540 2023-10-08T13:51:14.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.873+0000 7fccf23c2700 1 -- 172.21.15.57:0/1742778999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec100c60 con 0x7fccec0ff540 2023-10-08T13:51:14.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.874+0000 7fcce97fa700 1 -- 172.21.15.57:0/1742778999 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fccdc010860 con 0x7fccec0ff540 2023-10-08T13:51:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.875+0000 7fcce97fa700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fccd40a1560 0x7fccd40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.875+0000 7fcce97fa700 1 -- 172.21.15.57:0/1742778999 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7fccdc014070 con 0x7fccec0ff540 2023-10-08T13:51:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.875+0000 7fcceb7fe700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fccd40a1560 0x7fccd40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.876+0000 7fcceb7fe700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fccd40a1560 0x7fccd40a3a20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fccec1028e0 tx=0x7fcce000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:14.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:14.879+0000 7fcce97fa700 1 -- 172.21.15.57:0/1742778999 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fccdc096640 con 0x7fccec0ff540 2023-10-08T13:51:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:14 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2410284675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:14 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1338540901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:14 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/871492198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:15.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.082+0000 7fccf23c2700 1 -- 172.21.15.57:0/1742778999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccec10f4b0 con 0x7fccec0ff540 2023-10-08T13:51:15.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.084+0000 7fcce97fa700 1 -- 172.21.15.57:0/1742778999 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24753 (secure 0 0 0) 0x7fccdc099c60 con 0x7fccec0ff540 2023-10-08T13:51:15.084 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:15.085 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":90,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:50:15.770421+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:51:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 -- 172.21.15.57:0/1742778999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fccd40a1560 msgr2=0x7fccd40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fccd40a1560 0x7fccd40a3a20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fccec1028e0 tx=0x7fcce000b410 comp rx=0 tx=0).stop 2023-10-08T13:51:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 -- 172.21.15.57:0/1742778999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec0ff540 msgr2=0x7fccec1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec0ff540 0x7fccec1011b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e9f0 tx=0x7fccdc00ed00 comp rx=0 tx=0).stop 2023-10-08T13:51:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 -- 172.21.15.57:0/1742778999 shutdown_connections 2023-10-08T13:51:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fccec105600 0x7fccec101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fccd40a1560 0x7fccd40a3a20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccec0ff540 0x7fccec1011b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 --2- 172.21.15.57:0/1742778999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccec0fff00 0x7fccec1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 -- 172.21.15.57:0/1742778999 >> 172.21.15.57:0/1742778999 conn(0x7fccec0fb030 msgr2=0x7fccec0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 -- 172.21.15.57:0/1742778999 shutdown_connections 2023-10-08T13:51:15.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.086+0000 7fccf23c2700 1 -- 172.21.15.57:0/1742778999 wait complete. 2023-10-08T13:51:15.097 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2410284675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:15.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1338540901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:15.098 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/871492198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:15.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T13:51:15.099 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:51:15.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:14 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2410284675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:14 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1338540901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:15.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:14 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/871492198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:15.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.228+0000 7f1e72fa9700 1 -- 172.21.15.57:0/4026296822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e6c100720 msgr2=0x7f1e6c100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.228+0000 7f1e72fa9700 1 --2- 172.21.15.57:0/4026296822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e6c100720 0x7f1e6c100ba0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009990 tx=0x7f1e54009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 -- 172.21.15.57:0/4026296822 shutdown_connections 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 --2- 172.21.15.57:0/4026296822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e6c1010e0 0x7f1e6c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 --2- 172.21.15.57:0/4026296822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6c106730 0x7f1e6c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 --2- 172.21.15.57:0/4026296822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e6c100720 0x7f1e6c100ba0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 -- 172.21.15.57:0/4026296822 >> 172.21.15.57:0/4026296822 conn(0x7f1e6c0fc230 msgr2=0x7f1e6c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 -- 172.21.15.57:0/4026296822 shutdown_connections 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 -- 172.21.15.57:0/4026296822 wait complete. 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 Processor -- start 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 -- start start 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6c100720 0x7f1e6c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e6c1010e0 0x7f1e6c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e6c106730 0x7f1e6c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c19dcd0 con 0x7f1e6c106730 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c19de40 con 0x7f1e6c100720 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.229+0000 7f1e72fa9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c19e000 con 0x7f1e6c1010e0 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e71546700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e6c106730 0x7f1e6c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e71546700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e6c106730 0x7f1e6c19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55624/0 (socket says 172.21.15.57:55624) 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e71546700 1 -- 172.21.15.57:0/800763338 learned_addr learned my addr 172.21.15.57:0/800763338 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e70d45700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6c100720 0x7f1e6c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e6bfff700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e6c1010e0 0x7f1e6c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e71546700 1 -- 172.21.15.57:0/800763338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e6c1010e0 msgr2=0x7f1e6c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e71546700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e6c1010e0 0x7f1e6c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e71546700 1 -- 172.21.15.57:0/800763338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6c100720 msgr2=0x7f1e6c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e71546700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6c100720 0x7f1e6c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e71546700 1 -- 172.21.15.57:0/800763338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540095f0 con 0x7f1e6c106730 2023-10-08T13:51:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e71546700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e6c106730 0x7f1e6c19d690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1e64007ee0 tx=0x7f1e6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e69ffb700 1 -- 172.21.15.57:0/800763338 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64017070 con 0x7f1e6c106730 2023-10-08T13:51:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e69ffb700 1 -- 172.21.15.57:0/800763338 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e6400f040 con 0x7f1e6c106730 2023-10-08T13:51:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e72fa9700 1 -- 172.21.15.57:0/800763338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6c19e2e0 con 0x7f1e6c106730 2023-10-08T13:51:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e69ffb700 1 -- 172.21.15.57:0/800763338 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64005580 con 0x7f1e6c106730 2023-10-08T13:51:15.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.230+0000 7f1e72fa9700 1 -- 172.21.15.57:0/800763338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6c1a5b20 con 0x7f1e6c106730 2023-10-08T13:51:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.231+0000 7f1e72fa9700 1 -- 172.21.15.57:0/800763338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6c101e60 con 0x7f1e6c106730 2023-10-08T13:51:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.232+0000 7f1e69ffb700 1 -- 172.21.15.57:0/800763338 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1e6400db20 con 0x7f1e6c106730 2023-10-08T13:51:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.233+0000 7f1e69ffb700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e580a15e0 0x7f1e580a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.233+0000 7f1e70d45700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e580a15e0 0x7f1e580a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.233+0000 7f1e69ffb700 1 -- 172.21.15.57:0/800763338 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f1e640cc9a0 con 0x7f1e6c106730 2023-10-08T13:51:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.233+0000 7f1e70d45700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e580a15e0 0x7f1e580a3aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009be0 tx=0x7f1e5c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:15.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.237+0000 7f1e69ffb700 1 -- 172.21.15.57:0/800763338 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1e64095a50 con 0x7f1e6c106730 2023-10-08T13:51:15.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.438+0000 7f1e72fa9700 1 -- 172.21.15.57:0/800763338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e6c19a150 con 0x7f1e6c106730 2023-10-08T13:51:15.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.439+0000 7f1e69ffb700 1 -- 172.21.15.57:0/800763338 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e64099070 con 0x7f1e6c106730 2023-10-08T13:51:15.439 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:51:15.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 -- 172.21.15.57:0/800763338 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e580a15e0 msgr2=0x7f1e580a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e580a15e0 0x7f1e580a3aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009be0 tx=0x7f1e5c009260 comp rx=0 tx=0).stop 2023-10-08T13:51:15.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 -- 172.21.15.57:0/800763338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e6c106730 msgr2=0x7f1e6c19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e6c106730 0x7f1e6c19d690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1e64007ee0 tx=0x7f1e6400d3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 -- 172.21.15.57:0/800763338 shutdown_connections 2023-10-08T13:51:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e6c106730 0x7f1e6c19d690 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e580a15e0 0x7f1e580a3aa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6c100720 0x7f1e6c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 --2- 172.21.15.57:0/800763338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e6c1010e0 0x7f1e6c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 -- 172.21.15.57:0/800763338 >> 172.21.15.57:0/800763338 conn(0x7f1e6c0fc230 msgr2=0x7f1e6c072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 -- 172.21.15.57:0/800763338 shutdown_connections 2023-10-08T13:51:15.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.441+0000 7f1e72fa9700 1 -- 172.21.15.57:0/800763338 wait complete. 2023-10-08T13:51:15.453 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T13:51:15.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.583+0000 7f91103ad700 1 -- 172.21.15.57:0/385369507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91080ff610 msgr2=0x7f91080ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.583+0000 7f91103ad700 1 --2- 172.21.15.57:0/385369507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91080ff610 0x7f91080ffa90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009990 tx=0x7f90f8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.583+0000 7f91103ad700 1 -- 172.21.15.57:0/385369507 shutdown_connections 2023-10-08T13:51:15.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.583+0000 7f91103ad700 1 --2- 172.21.15.57:0/385369507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080fffd0 0x7f910810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.583+0000 7f91103ad700 1 --2- 172.21.15.57:0/385369507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91081056b0 0x7f9108105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.583+0000 7f91103ad700 1 --2- 172.21.15.57:0/385369507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91080ff610 0x7f91080ffa90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.583+0000 7f91103ad700 1 -- 172.21.15.57:0/385369507 >> 172.21.15.57:0/385369507 conn(0x7f9108076820 msgr2=0x7f9108076c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.583+0000 7f91103ad700 1 -- 172.21.15.57:0/385369507 shutdown_connections 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.583+0000 7f91103ad700 1 -- 172.21.15.57:0/385369507 wait complete. 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f91103ad700 1 Processor -- start 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f91103ad700 1 -- start start 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f91103ad700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91080ff610 0x7f91081978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f91103ad700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080fffd0 0x7f9108197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f91103ad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91081056b0 0x7f91081a2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f91103ad700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9108198660 con 0x7f91081056b0 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f91103ad700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91081a2eb0 con 0x7f91080ff610 2023-10-08T13:51:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f91103ad700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91081a2ff0 con 0x7f91080fffd0 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f910e94a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91081056b0 0x7f91081a2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f910e94a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91081056b0 0x7f91081a2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55638/0 (socket says 172.21.15.57:55638) 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f910e94a700 1 -- 172.21.15.57:0/3243851597 learned_addr learned my addr 172.21.15.57:0/3243851597 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f910e149700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91080ff610 0x7f91081978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f910d948700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080fffd0 0x7f9108197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f910e94a700 1 -- 172.21.15.57:0/3243851597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080fffd0 msgr2=0x7f9108197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f910e94a700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080fffd0 0x7f9108197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f910e94a700 1 -- 172.21.15.57:0/3243851597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91080ff610 msgr2=0x7f91081978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f910e94a700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91080ff610 0x7f91081978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.584+0000 7f910e94a700 1 -- 172.21.15.57:0/3243851597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f80095f0 con 0x7f91081056b0 2023-10-08T13:51:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.585+0000 7f910e94a700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91081056b0 0x7f91081a2970 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9100007d70 tx=0x7f910000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:15.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.585+0000 7f90ff7fe700 1 -- 172.21.15.57:0/3243851597 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910000ad40 con 0x7f91081056b0 2023-10-08T13:51:15.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.585+0000 7f91103ad700 1 -- 172.21.15.57:0/3243851597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91081a3250 con 0x7f91081056b0 2023-10-08T13:51:15.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.585+0000 7f90ff7fe700 1 -- 172.21.15.57:0/3243851597 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91000128e0 con 0x7f91081056b0 2023-10-08T13:51:15.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.585+0000 7f90ff7fe700 1 -- 172.21.15.57:0/3243851597 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000116b0 con 0x7f91081056b0 2023-10-08T13:51:15.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.585+0000 7f91103ad700 1 -- 172.21.15.57:0/3243851597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91081a3730 con 0x7f91081056b0 2023-10-08T13:51:15.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.586+0000 7f91103ad700 1 -- 172.21.15.57:0/3243851597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9108107c30 con 0x7f91081056b0 2023-10-08T13:51:15.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.587+0000 7f90ff7fe700 1 -- 172.21.15.57:0/3243851597 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f9100012420 con 0x7f91081056b0 2023-10-08T13:51:15.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.587+0000 7f90ff7fe700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f90f40a1560 0x7f90f40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.588+0000 7f910e149700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f90f40a1560 0x7f90f40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.588+0000 7f90ff7fe700 1 -- 172.21.15.57:0/3243851597 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9235+0+0 (secure 0 0 0) 0x7f91000cd810 con 0x7f91081056b0 2023-10-08T13:51:15.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.588+0000 7f910e149700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f90f40a1560 0x7f90f40a3a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9108100750 tx=0x7f910400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:15.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.591+0000 7f90ff7fe700 1 -- 172.21.15.57:0/3243851597 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9100096a30 con 0x7f91081056b0 2023-10-08T13:51:15.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.798+0000 7f91103ad700 1 -- 172.21.15.57:0/3243851597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9108067980 con 0x7f91081056b0 2023-10-08T13:51:15.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.799+0000 7f90ff7fe700 1 -- 172.21.15.57:0/3243851597 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9108067980 con 0x7f91081056b0 2023-10-08T13:51:15.799 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:51:15.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 -- 172.21.15.57:0/3243851597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f90f40a1560 msgr2=0x7f90f40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f90f40a1560 0x7f90f40a3a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9108100750 tx=0x7f910400b410 comp rx=0 tx=0).stop 2023-10-08T13:51:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 -- 172.21.15.57:0/3243851597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91081056b0 msgr2=0x7f91081a2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91081056b0 0x7f91081a2970 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9100007d70 tx=0x7f910000a340 comp rx=0 tx=0).stop 2023-10-08T13:51:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 -- 172.21.15.57:0/3243851597 shutdown_connections 2023-10-08T13:51:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91081056b0 0x7f91081a2970 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f90f40a1560 0x7f90f40a3a20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91080ff610 0x7f91081978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 --2- 172.21.15.57:0/3243851597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080fffd0 0x7f9108197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 -- 172.21.15.57:0/3243851597 >> 172.21.15.57:0/3243851597 conn(0x7f9108076820 msgr2=0x7f91080fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 -- 172.21.15.57:0/3243851597 shutdown_connections 2023-10-08T13:51:15.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.801+0000 7f91103ad700 1 -- 172.21.15.57:0/3243851597 wait complete. 2023-10-08T13:51:15.815 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T13:51:15.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.945+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/518789228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb41051c0 msgr2=0x7f5eb4105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.945+0000 7f5ebc5b2700 1 --2- 172.21.15.57:0/518789228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb41051c0 0x7f5eb4105640 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00099c0 tx=0x7f5eb0009cd0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.945+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/518789228 shutdown_connections 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.945+0000 7f5ebc5b2700 1 --2- 172.21.15.57:0/518789228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5eb4105b80 0x7f5eb410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.945+0000 7f5ebc5b2700 1 --2- 172.21.15.57:0/518789228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb4067ce0 0x7f5eb40680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.945+0000 7f5ebc5b2700 1 --2- 172.21.15.57:0/518789228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb41051c0 0x7f5eb4105640 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.945+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/518789228 >> 172.21.15.57:0/518789228 conn(0x7f5eb4076780 msgr2=0x7f5eb4076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.945+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/518789228 shutdown_connections 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.945+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/518789228 wait complete. 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5ebc5b2700 1 Processor -- start 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5ebc5b2700 1 -- start start 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5ebc5b2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4067ce0 0x7f5eb40ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5ebc5b2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5eb41051c0 0x7f5eb40ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5ebc5b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb4105b80 0x7f5eb4100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5ebc5b2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb41a4fc0 con 0x7f5eb4067ce0 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5ebc5b2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb41a5130 con 0x7f5eb4105b80 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5ebc5b2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb41a52f0 con 0x7f5eb41051c0 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eba34e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4067ce0 0x7f5eb40ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eba34e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4067ce0 0x7f5eb40ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55644/0 (socket says 172.21.15.57:55644) 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eba34e700 1 -- 172.21.15.57:0/564878212 learned_addr learned my addr 172.21.15.57:0/564878212 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eb9b4d700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5eb41051c0 0x7f5eb40ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eba34e700 1 -- 172.21.15.57:0/564878212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5eb41051c0 msgr2=0x7f5eb40ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5ebab4f700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb4105b80 0x7f5eb4100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eba34e700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5eb41051c0 0x7f5eb40ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eba34e700 1 -- 172.21.15.57:0/564878212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb4105b80 msgr2=0x7f5eb4100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eba34e700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb4105b80 0x7f5eb4100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eba34e700 1 -- 172.21.15.57:0/564878212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00095f0 con 0x7f5eb4067ce0 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eb9b4d700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5eb41051c0 0x7f5eb40ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5ebab4f700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb4105b80 0x7f5eb4100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.946+0000 7f5eba34e700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4067ce0 0x7f5eb40ff6b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400d950 tx=0x7f5ea400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.947+0000 7f5eab7fe700 1 -- 172.21.15.57:0/564878212 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4009b40 con 0x7f5eb4067ce0 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.947+0000 7f5eab7fe700 1 -- 172.21.15.57:0/564878212 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea4009ca0 con 0x7f5eb4067ce0 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.947+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/564878212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb41a55d0 con 0x7f5eb4067ce0 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.947+0000 7f5eab7fe700 1 -- 172.21.15.57:0/564878212 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400f620 con 0x7f5eb4067ce0 2023-10-08T13:51:15.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.947+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/564878212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb41a5ae0 con 0x7f5eb4067ce0 2023-10-08T13:51:15.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.948+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/564878212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb4108c70 con 0x7f5eb4067ce0 2023-10-08T13:51:15.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.949+0000 7f5eab7fe700 1 -- 172.21.15.57:0/564878212 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f5ea4010460 con 0x7f5eb4067ce0 2023-10-08T13:51:15.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.949+0000 7f5eab7fe700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ea00a1560 0x7f5ea00a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:15.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.950+0000 7f5eb9b4d700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ea00a1560 0x7f5ea00a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:15.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.950+0000 7f5eab7fe700 1 -- 172.21.15.57:0/564878212 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5ea40cd6a0 con 0x7f5eb4067ce0 2023-10-08T13:51:15.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.950+0000 7f5eb9b4d700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ea00a1560 0x7f5ea00a3a20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4100de0 tx=0x7f5eb0005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:15.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:15.954+0000 7f5eab7fe700 1 -- 172.21.15.57:0/564878212 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5ea4096780 con 0x7f5eb4067ce0 2023-10-08T13:51:16.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:15 smithi099 ceph-mon[103849]: pgmap v288: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:16.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-10-08T13:51:16.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:15 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-10-08T13:51:16.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:15 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1742778999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:16.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:15 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/800763338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:16.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.154+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/564878212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5eb4100b90 con 0x7f5eb4067ce0 2023-10-08T13:51:16.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.155+0000 7f5eab7fe700 1 -- 172.21.15.57:0/564878212 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ea4099da0 con 0x7f5eb4067ce0 2023-10-08T13:51:16.156 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T13:51:16.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/564878212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ea00a1560 msgr2=0x7f5ea00a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ea00a1560 0x7f5ea00a3a20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4100de0 tx=0x7f5eb0005fd0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/564878212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4067ce0 msgr2=0x7f5eb40ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4067ce0 0x7f5eb40ff6b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400d950 tx=0x7f5ea400dd10 comp rx=0 tx=0).stop 2023-10-08T13:51:16.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/564878212 shutdown_connections 2023-10-08T13:51:16.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5ea00a1560 0x7f5ea00a3a20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb4105b80 0x7f5eb4100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5eb4067ce0 0x7f5eb40ff6b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 --2- 172.21.15.57:0/564878212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5eb41051c0 0x7f5eb40ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/564878212 >> 172.21.15.57:0/564878212 conn(0x7f5eb4076780 msgr2=0x7f5eb40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:16.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/564878212 shutdown_connections 2023-10-08T13:51:16.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.157+0000 7f5ebc5b2700 1 -- 172.21.15.57:0/564878212 wait complete. 2023-10-08T13:51:16.168 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:15 smithi057 ceph-mon[99476]: pgmap v288: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:16.169 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-10-08T13:51:16.169 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:15 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-10-08T13:51:16.169 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:15 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1742778999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:16.169 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:15 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/800763338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:16.170 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T13:51:16.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:15 smithi115 ceph-mon[104408]: pgmap v288: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-08T13:51:16.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-10-08T13:51:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:15 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-10-08T13:51:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:15 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1742778999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:16.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:15 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/800763338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:16.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.299+0000 7f95a757e700 1 -- 172.21.15.57:0/985515343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a00ff620 msgr2=0x7f95a00ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.299+0000 7f95a757e700 1 --2- 172.21.15.57:0/985515343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a00ff620 0x7f95a00ffaa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f959c009990 tx=0x7f959c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.299+0000 7f95a757e700 1 -- 172.21.15.57:0/985515343 shutdown_connections 2023-10-08T13:51:16.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.299+0000 7f95a757e700 1 --2- 172.21.15.57:0/985515343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a00fffe0 0x7f95a010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.299+0000 7f95a757e700 1 --2- 172.21.15.57:0/985515343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a01056b0 0x7f95a0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.299+0000 7f95a757e700 1 --2- 172.21.15.57:0/985515343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a00ff620 0x7f95a00ffaa0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.299+0000 7f95a757e700 1 -- 172.21.15.57:0/985515343 >> 172.21.15.57:0/985515343 conn(0x7f95a0076780 msgr2=0x7f95a0076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.299+0000 7f95a757e700 1 -- 172.21.15.57:0/985515343 shutdown_connections 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.299+0000 7f95a757e700 1 -- 172.21.15.57:0/985515343 wait complete. 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a757e700 1 Processor -- start 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a757e700 1 -- start start 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a757e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a00ff620 0x7f95a0197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a757e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a00fffe0 0x7f95a0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a757e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a01056b0 0x7f95a01a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a757e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a0198700 con 0x7f95a00fffe0 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a757e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a01a2f10 con 0x7f95a00ff620 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a757e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a01a3050 con 0x7f95a01056b0 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a4b19700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a00fffe0 0x7f95a0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:16.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a4b19700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a00fffe0 0x7f95a0197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55660/0 (socket says 172.21.15.57:55660) 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a4b19700 1 -- 172.21.15.57:0/4265818274 learned_addr learned my addr 172.21.15.57:0/4265818274 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a531a700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a00ff620 0x7f95a0197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a5b1b700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a01056b0 0x7f95a01a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a4b19700 1 -- 172.21.15.57:0/4265818274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a01056b0 msgr2=0x7f95a01a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a4b19700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a01056b0 0x7f95a01a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a4b19700 1 -- 172.21.15.57:0/4265818274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a00ff620 msgr2=0x7f95a0197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a4b19700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a00ff620 0x7f95a0197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.300+0000 7f95a4b19700 1 -- 172.21.15.57:0/4265818274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0095f0 con 0x7f95a00fffe0 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.301+0000 7f95a4b19700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a00fffe0 0x7f95a0197e90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.301+0000 7f95967fc700 1 -- 172.21.15.57:0/4265818274 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c01d070 con 0x7f95a00fffe0 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.301+0000 7f95a757e700 1 -- 172.21.15.57:0/4265818274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a01a3280 con 0x7f95a00fffe0 2023-10-08T13:51:16.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.301+0000 7f95967fc700 1 -- 172.21.15.57:0/4265818274 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f959c022470 con 0x7f95a00fffe0 2023-10-08T13:51:16.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.301+0000 7f95967fc700 1 -- 172.21.15.57:0/4265818274 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c017650 con 0x7f95a00fffe0 2023-10-08T13:51:16.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.301+0000 7f95a757e700 1 -- 172.21.15.57:0/4265818274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a01a3760 con 0x7f95a00fffe0 2023-10-08T13:51:16.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.302+0000 7f95a757e700 1 -- 172.21.15.57:0/4265818274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a0107c30 con 0x7f95a00fffe0 2023-10-08T13:51:16.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.303+0000 7f95967fc700 1 -- 172.21.15.57:0/4265818274 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f959c0225e0 con 0x7f95a00fffe0 2023-10-08T13:51:16.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.303+0000 7f95967fc700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f958c0a1560 0x7f958c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:16.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.304+0000 7f95967fc700 1 -- 172.21.15.57:0/4265818274 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f959c0ceb20 con 0x7f95a00fffe0 2023-10-08T13:51:16.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.304+0000 7f95a531a700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f958c0a1560 0x7f958c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:16.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.304+0000 7f95a531a700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f958c0a1560 0x7f958c0a3a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95a0077ab0 tx=0x7f9590007400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:16.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.307+0000 7f95967fc700 1 -- 172.21.15.57:0/4265818274 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f959c097c30 con 0x7f95a00fffe0 2023-10-08T13:51:16.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.510+0000 7f95a757e700 1 -- 172.21.15.57:0/4265818274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f95a0100d50 con 0x7f95a00fffe0 2023-10-08T13:51:16.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.510+0000 7f95967fc700 1 -- 172.21.15.57:0/4265818274 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f959c09b250 con 0x7f95a00fffe0 2023-10-08T13:51:16.511 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:51:16.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 -- 172.21.15.57:0/4265818274 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f958c0a1560 msgr2=0x7f958c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f958c0a1560 0x7f958c0a3a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95a0077ab0 tx=0x7f9590007400 comp rx=0 tx=0).stop 2023-10-08T13:51:16.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 -- 172.21.15.57:0/4265818274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a00fffe0 msgr2=0x7f95a0197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a00fffe0 0x7f95a0197e90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c004730 comp rx=0 tx=0).stop 2023-10-08T13:51:16.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 -- 172.21.15.57:0/4265818274 shutdown_connections 2023-10-08T13:51:16.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a01056b0 0x7f95a01a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f958c0a1560 0x7f958c0a3a20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a00ff620 0x7f95a0197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 --2- 172.21.15.57:0/4265818274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95a00fffe0 0x7f95a0197e90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 -- 172.21.15.57:0/4265818274 >> 172.21.15.57:0/4265818274 conn(0x7f95a0076780 msgr2=0x7f95a00fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 -- 172.21.15.57:0/4265818274 shutdown_connections 2023-10-08T13:51:16.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.513+0000 7f95a757e700 1 -- 172.21.15.57:0/4265818274 wait complete. 2023-10-08T13:51:16.525 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:51:16.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.651+0000 7f76229c9700 1 -- 172.21.15.57:0/2580030420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f761c1066b0 msgr2=0x7f761c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.651+0000 7f76229c9700 1 --2- 172.21.15.57:0/2580030420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f761c1066b0 0x7f761c106a90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f760c009990 tx=0x7f760c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.652+0000 7f76229c9700 1 -- 172.21.15.57:0/2580030420 shutdown_connections 2023-10-08T13:51:16.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.652+0000 7f76229c9700 1 --2- 172.21.15.57:0/2580030420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f761c100fd0 0x7f761c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.652+0000 7f76229c9700 1 --2- 172.21.15.57:0/2580030420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f761c1066b0 0x7f761c106a90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.652+0000 7f76229c9700 1 --2- 172.21.15.57:0/2580030420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c100610 0x7f761c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.652+0000 7f76229c9700 1 -- 172.21.15.57:0/2580030420 >> 172.21.15.57:0/2580030420 conn(0x7f761c0fc160 msgr2=0x7f761c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:16.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.652+0000 7f76229c9700 1 -- 172.21.15.57:0/2580030420 shutdown_connections 2023-10-08T13:51:16.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.652+0000 7f76229c9700 1 -- 172.21.15.57:0/2580030420 wait complete. 2023-10-08T13:51:16.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.652+0000 7f76229c9700 1 Processor -- start 2023-10-08T13:51:16.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.652+0000 7f76229c9700 1 -- start start 2023-10-08T13:51:16.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f76229c9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f761c100610 0x7f761c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:16.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f76229c9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f761c100fd0 0x7f761c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:16.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f76229c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1066b0 0x7f761c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:16.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f76229c9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c1a4e70 con 0x7f761c100fd0 2023-10-08T13:51:16.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f76229c9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c1a4fe0 con 0x7f761c1066b0 2023-10-08T13:51:16.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f76229c9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c1a51a0 con 0x7f761c100610 2023-10-08T13:51:16.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f761b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f761c100fd0 0x7f761c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:16.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f761b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f761c100fd0 0x7f761c073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55674/0 (socket says 172.21.15.57:55674) 2023-10-08T13:51:16.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f761b7fe700 1 -- 172.21.15.57:0/3719966365 learned_addr learned my addr 172.21.15.57:0/3719966365 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:16.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f761bfff700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f761c100610 0x7f761c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:16.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f761b7fe700 1 -- 172.21.15.57:0/3719966365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f761c100610 msgr2=0x7f761c072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f761b7fe700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f761c100610 0x7f761c072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f7620f66700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1066b0 0x7f761c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:16.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f761b7fe700 1 -- 172.21.15.57:0/3719966365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1066b0 msgr2=0x7f761c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f761b7fe700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1066b0 0x7f761c073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.653+0000 7f761b7fe700 1 -- 172.21.15.57:0/3719966365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c0095f0 con 0x7f761c100fd0 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.654+0000 7f761b7fe700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f761c100fd0 0x7f761c073030 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f761000c920 tx=0x7f761000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.654+0000 7f76197fa700 1 -- 172.21.15.57:0/3719966365 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610007b00 con 0x7f761c100fd0 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.654+0000 7f76197fa700 1 -- 172.21.15.57:0/3719966365 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7610004cf0 con 0x7f761c100fd0 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.654+0000 7f76229c9700 1 -- 172.21.15.57:0/3719966365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761c1a5480 con 0x7f761c100fd0 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.654+0000 7f76197fa700 1 -- 172.21.15.57:0/3719966365 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610005610 con 0x7f761c100fd0 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.654+0000 7f76229c9700 1 -- 172.21.15.57:0/3719966365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761c1a59e0 con 0x7f761c100fd0 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.655+0000 7f76229c9700 1 -- 172.21.15.57:0/3719966365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761c108c10 con 0x7f761c100fd0 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.656+0000 7f76197fa700 1 -- 172.21.15.57:0/3719966365 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f7610004750 con 0x7f761c100fd0 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.656+0000 7f76197fa700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76040a15e0 0x7f76040a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:16.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.656+0000 7f76197fa700 1 -- 172.21.15.57:0/3719966365 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f76100cd250 con 0x7f761c100fd0 2023-10-08T13:51:16.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.656+0000 7f761bfff700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76040a15e0 0x7f76040a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:16.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.657+0000 7f761bfff700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76040a15e0 0x7f76040a3aa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b5c0 tx=0x7f760c01a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:16.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.661+0000 7f76197fa700 1 -- 172.21.15.57:0/3719966365 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7610096330 con 0x7f761c100fd0 2023-10-08T13:51:16.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.808+0000 7f76197fa700 1 -- 172.21.15.57:0/3719966365 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7610004a00 con 0x7f761c100fd0 2023-10-08T13:51:16.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.922+0000 7f76229c9700 1 -- 172.21.15.57:0/3719966365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f761c073fd0 con 0x7f761c100fd0 2023-10-08T13:51:16.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.923+0000 7f76197fa700 1 -- 172.21.15.57:0/3719966365 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f7610094840 con 0x7f761c100fd0 2023-10-08T13:51:16.923 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:16.923 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:50:07.719573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:51:16.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 -- 172.21.15.57:0/3719966365 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76040a15e0 msgr2=0x7f76040a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76040a15e0 0x7f76040a3aa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b5c0 tx=0x7f760c01a040 comp rx=0 tx=0).stop 2023-10-08T13:51:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 -- 172.21.15.57:0/3719966365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f761c100fd0 msgr2=0x7f761c073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f761c100fd0 0x7f761c073030 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f761000c920 tx=0x7f761000cce0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 -- 172.21.15.57:0/3719966365 shutdown_connections 2023-10-08T13:51:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f76040a15e0 0x7f76040a3aa0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c1066b0 0x7f761c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f761c100610 0x7f761c072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 --2- 172.21.15.57:0/3719966365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f761c100fd0 0x7f761c073030 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 -- 172.21.15.57:0/3719966365 >> 172.21.15.57:0/3719966365 conn(0x7f761c0fc160 msgr2=0x7f761c0fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:16.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 -- 172.21.15.57:0/3719966365 shutdown_connections 2023-10-08T13:51:16.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:16.925+0000 7f76229c9700 1 -- 172.21.15.57:0/3719966365 wait complete. 2023-10-08T13:51:16.927 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:51:16.938 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:51:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.066+0000 7f8a6cc48700 1 -- 172.21.15.57:0/2739489213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a68105610 msgr2=0x7f8a681059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.066+0000 7f8a6cc48700 1 --2- 172.21.15.57:0/2739489213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a68105610 0x7f8a681059f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800b0d0 tx=0x7f8a5800b3e0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.066+0000 7f8a6cc48700 1 -- 172.21.15.57:0/2739489213 shutdown_connections 2023-10-08T13:51:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.066+0000 7f8a6cc48700 1 --2- 172.21.15.57:0/2739489213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a680fff10 0x7f8a6810c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.066+0000 7f8a6cc48700 1 --2- 172.21.15.57:0/2739489213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a68105610 0x7f8a681059f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.066+0000 7f8a6cc48700 1 --2- 172.21.15.57:0/2739489213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a680ff550 0x7f8a680ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.066+0000 7f8a6cc48700 1 -- 172.21.15.57:0/2739489213 >> 172.21.15.57:0/2739489213 conn(0x7f8a680fb060 msgr2=0x7f8a680fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.066+0000 7f8a6cc48700 1 -- 172.21.15.57:0/2739489213 shutdown_connections 2023-10-08T13:51:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.066+0000 7f8a6cc48700 1 -- 172.21.15.57:0/2739489213 wait complete. 2023-10-08T13:51:17.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.067+0000 7f8a6cc48700 1 Processor -- start 2023-10-08T13:51:17.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.067+0000 7f8a6cc48700 1 -- start start 2023-10-08T13:51:17.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a6cc48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a680ff550 0x7f8a6819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a6cc48700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a680fff10 0x7f8a6819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a6cc48700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a68105610 0x7f8a68196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a6cc48700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68197100 con 0x7f8a680fff10 2023-10-08T13:51:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a6cc48700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68197270 con 0x7f8a680ff550 2023-10-08T13:51:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a6cc48700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68197430 con 0x7f8a68105610 2023-10-08T13:51:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a65d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a680fff10 0x7f8a6819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a65d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a680fff10 0x7f8a6819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55688/0 (socket says 172.21.15.57:55688) 2023-10-08T13:51:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a65d9b700 1 -- 172.21.15.57:0/1294449974 learned_addr learned my addr 172.21.15.57:0/1294449974 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a6659c700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a680ff550 0x7f8a6819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a66d9d700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a68105610 0x7f8a68196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a65d9b700 1 -- 172.21.15.57:0/1294449974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a68105610 msgr2=0x7f8a68196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a65d9b700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a68105610 0x7f8a68196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a65d9b700 1 -- 172.21.15.57:0/1294449974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a680ff550 msgr2=0x7f8a6819c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a65d9b700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a680ff550 0x7f8a6819c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a65d9b700 1 -- 172.21.15.57:0/1294449974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a58009d00 con 0x7f8a680fff10 2023-10-08T13:51:17.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.068+0000 7f8a65d9b700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a680fff10 0x7f8a6819ce60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000e9c0 tx=0x7f8a5000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.069+0000 7f8a5f7fe700 1 -- 172.21.15.57:0/1294449974 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500097b0 con 0x7f8a680fff10 2023-10-08T13:51:17.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.069+0000 7f8a6cc48700 1 -- 172.21.15.57:0/1294449974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a68197710 con 0x7f8a680fff10 2023-10-08T13:51:17.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.069+0000 7f8a5f7fe700 1 -- 172.21.15.57:0/1294449974 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a50004cf0 con 0x7f8a680fff10 2023-10-08T13:51:17.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.069+0000 7f8a5f7fe700 1 -- 172.21.15.57:0/1294449974 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50005600 con 0x7f8a680fff10 2023-10-08T13:51:17.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.069+0000 7f8a6cc48700 1 -- 172.21.15.57:0/1294449974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a681a3910 con 0x7f8a680fff10 2023-10-08T13:51:17.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.069+0000 7f8a6cc48700 1 -- 172.21.15.57:0/1294449974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a68100c70 con 0x7f8a680fff10 2023-10-08T13:51:17.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.071+0000 7f8a5f7fe700 1 -- 172.21.15.57:0/1294449974 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f8a5000cc60 con 0x7f8a680fff10 2023-10-08T13:51:17.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.071+0000 7f8a5f7fe700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8a540a14d0 0x7f8a540a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.071+0000 7f8a6659c700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8a540a14d0 0x7f8a540a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.071+0000 7f8a5f7fe700 1 -- 172.21.15.57:0/1294449974 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8a50014070 con 0x7f8a680fff10 2023-10-08T13:51:17.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.072+0000 7f8a6659c700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8a540a14d0 0x7f8a540a3990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009fd0 tx=0x7f8a5800b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.075+0000 7f8a5f7fe700 1 -- 172.21.15.57:0/1294449974 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8a50096870 con 0x7f8a680fff10 2023-10-08T13:51:17.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-10-08T13:51:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:16 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-10-08T13:51:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:16 smithi115 ceph-mon[104408]: osdmap e91: 12 total, 12 up, 12 in 2023-10-08T13:51:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:16 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3243851597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:16 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/564878212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:16 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4265818274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:17.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-10-08T13:51:17.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:16 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-10-08T13:51:17.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:16 smithi057 ceph-mon[99476]: osdmap e91: 12 total, 12 up, 12 in 2023-10-08T13:51:17.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:16 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3243851597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:17.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:16 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/564878212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:17.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:16 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4265818274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-10-08T13:51:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:16 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-10-08T13:51:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:16 smithi099 ceph-mon[103849]: osdmap e91: 12 total, 12 up, 12 in 2023-10-08T13:51:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:16 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3243851597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:16 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/564878212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:16 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4265818274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:17.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.279+0000 7f8a6cc48700 1 -- 172.21.15.57:0/1294449974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a6810f4a0 con 0x7f8a680fff10 2023-10-08T13:51:17.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.281+0000 7f8a5f7fe700 1 -- 172.21.15.57:0/1294449974 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24847 (secure 0 0 0) 0x7f8a50099e90 con 0x7f8a680fff10 2023-10-08T13:51:17.281 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:17.282 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:51:16.778201+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:51:17.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 -- 172.21.15.57:0/1294449974 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8a540a14d0 msgr2=0x7f8a540a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8a540a14d0 0x7f8a540a3990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009fd0 tx=0x7f8a5800b040 comp rx=0 tx=0).stop 2023-10-08T13:51:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 -- 172.21.15.57:0/1294449974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a680fff10 msgr2=0x7f8a6819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a680fff10 0x7f8a6819ce60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000e9c0 tx=0x7f8a5000ecd0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 -- 172.21.15.57:0/1294449974 shutdown_connections 2023-10-08T13:51:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a68105610 0x7f8a68196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8a540a14d0 0x7f8a540a3990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a680ff550 0x7f8a6819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 --2- 172.21.15.57:0/1294449974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a680fff10 0x7f8a6819ce60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 -- 172.21.15.57:0/1294449974 >> 172.21.15.57:0/1294449974 conn(0x7f8a680fb060 msgr2=0x7f8a68068c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 -- 172.21.15.57:0/1294449974 shutdown_connections 2023-10-08T13:51:17.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.283+0000 7f8a6cc48700 1 -- 172.21.15.57:0/1294449974 wait complete. 2023-10-08T13:51:17.296 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-08T13:49:44.186896+0000', 'modified': '2023-10-08T13:50:07.719573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24662, 'mds_1': 24652, 'mds_2': 24634, 'mds_3': 24622, 'mds_4': 24617}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14790': {'gid': 14790, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.57:6841/1847373212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6840', 'nonce': 1847373212}, {'type': 'v1', 'addr': '172.21.15.57:6841', 'nonce': 1847373212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24557': {'gid': 24557, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6833/2949059139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 2949059139}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 2949059139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24572': {'gid': 24572, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.57:6837/4079715365', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6836', 'nonce': 4079715365}, {'type': 'v1', 'addr': '172.21.15.57:6837', 'nonce': 4079715365}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24581': {'gid': 24581, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/2732351731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 2732351731}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 2732351731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24589': {'gid': 24589, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6833/3794414355', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 3794414355}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 3794414355}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24617': {'gid': 24617, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.99:6837/822922696', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 822922696}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 822922696}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24622': {'gid': 24622, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.115:6835/389121345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 389121345}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 389121345}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24634': {'gid': 24634, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.115:6837/383062444', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 383062444}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 383062444}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24652': {'gid': 24652, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.115:6839/2892511272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 2892511272}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 2892511272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24662': {'gid': 24662, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6839/687823376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 687823376}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 687823376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T13:51:17.296 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-08T13:51:17.297 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-08T13:51:17.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.423+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/3888566014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a4106750 msgr2=0x7fe5a4106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.423+0000 7fe5aa5fe700 1 --2- 172.21.15.57:0/3888566014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a4106750 0x7fe5a4106b30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe594009990 tx=0x7fe594009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.423+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/3888566014 shutdown_connections 2023-10-08T13:51:17.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.423+0000 7fe5aa5fe700 1 --2- 172.21.15.57:0/3888566014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a4101100 0x7fe5a410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.423+0000 7fe5aa5fe700 1 --2- 172.21.15.57:0/3888566014 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a4106750 0x7fe5a4106b30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.423+0000 7fe5aa5fe700 1 --2- 172.21.15.57:0/3888566014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5a4100740 0x7fe5a4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.423+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/3888566014 >> 172.21.15.57:0/3888566014 conn(0x7fe5a40fc230 msgr2=0x7fe5a40fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.423+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/3888566014 shutdown_connections 2023-10-08T13:51:17.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.423+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/3888566014 wait complete. 2023-10-08T13:51:17.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.424+0000 7fe5aa5fe700 1 Processor -- start 2023-10-08T13:51:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.424+0000 7fe5aa5fe700 1 -- start start 2023-10-08T13:51:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.424+0000 7fe5aa5fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5a4100740 0x7fe5a419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5aa5fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a4101100 0x7fe5a419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5aa5fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a4106750 0x7fe5a4196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5aa5fe700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a4196fe0 con 0x7fe5a4101100 2023-10-08T13:51:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5aa5fe700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a4197150 con 0x7fe5a4100740 2023-10-08T13:51:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5aa5fe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a4197310 con 0x7fe5a4106750 2023-10-08T13:51:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a4101100 0x7fe5a419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a4101100 0x7fe5a419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55702/0 (socket says 172.21.15.57:55702) 2023-10-08T13:51:17.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a37fe700 1 -- 172.21.15.57:0/4000128101 learned_addr learned my addr 172.21.15.57:0/4000128101 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a4106750 0x7fe5a4196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5a4100740 0x7fe5a419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a37fe700 1 -- 172.21.15.57:0/4000128101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a4106750 msgr2=0x7fe5a4196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a4106750 0x7fe5a4196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a37fe700 1 -- 172.21.15.57:0/4000128101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5a4100740 msgr2=0x7fe5a419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5a4100740 0x7fe5a419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a37fe700 1 -- 172.21.15.57:0/4000128101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5940095f0 con 0x7fe5a4101100 2023-10-08T13:51:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a4106750 0x7fe5a4196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:17.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a4101100 0x7fe5a419cd90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00c920 tx=0x7fe58c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a17fa700 1 -- 172.21.15.57:0/4000128101 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c007b00 con 0x7fe5a4101100 2023-10-08T13:51:17.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a17fa700 1 -- 172.21.15.57:0/4000128101 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe58c004cf0 con 0x7fe5a4101100 2023-10-08T13:51:17.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5a17fa700 1 -- 172.21.15.57:0/4000128101 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c005610 con 0x7fe5a4101100 2023-10-08T13:51:17.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/4000128101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a41975f0 con 0x7fe5a4101100 2023-10-08T13:51:17.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.425+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/4000128101 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a41a3840 con 0x7fe5a4101100 2023-10-08T13:51:17.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.428+0000 7fe5a17fa700 1 -- 172.21.15.57:0/4000128101 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe58c004750 con 0x7fe5a4101100 2023-10-08T13:51:17.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.428+0000 7fe5a17fa700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe5900a14d0 0x7fe5900a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.428+0000 7fe5a17fa700 1 -- 172.21.15.57:0/4000128101 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe58c0cd130 con 0x7fe5a4101100 2023-10-08T13:51:17.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.428+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe5900a14d0 0x7fe5900a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.429+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe5900a14d0 0x7fe5900a3990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe594005850 tx=0x7fe59401a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.429+0000 7fe5b67e7b80 1 Processor -- start 2023-10-08T13:51:17.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.429+0000 7fe5b67e7b80 1 -- start start 2023-10-08T13:51:17.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.429+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6ce30 0x564490d6d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.429+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6d750 0x564490d780c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.429+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 0x564490d7a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.429+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490cd6410 con 0x564490d6d750 2023-10-08T13:51:17.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.429+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490cc7bd0 con 0x564490d6ce30 2023-10-08T13:51:17.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.429+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490cc54f0 con 0x564490d78600 2023-10-08T13:51:17.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.429+0000 7fe5a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6d750 0x564490d780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe5a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6d750 0x564490d780c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55710/0 (socket says 172.21.15.57:55710) 2023-10-08T13:51:17.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe5a37fe700 1 -- 172.21.15.57:0/2008580704 learned_addr learned my addr 172.21.15.57:0/2008580704 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/2008580704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6ce30 0x564490d6d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/2008580704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 0x564490d7a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe5a37fe700 1 -- 172.21.15.57:0/2008580704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 msgr2=0x564490d7a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/2008580704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 0x564490d7a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe5a37fe700 1 -- 172.21.15.57:0/2008580704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6ce30 msgr2=0x564490d6d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/2008580704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6ce30 0x564490d6d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe5a37fe700 1 -- 172.21.15.57:0/2008580704 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564490aebd90 con 0x564490d6d750 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/2008580704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6d750 0x564490d780c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe58c099fc0 tx=0x7fe58c099bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a215cce8022aa683 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe59a7fc700 1 -- 172.21.15.57:0/2008580704 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c09e180 con 0x564490d6d750 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe59a7fc700 1 -- 172.21.15.57:0/2008580704 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe58c0a23c0 con 0x564490d6d750 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.430+0000 7fe59a7fc700 1 -- 172.21.15.57:0/2008580704 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c004b50 con 0x564490d6d750 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/2008580704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6d750 msgr2=0x564490d780c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/2008580704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6d750 0x564490d780c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe58c099fc0 tx=0x7fe58c099bc0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/2008580704 shutdown_connections 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/2008580704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 0x564490d7a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/2008580704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6ce30 0x564490d6d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/2008580704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6d750 0x564490d780c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/2008580704 >> 172.21.15.57:0/2008580704 conn(0x564490ce2210 msgr2=0x564490d69280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/2008580704 shutdown_connections 2023-10-08T13:51:17.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/2008580704 wait complete. 2023-10-08T13:51:17.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 -- start start 2023-10-08T13:51:17.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.431+0000 7fe5b67e7b80 1 Processor -- start 2023-10-08T13:51:17.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6ce30 0x564490d78050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d75390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 0x564490d75ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490cc0de0 con 0x564490d6ce30 2023-10-08T13:51:17.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490c5d6c0 con 0x564490d6d750 2023-10-08T13:51:17.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490c5d360 con 0x564490d78600 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5b67e7b80 10 client.0 _ll_register_callbacks cb 0x564490aba7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6ce30 0x564490d78050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6ce30 0x564490d78050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55714/0 (socket says 172.21.15.57:55714) 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a3fff700 1 -- 172.21.15.57:0/3702123345 learned_addr learned my addr 172.21.15.57:0/3702123345 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 0x564490d75ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d75390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a3fff700 1 -- 172.21.15.57:0/3702123345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 msgr2=0x564490d75ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 0x564490d75ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a3fff700 1 -- 172.21.15.57:0/3702123345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 msgr2=0x564490d75390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d75390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a3fff700 1 -- 172.21.15.57:0/3702123345 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a41a3840 con 0x564490d6ce30 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 0x564490d75ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6ce30 0x564490d78050 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe5940052b0 tx=0x7fe594009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c5827e7ae03831 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5837fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5837fe700 1 -- 172.21.15.57:0/3702123345 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59401d040 con 0x564490d6ce30 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5837fe700 1 -- 172.21.15.57:0/3702123345 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe59400f460 con 0x564490d6ce30 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.432+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/3702123345 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564490c88e60 con 0x564490d6ce30 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5837fe700 1 -- 172.21.15.57:0/3702123345 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe59400bb00 con 0x564490d6ce30 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5b67e7b80 10 client.15582 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/3702123345 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564490ad4c10 con 0x564490d6ce30 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5837fe700 1 -- 172.21.15.57:0/3702123345 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564490c88e60 con 0x564490d6ce30 2023-10-08T13:51:17.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5837fe700 1 -- 172.21.15.57:0/3702123345 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe5940246b0 con 0x564490d6ce30 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5b67e7b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5b67e7b80 10 client.15582 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5b67e7b80 20 client.15582 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x564490d88170 0x564490d86c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5b67e7b80 4 client.15582 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/3702123345 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564490cb9dc0 con 0x564490d88170 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.433+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x564490d88170 0x564490d86c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.434+0000 7fe5837fe700 10 client.15582 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.434+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x564490d88170 0x564490d86c80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c15633c27f6ad487 server_cookie=35f13a20887a2986 in_seq=0 out_seq=0 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.434+0000 7fe5837fe700 1 -- 172.21.15.57:0/3702123345 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564490cb9dc0 con 0x564490d88170 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.434+0000 7fe5837fe700 10 client.15582 handle_command_reply: tid=0 2023-10-08T13:51:17.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.434+0000 7fe5b67e7b80 1 client.15582 shutdown 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/3702123345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6ce30 msgr2=0x564490d78050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6ce30 0x564490d78050 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe5940052b0 tx=0x7fe594009310 comp rx=0 tx=0).stop 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/3702123345 shutdown_connections 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x564490d88170 0x564490d86c80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d78600 0x564490d75ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d6ce30 0x564490d78050 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/3702123345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d75390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5837fe700 0 client.15582 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/3702123345 >> 172.21.15.57:0/3702123345 conn(0x564490ce2210 msgr2=0x564490d68e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/3702123345 shutdown_connections 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/3702123345 wait complete. 2023-10-08T13:51:17.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.435+0000 7fe5b67e7b80 20 client.15582 trim_cache size 0 max 16384 2023-10-08T13:51:17.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5b67e7b80 1 Processor -- start 2023-10-08T13:51:17.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5b67e7b80 1 -- start start 2023-10-08T13:51:17.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d6ce30 0x564490d87390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d90ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d77b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490b418d0 con 0x564490d78600 2023-10-08T13:51:17.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490c674d0 con 0x564490d6d750 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490b4dde0 con 0x564490d6ce30 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a8b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d77b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a8b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d77b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55726/0 (socket says 172.21.15.57:55726) 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a8b9b700 1 -- 172.21.15.57:0/1224923228 learned_addr learned my addr 172.21.15.57:0/1224923228 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/1224923228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d6ce30 0x564490d87390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/1224923228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d90ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a8b9b700 1 -- 172.21.15.57:0/1224923228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d6ce30 msgr2=0x564490d87390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/1224923228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d6ce30 0x564490d87390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a8b9b700 1 -- 172.21.15.57:0/1224923228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 msgr2=0x564490d90ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/1224923228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d90ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a8b9b700 1 -- 172.21.15.57:0/1224923228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564490b12200 con 0x564490d78600 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.453+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/1224923228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d90ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/1224923228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d77b80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009cb0 tx=0x7fe59c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35c5e4efcf96b30 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe59affd700 1 -- 172.21.15.57:0/1224923228 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00e070 con 0x564490d78600 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe59affd700 1 -- 172.21.15.57:0/1224923228 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe59c016400 con 0x564490d78600 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe59affd700 1 -- 172.21.15.57:0/1224923228 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0155c0 con 0x564490d78600 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/1224923228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 msgr2=0x564490d77b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/1224923228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d77b80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009cb0 tx=0x7fe59c009040 comp rx=0 tx=0).stop 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/1224923228 shutdown_connections 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/1224923228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d77b80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/1224923228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d6ce30 0x564490d87390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/1224923228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d90ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/1224923228 >> 172.21.15.57:0/1224923228 conn(0x564490ce2210 msgr2=0x564490d86c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/1224923228 shutdown_connections 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.454+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/1224923228 wait complete. 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5b67e7b80 1 -- start start 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5b67e7b80 1 Processor -- start 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d6ce30 0x564490d90c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d8b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5b67e7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d8b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490d8bef0 con 0x564490d78600 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490ddbb60 con 0x564490d6d750 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5b67e7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564490ddbca0 con 0x564490d6ce30 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5b67e7b80 10 client.0 _ll_register_callbacks cb 0x7fe59400b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5a8b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d8b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5a8b9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d8b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55730/0 (socket says 172.21.15.57:55730) 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5a8b9b700 1 -- 172.21.15.57:0/49478697 learned_addr learned my addr 172.21.15.57:0/49478697 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.455+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d6ce30 0x564490d90c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5a37fe700 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d8b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5a8b9b700 1 -- 172.21.15.57:0/49478697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d6ce30 msgr2=0x564490d90c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d6ce30 0x564490d90c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5a8b9b700 1 -- 172.21.15.57:0/49478697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 msgr2=0x564490d8b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d8b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5a8b9b700 1 -- 172.21.15.57:0/49478697 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c011040 con 0x564490d78600 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5a8b9b700 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d8b7e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe59c016d00 tx=0x7fe59c016b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e5f4b22cb39dc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5837fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5837fe700 1 -- 172.21.15.57:0/49478697 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00e040 con 0x564490d78600 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5837fe700 1 -- 172.21.15.57:0/49478697 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe59c0168f0 con 0x564490d78600 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/49478697 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe58c0a23c0 con 0x564490d78600 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5837fe700 1 -- 172.21.15.57:0/49478697 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe59c007a40 con 0x564490d78600 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5b67e7b80 10 client.15588 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/49478697 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564490aea260 con 0x564490d78600 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.456+0000 7fe5837fe700 1 -- 172.21.15.57:0/49478697 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe59c032020 con 0x564490d78600 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5837fe700 1 -- 172.21.15.57:0/49478697 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe59c005040 con 0x564490d78600 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5b67e7b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5b67e7b80 10 client.15588 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5b67e7b80 20 client.15588 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:17.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x564490de0e40 0x564490ddd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5b67e7b80 4 client.15588 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/49478697 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564490cb9ae0 con 0x564490de0e40 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x564490de0e40 0x564490ddd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5837fe700 10 client.15588 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5a3fff700 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x564490de0e40 0x564490ddd260 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10e77b02d6dc6adc server_cookie=933d9bb3167dfd51 in_seq=0 out_seq=0 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5837fe700 1 -- 172.21.15.57:0/49478697 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564490cb9ae0 con 0x564490de0e40 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5837fe700 10 client.15588 handle_command_reply: tid=0 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.457+0000 7fe5b67e7b80 1 client.15588 shutdown 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/49478697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 msgr2=0x564490d8b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d8b7e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe59c016d00 tx=0x7fe59c016b20 comp rx=0 tx=0).stop 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/49478697 shutdown_connections 2023-10-08T13:51:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x564490de0e40 0x564490ddd260 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564490d78600 0x564490d8b7e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564490d6ce30 0x564490d90c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5837fe700 0 client.15588 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:51:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 1 --2- 172.21.15.57:0/49478697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564490d6d750 0x564490d8b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/49478697 >> 172.21.15.57:0/49478697 conn(0x564490ce2210 msgr2=0x564490d68c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/49478697 shutdown_connections 2023-10-08T13:51:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 1 -- 172.21.15.57:0/49478697 wait complete. 2023-10-08T13:51:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5b67e7b80 20 client.15588 trim_cache size 0 max 16384 2023-10-08T13:51:17.465 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 4, 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24617, 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 70.143655445999997, 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 27, 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 92, 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 88, 2023-10-08T13:51:17.466 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 107.569731889 2023-10-08T13:51:17.467 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:51:17.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/4000128101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe5900a14d0 msgr2=0x7fe5900a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe5900a14d0 0x7fe5900a3990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe594005850 tx=0x7fe59401a040 comp rx=0 tx=0).stop 2023-10-08T13:51:17.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/4000128101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a4101100 msgr2=0x7fe5a419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a4101100 0x7fe5a419cd90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00c920 tx=0x7fe58c00cce0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/4000128101 shutdown_connections 2023-10-08T13:51:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a4106750 0x7fe5a4196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe5900a14d0 0x7fe5900a3990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5a4100740 0x7fe5a419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 --2- 172.21.15.57:0/4000128101 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5a4101100 0x7fe5a419cd90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/4000128101 >> 172.21.15.57:0/4000128101 conn(0x7fe5a40fc230 msgr2=0x7fe5a40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/4000128101 shutdown_connections 2023-10-08T13:51:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.458+0000 7fe5aa5fe700 1 -- 172.21.15.57:0/4000128101 wait complete. 2023-10-08T13:51:17.471 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-08T13:51:17.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.599+0000 7fa06c629700 1 -- 172.21.15.57:0/4116756458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0640fffe0 msgr2=0x7fa06410c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.599+0000 7fa06c629700 1 --2- 172.21.15.57:0/4116756458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0640fffe0 0x7fa06410c4e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00b0a0 tx=0x7fa05c00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.599+0000 7fa06c629700 1 -- 172.21.15.57:0/4116756458 shutdown_connections 2023-10-08T13:51:17.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.599+0000 7fa06c629700 1 --2- 172.21.15.57:0/4116756458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0640fffe0 0x7fa06410c4e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.599+0000 7fa06c629700 1 --2- 172.21.15.57:0/4116756458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0641056b0 0x7fa064105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.599+0000 7fa06c629700 1 --2- 172.21.15.57:0/4116756458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0640ff620 0x7fa0640ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.599+0000 7fa06c629700 1 -- 172.21.15.57:0/4116756458 >> 172.21.15.57:0/4116756458 conn(0x7fa064076780 msgr2=0x7fa064076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa06c629700 1 -- 172.21.15.57:0/4116756458 shutdown_connections 2023-10-08T13:51:17.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa06c629700 1 -- 172.21.15.57:0/4116756458 wait complete. 2023-10-08T13:51:17.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa06c629700 1 Processor -- start 2023-10-08T13:51:17.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa06c629700 1 -- start start 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa06c629700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0640ff620 0x7fa064101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa06c629700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0640fffe0 0x7fa064101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa06c629700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0641056b0 0x7fa064101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa06c629700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0641a4ee0 con 0x7fa0640fffe0 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa06c629700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0641a5050 con 0x7fa0641056b0 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa06c629700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0641a5210 con 0x7fa0640ff620 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.600+0000 7fa069bc4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0640fffe0 0x7fa064101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa069bc4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0640fffe0 0x7fa064101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55744/0 (socket says 172.21.15.57:55744) 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa069bc4700 1 -- 172.21.15.57:0/3915190155 learned_addr learned my addr 172.21.15.57:0/3915190155 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0640ff620 0x7fa064101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa06abc6700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0641056b0 0x7fa064101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa069bc4700 1 -- 172.21.15.57:0/3915190155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0640ff620 msgr2=0x7fa064101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa069bc4700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0640ff620 0x7fa064101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa069bc4700 1 -- 172.21.15.57:0/3915190155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0641056b0 msgr2=0x7fa064101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa069bc4700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0641056b0 0x7fa064101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa069bc4700 1 -- 172.21.15.57:0/3915190155 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c009d00 con 0x7fa0640fffe0 2023-10-08T13:51:17.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa069bc4700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0640fffe0 0x7fa064101a80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa06000d950 tx=0x7fa06000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3915190155 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060009940 con 0x7fa0640fffe0 2023-10-08T13:51:17.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3915190155 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa060010460 con 0x7fa0640fffe0 2023-10-08T13:51:17.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3915190155 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000f620 con 0x7fa0640fffe0 2023-10-08T13:51:17.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa06c629700 1 -- 172.21.15.57:0/3915190155 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0641a54f0 con 0x7fa0640fffe0 2023-10-08T13:51:17.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.601+0000 7fa06c629700 1 -- 172.21.15.57:0/3915190155 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0641a59b0 con 0x7fa0640fffe0 2023-10-08T13:51:17.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.603+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3915190155 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fa0600108d0 con 0x7fa0640fffe0 2023-10-08T13:51:17.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.604+0000 7fa05b7fe700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0500a14d0 0x7fa0500a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.604+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3915190155 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa0600cd050 con 0x7fa0640fffe0 2023-10-08T13:51:17.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.604+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0500a14d0 0x7fa0500a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.604+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0500a14d0 0x7fa0500a3990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa064077ab0 tx=0x7fa05400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa078812b80 1 Processor -- start 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa078812b80 1 -- start start 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa078812b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde1e30 0x560aafde2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa078812b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde2750 0x560aafded0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa078812b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafded600 0x560aafdef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa078812b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafd4b410 con 0x560aafde1e30 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa078812b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafd3cbd0 con 0x560aafded600 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa078812b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafd3a4f0 con 0x560aafde2750 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa06a3c5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde1e30 0x560aafde2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa06a3c5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde1e30 0x560aafde2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55750/0 (socket says 172.21.15.57:55750) 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.605+0000 7fa06a3c5700 1 -- 172.21.15.57:0/1810059587 learned_addr learned my addr 172.21.15.57:0/1810059587 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa069bc4700 1 --2- 172.21.15.57:0/1810059587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde2750 0x560aafded0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa06abc6700 1 --2- 172.21.15.57:0/1810059587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafded600 0x560aafdef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa06a3c5700 1 -- 172.21.15.57:0/1810059587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde2750 msgr2=0x560aafded0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/1810059587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde2750 0x560aafded0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa06a3c5700 1 -- 172.21.15.57:0/1810059587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafded600 msgr2=0x560aafdef9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/1810059587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafded600 0x560aafdef9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa06a3c5700 1 -- 172.21.15.57:0/1810059587 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aafb60d90 con 0x560aafde1e30 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa06abc6700 1 --2- 172.21.15.57:0/1810059587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafded600 0x560aafdef9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/1810059587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde1e30 0x560aafde2210 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa054010e30 tx=0x7fa054013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc43b8f31377a5e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa058ff9700 1 -- 172.21.15.57:0/1810059587 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054011db0 con 0x560aafde1e30 2023-10-08T13:51:17.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa058ff9700 1 -- 172.21.15.57:0/1810059587 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa054015070 con 0x560aafde1e30 2023-10-08T13:51:17.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.606+0000 7fa058ff9700 1 -- 172.21.15.57:0/1810059587 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05401c610 con 0x560aafde1e30 2023-10-08T13:51:17.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 -- 172.21.15.57:0/1810059587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde1e30 msgr2=0x560aafde2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 --2- 172.21.15.57:0/1810059587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde1e30 0x560aafde2210 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa054010e30 tx=0x7fa054013000 comp rx=0 tx=0).stop 2023-10-08T13:51:17.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 -- 172.21.15.57:0/1810059587 shutdown_connections 2023-10-08T13:51:17.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 --2- 172.21.15.57:0/1810059587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafded600 0x560aafdef9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 --2- 172.21.15.57:0/1810059587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde1e30 0x560aafde2210 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 --2- 172.21.15.57:0/1810059587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde2750 0x560aafded0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 -- 172.21.15.57:0/1810059587 >> 172.21.15.57:0/1810059587 conn(0x560aafd57210 msgr2=0x560aafdde280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 -- 172.21.15.57:0/1810059587 shutdown_connections 2023-10-08T13:51:17.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 -- 172.21.15.57:0/1810059587 wait complete. 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 -- start start 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.607+0000 7fa078812b80 1 Processor -- start 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa078812b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafded000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa078812b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafde2750 0x560aafdea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa078812b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafdea900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa078812b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafd35de0 con 0x560aafded600 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa078812b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafcd26c0 con 0x560aafde2750 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa078812b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafcd2360 con 0x560aafde1e30 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa078812b80 10 client.0 _ll_register_callbacks cb 0x560aafb2f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06abc6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafdea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06abc6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafdea900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55760/0 (socket says 172.21.15.57:55760) 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06abc6700 1 -- 172.21.15.57:0/3970416526 learned_addr learned my addr 172.21.15.57:0/3970416526 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa069bc4700 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafde2750 0x560aafdea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafded000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06abc6700 1 -- 172.21.15.57:0/3970416526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 msgr2=0x560aafded000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06abc6700 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafded000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06abc6700 1 -- 172.21.15.57:0/3970416526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafde2750 msgr2=0x560aafdea2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06abc6700 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafde2750 0x560aafdea2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06abc6700 1 -- 172.21.15.57:0/3970416526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa054010ae0 con 0x560aafded600 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafded000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa069bc4700 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafde2750 0x560aafdea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa06abc6700 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafdea900 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa05c015000 tx=0x7fa05c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4213d619424a49 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.608+0000 7fa04dffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa04dffb700 1 -- 172.21.15.57:0/3970416526 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00e040 con 0x560aafded600 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa078812b80 1 -- 172.21.15.57:0/3970416526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560aafcfde60 con 0x560aafded600 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa04dffb700 1 -- 172.21.15.57:0/3970416526 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa05c007970 con 0x560aafded600 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa04dffb700 1 -- 172.21.15.57:0/3970416526 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa05c01c440 con 0x560aafded600 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa078812b80 10 client.15600 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa078812b80 1 -- 172.21.15.57:0/3970416526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560aafb49c10 con 0x560aafded600 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa04dffb700 1 -- 172.21.15.57:0/3970416526 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560aafcfde60 con 0x560aafded600 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa04dffb700 1 -- 172.21.15.57:0/3970416526 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa05c019050 con 0x560aafded600 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa078812b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa078812b80 10 client.15600 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa078812b80 20 client.15600 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:17.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa078812b80 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x560aafdfd1e0 0x560aafdfbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa078812b80 4 client.15600 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.609+0000 7fa078812b80 1 -- 172.21.15.57:0/3970416526 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560aafd2edc0 con 0x560aafdfd1e0 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.610+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x560aafdfd1e0 0x560aafdfbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.610+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x560aafdfd1e0 0x560aafdfbc60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61e4ce6061c07468 server_cookie=6f07cf8597750739 in_seq=0 out_seq=0 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.610+0000 7fa04dffb700 10 client.15600 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa04dffb700 1 -- 172.21.15.57:0/3970416526 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560aafd2edc0 con 0x560aafdfd1e0 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa04dffb700 10 client.15600 handle_command_reply: tid=0 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 client.15600 shutdown 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 -- 172.21.15.57:0/3970416526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 msgr2=0x560aafdea900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafdea900 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa05c015000 tx=0x7fa05c004db0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 -- 172.21.15.57:0/3970416526 shutdown_connections 2023-10-08T13:51:17.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x560aafdfd1e0 0x560aafdfbc60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafdea900 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa04dffb700 0 client.15600 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:51:17.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafded000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 --2- 172.21.15.57:0/3970416526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafde2750 0x560aafdea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 -- 172.21.15.57:0/3970416526 >> 172.21.15.57:0/3970416526 conn(0x560aafd57210 msgr2=0x560aafddde70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 -- 172.21.15.57:0/3970416526 shutdown_connections 2023-10-08T13:51:17.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 1 -- 172.21.15.57:0/3970416526 wait complete. 2023-10-08T13:51:17.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.611+0000 7fa078812b80 20 client.15600 trim_cache size 0 max 16384 2023-10-08T13:51:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa078812b80 1 Processor -- start 2023-10-08T13:51:17.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa078812b80 1 -- start start 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa078812b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafdecb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa078812b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafde2750 0x560aafde55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa078812b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafde5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa078812b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafbb68d0 con 0x560aafded600 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa078812b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafcdc500 con 0x560aafde2750 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa078812b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafcf4590 con 0x560aafde1e30 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa06abc6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafde5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa06abc6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafde5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55768/0 (socket says 172.21.15.57:55768) 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa06abc6700 1 -- 172.21.15.57:0/664540221 learned_addr learned my addr 172.21.15.57:0/664540221 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/664540221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafdecb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa06abc6700 1 -- 172.21.15.57:0/664540221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 msgr2=0x560aafdecb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa06abc6700 1 --2- 172.21.15.57:0/664540221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafdecb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa06abc6700 1 -- 172.21.15.57:0/664540221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafde2750 msgr2=0x560aafde55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa06abc6700 1 --2- 172.21.15.57:0/664540221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafde2750 0x560aafde55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa06abc6700 1 -- 172.21.15.57:0/664540221 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aafb87200 con 0x560aafded600 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa06abc6700 1 --2- 172.21.15.57:0/664540221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafde5b00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009a20 tx=0x7fa05c00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c36dca37acb1f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa0597fa700 1 -- 172.21.15.57:0/664540221 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00e040 con 0x560aafded600 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa0597fa700 1 -- 172.21.15.57:0/664540221 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa05c0050d0 con 0x560aafded600 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.629+0000 7fa0597fa700 1 -- 172.21.15.57:0/664540221 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c01c860 con 0x560aafded600 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 -- 172.21.15.57:0/664540221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 msgr2=0x560aafde5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 --2- 172.21.15.57:0/664540221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafde5b00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009a20 tx=0x7fa05c00b930 comp rx=0 tx=0).stop 2023-10-08T13:51:17.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 -- 172.21.15.57:0/664540221 shutdown_connections 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 --2- 172.21.15.57:0/664540221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafded600 0x560aafde5b00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 --2- 172.21.15.57:0/664540221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafdecb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 --2- 172.21.15.57:0/664540221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafde2750 0x560aafde55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 -- 172.21.15.57:0/664540221 >> 172.21.15.57:0/664540221 conn(0x560aafd57210 msgr2=0x560aafddd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 -- 172.21.15.57:0/664540221 shutdown_connections 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 -- 172.21.15.57:0/664540221 wait complete. 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 -- start start 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.630+0000 7fa078812b80 1 Processor -- start 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa078812b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafe05b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa078812b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde2750 0x560aafdef040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa078812b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafded600 0x560aafded9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa078812b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafdef720 con 0x560aafde2750 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa078812b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafdedf20 con 0x560aafded600 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa078812b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aafdee0e0 con 0x560aafde1e30 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa078812b80 10 client.0 _ll_register_callbacks cb 0x7fa05c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa069bc4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde2750 0x560aafdef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa069bc4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde2750 0x560aafdef040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55772/0 (socket says 172.21.15.57:55772) 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa069bc4700 1 -- 172.21.15.57:0/2768220115 learned_addr learned my addr 172.21.15.57:0/2768220115 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafe05b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa069bc4700 1 -- 172.21.15.57:0/2768220115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 msgr2=0x560aafe05b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa069bc4700 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafe05b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa069bc4700 1 -- 172.21.15.57:0/2768220115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafded600 msgr2=0x560aafded9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa069bc4700 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafded600 0x560aafded9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa069bc4700 1 -- 172.21.15.57:0/2768220115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560aafb49c10 con 0x560aafde2750 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa069bc4700 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde2750 0x560aafdef040 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa060010fd0 tx=0x7fa060099f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324f124e813a2d18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa04dffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa04dffb700 1 -- 172.21.15.57:0/2768220115 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060014070 con 0x560aafde2750 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa04dffb700 1 -- 172.21.15.57:0/2768220115 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa06009a350 con 0x560aafde2750 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.631+0000 7fa078812b80 1 -- 172.21.15.57:0/2768220115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa054015070 con 0x560aafde2750 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa04dffb700 1 -- 172.21.15.57:0/2768220115 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0600b8170 con 0x560aafde2750 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa078812b80 10 client.15606 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa078812b80 1 -- 172.21.15.57:0/2768220115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560aafb5f260 con 0x560aafde2750 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa04dffb700 1 -- 172.21.15.57:0/2768220115 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa060063080 con 0x560aafde2750 2023-10-08T13:51:17.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa04dffb700 1 -- 172.21.15.57:0/2768220115 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa0600cab90 con 0x560aafde2750 2023-10-08T13:51:17.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa078812b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:17.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa078812b80 10 client.15606 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-08T13:51:17.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa078812b80 20 client.15606 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:17.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa078812b80 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x560aafe55540 0x560aafe51960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa078812b80 4 client.15606 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:51:17.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa078812b80 1 -- 172.21.15.57:0/2768220115 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560aafd2eae0 con 0x560aafe55540 2023-10-08T13:51:17.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.632+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x560aafe55540 0x560aafe51960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa04dffb700 10 client.15606 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:51:17.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa06a3c5700 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x560aafe55540 0x560aafe51960 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3d7a1de342c196c server_cookie=4b2b790579801fcb in_seq=0 out_seq=0 2023-10-08T13:51:17.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa04dffb700 1 -- 172.21.15.57:0/2768220115 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560aafd2eae0 con 0x560aafe55540 2023-10-08T13:51:17.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa04dffb700 10 client.15606 handle_command_reply: tid=0 2023-10-08T13:51:17.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 client.15606 shutdown 2023-10-08T13:51:17.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 -- 172.21.15.57:0/2768220115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde2750 msgr2=0x560aafdef040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde2750 0x560aafdef040 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa060010fd0 tx=0x7fa060099f70 comp rx=0 tx=0).stop 2023-10-08T13:51:17.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 -- 172.21.15.57:0/2768220115 shutdown_connections 2023-10-08T13:51:17.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x560aafe55540 0x560aafe51960 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aafded600 0x560aafded9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560aafde1e30 0x560aafe05b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 --2- 172.21.15.57:0/2768220115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560aafde2750 0x560aafdef040 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa04dffb700 0 client.15606 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:51:17.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 -- 172.21.15.57:0/2768220115 >> 172.21.15.57:0/2768220115 conn(0x560aafd57210 msgr2=0x560aafde55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 -- 172.21.15.57:0/2768220115 shutdown_connections 2023-10-08T13:51:17.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 1 -- 172.21.15.57:0/2768220115 wait complete. 2023-10-08T13:51:17.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.633+0000 7fa078812b80 20 client.15606 trim_cache size 0 max 16384 2023-10-08T13:51:17.642 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:51:17.643 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:51:17.643 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 3, 2023-10-08T13:51:17.643 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24622, 2023-10-08T13:51:17.643 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:51:17.643 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:51:17.643 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:51:17.643 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 71.329359660999998, 2023-10-08T13:51:17.643 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 27, 2023-10-08T13:51:17.643 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 92, 2023-10-08T13:51:17.644 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 88, 2023-10-08T13:51:17.644 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 112.30591098399999 2023-10-08T13:51:17.644 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:51:17.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 -- 172.21.15.57:0/3915190155 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0500a14d0 msgr2=0x7fa0500a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0500a14d0 0x7fa0500a3990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa064077ab0 tx=0x7fa05400b410 comp rx=0 tx=0).stop 2023-10-08T13:51:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 -- 172.21.15.57:0/3915190155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0640fffe0 msgr2=0x7fa064101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0640fffe0 0x7fa064101a80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa06000d950 tx=0x7fa06000dc60 comp rx=0 tx=0).stop 2023-10-08T13:51:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 -- 172.21.15.57:0/3915190155 shutdown_connections 2023-10-08T13:51:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa0500a14d0 0x7fa0500a3990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0641056b0 0x7fa064101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0640ff620 0x7fa064101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 --2- 172.21.15.57:0/3915190155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0640fffe0 0x7fa064101a80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 -- 172.21.15.57:0/3915190155 >> 172.21.15.57:0/3915190155 conn(0x7fa064076780 msgr2=0x7fa064103820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 -- 172.21.15.57:0/3915190155 shutdown_connections 2023-10-08T13:51:17.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.634+0000 7fa06c629700 1 -- 172.21.15.57:0/3915190155 wait complete. 2023-10-08T13:51:17.647 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T13:51:17.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.773+0000 7fe4dc94f700 1 -- 172.21.15.57:0/396024291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d80fffe0 msgr2=0x7fe4d810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.773+0000 7fe4dc94f700 1 --2- 172.21.15.57:0/396024291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d80fffe0 0x7fe4d810c4e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000b0a0 tx=0x7fe4d000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.774+0000 7fe4dc94f700 1 -- 172.21.15.57:0/396024291 shutdown_connections 2023-10-08T13:51:17.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.774+0000 7fe4dc94f700 1 --2- 172.21.15.57:0/396024291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d80fffe0 0x7fe4d810c4e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.774+0000 7fe4dc94f700 1 --2- 172.21.15.57:0/396024291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4d81056b0 0x7fe4d8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.774+0000 7fe4dc94f700 1 --2- 172.21.15.57:0/396024291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4d80ff620 0x7fe4d80ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.774+0000 7fe4dc94f700 1 -- 172.21.15.57:0/396024291 >> 172.21.15.57:0/396024291 conn(0x7fe4d8076780 msgr2=0x7fe4d8076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.774+0000 7fe4dc94f700 1 -- 172.21.15.57:0/396024291 shutdown_connections 2023-10-08T13:51:17.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.774+0000 7fe4dc94f700 1 -- 172.21.15.57:0/396024291 wait complete. 2023-10-08T13:51:17.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.774+0000 7fe4dc94f700 1 Processor -- start 2023-10-08T13:51:17.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.774+0000 7fe4dc94f700 1 -- start start 2023-10-08T13:51:17.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4dc94f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4d80ff620 0x7fe4d8197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4dc94f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4d80fffe0 0x7fe4d8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4dc94f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d81056b0 0x7fe4d81a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4dc94f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8198730 con 0x7fe4d81056b0 2023-10-08T13:51:17.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4dc94f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d81a2f30 con 0x7fe4d80fffe0 2023-10-08T13:51:17.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4dc94f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d81a3070 con 0x7fe4d80ff620 2023-10-08T13:51:17.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d81056b0 0x7fe4d81a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d81056b0 0x7fe4d81a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55788/0 (socket says 172.21.15.57:55788) 2023-10-08T13:51:17.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d6d9d700 1 -- 172.21.15.57:0/2668268353 learned_addr learned my addr 172.21.15.57:0/2668268353 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d659c700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4d80ff620 0x7fe4d8197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d5d9b700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4d80fffe0 0x7fe4d8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d6d9d700 1 -- 172.21.15.57:0/2668268353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4d80ff620 msgr2=0x7fe4d8197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d6d9d700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4d80ff620 0x7fe4d8197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d6d9d700 1 -- 172.21.15.57:0/2668268353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4d80fffe0 msgr2=0x7fe4d8197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d6d9d700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4d80fffe0 0x7fe4d8197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d6d9d700 1 -- 172.21.15.57:0/2668268353 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d0009d00 con 0x7fe4d81056b0 2023-10-08T13:51:17.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4d6d9d700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d81056b0 0x7fe4d81a29f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000b070 tx=0x7fe4d0007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4c77fe700 1 -- 172.21.15.57:0/2668268353 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000e040 con 0x7fe4d81056b0 2023-10-08T13:51:17.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4dc94f700 1 -- 172.21.15.57:0/2668268353 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d81a32a0 con 0x7fe4d81056b0 2023-10-08T13:51:17.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4c77fe700 1 -- 172.21.15.57:0/2668268353 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d0012dc0 con 0x7fe4d81056b0 2023-10-08T13:51:17.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.775+0000 7fe4c77fe700 1 -- 172.21.15.57:0/2668268353 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001b450 con 0x7fe4d81056b0 2023-10-08T13:51:17.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.776+0000 7fe4dc94f700 1 -- 172.21.15.57:0/2668268353 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d81a36c0 con 0x7fe4d81056b0 2023-10-08T13:51:17.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.777+0000 7fe4c77fe700 1 -- 172.21.15.57:0/2668268353 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe4d001d680 con 0x7fe4d81056b0 2023-10-08T13:51:17.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.778+0000 7fe4c77fe700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4c00a14d0 0x7fe4c00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.778+0000 7fe4c77fe700 1 -- 172.21.15.57:0/2668268353 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe4d00cf130 con 0x7fe4d81056b0 2023-10-08T13:51:17.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.778+0000 7fe4d659c700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4c00a14d0 0x7fe4c00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.779+0000 7fe4d659c700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4c00a14d0 0x7fe4c00a3990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8077ab0 tx=0x7fe4c800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.779+0000 7fe4e8b38b80 1 Processor -- start 2023-10-08T13:51:17.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.779+0000 7fe4e8b38b80 1 -- start start 2023-10-08T13:51:17.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.779+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e01033210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.779+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01033750 0x559e0103e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.779+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e010409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.779+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e00f9c410 con 0x559e01032e30 2023-10-08T13:51:17.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.779+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e00f8dbd0 con 0x559e01033750 2023-10-08T13:51:17.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.779+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e00f8b4f0 con 0x559e0103e600 2023-10-08T13:51:17.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.779+0000 7fe4d659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e01033210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4d659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e01033210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55792/0 (socket says 172.21.15.57:55792) 2023-10-08T13:51:17.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4d659c700 1 -- 172.21.15.57:0/591412271 learned_addr learned my addr 172.21.15.57:0/591412271 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4d6d9d700 1 --2- 172.21.15.57:0/591412271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e010409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4d5d9b700 1 --2- 172.21.15.57:0/591412271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01033750 0x559e0103e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4d659c700 1 -- 172.21.15.57:0/591412271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 msgr2=0x559e010409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4d659c700 1 --2- 172.21.15.57:0/591412271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e010409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4d659c700 1 -- 172.21.15.57:0/591412271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01033750 msgr2=0x559e0103e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4d659c700 1 --2- 172.21.15.57:0/591412271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01033750 0x559e0103e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4d659c700 1 -- 172.21.15.57:0/591412271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e00db1d90 con 0x559e01032e30 2023-10-08T13:51:17.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4d659c700 1 --2- 172.21.15.57:0/591412271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e01033210 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8010e30 tx=0x7fe4c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4ff14781950d8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4c4ff9700 1 -- 172.21.15.57:0/591412271 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8011db0 con 0x559e01032e30 2023-10-08T13:51:17.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4c4ff9700 1 -- 172.21.15.57:0/591412271 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4c8015070 con 0x559e01032e30 2023-10-08T13:51:17.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.780+0000 7fe4c4ff9700 1 -- 172.21.15.57:0/591412271 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c801c610 con 0x559e01032e30 2023-10-08T13:51:17.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/591412271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 msgr2=0x559e01033210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/591412271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e01033210 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8010e30 tx=0x7fe4c8013000 comp rx=0 tx=0).stop 2023-10-08T13:51:17.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/591412271 shutdown_connections 2023-10-08T13:51:17.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/591412271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e010409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/591412271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e01033210 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/591412271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01033750 0x559e0103e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/591412271 >> 172.21.15.57:0/591412271 conn(0x559e00fa8210 msgr2=0x559e0102f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/591412271 shutdown_connections 2023-10-08T13:51:17.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/591412271 wait complete. 2023-10-08T13:51:17.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 -- start start 2023-10-08T13:51:17.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.781+0000 7fe4e8b38b80 1 Processor -- start 2023-10-08T13:51:17.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 0x559e0103e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e0103b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e0103b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e00f86de0 con 0x559e01033750 2023-10-08T13:51:17.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e00f236c0 con 0x559e01032e30 2023-10-08T13:51:17.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e00f23360 con 0x559e0103e600 2023-10-08T13:51:17.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4e8b38b80 10 client.0 _ll_register_callbacks cb 0x559e00d807b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:17.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e0103b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e0103b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55800/0 (socket says 172.21.15.57:55800) 2023-10-08T13:51:17.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d5d9b700 1 -- 172.21.15.57:0/1860479064 learned_addr learned my addr 172.21.15.57:0/1860479064 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d659c700 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 0x559e0103e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d6d9d700 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e0103b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d5d9b700 1 -- 172.21.15.57:0/1860479064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 msgr2=0x559e0103b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d5d9b700 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e0103b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d5d9b700 1 -- 172.21.15.57:0/1860479064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 msgr2=0x559e0103e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d5d9b700 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 0x559e0103e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d5d9b700 1 -- 172.21.15.57:0/1860479064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c8010ae0 con 0x559e01033750 2023-10-08T13:51:17.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4d5d9b700 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e0103b2a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0107c0 tx=0x7fe4cc010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f3c9e090c9f64d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.782+0000 7fe4bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:17.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4bdffb700 1 -- 172.21.15.57:0/1860479064 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc011970 con 0x559e01033750 2023-10-08T13:51:17.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4bdffb700 1 -- 172.21.15.57:0/1860479064 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe4cc017070 con 0x559e01033750 2023-10-08T13:51:17.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/1860479064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e00f4ee60 con 0x559e01033750 2023-10-08T13:51:17.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4bdffb700 1 -- 172.21.15.57:0/1860479064 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4cc011ad0 con 0x559e01033750 2023-10-08T13:51:17.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4e8b38b80 10 client.15618 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/1860479064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559e00d9ac10 con 0x559e01033750 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4bdffb700 1 -- 172.21.15.57:0/1860479064 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e00f4ee60 con 0x559e01033750 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4bdffb700 1 -- 172.21.15.57:0/1860479064 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe4cc01a380 con 0x559e01033750 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4e8b38b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4e8b38b80 10 client.15618 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4e8b38b80 20 client.15618 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x559e0104e1c0 0x559e0104cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4e8b38b80 4 client.15618 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.783+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/1860479064 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e00f7fdc0 con 0x559e0104e1c0 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.784+0000 7fe4d659c700 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x559e0104e1c0 0x559e0104cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.784+0000 7fe4bdffb700 10 client.15618 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.784+0000 7fe4d659c700 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x559e0104e1c0 0x559e0104cc90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2345118689efcd9 server_cookie=eb1733f0544cd84d in_seq=0 out_seq=0 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4bdffb700 1 -- 172.21.15.57:0/1860479064 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559e00f7fdc0 con 0x559e0104e1c0 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4bdffb700 10 client.15618 handle_command_reply: tid=0 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 client.15618 shutdown 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/1860479064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 msgr2=0x559e0103b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e0103b2a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0107c0 tx=0x7fe4cc010ad0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/1860479064 shutdown_connections 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x559e0104e1c0 0x559e0104cc90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e0103b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4bdffb700 0 client.15618 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:51:17.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 0x559e0103e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/1860479064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e0103b2a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/1860479064 >> 172.21.15.57:0/1860479064 conn(0x559e00fa8210 msgr2=0x559e0102ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/1860479064 shutdown_connections 2023-10-08T13:51:17.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/1860479064 wait complete. 2023-10-08T13:51:17.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.785+0000 7fe4e8b38b80 20 client.15618 trim_cache size 0 max 16384 2023-10-08T13:51:17.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.803+0000 7fe4e8b38b80 1 Processor -- start 2023-10-08T13:51:17.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4e8b38b80 1 -- start start 2023-10-08T13:51:17.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e0104a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e01033750 0x559e0103db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e0103e600 0x559e0102ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e00e078d0 con 0x559e01032e30 2023-10-08T13:51:17.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e00f82ed0 con 0x559e0103e600 2023-10-08T13:51:17.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e00f2d4d0 con 0x559e01033750 2023-10-08T13:51:17.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e0104a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e0104a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55806/0 (socket says 172.21.15.57:55806) 2023-10-08T13:51:17.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d659c700 1 -- 172.21.15.57:0/268272044 learned_addr learned my addr 172.21.15.57:0/268272044 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d5d9b700 1 --2- 172.21.15.57:0/268272044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e01033750 0x559e0103db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d6d9d700 1 --2- 172.21.15.57:0/268272044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e0103e600 0x559e0102ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d659c700 1 -- 172.21.15.57:0/268272044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e01033750 msgr2=0x559e0103db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d659c700 1 --2- 172.21.15.57:0/268272044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e01033750 0x559e0103db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d659c700 1 -- 172.21.15.57:0/268272044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e0103e600 msgr2=0x559e0102ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d659c700 1 --2- 172.21.15.57:0/268272044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e0103e600 0x559e0102ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d659c700 1 -- 172.21.15.57:0/268272044 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e00dd8200 con 0x559e01032e30 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d6d9d700 1 --2- 172.21.15.57:0/268272044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e0103e600 0x559e0102ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4d659c700 1 --2- 172.21.15.57:0/268272044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e0104a170 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8010e30 tx=0x7fe4c800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afcc816a776ff94 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.804+0000 7fe4c57fa700 1 -- 172.21.15.57:0/268272044 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80325c0 con 0x559e01032e30 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4c57fa700 1 -- 172.21.15.57:0/268272044 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4c802d040 con 0x559e01032e30 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4c57fa700 1 -- 172.21.15.57:0/268272044 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c801e820 con 0x559e01032e30 2023-10-08T13:51:17.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/268272044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 msgr2=0x559e0104a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/268272044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e0104a170 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8010e30 tx=0x7fe4c800bf90 comp rx=0 tx=0).stop 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/268272044 shutdown_connections 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/268272044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e0103e600 0x559e0102ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/268272044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01032e30 0x559e0104a170 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/268272044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e01033750 0x559e0103db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/268272044 >> 172.21.15.57:0/268272044 conn(0x559e00fa8210 msgr2=0x559e01039bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/268272044 shutdown_connections 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.805+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/268272044 wait complete. 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4e8b38b80 1 -- start start 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4e8b38b80 1 Processor -- start 2023-10-08T13:51:17.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 0x559e01038eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e010375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4e8b38b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e01037af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e010381e0 con 0x559e01033750 2023-10-08T13:51:17.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e01038350 con 0x559e01032e30 2023-10-08T13:51:17.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4e8b38b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e010a5be0 con 0x559e0103e600 2023-10-08T13:51:17.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4e8b38b80 10 client.0 _ll_register_callbacks cb 0x7fe4cc00cc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:17.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e010375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e010375b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55814/0 (socket says 172.21.15.57:55814) 2023-10-08T13:51:17.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d5d9b700 1 -- 172.21.15.57:0/3573553722 learned_addr learned my addr 172.21.15.57:0/3573553722 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d659c700 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 0x559e01038eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d5d9b700 1 -- 172.21.15.57:0/3573553722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 msgr2=0x559e01037af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d5d9b700 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e01037af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d5d9b700 1 -- 172.21.15.57:0/3573553722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 msgr2=0x559e01038eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d5d9b700 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 0x559e01038eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d5d9b700 1 -- 172.21.15.57:0/3573553722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e00db1d90 con 0x559e01033750 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d659c700 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 0x559e01038eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4d5d9b700 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e010375b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc01a7b0 tx=0x7fe4cc01a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e7103ac01a4c0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.806+0000 7fe4bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4bdffb700 1 -- 172.21.15.57:0/3573553722 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc004220 con 0x559e01033750 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/3573553722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4c8015070 con 0x559e01033750 2023-10-08T13:51:17.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4bdffb700 1 -- 172.21.15.57:0/3573553722 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe4cc017040 con 0x559e01033750 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4bdffb700 1 -- 172.21.15.57:0/3573553722 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4cc004380 con 0x559e01033750 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4e8b38b80 10 client.15624 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/3573553722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559e00db0260 con 0x559e01033750 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4bdffb700 1 -- 172.21.15.57:0/3573553722 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4c8015070 con 0x559e01033750 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4bdffb700 1 -- 172.21.15.57:0/3573553722 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe4cc02f030 con 0x559e01033750 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4e8b38b80 10 client.15624 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4e8b38b80 10 client.15624 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4e8b38b80 20 client.15624 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x559e010aae60 0x559e010a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4e8b38b80 4 client.15624 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:51:17.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.807+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/3573553722 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559e00f7fae0 con 0x559e010aae60 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4d659c700 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x559e010aae60 0x559e010a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4bdffb700 10 client.15624 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4d659c700 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x559e010aae60 0x559e010a7260 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65558f85132e3ba2 server_cookie=3411132049b9e19a in_seq=0 out_seq=0 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4bdffb700 1 -- 172.21.15.57:0/3573553722 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x559e00f7fae0 con 0x559e010aae60 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4bdffb700 10 client.15624 handle_command_reply: tid=0 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4e8b38b80 1 client.15624 shutdown 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/3573553722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 msgr2=0x559e010375b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e010375b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc01a7b0 tx=0x7fe4cc01a9d0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/3573553722 shutdown_connections 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x559e010aae60 0x559e010a7260 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e0103e600 0x559e01037af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e01032e30 0x559e01038eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4bdffb700 0 client.15624 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:51:17.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4e8b38b80 1 --2- 172.21.15.57:0/3573553722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559e01033750 0x559e010375b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.808+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/3573553722 >> 172.21.15.57:0/3573553722 conn(0x559e00fa8210 msgr2=0x559e0102eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/3573553722 shutdown_connections 2023-10-08T13:51:17.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4e8b38b80 1 -- 172.21.15.57:0/3573553722 wait complete. 2023-10-08T13:51:17.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4e8b38b80 20 client.15624 trim_cache size 0 max 16384 2023-10-08T13:51:17.817 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:51:17.818 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:51:17.818 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 2, 2023-10-08T13:51:17.818 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24634, 2023-10-08T13:51:17.818 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:51:17.818 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:51:17.818 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:51:17.818 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 72.520917808999997, 2023-10-08T13:51:17.818 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 27, 2023-10-08T13:51:17.818 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 92, 2023-10-08T13:51:17.818 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 88, 2023-10-08T13:51:17.819 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 110.33360180299999 2023-10-08T13:51:17.819 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:51:17.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 -- 172.21.15.57:0/2668268353 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4c00a14d0 msgr2=0x7fe4c00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4c00a14d0 0x7fe4c00a3990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8077ab0 tx=0x7fe4c800b410 comp rx=0 tx=0).stop 2023-10-08T13:51:17.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 -- 172.21.15.57:0/2668268353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d81056b0 msgr2=0x7fe4d81a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d81056b0 0x7fe4d81a29f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000b070 tx=0x7fe4d0007b70 comp rx=0 tx=0).stop 2023-10-08T13:51:17.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 -- 172.21.15.57:0/2668268353 shutdown_connections 2023-10-08T13:51:17.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4d81056b0 0x7fe4d81a29f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4d80ff620 0x7fe4d8197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4c00a14d0 0x7fe4c00a3990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 --2- 172.21.15.57:0/2668268353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4d80fffe0 0x7fe4d8197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 -- 172.21.15.57:0/2668268353 >> 172.21.15.57:0/2668268353 conn(0x7fe4d8076780 msgr2=0x7fe4d80fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 -- 172.21.15.57:0/2668268353 shutdown_connections 2023-10-08T13:51:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.809+0000 7fe4dc94f700 1 -- 172.21.15.57:0/2668268353 wait complete. 2023-10-08T13:51:17.822 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-08T13:51:17.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.949+0000 7f2601fb4700 1 -- 172.21.15.57:0/2042974694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25fc0fffe0 msgr2=0x7f25fc10c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.949+0000 7f2601fb4700 1 --2- 172.21.15.57:0/2042974694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25fc0fffe0 0x7f25fc10c4e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b0a0 tx=0x7f25f400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.949+0000 7f2601fb4700 1 -- 172.21.15.57:0/2042974694 shutdown_connections 2023-10-08T13:51:17.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.949+0000 7f2601fb4700 1 --2- 172.21.15.57:0/2042974694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25fc0fffe0 0x7f25fc10c4e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.949+0000 7f2601fb4700 1 --2- 172.21.15.57:0/2042974694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25fc1056b0 0x7f25fc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.949+0000 7f2601fb4700 1 --2- 172.21.15.57:0/2042974694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25fc0ff620 0x7f25fc0ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.949+0000 7f2601fb4700 1 -- 172.21.15.57:0/2042974694 >> 172.21.15.57:0/2042974694 conn(0x7f25fc076780 msgr2=0x7f25fc076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.949+0000 7f2601fb4700 1 -- 172.21.15.57:0/2042974694 shutdown_connections 2023-10-08T13:51:17.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.949+0000 7f2601fb4700 1 -- 172.21.15.57:0/2042974694 wait complete. 2023-10-08T13:51:17.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f2601fb4700 1 Processor -- start 2023-10-08T13:51:17.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f2601fb4700 1 -- start start 2023-10-08T13:51:17.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f2601fb4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25fc0ff620 0x7f25fc197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f2601fb4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25fc0fffe0 0x7f25fc197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f2601fb4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25fc1056b0 0x7f25fc1a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f2601fb4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1986a0 con 0x7f25fc0fffe0 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f2601fb4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1a2ef0 con 0x7f25fc0ff620 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f2601fb4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1a3030 con 0x7f25fc1056b0 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f25faffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25fc0fffe0 0x7f25fc197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f25faffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25fc0fffe0 0x7f25fc197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55828/0 (socket says 172.21.15.57:55828) 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.950+0000 7f25faffd700 1 -- 172.21.15.57:0/778543328 learned_addr learned my addr 172.21.15.57:0/778543328 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25fbfff700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25fc1056b0 0x7f25fc1a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25fc0ff620 0x7f25fc197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25faffd700 1 -- 172.21.15.57:0/778543328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25fc1056b0 msgr2=0x7f25fc1a29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25faffd700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25fc1056b0 0x7f25fc1a29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25faffd700 1 -- 172.21.15.57:0/778543328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25fc0ff620 msgr2=0x7f25fc197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25faffd700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25fc0ff620 0x7f25fc197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25faffd700 1 -- 172.21.15.57:0/778543328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f4009d00 con 0x7f25fc0fffe0 2023-10-08T13:51:17.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25fbfff700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25fc1056b0 0x7f25fc1a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:17.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25faffd700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25fc0fffe0 0x7f25fc197e70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f25e400d950 tx=0x7f25e400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25f8ff9700 1 -- 172.21.15.57:0/778543328 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4009b40 con 0x7f25fc0fffe0 2023-10-08T13:51:17.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f2601fb4700 1 -- 172.21.15.57:0/778543328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc1a3290 con 0x7f25fc0fffe0 2023-10-08T13:51:17.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25f8ff9700 1 -- 172.21.15.57:0/778543328 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e4009ca0 con 0x7f25fc0fffe0 2023-10-08T13:51:17.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f25f8ff9700 1 -- 172.21.15.57:0/778543328 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e400f620 con 0x7f25fc0fffe0 2023-10-08T13:51:17.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.951+0000 7f2601fb4700 1 -- 172.21.15.57:0/778543328 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc1a3770 con 0x7f25fc0fffe0 2023-10-08T13:51:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.953+0000 7f25f8ff9700 1 -- 172.21.15.57:0/778543328 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f25e4010460 con 0x7f25fc0fffe0 2023-10-08T13:51:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.954+0000 7f25f8ff9700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25e80a14d0 0x7f25e80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.954+0000 7f25f8ff9700 1 -- 172.21.15.57:0/778543328 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f25e40cd510 con 0x7f25fc0fffe0 2023-10-08T13:51:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.954+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25e80a14d0 0x7f25e80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.954+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25e80a14d0 0x7f25e80a3990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f25fc077ab0 tx=0x7f25ec009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f260e19db80 1 Processor -- start 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f260e19db80 1 -- start start 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f260e19db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e40ee30 0x55fe4e40f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f260e19db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e41e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f260e19db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e41ea10 0x55fe4e420e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f260e19db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e378410 con 0x55fe4e41ea10 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f260e19db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e369bd0 con 0x55fe4e40f750 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f260e19db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e3674f0 con 0x55fe4e40ee30 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25fbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e41ea10 0x55fe4e420e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25fbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e41ea10 0x55fe4e420e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55838/0 (socket says 172.21.15.57:55838) 2023-10-08T13:51:17.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25fbfff700 1 -- 172.21.15.57:0/3453643046 learned_addr learned my addr 172.21.15.57:0/3453643046 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25faffd700 1 --2- 172.21.15.57:0/3453643046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e41e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/3453643046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e40ee30 0x55fe4e40f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25fbfff700 1 -- 172.21.15.57:0/3453643046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e40ee30 msgr2=0x55fe4e40f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25fbfff700 1 --2- 172.21.15.57:0/3453643046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e40ee30 0x55fe4e40f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25fbfff700 1 -- 172.21.15.57:0/3453643046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 msgr2=0x55fe4e41e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25fbfff700 1 --2- 172.21.15.57:0/3453643046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e41e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25fbfff700 1 -- 172.21.15.57:0/3453643046 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe4e18dd90 con 0x55fe4e41ea10 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.955+0000 7f25faffd700 1 --2- 172.21.15.57:0/3453643046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e41e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.956+0000 7f25fbfff700 1 --2- 172.21.15.57:0/3453643046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e41ea10 0x55fe4e420e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b070 tx=0x7f25f4009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e17c8b35eac184 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.956+0000 7f25f1ffb700 1 -- 172.21.15.57:0/3453643046 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f400e040 con 0x55fe4e41ea10 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.956+0000 7f25f1ffb700 1 -- 172.21.15.57:0/3453643046 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25f4007e80 con 0x55fe4e41ea10 2023-10-08T13:51:17.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.956+0000 7f25f1ffb700 1 -- 172.21.15.57:0/3453643046 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f401d470 con 0x55fe4e41ea10 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.956+0000 7f260e19db80 1 -- 172.21.15.57:0/3453643046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e41ea10 msgr2=0x55fe4e420e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.956+0000 7f260e19db80 1 --2- 172.21.15.57:0/3453643046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e41ea10 0x55fe4e420e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b070 tx=0x7f25f4009450 comp rx=0 tx=0).stop 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.957+0000 7f260e19db80 1 -- 172.21.15.57:0/3453643046 shutdown_connections 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.957+0000 7f260e19db80 1 --2- 172.21.15.57:0/3453643046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e41ea10 0x55fe4e420e00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.957+0000 7f260e19db80 1 --2- 172.21.15.57:0/3453643046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e40ee30 0x55fe4e40f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.957+0000 7f260e19db80 1 --2- 172.21.15.57:0/3453643046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e41e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.957+0000 7f260e19db80 1 -- 172.21.15.57:0/3453643046 >> 172.21.15.57:0/3453643046 conn(0x55fe4e384210 msgr2=0x55fe4e40b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.957+0000 7f260e19db80 1 -- 172.21.15.57:0/3453643046 shutdown_connections 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.957+0000 7f260e19db80 1 -- 172.21.15.57:0/3453643046 wait complete. 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.957+0000 7f260e19db80 1 -- start start 2023-10-08T13:51:17.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.957+0000 7f260e19db80 1 Processor -- start 2023-10-08T13:51:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f260e19db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e41e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f260e19db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e41b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f260e19db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 0x55fe4e41bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f260e19db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e362de0 con 0x55fe4e40ee30 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f260e19db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e2ff6c0 con 0x55fe4e40f750 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f260e19db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e2ff360 con 0x55fe4e41ea10 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f260e19db80 10 client.0 _ll_register_callbacks cb 0x55fe4e15c7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e41e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e41e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55840/0 (socket says 172.21.15.57:55840) 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fb7fe700 1 -- 172.21.15.57:0/446263034 learned_addr learned my addr 172.21.15.57:0/446263034 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fbfff700 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 0x55fe4e41bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25faffd700 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e41b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fb7fe700 1 -- 172.21.15.57:0/446263034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 msgr2=0x55fe4e41bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 0x55fe4e41bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fb7fe700 1 -- 172.21.15.57:0/446263034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 msgr2=0x55fe4e41b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e41b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fb7fe700 1 -- 172.21.15.57:0/446263034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc10cde0 con 0x55fe4e40ee30 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fbfff700 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 0x55fe4e41bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25faffd700 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e41b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e41e380 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0126b0 tx=0x7f25ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee7656b97ca590b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.958+0000 7f25daffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f25daffd700 1 -- 172.21.15.57:0/446263034 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec01f830 con 0x55fe4e40ee30 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f25daffd700 1 -- 172.21.15.57:0/446263034 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f25ec018070 con 0x55fe4e40ee30 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f260e19db80 1 -- 172.21.15.57:0/446263034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe4e35bdc0 con 0x55fe4e40ee30 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f25daffd700 1 -- 172.21.15.57:0/446263034 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ec012e50 con 0x55fe4e40ee30 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f260e19db80 10 client.15636 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:17.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f260e19db80 1 -- 172.21.15.57:0/446263034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fe4e176c10 con 0x55fe4e40ee30 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f25daffd700 1 -- 172.21.15.57:0/446263034 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe4e35bdc0 con 0x55fe4e40ee30 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f25daffd700 1 -- 172.21.15.57:0/446263034 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f25ec01cb50 con 0x55fe4e40ee30 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f260e19db80 10 client.15636 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f260e19db80 10 client.15636 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f260e19db80 20 client.15636 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f260e19db80 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55fe4e42e500 0x55fe4e42cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f260e19db80 4 client.15636 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.959+0000 7f260e19db80 1 -- 172.21.15.57:0/446263034 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe4e35bae0 con 0x55fe4e42e500 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.960+0000 7f25faffd700 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55fe4e42e500 0x55fe4e42cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.960+0000 7f25faffd700 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55fe4e42e500 0x55fe4e42cfd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe9c02bd6e839a94 server_cookie=1d65263741cbd259 in_seq=0 out_seq=0 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.960+0000 7f25daffd700 10 client.15636 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:51:17.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f25daffd700 1 -- 172.21.15.57:0/446263034 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe4e35bae0 con 0x55fe4e42e500 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f25daffd700 10 client.15636 handle_command_reply: tid=0 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 client.15636 shutdown 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 -- 172.21.15.57:0/446263034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 msgr2=0x55fe4e41e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e41e380 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0126b0 tx=0x7f25ec0047c0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 -- 172.21.15.57:0/446263034 shutdown_connections 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55fe4e42e500 0x55fe4e42cfd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 0x55fe4e41bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e41e380 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 --2- 172.21.15.57:0/446263034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e41b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f25daffd700 0 client.15636 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 -- 172.21.15.57:0/446263034 >> 172.21.15.57:0/446263034 conn(0x55fe4e384210 msgr2=0x55fe4e40ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 -- 172.21.15.57:0/446263034 shutdown_connections 2023-10-08T13:51:17.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 1 -- 172.21.15.57:0/446263034 wait complete. 2023-10-08T13:51:17.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.961+0000 7f260e19db80 20 client.15636 trim_cache size 0 max 16384 2023-10-08T13:51:17.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f260e19db80 1 Processor -- start 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f260e19db80 1 -- start start 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f260e19db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e42a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f260e19db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e40f750 0x55fe4e436e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f260e19db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e41ea10 0x55fe4e42cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f260e19db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e1e38d0 con 0x55fe4e40ee30 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f260e19db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e321590 con 0x55fe4e41ea10 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f260e19db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e364070 con 0x55fe4e40f750 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25fb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e42a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25fb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e42a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55848/0 (socket says 172.21.15.57:55848) 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25fb7fe700 1 -- 172.21.15.57:0/3889686575 learned_addr learned my addr 172.21.15.57:0/3889686575 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25faffd700 1 --2- 172.21.15.57:0/3889686575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e40f750 0x55fe4e436e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25fbfff700 1 --2- 172.21.15.57:0/3889686575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e41ea10 0x55fe4e42cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25fb7fe700 1 -- 172.21.15.57:0/3889686575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e40f750 msgr2=0x55fe4e436e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/3889686575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e40f750 0x55fe4e436e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25fb7fe700 1 -- 172.21.15.57:0/3889686575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e41ea10 msgr2=0x55fe4e42cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/3889686575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e41ea10 0x55fe4e42cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25fb7fe700 1 -- 172.21.15.57:0/3889686575 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe4e1b4200 con 0x55fe4e40ee30 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.979+0000 7f25fbfff700 1 --2- 172.21.15.57:0/3889686575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e41ea10 0x55fe4e42cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/3889686575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e42a520 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0097d0 tx=0x7f25ec0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7a0bb0c5450d21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f25f27fc700 1 -- 172.21.15.57:0/3889686575 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec0092d0 con 0x55fe4e40ee30 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f25f27fc700 1 -- 172.21.15.57:0/3889686575 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25ec016070 con 0x55fe4e40ee30 2023-10-08T13:51:17.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f25f27fc700 1 -- 172.21.15.57:0/3889686575 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec0038e0 con 0x55fe4e40ee30 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f260e19db80 1 -- 172.21.15.57:0/3889686575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 msgr2=0x55fe4e42a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f260e19db80 1 --2- 172.21.15.57:0/3889686575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e42a520 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0097d0 tx=0x7f25ec0046f0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f260e19db80 1 -- 172.21.15.57:0/3889686575 shutdown_connections 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f260e19db80 1 --2- 172.21.15.57:0/3889686575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e41ea10 0x55fe4e42cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f260e19db80 1 --2- 172.21.15.57:0/3889686575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e42a520 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f260e19db80 1 --2- 172.21.15.57:0/3889686575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e40f750 0x55fe4e436e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f260e19db80 1 -- 172.21.15.57:0/3889686575 >> 172.21.15.57:0/3889686575 conn(0x55fe4e384210 msgr2=0x55fe4e419f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f260e19db80 1 -- 172.21.15.57:0/3889686575 shutdown_connections 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.980+0000 7f260e19db80 1 -- 172.21.15.57:0/3889686575 wait complete. 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f260e19db80 1 -- start start 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f260e19db80 1 Processor -- start 2023-10-08T13:51:17.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f260e19db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e436e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f260e19db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e417d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f260e19db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 0x55fe4e418250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f260e19db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e4187d0 con 0x55fe4e40ee30 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f260e19db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e418940 con 0x55fe4e40f750 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f260e19db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4e418b00 con 0x55fe4e41ea10 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f260e19db80 10 client.0 _ll_register_callbacks cb 0x7f25ec011e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f25fb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e436e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f25fb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e436e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55854/0 (socket says 172.21.15.57:55854) 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f25fb7fe700 1 -- 172.21.15.57:0/1542213219 learned_addr learned my addr 172.21.15.57:0/1542213219 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.981+0000 7f25faffd700 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e417d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25fbfff700 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 0x55fe4e418250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25fb7fe700 1 -- 172.21.15.57:0/1542213219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 msgr2=0x55fe4e418250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 0x55fe4e418250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25fb7fe700 1 -- 172.21.15.57:0/1542213219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 msgr2=0x55fe4e417d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e417d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25fb7fe700 1 -- 172.21.15.57:0/1542213219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe4e176c10 con 0x55fe4e40ee30 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25fb7fe700 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e436e80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f25ec01e440 tx=0x7f25ec009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1297464ccca087 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25daffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25daffd700 1 -- 172.21.15.57:0/1542213219 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec005250 con 0x55fe4e40ee30 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25daffd700 1 -- 172.21.15.57:0/1542213219 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f25ec018050 con 0x55fe4e40ee30 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f260e19db80 1 -- 172.21.15.57:0/1542213219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe4e32ae60 con 0x55fe4e40ee30 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25daffd700 1 -- 172.21.15.57:0/1542213219 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ec0053b0 con 0x55fe4e40ee30 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f260e19db80 10 client.15642 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:17.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f260e19db80 1 -- 172.21.15.57:0/1542213219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fe4e18c260 con 0x55fe4e40ee30 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.982+0000 7f25daffd700 1 -- 172.21.15.57:0/1542213219 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe4e32ae60 con 0x55fe4e40ee30 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f25daffd700 1 -- 172.21.15.57:0/1542213219 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f25ec03b070 con 0x55fe4e40ee30 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f260e19db80 10 client.15642 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f260e19db80 10 client.15642 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f260e19db80 20 client.15642 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f260e19db80 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55fe4e486aa0 0x55fe4e482f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f260e19db80 4 client.15642 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f260e19db80 1 -- 172.21.15.57:0/1542213219 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fe4e1aef20 con 0x55fe4e486aa0 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f25faffd700 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55fe4e486aa0 0x55fe4e482f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f25daffd700 10 client.15642 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:51:17.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f25faffd700 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55fe4e486aa0 0x55fe4e482f10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e08df96b8fafd40c server_cookie=48a9106e04485aed in_seq=0 out_seq=0 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f25daffd700 1 -- 172.21.15.57:0/1542213219 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fe4e1aef20 con 0x55fe4e486aa0 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.983+0000 7f25daffd700 10 client.15642 handle_command_reply: tid=0 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 client.15642 shutdown 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 -- 172.21.15.57:0/1542213219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 msgr2=0x55fe4e436e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e436e80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f25ec01e440 tx=0x7f25ec009340 comp rx=0 tx=0).stop 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 -- 172.21.15.57:0/1542213219 shutdown_connections 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55fe4e486aa0 0x55fe4e482f10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe4e41ea10 0x55fe4e418250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f25daffd700 0 client.15642 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fe4e40ee30 0x55fe4e436e80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 --2- 172.21.15.57:0/1542213219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4e40f750 0x55fe4e417d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 -- 172.21.15.57:0/1542213219 >> 172.21.15.57:0/1542213219 conn(0x55fe4e384210 msgr2=0x55fe4e419e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 -- 172.21.15.57:0/1542213219 shutdown_connections 2023-10-08T13:51:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 1 -- 172.21.15.57:0/1542213219 wait complete. 2023-10-08T13:51:17.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f260e19db80 20 client.15642 trim_cache size 0 max 16384 2023-10-08T13:51:17.992 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:51:17.992 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:51:17.992 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 1, 2023-10-08T13:51:17.992 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24652, 2023-10-08T13:51:17.992 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:51:17.992 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:51:17.993 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:51:17.993 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 73.710618397000005, 2023-10-08T13:51:17.993 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 27, 2023-10-08T13:51:17.993 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 92, 2023-10-08T13:51:17.993 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 88, 2023-10-08T13:51:17.993 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 105.918327152 2023-10-08T13:51:17.993 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:51:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f2601fb4700 1 -- 172.21.15.57:0/778543328 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25e80a14d0 msgr2=0x7f25e80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f2601fb4700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25e80a14d0 0x7f25e80a3990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f25fc077ab0 tx=0x7f25ec009260 comp rx=0 tx=0).stop 2023-10-08T13:51:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f2601fb4700 1 -- 172.21.15.57:0/778543328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25fc0fffe0 msgr2=0x7f25fc197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f2601fb4700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25fc0fffe0 0x7f25fc197e70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f25e400d950 tx=0x7f25e400dd10 comp rx=0 tx=0).stop 2023-10-08T13:51:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f2601fb4700 1 -- 172.21.15.57:0/778543328 shutdown_connections 2023-10-08T13:51:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f2601fb4700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25fc1056b0 0x7f25fc1a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f2601fb4700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f25e80a14d0 0x7f25e80a3990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f2601fb4700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25fc0ff620 0x7f25fc197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f2601fb4700 1 --2- 172.21.15.57:0/778543328 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25fc0fffe0 0x7f25fc197e70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:17.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.984+0000 7f2601fb4700 1 -- 172.21.15.57:0/778543328 >> 172.21.15.57:0/778543328 conn(0x7f25fc076780 msgr2=0x7f25fc0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:17.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.985+0000 7f2601fb4700 1 -- 172.21.15.57:0/778543328 shutdown_connections 2023-10-08T13:51:17.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:17.985+0000 7f2601fb4700 1 -- 172.21.15.57:0/778543328 wait complete. 2023-10-08T13:51:17.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:17 smithi057 ceph-mon[99476]: pgmap v290: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-10-08T13:51:17.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:17 smithi057 ceph-mon[99476]: osdmap e92: 12 total, 12 up, 12 in 2023-10-08T13:51:17.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:17 smithi057 ceph-mon[99476]: mgrmap e22: x(active, since 7m), standbys: z, y 2023-10-08T13:51:17.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:17 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3719966365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:17.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:17 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1294449974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:17.997 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T13:51:18.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.124+0000 7f3a24ca9700 1 -- 172.21.15.57:0/829341260 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a20104130 msgr2=0x7f3a2010bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:18.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.124+0000 7f3a24ca9700 1 --2- 172.21.15.57:0/829341260 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a20104130 0x7f3a2010bcb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800b0a0 tx=0x7f3a1800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.124+0000 7f3a24ca9700 1 -- 172.21.15.57:0/829341260 shutdown_connections 2023-10-08T13:51:18.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.124+0000 7f3a24ca9700 1 --2- 172.21.15.57:0/829341260 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a20104130 0x7f3a2010bcb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.124+0000 7f3a24ca9700 1 --2- 172.21.15.57:0/829341260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a200ff060 0x7f3a200ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.124+0000 7f3a24ca9700 1 --2- 172.21.15.57:0/829341260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a200ffa10 0x7f3a20103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.124+0000 7f3a24ca9700 1 -- 172.21.15.57:0/829341260 >> 172.21.15.57:0/829341260 conn(0x7f3a20075c20 msgr2=0x7f3a200fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.124+0000 7f3a24ca9700 1 -- 172.21.15.57:0/829341260 shutdown_connections 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.124+0000 7f3a24ca9700 1 -- 172.21.15.57:0/829341260 wait complete. 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a24ca9700 1 Processor -- start 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a24ca9700 1 -- start start 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a24ca9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a200ff060 0x7f3a2019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a24ca9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20104130 0x7f3a2019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a24ca9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a20194940 0x7f3a20194df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a24ca9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201953c0 con 0x7f3a200ff060 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a24ca9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a20195530 con 0x7f3a20194940 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a24ca9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201956f0 con 0x7f3a20104130 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a200ff060 0x7f3a2019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a200ff060 0x7f3a2019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55868/0 (socket says 172.21.15.57:55868) 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a1e59c700 1 -- 172.21.15.57:0/3136881292 learned_addr learned my addr 172.21.15.57:0/3136881292 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a17fff700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20104130 0x7f3a2019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a1ed9d700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a20194940 0x7f3a20194df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a1e59c700 1 -- 172.21.15.57:0/3136881292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20104130 msgr2=0x7f3a2019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20104130 0x7f3a2019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a1e59c700 1 -- 172.21.15.57:0/3136881292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a20194940 msgr2=0x7f3a20194df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a20194940 0x7f3a20194df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.125+0000 7f3a1e59c700 1 -- 172.21.15.57:0/3136881292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a18009d00 con 0x7f3a200ff060 2023-10-08T13:51:18.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.126+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a200ff060 0x7f3a2019a810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000ad80 tx=0x7f3a100084b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:18.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.126+0000 7f3a177fe700 1 -- 172.21.15.57:0/3136881292 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10012070 con 0x7f3a200ff060 2023-10-08T13:51:18.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.126+0000 7f3a177fe700 1 -- 172.21.15.57:0/3136881292 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a10018420 con 0x7f3a200ff060 2023-10-08T13:51:18.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.126+0000 7f3a177fe700 1 -- 172.21.15.57:0/3136881292 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10016580 con 0x7f3a200ff060 2023-10-08T13:51:18.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.126+0000 7f3a24ca9700 1 -- 172.21.15.57:0/3136881292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a201a16e0 con 0x7f3a200ff060 2023-10-08T13:51:18.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.126+0000 7f3a24ca9700 1 -- 172.21.15.57:0/3136881292 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a201a1bc0 con 0x7f3a200ff060 2023-10-08T13:51:18.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.129+0000 7f3a177fe700 1 -- 172.21.15.57:0/3136881292 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f3a10016070 con 0x7f3a200ff060 2023-10-08T13:51:18.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.129+0000 7f3a177fe700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a0c0a1620 0x7f3a0c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.129+0000 7f3a177fe700 1 -- 172.21.15.57:0/3136881292 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3a100ce3f0 con 0x7f3a200ff060 2023-10-08T13:51:18.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.129+0000 7f3a17fff700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a0c0a1620 0x7f3a0c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a17fff700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a0c0a1620 0x7f3a0c0a3ae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3a20195b00 tx=0x7f3a0800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:18.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a30e92b80 1 Processor -- start 2023-10-08T13:51:18.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a30e92b80 1 -- start start 2023-10-08T13:51:18.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f72185f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72185f750 0x55f72186a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72186a600 0x55f72186c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7217c8410 con 0x55f72185ee30 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7217b9bd0 con 0x55f72185f750 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7217b74f0 con 0x55f72186a600 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f72185f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f72185f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55876/0 (socket says 172.21.15.57:55876) 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a1e59c700 1 -- 172.21.15.57:0/2522878108 learned_addr learned my addr 172.21.15.57:0/2522878108 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.130+0000 7f3a17fff700 1 --2- 172.21.15.57:0/2522878108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72185f750 0x55f72186a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a1ed9d700 1 --2- 172.21.15.57:0/2522878108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72186a600 0x55f72186c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a1e59c700 1 -- 172.21.15.57:0/2522878108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72186a600 msgr2=0x55f72186c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/2522878108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72186a600 0x55f72186c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a1e59c700 1 -- 172.21.15.57:0/2522878108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72185f750 msgr2=0x55f72186a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/2522878108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72185f750 0x55f72186a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a1e59c700 1 -- 172.21.15.57:0/2522878108 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7215ddd90 con 0x55f72185ee30 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a1ed9d700 1 --2- 172.21.15.57:0/2522878108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72186a600 0x55f72186c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/2522878108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f72185f210 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a10097d60 tx=0x7f3a10005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c2572806c7028f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a14ff9700 1 -- 172.21.15.57:0/2522878108 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100b1070 con 0x55f72185ee30 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a14ff9700 1 -- 172.21.15.57:0/2522878108 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1009b2f0 con 0x55f72185ee30 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.131+0000 7f3a14ff9700 1 -- 172.21.15.57:0/2522878108 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1009f120 con 0x55f72185ee30 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 -- 172.21.15.57:0/2522878108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 msgr2=0x55f72185f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/2522878108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f72185f210 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a10097d60 tx=0x7f3a10005910 comp rx=0 tx=0).stop 2023-10-08T13:51:18.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 -- 172.21.15.57:0/2522878108 shutdown_connections 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/2522878108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72186a600 0x55f72186c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/2522878108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f72185f210 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/2522878108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72185f750 0x55f72186a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 -- 172.21.15.57:0/2522878108 >> 172.21.15.57:0/2522878108 conn(0x55f7217d4210 msgr2=0x55f72185b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 -- 172.21.15.57:0/2522878108 shutdown_connections 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 -- 172.21.15.57:0/2522878108 wait complete. 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 -- start start 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 Processor -- start 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.132+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f721869ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185f750 0x55f721867330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 0x55f7218679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7217b2de0 con 0x55f72185ee30 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72174f6c0 con 0x55f72186a600 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72174f360 con 0x55f72185f750 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f721869ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a30e92b80 10 client.0 _ll_register_callbacks cb 0x55f7215ac7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f721869ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55882/0 (socket says 172.21.15.57:55882) 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1e59c700 1 -- 172.21.15.57:0/735464392 learned_addr learned my addr 172.21.15.57:0/735464392 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a17fff700 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185f750 0x55f721867330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1ed9d700 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 0x55f7218679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1e59c700 1 -- 172.21.15.57:0/735464392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185f750 msgr2=0x55f721867330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185f750 0x55f721867330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1e59c700 1 -- 172.21.15.57:0/735464392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 msgr2=0x55f7218679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 0x55f7218679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1e59c700 1 -- 172.21.15.57:0/735464392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a201a1bc0 con 0x55f72185ee30 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1ed9d700 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 0x55f7218679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f721869ff0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3a100a3150 tx=0x7f3a100cba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb90b2ae6346533d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a05ffb700 1 -- 172.21.15.57:0/735464392 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100b1040 con 0x55f72185ee30 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a05ffb700 1 -- 172.21.15.57:0/735464392 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3a100a3680 con 0x55f72185ee30 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a30e92b80 1 -- 172.21.15.57:0/735464392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f72177ae60 con 0x55f72185ee30 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.133+0000 7f3a05ffb700 1 -- 172.21.15.57:0/735464392 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a1009f120 con 0x55f72185ee30 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a30e92b80 10 client.15654 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:18.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a30e92b80 1 -- 172.21.15.57:0/735464392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f7215c6c10 con 0x55f72185ee30 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a05ffb700 1 -- 172.21.15.57:0/735464392 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f72177ae60 con 0x55f72185ee30 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a05ffb700 1 -- 172.21.15.57:0/735464392 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3a100a31f0 con 0x55f72185ee30 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a30e92b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a30e92b80 10 client.15654 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a30e92b80 20 client.15654 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f721879e90 0x55f721878a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a30e92b80 4 client.15654 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a30e92b80 1 -- 172.21.15.57:0/735464392 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7217abdc0 con 0x55f721879e90 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.134+0000 7f3a17fff700 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f721879e90 0x55f721878a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.135+0000 7f3a05ffb700 10 client.15654 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.135+0000 7f3a17fff700 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f721879e90 0x55f721878a30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=101c948248420767 server_cookie=14fbdd13f2ef7af5 in_seq=0 out_seq=0 2023-10-08T13:51:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.135+0000 7f3a05ffb700 1 -- 172.21.15.57:0/735464392 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f7217abdc0 con 0x55f721879e90 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.135+0000 7f3a05ffb700 10 client.15654 handle_command_reply: tid=0 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.135+0000 7f3a30e92b80 1 client.15654 shutdown 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.135+0000 7f3a30e92b80 1 -- 172.21.15.57:0/735464392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 msgr2=0x55f721869ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.135+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f721869ff0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3a100a3150 tx=0x7f3a100cba80 comp rx=0 tx=0).stop 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.136+0000 7f3a30e92b80 1 -- 172.21.15.57:0/735464392 shutdown_connections 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.136+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f721879e90 0x55f721878a30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.136+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 0x55f7218679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.136+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185ee30 0x55f721869ff0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.136+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/735464392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185f750 0x55f721867330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.136+0000 7f3a05ffb700 0 client.15654 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.136+0000 7f3a30e92b80 1 -- 172.21.15.57:0/735464392 >> 172.21.15.57:0/735464392 conn(0x55f7217d4210 msgr2=0x55f72185ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:18.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.136+0000 7f3a30e92b80 1 -- 172.21.15.57:0/735464392 shutdown_connections 2023-10-08T13:51:18.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.136+0000 7f3a30e92b80 1 -- 172.21.15.57:0/735464392 wait complete. 2023-10-08T13:51:18.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.136+0000 7f3a30e92b80 20 client.15654 trim_cache size 0 max 16384 2023-10-08T13:51:18.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.153+0000 7f3a30e92b80 1 Processor -- start 2023-10-08T13:51:18.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.153+0000 7f3a30e92b80 1 -- start start 2023-10-08T13:51:18.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.153+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72185ee30 0x55f721876170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.153+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185f750 0x55f72185ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.153+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72186a600 0x55f72185b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7216338d0 con 0x55f72186a600 2023-10-08T13:51:18.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7217aef00 con 0x55f72185ee30 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f72163fde0 con 0x55f72185f750 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a1ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72186a600 0x55f72185b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a1ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72186a600 0x55f72185b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55894/0 (socket says 172.21.15.57:55894) 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a1ed9d700 1 -- 172.21.15.57:0/695958648 learned_addr learned my addr 172.21.15.57:0/695958648 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a17fff700 1 --2- 172.21.15.57:0/695958648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185f750 0x55f72185ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/695958648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72185ee30 0x55f721876170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a1ed9d700 1 -- 172.21.15.57:0/695958648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185f750 msgr2=0x55f72185ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a1ed9d700 1 --2- 172.21.15.57:0/695958648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185f750 0x55f72185ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a1ed9d700 1 -- 172.21.15.57:0/695958648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72185ee30 msgr2=0x55f721876170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a1ed9d700 1 --2- 172.21.15.57:0/695958648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72185ee30 0x55f721876170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a1ed9d700 1 -- 172.21.15.57:0/695958648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f721604200 con 0x55f72186a600 2023-10-08T13:51:18.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a1ed9d700 1 --2- 172.21.15.57:0/695958648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72186a600 0x55f72185b260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006180 tx=0x7f3a180061b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d1b6dc61331d33 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a157fa700 1 -- 172.21.15.57:0/695958648 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800e040 con 0x55f72186a600 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a157fa700 1 -- 172.21.15.57:0/695958648 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a18020070 con 0x55f72186a600 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.154+0000 7f3a157fa700 1 -- 172.21.15.57:0/695958648 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180079a0 con 0x55f72186a600 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 -- 172.21.15.57:0/695958648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72186a600 msgr2=0x55f72185b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/695958648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72186a600 0x55f72185b260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006180 tx=0x7f3a180061b0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 -- 172.21.15.57:0/695958648 shutdown_connections 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/695958648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72186a600 0x55f72185b260 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/695958648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72185ee30 0x55f721876170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/695958648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185f750 0x55f72185ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 -- 172.21.15.57:0/695958648 >> 172.21.15.57:0/695958648 conn(0x55f7217d4210 msgr2=0x55f721862660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 -- 172.21.15.57:0/695958648 shutdown_connections 2023-10-08T13:51:18.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 -- 172.21.15.57:0/695958648 wait complete. 2023-10-08T13:51:18.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 -- start start 2023-10-08T13:51:18.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 Processor -- start 2023-10-08T13:51:18.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185ee30 0x55f7218639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.155+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185f750 0x55f721863ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a30e92b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 0x55f721864430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7218d1960 con 0x55f72185f750 2023-10-08T13:51:18.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7218d1ad0 con 0x55f72186a600 2023-10-08T13:51:18.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a30e92b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7218d1c90 con 0x55f72185ee30 2023-10-08T13:51:18.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a30e92b80 10 client.0 _ll_register_callbacks cb 0x7f3a1009b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:18.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185f750 0x55f721863ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185f750 0x55f721863ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55898/0 (socket says 172.21.15.57:55898) 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a1e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185ee30 0x55f7218639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a17fff700 1 -- 172.21.15.57:0/1408774651 learned_addr learned my addr 172.21.15.57:0/1408774651 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a1ed9d700 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 0x55f721864430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a17fff700 1 -- 172.21.15.57:0/1408774651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185ee30 msgr2=0x55f7218639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a17fff700 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185ee30 0x55f7218639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a17fff700 1 -- 172.21.15.57:0/1408774651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 msgr2=0x55f721864430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a17fff700 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 0x55f721864430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a17fff700 1 -- 172.21.15.57:0/1408774651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2010c520 con 0x55f72185f750 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a17fff700 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185f750 0x55f721863ef0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3a08005950 tx=0x7f3a080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b2fd1ee9478203 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:18.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a05ffb700 1 -- 172.21.15.57:0/1408774651 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08027450 con 0x55f72185f750 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a05ffb700 1 -- 172.21.15.57:0/1408774651 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3a08017070 con 0x55f72185f750 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a30e92b80 1 -- 172.21.15.57:0/1408774651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a1009b2f0 con 0x55f72185f750 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.156+0000 7f3a05ffb700 1 -- 172.21.15.57:0/1408774651 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a080275b0 con 0x55f72185f750 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a30e92b80 10 client.15660 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a30e92b80 1 -- 172.21.15.57:0/1408774651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f7215dc260 con 0x55f72185f750 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a05ffb700 1 -- 172.21.15.57:0/1408774651 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1009b2f0 con 0x55f72185f750 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a05ffb700 1 -- 172.21.15.57:0/1408774651 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3a0801f660 con 0x55f72185f750 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a30e92b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a30e92b80 10 client.15660 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a30e92b80 20 client.15660 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:18.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f7218d7100 0x55f7218d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a30e92b80 4 client.15660 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a30e92b80 1 -- 172.21.15.57:0/1408774651 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f7217abae0 con 0x55f7218d7100 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f7218d7100 0x55f7218d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a05ffb700 10 client.15660 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.157+0000 7f3a1e59c700 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f7218d7100 0x55f7218d3540 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=800005080296db27 server_cookie=cb8aae95cd7058fc in_seq=0 out_seq=0 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a05ffb700 1 -- 172.21.15.57:0/1408774651 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f7217abae0 con 0x55f7218d7100 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a05ffb700 10 client.15660 handle_command_reply: tid=0 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 client.15660 shutdown 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 -- 172.21.15.57:0/1408774651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185f750 msgr2=0x55f721863ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185f750 0x55f721863ef0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3a08005950 tx=0x7f3a080047c0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 -- 172.21.15.57:0/1408774651 shutdown_connections 2023-10-08T13:51:18.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f7218d7100 0x55f7218d3540 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f72186a600 0x55f721864430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72185ee30 0x55f7218639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a05ffb700 0 client.15660 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:18.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 --2- 172.21.15.57:0/1408774651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f72185f750 0x55f721863ef0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 -- 172.21.15.57:0/1408774651 >> 172.21.15.57:0/1408774651 conn(0x55f7217d4210 msgr2=0x55f721878a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:18.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 -- 172.21.15.57:0/1408774651 shutdown_connections 2023-10-08T13:51:18.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 1 -- 172.21.15.57:0/1408774651 wait complete. 2023-10-08T13:51:18.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a30e92b80 20 client.15660 trim_cache size 0 max 16384 2023-10-08T13:51:18.165 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "5f25708e-65e0-11ee-8db5-212e2dc638e7", 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24662, 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 93.043238807999998, 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 27, 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 92, 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 88, 2023-10-08T13:51:18.166 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 98.972634794000001 2023-10-08T13:51:18.167 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:51:18.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a24ca9700 1 -- 172.21.15.57:0/3136881292 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a0c0a1620 msgr2=0x7f3a0c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:18.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a24ca9700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a0c0a1620 0x7f3a0c0a3ae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3a20195b00 tx=0x7f3a0800b410 comp rx=0 tx=0).stop 2023-10-08T13:51:18.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a24ca9700 1 -- 172.21.15.57:0/3136881292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a200ff060 msgr2=0x7f3a2019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:18.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.158+0000 7f3a24ca9700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a200ff060 0x7f3a2019a810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000ad80 tx=0x7f3a100084b0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.159+0000 7f3a24ca9700 1 -- 172.21.15.57:0/3136881292 shutdown_connections 2023-10-08T13:51:18.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.159+0000 7f3a24ca9700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a0c0a1620 0x7f3a0c0a3ae0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.159+0000 7f3a24ca9700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a20194940 0x7f3a20194df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.159+0000 7f3a24ca9700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a200ff060 0x7f3a2019a810 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.159+0000 7f3a24ca9700 1 --2- 172.21.15.57:0/3136881292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20104130 0x7f3a2019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:18.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.159+0000 7f3a24ca9700 1 -- 172.21.15.57:0/3136881292 >> 172.21.15.57:0/3136881292 conn(0x7f3a20075c20 msgr2=0x7f3a201023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:18.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.159+0000 7f3a24ca9700 1 -- 172.21.15.57:0/3136881292 shutdown_connections 2023-10-08T13:51:18.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:18.159+0000 7f3a24ca9700 1 -- 172.21.15.57:0/3136881292 wait complete. 2023-10-08T13:51:18.171 INFO:teuthology.run_tasks:Running task kclient... 2023-10-08T13:51:18.190 INFO:tasks.kclient:Mounting kernel clients... 2023-10-08T13:51:18.190 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-08T13:51:18.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:18.190 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-08T13:51:18.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:17 smithi115 ceph-mon[104408]: pgmap v290: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-10-08T13:51:18.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:17 smithi115 ceph-mon[104408]: osdmap e92: 12 total, 12 up, 12 in 2023-10-08T13:51:18.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:17 smithi115 ceph-mon[104408]: mgrmap e22: x(active, since 7m), standbys: z, y 2023-10-08T13:51:18.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:17 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3719966365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:18.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:17 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1294449974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:18.209 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-08T13:51:18.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:18.210 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T13:51:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:17 smithi099 ceph-mon[103849]: pgmap v290: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-10-08T13:51:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:17 smithi099 ceph-mon[103849]: osdmap e92: 12 total, 12 up, 12 in 2023-10-08T13:51:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:17 smithi099 ceph-mon[103849]: mgrmap e22: x(active, since 7m), standbys: z, y 2023-10-08T13:51:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:17 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3719966365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:17 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1294449974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:18.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:18.288 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2023-10-08T13:51:18.388 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-10-08T13:51:18.388 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T13:51:18.389 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T13:51:18.389 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-08T13:51:18.958 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T13:51:20.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.168+0000 7f0e2af3c700 1 -- 172.21.15.57:0/817888205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e24105690 msgr2=0x7f0e24105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:20.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.168+0000 7f0e2af3c700 1 --2- 172.21.15.57:0/817888205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e24105690 0x7f0e24105a70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009990 tx=0x7f0e18009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.168+0000 7f0e2af3c700 1 -- 172.21.15.57:0/817888205 shutdown_connections 2023-10-08T13:51:20.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.168+0000 7f0e2af3c700 1 --2- 172.21.15.57:0/817888205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e24100110 0x7f0e2410c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.168+0000 7f0e2af3c700 1 --2- 172.21.15.57:0/817888205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e24105690 0x7f0e24105a70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.168+0000 7f0e2af3c700 1 --2- 172.21.15.57:0/817888205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e240ff750 0x7f0e240ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.168+0000 7f0e2af3c700 1 -- 172.21.15.57:0/817888205 >> 172.21.15.57:0/817888205 conn(0x7f0e240757d0 msgr2=0x7f0e24075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.168+0000 7f0e2af3c700 1 -- 172.21.15.57:0/817888205 shutdown_connections 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.168+0000 7f0e2af3c700 1 -- 172.21.15.57:0/817888205 wait complete. 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.168+0000 7f0e2af3c700 1 Processor -- start 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e2af3c700 1 -- start start 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e2af3c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240ff750 0x7f0e24195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e2af3c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e24100110 0x7f0e24195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e2af3c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e24105690 0x7f0e2419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e2af3c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2419a940 con 0x7f0e24100110 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e2af3c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2419aab0 con 0x7f0e24105690 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e2af3c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2419ac70 con 0x7f0e240ff750 2023-10-08T13:51:20.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e23fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e24100110 0x7f0e24195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e23fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e24100110 0x7f0e24195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55910/0 (socket says 172.21.15.57:55910) 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e23fff700 1 -- 172.21.15.57:0/3168304759 learned_addr learned my addr 172.21.15.57:0/3168304759 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e28cd8700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240ff750 0x7f0e24195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e294d9700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e24105690 0x7f0e2419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e23fff700 1 -- 172.21.15.57:0/3168304759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240ff750 msgr2=0x7f0e24195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e23fff700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240ff750 0x7f0e24195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e23fff700 1 -- 172.21.15.57:0/3168304759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e24105690 msgr2=0x7f0e2419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e23fff700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e24105690 0x7f0e2419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e23fff700 1 -- 172.21.15.57:0/3168304759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180095f0 con 0x7f0e24100110 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e294d9700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e24105690 0x7f0e2419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:20.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.169+0000 7f0e23fff700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e24100110 0x7f0e24195fe0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000c9b0 tx=0x7f0e1000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:20.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.170+0000 7f0e21ffb700 1 -- 172.21.15.57:0/3168304759 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10007b00 con 0x7f0e24100110 2023-10-08T13:51:20.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.170+0000 7f0e2af3c700 1 -- 172.21.15.57:0/3168304759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2419af50 con 0x7f0e24100110 2023-10-08T13:51:20.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.170+0000 7f0e21ffb700 1 -- 172.21.15.57:0/3168304759 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e10007c60 con 0x7f0e24100110 2023-10-08T13:51:20.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.170+0000 7f0e21ffb700 1 -- 172.21.15.57:0/3168304759 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10018660 con 0x7f0e24100110 2023-10-08T13:51:20.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.170+0000 7f0e2af3c700 1 -- 172.21.15.57:0/3168304759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e241a27f0 con 0x7f0e24100110 2023-10-08T13:51:20.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.171+0000 7f0e2af3c700 1 -- 172.21.15.57:0/3168304759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e24107c90 con 0x7f0e24100110 2023-10-08T13:51:20.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.172+0000 7f0e21ffb700 1 -- 172.21.15.57:0/3168304759 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f0e1000f450 con 0x7f0e24100110 2023-10-08T13:51:20.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.172+0000 7f0e21ffb700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a14d0 0x7f0e140a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:20.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.172+0000 7f0e28cd8700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a14d0 0x7f0e140a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:20.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.172+0000 7f0e21ffb700 1 -- 172.21.15.57:0/3168304759 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0e100cd130 con 0x7f0e24100110 2023-10-08T13:51:20.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.173+0000 7f0e28cd8700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a14d0 0x7f0e140a3990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800b5c0 tx=0x7f0e18005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:20.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.177+0000 7f0e21ffb700 1 -- 172.21.15.57:0/3168304759 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0e10096370 con 0x7f0e24100110 2023-10-08T13:51:20.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:19 smithi115 ceph-mon[104408]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 10 op/s 2023-10-08T13:51:20.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:19 smithi057 ceph-mon[99476]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 10 op/s 2023-10-08T13:51:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:19 smithi099 ceph-mon[103849]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 10 op/s 2023-10-08T13:51:20.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.422+0000 7f0e2af3c700 1 -- 172.21.15.57:0/3168304759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0e2404e4c0 con 0x7f0e24100110 2023-10-08T13:51:20.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.422+0000 7f0e21ffb700 1 -- 172.21.15.57:0/3168304759 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0e10099990 con 0x7f0e24100110 2023-10-08T13:51:20.422 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T13:51:20.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.424+0000 7f0e2af3c700 1 -- 172.21.15.57:0/3168304759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a14d0 msgr2=0x7f0e140a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.424+0000 7f0e2af3c700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a14d0 0x7f0e140a3990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800b5c0 tx=0x7f0e18005960 comp rx=0 tx=0).stop 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.424+0000 7f0e2af3c700 1 -- 172.21.15.57:0/3168304759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e24100110 msgr2=0x7f0e24195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.424+0000 7f0e2af3c700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e24100110 0x7f0e24195fe0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000c9b0 tx=0x7f0e1000ccc0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.424+0000 7f0e2af3c700 1 -- 172.21.15.57:0/3168304759 shutdown_connections 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.424+0000 7f0e2af3c700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0e140a14d0 0x7f0e140a3990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.424+0000 7f0e2af3c700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e24105690 0x7f0e2419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.424+0000 7f0e2af3c700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240ff750 0x7f0e24195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.424+0000 7f0e2af3c700 1 --2- 172.21.15.57:0/3168304759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e24100110 0x7f0e24195fe0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.424+0000 7f0e2af3c700 1 -- 172.21.15.57:0/3168304759 >> 172.21.15.57:0/3168304759 conn(0x7f0e240757d0 msgr2=0x7f0e240fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.425+0000 7f0e2af3c700 1 -- 172.21.15.57:0/3168304759 shutdown_connections 2023-10-08T13:51:20.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:20.425+0000 7f0e2af3c700 1 -- 172.21.15.57:0/3168304759 wait complete. 2023-10-08T13:51:20.811 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T13:51:20.811 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T13:51:20.811 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-10-08T13:51:20.811 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T13:51:20.811 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T13:51:20.811 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T13:51:20.811 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T13:51:20.811 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T13:51:20.812 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T13:51:20.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:20.812 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-10-08T13:51:20.831 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T13:51:20.831 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T13:51:20.831 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T13:51:20.831 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-08T13:51:20.831 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-10-08T13:51:20.831 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-08T13:51:20.831 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T13:51:20.831 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-10-08T13:51:20.831 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T13:51:20.832 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-10-08T13:51:20.832 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T13:51:20.832 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T13:51:20.832 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-08T13:51:20.832 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 41821sec preferred_lft 41821sec 2023-10-08T13:51:20.832 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-08T13:51:20.832 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-08T13:51:20.832 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T13:51:20.832 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T13:51:20.832 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T13:51:20.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:20.833 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-08T13:51:20.833 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-08T13:51:20.833 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add name ceph-brx type bridge 2023-10-08T13:51:20.833 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr flush dev ceph-brx 2023-10-08T13:51:20.833 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set ceph-brx up 2023-10-08T13:51:20.833 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T13:51:20.833 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-08T13:51:20.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:20.953 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T13:51:20.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:20 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3168304759' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T13:51:20.981 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-10-08T13:51:20.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:20.983 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-10-08T13:51:21.043 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-10-08T13:51:21.043 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T13:51:21.043 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-08T13:51:21.044 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-08T13:51:21.044 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T13:51:21.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:21.044 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-08T13:51:21.044 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-08T13:51:21.045 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-08T13:51:21.045 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-08T13:51:21.045 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-08T13:51:21.045 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-08T13:51:21.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:21.151 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-08T13:51:21.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:21.169 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2023-10-08T13:51:21.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:20 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3168304759' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T13:51:21.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:21.226 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-08T13:51:21.226 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-08T13:51:21.226 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T13:51:21.226 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-08T13:51:21.226 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-08T13:51:21.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:20 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3168304759' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T13:51:21.323 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-08T13:51:21.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:21.324 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-08T13:51:21.324 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-08T13:51:21.324 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-08T13:51:21.324 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T13:51:21.324 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-08T13:51:21.324 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-08T13:51:21.324 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-08T13:51:21.324 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-08T13:51:21.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:21.510 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-08T13:51:21.510 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-08T13:51:21.510 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-10-08T13:51:21.510 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T13:51:21.510 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-08T13:51:21.634 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T13:51:21.634 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T13:51:21.658 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T13:51:21.658 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T13:51:21.659 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T13:51:21.715 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-10-08T13:51:21.716 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-10-08T13:51:21.716 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725 2023-10-08T13:51:21.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:21.716 DEBUG:teuthology.orchestra.run.smithi057:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2023-10-08T13:51:21.802 INFO:teuthology.orchestra.run.smithi057.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2023-10-08T13:51:21.802 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2023-10-08T13:51:21.973 INFO:teuthology.orchestra.run.smithi057.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2023-10-08T13:51:21.974 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2023-10-08T13:51:21.974 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: resolved to: "172.21.15.57:3300,172.21.15.99:3300,172.21.15.115:3300" 2023-10-08T13:51:21.974 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: trying mount with new device syntax: 0@5f25708e-65e0-11ee-8db5-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725 2023-10-08T13:51:21.974 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.57:3300/172.21.15.99:3300/172.21.15.115:3300" will pass to kernel 2023-10-08T13:51:21.974 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T13:51:21.974 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T13:51:21.974 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.57:3300,172.21.15.99:3300,172.21.15.115:3300:/volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725 2023-10-08T13:51:21.974 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mds_namespace=cephfs,fsid=5f25708e-65e0-11ee-8db5-212e2dc638e7" will pass to kernel 2023-10-08T13:51:21.989 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T13:51:21.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:21.989 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T13:51:22.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:22.019 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T13:51:22.092 INFO:teuthology.orchestra.run.smithi057.stdout:client25349 2023-10-08T13:51:22.093 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:51:22.093 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T13:51:22.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:22.151 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T13:51:22.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:21 smithi115 ceph-mon[104408]: pgmap v293: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2023-10-08T13:51:22.224 INFO:teuthology.orchestra.run.smithi057.stdout:client25349 2023-10-08T13:51:22.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:22.225 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T13:51:22.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:21 smithi057 ceph-mon[99476]: pgmap v293: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2023-10-08T13:51:22.270 INFO:teuthology.orchestra.run.smithi057.stdout:client25349 2023-10-08T13:51:22.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T13:51:22.270 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7.client25349/status') 2023-10-08T13:51:22.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:21 smithi099 ceph-mon[103849]: pgmap v293: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2023-10-08T13:51:22.351 INFO:teuthology.orchestra.run.smithi057.stdout:instance: client.25349 (3)172.21.15.57:0/3893508067 2023-10-08T13:51:22.351 INFO:teuthology.orchestra.run.smithi057.stdout:blocklisted: false 2023-10-08T13:51:22.352 INFO:teuthology.orchestra.run.smithi057.stderr:0+1 records in 2023-10-08T13:51:22.352 INFO:teuthology.orchestra.run.smithi057.stderr:0+1 records out 2023-10-08T13:51:22.352 INFO:teuthology.orchestra.run.smithi057.stderr:71 bytes copied, 2.5588e-05 s, 2.8 MB/s 2023-10-08T13:51:22.353 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-08T13:51:22.364 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-08T13:51:22.528 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:51:22.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 -- 172.21.15.57:0/405806356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ca80fff00 msgr2=0x7f2ca810c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:22.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 --2- 172.21.15.57:0/405806356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ca80fff00 0x7f2ca810c3c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000b0d0 tx=0x7f2ca000b3e0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 -- 172.21.15.57:0/405806356 shutdown_connections 2023-10-08T13:51:22.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 --2- 172.21.15.57:0/405806356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ca80fff00 0x7f2ca810c3c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 --2- 172.21.15.57:0/405806356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ca8105600 0x7f2ca81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 --2- 172.21.15.57:0/405806356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ca80ff540 0x7f2ca80ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 -- 172.21.15.57:0/405806356 >> 172.21.15.57:0/405806356 conn(0x7f2ca80fb030 msgr2=0x7f2ca80fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 -- 172.21.15.57:0/405806356 shutdown_connections 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 -- 172.21.15.57:0/405806356 wait complete. 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 Processor -- start 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.654+0000 7f2caed39700 1 -- start start 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2caed39700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ca80ff540 0x7f2ca81011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2caed39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ca80fff00 0x7f2ca81016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2caed39700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ca8105600 0x7f2ca8101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2caed39700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca81a4fc0 con 0x7f2ca8105600 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2caed39700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca81a5130 con 0x7f2ca80fff00 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2caed39700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca81a52f0 con 0x7f2ca80ff540 2023-10-08T13:51:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2cad2d6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ca8105600 0x7f2ca8101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2ca7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ca80fff00 0x7f2ca81016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2cad2d6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ca8105600 0x7f2ca8101c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55930/0 (socket says 172.21.15.57:55930) 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2cad2d6700 1 -- 172.21.15.57:0/3952530473 learned_addr learned my addr 172.21.15.57:0/3952530473 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2cacad5700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ca80ff540 0x7f2ca81011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2cad2d6700 1 -- 172.21.15.57:0/3952530473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ca80ff540 msgr2=0x7f2ca81011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2cad2d6700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ca80ff540 0x7f2ca81011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2cad2d6700 1 -- 172.21.15.57:0/3952530473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ca80fff00 msgr2=0x7f2ca81016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2cad2d6700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ca80fff00 0x7f2ca81016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2cad2d6700 1 -- 172.21.15.57:0/3952530473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0009d00 con 0x7f2ca8105600 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2cad2d6700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ca8105600 0x7f2ca8101c30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000d010 tx=0x7f2ca0009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2ca5ffb700 1 -- 172.21.15.57:0/3952530473 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000e040 con 0x7f2ca8105600 2023-10-08T13:51:22.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2ca5ffb700 1 -- 172.21.15.57:0/3952530473 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca0005070 con 0x7f2ca8105600 2023-10-08T13:51:22.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2caed39700 1 -- 172.21.15.57:0/3952530473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca81a5570 con 0x7f2ca8105600 2023-10-08T13:51:22.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.655+0000 7f2ca5ffb700 1 -- 172.21.15.57:0/3952530473 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca001b6d0 con 0x7f2ca8105600 2023-10-08T13:51:22.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.656+0000 7f2caed39700 1 -- 172.21.15.57:0/3952530473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca81a5990 con 0x7f2ca8105600 2023-10-08T13:51:22.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.657+0000 7f2caed39700 1 -- 172.21.15.57:0/3952530473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca8100c60 con 0x7f2ca8105600 2023-10-08T13:51:22.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.658+0000 7f2ca5ffb700 1 -- 172.21.15.57:0/3952530473 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f2ca00054e0 con 0x7f2ca8105600 2023-10-08T13:51:22.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.659+0000 7f2ca5ffb700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2c980a14d0 0x7f2c980a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:22.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.659+0000 7f2ca5ffb700 1 -- 172.21.15.57:0/3952530473 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f2ca00ce630 con 0x7f2ca8105600 2023-10-08T13:51:22.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.659+0000 7f2cacad5700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2c980a14d0 0x7f2c980a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:22.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.659+0000 7f2cacad5700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2c980a14d0 0x7f2c980a3990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c007c60 tx=0x7f2c9c0073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:22.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.662+0000 7f2ca5ffb700 1 -- 172.21.15.57:0/3952530473 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2ca00978a0 con 0x7f2ca8105600 2023-10-08T13:51:22.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.866+0000 7f2caed39700 1 -- 172.21.15.57:0/3952530473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ca8067450 con 0x7f2ca8105600 2023-10-08T13:51:22.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.868+0000 7f2ca5ffb700 1 -- 172.21.15.57:0/3952530473 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24847 (secure 0 0 0) 0x7f2ca0097020 con 0x7f2ca8105600 2023-10-08T13:51:22.869 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:22.869 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:51:16.778201+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:51:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 -- 172.21.15.57:0/3952530473 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2c980a14d0 msgr2=0x7f2c980a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2c980a14d0 0x7f2c980a3990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c007c60 tx=0x7f2c9c0073d0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 -- 172.21.15.57:0/3952530473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ca8105600 msgr2=0x7f2ca8101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ca8105600 0x7f2ca8101c30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000d010 tx=0x7f2ca0009440 comp rx=0 tx=0).stop 2023-10-08T13:51:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 -- 172.21.15.57:0/3952530473 shutdown_connections 2023-10-08T13:51:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ca8105600 0x7f2ca8101c30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ca80ff540 0x7f2ca81011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2c980a14d0 0x7f2c980a3990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 --2- 172.21.15.57:0/3952530473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ca80fff00 0x7f2ca81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:22.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 -- 172.21.15.57:0/3952530473 >> 172.21.15.57:0/3952530473 conn(0x7f2ca80fb030 msgr2=0x7f2ca80fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:22.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 -- 172.21.15.57:0/3952530473 shutdown_connections 2023-10-08T13:51:22.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:22.870+0000 7f2caed39700 1 -- 172.21.15.57:0/3952530473 wait complete. 2023-10-08T13:51:22.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T13:51:22.884 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:51:23.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 -- 172.21.15.57:0/190272146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20240ff620 msgr2=0x7f20240ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 --2- 172.21.15.57:0/190272146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20240ff620 0x7f20240ffaa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2010009990 tx=0x7f2010009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 -- 172.21.15.57:0/190272146 shutdown_connections 2023-10-08T13:51:23.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 --2- 172.21.15.57:0/190272146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20240fffe0 0x7f202410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 --2- 172.21.15.57:0/190272146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20241056b0 0x7f2024105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 --2- 172.21.15.57:0/190272146 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20240ff620 0x7f20240ffaa0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 -- 172.21.15.57:0/190272146 >> 172.21.15.57:0/190272146 conn(0x7f2024076780 msgr2=0x7f2024076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:23.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 -- 172.21.15.57:0/190272146 shutdown_connections 2023-10-08T13:51:23.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 -- 172.21.15.57:0/190272146 wait complete. 2023-10-08T13:51:23.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 Processor -- start 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.013+0000 7f202aedb700 1 -- start start 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f202aedb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20240ff620 0x7f20241979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f202aedb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20240fffe0 0x7f2024197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f202aedb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20241056b0 0x7f20241a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f202aedb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2024198740 con 0x7f20240ff620 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f202aedb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241a2f70 con 0x7f20241056b0 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f202aedb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241a30b0 con 0x7f20240fffe0 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2028c77700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20240ff620 0x7f20241979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2028c77700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20240ff620 0x7f20241979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55940/0 (socket says 172.21.15.57:55940) 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2028c77700 1 -- 172.21.15.57:0/1442807851 learned_addr learned my addr 172.21.15.57:0/1442807851 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2023fff700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20240fffe0 0x7f2024197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2029478700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20241056b0 0x7f20241a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2028c77700 1 -- 172.21.15.57:0/1442807851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20240fffe0 msgr2=0x7f2024197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2028c77700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20240fffe0 0x7f2024197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2028c77700 1 -- 172.21.15.57:0/1442807851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20241056b0 msgr2=0x7f20241a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2028c77700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20241056b0 0x7f20241a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2028c77700 1 -- 172.21.15.57:0/1442807851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100095f0 con 0x7f20240ff620 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2028c77700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20240ff620 0x7f20241979b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f201800c9a0 tx=0x7f201800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2021ffb700 1 -- 172.21.15.57:0/1442807851 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018004220 con 0x7f20240ff620 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f202aedb700 1 -- 172.21.15.57:0/1442807851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20241a3310 con 0x7f20240ff620 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2021ffb700 1 -- 172.21.15.57:0/1442807851 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2018007d20 con 0x7f20240ff620 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.014+0000 7f2021ffb700 1 -- 172.21.15.57:0/1442807851 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018003d00 con 0x7f20240ff620 2023-10-08T13:51:23.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.015+0000 7f202aedb700 1 -- 172.21.15.57:0/1442807851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20241a3820 con 0x7f20240ff620 2023-10-08T13:51:23.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.016+0000 7f202aedb700 1 -- 172.21.15.57:0/1442807851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2024107c30 con 0x7f20240ff620 2023-10-08T13:51:23.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.016+0000 7f2021ffb700 1 -- 172.21.15.57:0/1442807851 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f2018004380 con 0x7f20240ff620 2023-10-08T13:51:23.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.017+0000 7f2021ffb700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f20140a14d0 0x7f20140a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.017+0000 7f2023fff700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f20140a14d0 0x7f20140a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.017+0000 7f2021ffb700 1 -- 172.21.15.57:0/1442807851 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f20180ccdb0 con 0x7f20240ff620 2023-10-08T13:51:23.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.017+0000 7f2023fff700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f20140a14d0 0x7f20140a3990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2010005850 tx=0x7f201001a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:23.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.021+0000 7f2021ffb700 1 -- 172.21.15.57:0/1442807851 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2018095ff0 con 0x7f20240ff620 2023-10-08T13:51:23.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.223+0000 7f202aedb700 1 -- 172.21.15.57:0/1442807851 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2024067dd0 con 0x7f20240ff620 2023-10-08T13:51:23.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.224+0000 7f2021ffb700 1 -- 172.21.15.57:0/1442807851 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2018099610 con 0x7f20240ff620 2023-10-08T13:51:23.225 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:51:23.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 -- 172.21.15.57:0/1442807851 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f20140a14d0 msgr2=0x7f20140a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f20140a14d0 0x7f20140a3990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2010005850 tx=0x7f201001a040 comp rx=0 tx=0).stop 2023-10-08T13:51:23.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 -- 172.21.15.57:0/1442807851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20240ff620 msgr2=0x7f20241979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20240ff620 0x7f20241979b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f201800c9a0 tx=0x7f201800cd60 comp rx=0 tx=0).stop 2023-10-08T13:51:23.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 -- 172.21.15.57:0/1442807851 shutdown_connections 2023-10-08T13:51:23.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f20140a14d0 0x7f20140a3990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20241056b0 0x7f20241a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20240ff620 0x7f20241979b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 --2- 172.21.15.57:0/1442807851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20240fffe0 0x7f2024197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 -- 172.21.15.57:0/1442807851 >> 172.21.15.57:0/1442807851 conn(0x7f2024076780 msgr2=0x7f20240fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:23.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 -- 172.21.15.57:0/1442807851 shutdown_connections 2023-10-08T13:51:23.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.226+0000 7f202aedb700 1 -- 172.21.15.57:0/1442807851 wait complete. 2023-10-08T13:51:23.239 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T13:51:23.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.368+0000 7f2681010700 1 -- 172.21.15.57:0/3345574691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c069790 msgr2=0x7f267c10d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.368+0000 7f2681010700 1 --2- 172.21.15.57:0/3345574691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c069790 0x7f267c10d600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f267400b0a0 tx=0x7f267400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.368+0000 7f2681010700 1 -- 172.21.15.57:0/3345574691 shutdown_connections 2023-10-08T13:51:23.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.368+0000 7f2681010700 1 --2- 172.21.15.57:0/3345574691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c069790 0x7f267c10d600 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.368+0000 7f2681010700 1 --2- 172.21.15.57:0/3345574691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c1067f0 0x7f267c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.368+0000 7f2681010700 1 --2- 172.21.15.57:0/3345574691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c068dd0 0x7f267c069250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.368+0000 7f2681010700 1 -- 172.21.15.57:0/3345574691 >> 172.21.15.57:0/3345574691 conn(0x7f267c076820 msgr2=0x7f267c076c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:23.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.368+0000 7f2681010700 1 -- 172.21.15.57:0/3345574691 shutdown_connections 2023-10-08T13:51:23.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.368+0000 7f2681010700 1 -- 172.21.15.57:0/3345574691 wait complete. 2023-10-08T13:51:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f2681010700 1 Processor -- start 2023-10-08T13:51:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f2681010700 1 -- start start 2023-10-08T13:51:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f2681010700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c068dd0 0x7f267c19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f2681010700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c069790 0x7f267c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f2681010700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c1067f0 0x7f267c196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f2681010700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c197150 con 0x7f267c068dd0 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f2681010700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c1972c0 con 0x7f267c069790 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f2681010700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c197480 con 0x7f267c1067f0 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f267ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c068dd0 0x7f267c19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f267ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c068dd0 0x7f267c19c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55954/0 (socket says 172.21.15.57:55954) 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f267ad9d700 1 -- 172.21.15.57:0/115508346 learned_addr learned my addr 172.21.15.57:0/115508346 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f267b59e700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c1067f0 0x7f267c196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.369+0000 7f267a59c700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c069790 0x7f267c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f267ad9d700 1 -- 172.21.15.57:0/115508346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c1067f0 msgr2=0x7f267c196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f267ad9d700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c1067f0 0x7f267c196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f267ad9d700 1 -- 172.21.15.57:0/115508346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c069790 msgr2=0x7f267c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f267ad9d700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c069790 0x7f267c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f267ad9d700 1 -- 172.21.15.57:0/115508346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2674009d00 con 0x7f267c068dd0 2023-10-08T13:51:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f267ad9d700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c068dd0 0x7f267c19c9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f267000e9a0 tx=0x7f267000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f2663fff700 1 -- 172.21.15.57:0/115508346 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670009790 con 0x7f267c068dd0 2023-10-08T13:51:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f2681010700 1 -- 172.21.15.57:0/115508346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c197760 con 0x7f267c068dd0 2023-10-08T13:51:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f2663fff700 1 -- 172.21.15.57:0/115508346 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f267000cbd0 con 0x7f267c068dd0 2023-10-08T13:51:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f2663fff700 1 -- 172.21.15.57:0/115508346 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670018680 con 0x7f267c068dd0 2023-10-08T13:51:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.370+0000 7f2681010700 1 -- 172.21.15.57:0/115508346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c1a39b0 con 0x7f267c068dd0 2023-10-08T13:51:23.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.371+0000 7f2681010700 1 -- 172.21.15.57:0/115508346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c108d50 con 0x7f267c068dd0 2023-10-08T13:51:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.372+0000 7f2663fff700 1 -- 172.21.15.57:0/115508346 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f267000cd40 con 0x7f267c068dd0 2023-10-08T13:51:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.373+0000 7f2663fff700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f26640a14d0 0x7f26640a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.373+0000 7f2663fff700 1 -- 172.21.15.57:0/115508346 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f2670014070 con 0x7f267c068dd0 2023-10-08T13:51:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.373+0000 7f267a59c700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f26640a14d0 0x7f26640a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.373+0000 7f267a59c700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f26640a14d0 0x7f26640a3990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f266c005950 tx=0x7f266c0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:23.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.376+0000 7f2663fff700 1 -- 172.21.15.57:0/115508346 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2670096910 con 0x7f267c068dd0 2023-10-08T13:51:23.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.577+0000 7f2681010700 1 -- 172.21.15.57:0/115508346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f267c197fe0 con 0x7f267c068dd0 2023-10-08T13:51:23.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.577+0000 7f2663fff700 1 -- 172.21.15.57:0/115508346 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2670099f30 con 0x7f267c068dd0 2023-10-08T13:51:23.578 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:51:23.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 -- 172.21.15.57:0/115508346 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f26640a14d0 msgr2=0x7f26640a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f26640a14d0 0x7f26640a3990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f266c005950 tx=0x7f266c0058e0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 -- 172.21.15.57:0/115508346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c068dd0 msgr2=0x7f267c19c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c068dd0 0x7f267c19c9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f267000e9a0 tx=0x7f267000ecb0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 -- 172.21.15.57:0/115508346 shutdown_connections 2023-10-08T13:51:23.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c1067f0 0x7f267c196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f267c068dd0 0x7f267c19c9c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f26640a14d0 0x7f26640a3990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 --2- 172.21.15.57:0/115508346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c069790 0x7f267c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 -- 172.21.15.57:0/115508346 >> 172.21.15.57:0/115508346 conn(0x7f267c076820 msgr2=0x7f267c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:23.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 -- 172.21.15.57:0/115508346 shutdown_connections 2023-10-08T13:51:23.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.579+0000 7f2681010700 1 -- 172.21.15.57:0/115508346 wait complete. 2023-10-08T13:51:23.592 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T13:51:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.721+0000 7f5306cd4700 1 -- 172.21.15.57:0/2818378059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5300068a10 msgr2=0x7f5300068e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.721+0000 7f5306cd4700 1 --2- 172.21.15.57:0/2818378059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5300068a10 0x7f5300068e90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009990 tx=0x7f52f0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.721+0000 7f5306cd4700 1 -- 172.21.15.57:0/2818378059 shutdown_connections 2023-10-08T13:51:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.721+0000 7f5306cd4700 1 --2- 172.21.15.57:0/2818378059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53000693d0 0x7f530010d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.721+0000 7f5306cd4700 1 --2- 172.21.15.57:0/2818378059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5300106780 0x7f5300106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.721+0000 7f5306cd4700 1 --2- 172.21.15.57:0/2818378059 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5300068a10 0x7f5300068e90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.721+0000 7f5306cd4700 1 -- 172.21.15.57:0/2818378059 >> 172.21.15.57:0/2818378059 conn(0x7f53000767c0 msgr2=0x7f5300076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.721+0000 7f5306cd4700 1 -- 172.21.15.57:0/2818378059 shutdown_connections 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.721+0000 7f5306cd4700 1 -- 172.21.15.57:0/2818378059 wait complete. 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5306cd4700 1 Processor -- start 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5306cd4700 1 -- start start 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5306cd4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5300068a10 0x7f53000ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5306cd4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53000693d0 0x7f53000ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5306cd4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5300106780 0x7f5300100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5306cd4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53001a4f80 con 0x7f5300068a10 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5306cd4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53001a50f0 con 0x7f5300106780 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5306cd4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53001a52b0 con 0x7f53000693d0 2023-10-08T13:51:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5304a70700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5300068a10 0x7f53000ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5304a70700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5300068a10 0x7f53000ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55968/0 (socket says 172.21.15.57:55968) 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5304a70700 1 -- 172.21.15.57:0/2156132711 learned_addr learned my addr 172.21.15.57:0/2156132711 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f52fffff700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53000693d0 0x7f53000ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5305271700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5300106780 0x7f5300100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5304a70700 1 -- 172.21.15.57:0/2156132711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53000693d0 msgr2=0x7f53000ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5304a70700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53000693d0 0x7f53000ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5304a70700 1 -- 172.21.15.57:0/2156132711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5300106780 msgr2=0x7f5300100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5304a70700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5300106780 0x7f5300100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5304a70700 1 -- 172.21.15.57:0/2156132711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f00095f0 con 0x7f5300068a10 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.722+0000 7f5305271700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5300106780 0x7f5300100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:23.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.723+0000 7f5304a70700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5300068a10 0x7f53000ff6b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f52e800e9a0 tx=0x7f52e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:23.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.723+0000 7f52fdffb700 1 -- 172.21.15.57:0/2156132711 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8009790 con 0x7f5300068a10 2023-10-08T13:51:23.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.723+0000 7f52fdffb700 1 -- 172.21.15.57:0/2156132711 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e800cbd0 con 0x7f5300068a10 2023-10-08T13:51:23.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.723+0000 7f5306cd4700 1 -- 172.21.15.57:0/2156132711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53001a5590 con 0x7f5300068a10 2023-10-08T13:51:23.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.723+0000 7f52fdffb700 1 -- 172.21.15.57:0/2156132711 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8018680 con 0x7f5300068a10 2023-10-08T13:51:23.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.723+0000 7f5306cd4700 1 -- 172.21.15.57:0/2156132711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53001a5a70 con 0x7f5300068a10 2023-10-08T13:51:23.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.724+0000 7f5306cd4700 1 -- 172.21.15.57:0/2156132711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5300108cb0 con 0x7f5300068a10 2023-10-08T13:51:23.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.725+0000 7f52fdffb700 1 -- 172.21.15.57:0/2156132711 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f52e800cd40 con 0x7f5300068a10 2023-10-08T13:51:23.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.726+0000 7f52fdffb700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a14d0 0x7f52ec0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:23.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.726+0000 7f52fdffb700 1 -- 172.21.15.57:0/2156132711 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f52e8014070 con 0x7f5300068a10 2023-10-08T13:51:23.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.726+0000 7f52fffff700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a14d0 0x7f52ec0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:23.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.726+0000 7f52fffff700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a14d0 0x7f52ec0a3990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f52f000b2e0 tx=0x7f52f0005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:23.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.730+0000 7f52fdffb700 1 -- 172.21.15.57:0/2156132711 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f52e8052100 con 0x7f5300068a10 2023-10-08T13:51:23.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.933+0000 7f5306cd4700 1 -- 172.21.15.57:0/2156132711 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5300100b90 con 0x7f5300068a10 2023-10-08T13:51:23.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.934+0000 7f52fdffb700 1 -- 172.21.15.57:0/2156132711 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52e80965d0 con 0x7f5300068a10 2023-10-08T13:51:23.935 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T13:51:23.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 -- 172.21.15.57:0/2156132711 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a14d0 msgr2=0x7f52ec0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a14d0 0x7f52ec0a3990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f52f000b2e0 tx=0x7f52f0005960 comp rx=0 tx=0).stop 2023-10-08T13:51:23.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 -- 172.21.15.57:0/2156132711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5300068a10 msgr2=0x7f53000ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:23.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5300068a10 0x7f53000ff6b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f52e800e9a0 tx=0x7f52e800ecb0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 -- 172.21.15.57:0/2156132711 shutdown_connections 2023-10-08T13:51:23.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a14d0 0x7f52ec0a3990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5300106780 0x7f5300100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5300068a10 0x7f53000ff6b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 --2- 172.21.15.57:0/2156132711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53000693d0 0x7f53000ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:23.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 -- 172.21.15.57:0/2156132711 >> 172.21.15.57:0/2156132711 conn(0x7f53000767c0 msgr2=0x7f53000fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:23.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 -- 172.21.15.57:0/2156132711 shutdown_connections 2023-10-08T13:51:23.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:23.936+0000 7f5306cd4700 1 -- 172.21.15.57:0/2156132711 wait complete. 2023-10-08T13:51:23.948 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:23 smithi057 ceph-mon[99476]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2023-10-08T13:51:23.948 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:23 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3952530473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:23.948 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:23 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1442807851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:23.948 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:23 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/115508346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:23.949 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T13:51:24.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.077+0000 7fc473612700 1 -- 172.21.15.57:0/1935252470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c0fffe0 msgr2=0x7fc46c10c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.077+0000 7fc473612700 1 --2- 172.21.15.57:0/1935252470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c0fffe0 0x7fc46c10c4e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc46400b0a0 tx=0x7fc46400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.077+0000 7fc473612700 1 -- 172.21.15.57:0/1935252470 shutdown_connections 2023-10-08T13:51:24.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.077+0000 7fc473612700 1 --2- 172.21.15.57:0/1935252470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c0fffe0 0x7fc46c10c4e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.077+0000 7fc473612700 1 --2- 172.21.15.57:0/1935252470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46c1056b0 0x7fc46c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.077+0000 7fc473612700 1 --2- 172.21.15.57:0/1935252470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc46c0ff620 0x7fc46c0ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.077+0000 7fc473612700 1 -- 172.21.15.57:0/1935252470 >> 172.21.15.57:0/1935252470 conn(0x7fc46c076780 msgr2=0x7fc46c076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:24.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.077+0000 7fc473612700 1 -- 172.21.15.57:0/1935252470 shutdown_connections 2023-10-08T13:51:24.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.077+0000 7fc473612700 1 -- 172.21.15.57:0/1935252470 wait complete. 2023-10-08T13:51:24.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc473612700 1 Processor -- start 2023-10-08T13:51:24.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc473612700 1 -- start start 2023-10-08T13:51:24.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc473612700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc46c0ff620 0x7fc46c195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc473612700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c0fffe0 0x7fc46c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc473612700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46c1056b0 0x7fc46c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc473612700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c19a910 con 0x7fc46c0fffe0 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc473612700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c19aa80 con 0x7fc46c1056b0 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc473612700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c19ac40 con 0x7fc46c0ff620 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc470bad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c0fffe0 0x7fc46c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc471baf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46c1056b0 0x7fc46c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc470bad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c0fffe0 0x7fc46c195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55984/0 (socket says 172.21.15.57:55984) 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc471baf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46c1056b0 0x7fc46c19a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:57412/0 (socket says 172.21.15.57:57412) 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc470bad700 1 -- 172.21.15.57:0/3841856932 learned_addr learned my addr 172.21.15.57:0/3841856932 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.078+0000 7fc4713ae700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc46c0ff620 0x7fc46c195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc470bad700 1 -- 172.21.15.57:0/3841856932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc46c0ff620 msgr2=0x7fc46c195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc470bad700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc46c0ff620 0x7fc46c195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc470bad700 1 -- 172.21.15.57:0/3841856932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46c1056b0 msgr2=0x7fc46c19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc470bad700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46c1056b0 0x7fc46c19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc470bad700 1 -- 172.21.15.57:0/3841856932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc464009d00 con 0x7fc46c0fffe0 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc471baf700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46c1056b0 0x7fc46c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc470bad700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c0fffe0 0x7fc46c195f70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc46800c920 tx=0x7fc46800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc4627fc700 1 -- 172.21.15.57:0/3841856932 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468004a30 con 0x7fc46c0fffe0 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc4627fc700 1 -- 172.21.15.57:0/3841856932 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4680054e0 con 0x7fc46c0fffe0 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc4627fc700 1 -- 172.21.15.57:0/3841856932 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468007c10 con 0x7fc46c0fffe0 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc473612700 1 -- 172.21.15.57:0/3841856932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46c19af20 con 0x7fc46c0fffe0 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.079+0000 7fc473612700 1 -- 172.21.15.57:0/3841856932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c1a27d0 con 0x7fc46c0fffe0 2023-10-08T13:51:24.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.080+0000 7fc473612700 1 -- 172.21.15.57:0/3841856932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c107c30 con 0x7fc46c0fffe0 2023-10-08T13:51:24.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.081+0000 7fc4627fc700 1 -- 172.21.15.57:0/3841856932 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc468005000 con 0x7fc46c0fffe0 2023-10-08T13:51:24.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.082+0000 7fc4627fc700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4580a14d0 0x7fc4580a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.082+0000 7fc4627fc700 1 -- 172.21.15.57:0/3841856932 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fc4680cd0d0 con 0x7fc46c0fffe0 2023-10-08T13:51:24.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.082+0000 7fc4713ae700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4580a14d0 0x7fc4580a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.082+0000 7fc4713ae700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4580a14d0 0x7fc4580a3990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc46c077ab0 tx=0x7fc45c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:24.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.085+0000 7fc4627fc700 1 -- 172.21.15.57:0/3841856932 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc4680962b0 con 0x7fc46c0fffe0 2023-10-08T13:51:24.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:23 smithi115 ceph-mon[104408]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2023-10-08T13:51:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:23 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3952530473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:23 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1442807851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:24.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:23 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/115508346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:23 smithi099 ceph-mon[103849]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2023-10-08T13:51:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:23 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3952530473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:23 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1442807851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:23 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/115508346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:24.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.289+0000 7fc473612700 1 -- 172.21.15.57:0/3841856932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc46c196d30 con 0x7fc46c0fffe0 2023-10-08T13:51:24.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.289+0000 7fc4627fc700 1 -- 172.21.15.57:0/3841856932 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc46801d0a0 con 0x7fc46c0fffe0 2023-10-08T13:51:24.290 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:51:24.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 -- 172.21.15.57:0/3841856932 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4580a14d0 msgr2=0x7fc4580a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4580a14d0 0x7fc4580a3990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc46c077ab0 tx=0x7fc45c00b410 comp rx=0 tx=0).stop 2023-10-08T13:51:24.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 -- 172.21.15.57:0/3841856932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c0fffe0 msgr2=0x7fc46c195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c0fffe0 0x7fc46c195f70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc46800c920 tx=0x7fc46800cc30 comp rx=0 tx=0).stop 2023-10-08T13:51:24.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 -- 172.21.15.57:0/3841856932 shutdown_connections 2023-10-08T13:51:24.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc4580a14d0 0x7fc4580a3990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46c1056b0 0x7fc46c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc46c0ff620 0x7fc46c195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 --2- 172.21.15.57:0/3841856932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc46c0fffe0 0x7fc46c195f70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 -- 172.21.15.57:0/3841856932 >> 172.21.15.57:0/3841856932 conn(0x7fc46c076780 msgr2=0x7fc46c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:24.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 -- 172.21.15.57:0/3841856932 shutdown_connections 2023-10-08T13:51:24.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.291+0000 7fc473612700 1 -- 172.21.15.57:0/3841856932 wait complete. 2023-10-08T13:51:24.305 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:51:24.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.430+0000 7ffa5170b700 1 -- 172.21.15.57:0/752810484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa4c0ff8f0 msgr2=0x7ffa4c10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.430+0000 7ffa5170b700 1 --2- 172.21.15.57:0/752810484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa4c0ff8f0 0x7ffa4c10d4f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400b0a0 tx=0x7ffa4400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.430+0000 7ffa5170b700 1 -- 172.21.15.57:0/752810484 shutdown_connections 2023-10-08T13:51:24.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.430+0000 7ffa5170b700 1 --2- 172.21.15.57:0/752810484 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa4c0ff8f0 0x7ffa4c10d4f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.430+0000 7ffa5170b700 1 --2- 172.21.15.57:0/752810484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa4c106870 0x7ffa4c106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.430+0000 7ffa5170b700 1 --2- 172.21.15.57:0/752810484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0fef30 0x7ffa4c0ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.430+0000 7ffa5170b700 1 -- 172.21.15.57:0/752810484 >> 172.21.15.57:0/752810484 conn(0x7ffa4c074820 msgr2=0x7ffa4c074c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:24.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.431+0000 7ffa5170b700 1 -- 172.21.15.57:0/752810484 shutdown_connections 2023-10-08T13:51:24.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.431+0000 7ffa5170b700 1 -- 172.21.15.57:0/752810484 wait complete. 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.431+0000 7ffa5170b700 1 Processor -- start 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.431+0000 7ffa5170b700 1 -- start start 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.431+0000 7ffa5170b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa4c0fef30 0x7ffa4c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.431+0000 7ffa5170b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0ff8f0 0x7ffa4c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.431+0000 7ffa5170b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa4c106870 0x7ffa4c196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.431+0000 7ffa5170b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c197090 con 0x7ffa4c0fef30 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.431+0000 7ffa5170b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c197200 con 0x7ffa4c106870 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa5170b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1973c0 con 0x7ffa4c0ff8f0 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa4c0fef30 0x7ffa4c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa4c0fef30 0x7ffa4c19c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55996/0 (socket says 172.21.15.57:55996) 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4affd700 1 -- 172.21.15.57:0/3872789766 learned_addr learned my addr 172.21.15.57:0/3872789766 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:24.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4b7fe700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa4c106870 0x7ffa4c196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4a7fc700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0ff8f0 0x7ffa4c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4affd700 1 -- 172.21.15.57:0/3872789766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0ff8f0 msgr2=0x7ffa4c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4affd700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0ff8f0 0x7ffa4c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4affd700 1 -- 172.21.15.57:0/3872789766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa4c106870 msgr2=0x7ffa4c196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4affd700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa4c106870 0x7ffa4c196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4affd700 1 -- 172.21.15.57:0/3872789766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa44009d00 con 0x7ffa4c0fef30 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4a7fc700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0ff8f0 0x7ffa4c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa4affd700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa4c0fef30 0x7ffa4c19c8a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00d950 tx=0x7ffa3c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa33fff700 1 -- 172.21.15.57:0/3872789766 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c009940 con 0x7ffa4c0fef30 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa33fff700 1 -- 172.21.15.57:0/3872789766 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa3c010460 con 0x7ffa4c0fef30 2023-10-08T13:51:24.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa33fff700 1 -- 172.21.15.57:0/3872789766 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00f620 con 0x7ffa4c0fef30 2023-10-08T13:51:24.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa5170b700 1 -- 172.21.15.57:0/3872789766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c1976a0 con 0x7ffa4c0fef30 2023-10-08T13:51:24.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.432+0000 7ffa5170b700 1 -- 172.21.15.57:0/3872789766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c1a3890 con 0x7ffa4c0fef30 2023-10-08T13:51:24.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.433+0000 7ffa5170b700 1 -- 172.21.15.57:0/3872789766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c108bb0 con 0x7ffa4c0fef30 2023-10-08T13:51:24.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.435+0000 7ffa33fff700 1 -- 172.21.15.57:0/3872789766 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ffa3c0108d0 con 0x7ffa4c0fef30 2023-10-08T13:51:24.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.436+0000 7ffa33fff700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa340a14d0 0x7ffa340a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.436+0000 7ffa33fff700 1 -- 172.21.15.57:0/3872789766 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ffa3c0ccfd0 con 0x7ffa4c0fef30 2023-10-08T13:51:24.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.436+0000 7ffa4a7fc700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa340a14d0 0x7ffa340a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.436+0000 7ffa4a7fc700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa340a14d0 0x7ffa340a3990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffa40005fd0 tx=0x7ffa40005f60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:24.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.439+0000 7ffa33fff700 1 -- 172.21.15.57:0/3872789766 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffa3c0961b0 con 0x7ffa4c0fef30 2023-10-08T13:51:24.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.644+0000 7ffa5170b700 1 -- 172.21.15.57:0/3872789766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa4c068790 con 0x7ffa4c0fef30 2023-10-08T13:51:24.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.645+0000 7ffa33fff700 1 -- 172.21.15.57:0/3872789766 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24847 (secure 0 0 0) 0x7ffa3c016090 con 0x7ffa4c0fef30 2023-10-08T13:51:24.647 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:24.647 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:51:16.778201+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:51:24.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 -- 172.21.15.57:0/3872789766 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa340a14d0 msgr2=0x7ffa340a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa340a14d0 0x7ffa340a3990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffa40005fd0 tx=0x7ffa40005f60 comp rx=0 tx=0).stop 2023-10-08T13:51:24.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 -- 172.21.15.57:0/3872789766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa4c0fef30 msgr2=0x7ffa4c19c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa4c0fef30 0x7ffa4c19c8a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00d950 tx=0x7ffa3c00dc60 comp rx=0 tx=0).stop 2023-10-08T13:51:24.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 -- 172.21.15.57:0/3872789766 shutdown_connections 2023-10-08T13:51:24.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ffa340a14d0 0x7ffa340a3990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa4c106870 0x7ffa4c196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa4c0fef30 0x7ffa4c19c8a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 --2- 172.21.15.57:0/3872789766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0ff8f0 0x7ffa4c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 -- 172.21.15.57:0/3872789766 >> 172.21.15.57:0/3872789766 conn(0x7ffa4c074820 msgr2=0x7ffa4c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:24.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 -- 172.21.15.57:0/3872789766 shutdown_connections 2023-10-08T13:51:24.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.648+0000 7ffa5170b700 1 -- 172.21.15.57:0/3872789766 wait complete. 2023-10-08T13:51:24.661 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T13:51:24.662 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:51:24.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.789+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3500221849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01f81056b0 msgr2=0x7f01f8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.789+0000 7f01ffd9c700 1 --2- 172.21.15.57:0/3500221849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01f81056b0 0x7f01f8105a90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009990 tx=0x7f01e8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.789+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3500221849 shutdown_connections 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.789+0000 7f01ffd9c700 1 --2- 172.21.15.57:0/3500221849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f80fffe0 0x7f01f810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.789+0000 7f01ffd9c700 1 --2- 172.21.15.57:0/3500221849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01f81056b0 0x7f01f8105a90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.789+0000 7f01ffd9c700 1 --2- 172.21.15.57:0/3500221849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f80ff620 0x7f01f80ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.789+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3500221849 >> 172.21.15.57:0/3500221849 conn(0x7f01f8076780 msgr2=0x7f01f8076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.789+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3500221849 shutdown_connections 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.789+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3500221849 wait complete. 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01ffd9c700 1 Processor -- start 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01ffd9c700 1 -- start start 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01ffd9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f80ff620 0x7f01f8197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01ffd9c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f80fffe0 0x7f01f8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01ffd9c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01f81056b0 0x7f01f81a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01ffd9c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f8198710 con 0x7f01f81056b0 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01ffd9c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f81a2f40 con 0x7f01f80ff620 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01ffd9c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f81a3080 con 0x7f01f80fffe0 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fe339700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01f81056b0 0x7f01f81a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fe339700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01f81056b0 0x7f01f81a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56014/0 (socket says 172.21.15.57:56014) 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fdb38700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f80ff620 0x7f01f8197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fe339700 1 -- 172.21.15.57:0/3604557072 learned_addr learned my addr 172.21.15.57:0/3604557072 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fd337700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f80fffe0 0x7f01f8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fe339700 1 -- 172.21.15.57:0/3604557072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f80fffe0 msgr2=0x7f01f8197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fe339700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f80fffe0 0x7f01f8197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fe339700 1 -- 172.21.15.57:0/3604557072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f80ff620 msgr2=0x7f01f8197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:24.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fe339700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f80ff620 0x7f01f8197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:24.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fe339700 1 -- 172.21.15.57:0/3604557072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80095f0 con 0x7f01f81056b0 2023-10-08T13:51:24.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.790+0000 7f01fd337700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f80fffe0 0x7f01f8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:24.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.791+0000 7f01fe339700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01f81056b0 0x7f01f81a2a00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f01f0007f10 tx=0x7f01f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:24.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.791+0000 7f01eeffd700 1 -- 172.21.15.57:0/3604557072 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0017070 con 0x7f01f81056b0 2023-10-08T13:51:24.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.791+0000 7f01eeffd700 1 -- 172.21.15.57:0/3604557072 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01f000f040 con 0x7f01f81056b0 2023-10-08T13:51:24.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.791+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3604557072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f81a32e0 con 0x7f01f81056b0 2023-10-08T13:51:24.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.791+0000 7f01eeffd700 1 -- 172.21.15.57:0/3604557072 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0005740 con 0x7f01f81056b0 2023-10-08T13:51:24.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.791+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3604557072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f81a37a0 con 0x7f01f81056b0 2023-10-08T13:51:24.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.792+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3604557072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f8107c10 con 0x7f01f81056b0 2023-10-08T13:51:24.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.793+0000 7f01eeffd700 1 -- 172.21.15.57:0/3604557072 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f01f00058a0 con 0x7f01f81056b0 2023-10-08T13:51:24.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.794+0000 7f01eeffd700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f01e40a14d0 0x7f01e40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:24.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.794+0000 7f01fdb38700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f01e40a14d0 0x7f01e40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:24.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.794+0000 7f01eeffd700 1 -- 172.21.15.57:0/3604557072 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f01f00cd000 con 0x7f01f81056b0 2023-10-08T13:51:24.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.794+0000 7f01fdb38700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f01e40a14d0 0x7f01e40a3990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f01e8006010 tx=0x7f01e8005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:24.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:24.798+0000 7f01eeffd700 1 -- 172.21.15.57:0/3604557072 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f01f0096240 con 0x7f01f81056b0 2023-10-08T13:51:24.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:24 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2156132711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:24.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:24 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3841856932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:24.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:24 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3872789766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:25.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.000+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3604557072 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01f8067dd0 con 0x7f01f81056b0 2023-10-08T13:51:25.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.001+0000 7f01eeffd700 1 -- 172.21.15.57:0/3604557072 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01f0099860 con 0x7f01f81056b0 2023-10-08T13:51:25.002 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:51:25.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3604557072 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f01e40a14d0 msgr2=0x7f01e40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f01e40a14d0 0x7f01e40a3990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f01e8006010 tx=0x7f01e8005960 comp rx=0 tx=0).stop 2023-10-08T13:51:25.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3604557072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01f81056b0 msgr2=0x7f01f81a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01f81056b0 0x7f01f81a2a00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f01f0007f10 tx=0x7f01f000d3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3604557072 shutdown_connections 2023-10-08T13:51:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f01f81056b0 0x7f01f81a2a00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f01e40a14d0 0x7f01e40a3990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01f80ff620 0x7f01f8197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 --2- 172.21.15.57:0/3604557072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f80fffe0 0x7f01f8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3604557072 >> 172.21.15.57:0/3604557072 conn(0x7f01f8076780 msgr2=0x7f01f80fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3604557072 shutdown_connections 2023-10-08T13:51:25.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.003+0000 7f01ffd9c700 1 -- 172.21.15.57:0/3604557072 wait complete. 2023-10-08T13:51:25.017 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T13:51:25.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.145+0000 7faead62f700 1 -- 172.21.15.57:0/2019736371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faea80ff620 msgr2=0x7faea80ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.145+0000 7faead62f700 1 --2- 172.21.15.57:0/2019736371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faea80ff620 0x7faea80ffaa0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fae90009990 tx=0x7fae90009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.145+0000 7faead62f700 1 -- 172.21.15.57:0/2019736371 shutdown_connections 2023-10-08T13:51:25.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.145+0000 7faead62f700 1 --2- 172.21.15.57:0/2019736371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faea80fffe0 0x7faea810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.145+0000 7faead62f700 1 --2- 172.21.15.57:0/2019736371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea81056b0 0x7faea8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.145+0000 7faead62f700 1 --2- 172.21.15.57:0/2019736371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faea80ff620 0x7faea80ffaa0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.145+0000 7faead62f700 1 -- 172.21.15.57:0/2019736371 >> 172.21.15.57:0/2019736371 conn(0x7faea8076780 msgr2=0x7faea8076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:25.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.145+0000 7faead62f700 1 -- 172.21.15.57:0/2019736371 shutdown_connections 2023-10-08T13:51:25.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.145+0000 7faead62f700 1 -- 172.21.15.57:0/2019736371 wait complete. 2023-10-08T13:51:25.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.146+0000 7faead62f700 1 Processor -- start 2023-10-08T13:51:25.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.146+0000 7faead62f700 1 -- start start 2023-10-08T13:51:25.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faead62f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faea80ff620 0x7faea819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faead62f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea80fffe0 0x7faea819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faead62f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faea81056b0 0x7faea8195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faead62f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea8195ec0 con 0x7faea80ff620 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faead62f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea8196030 con 0x7faea80fffe0 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faead62f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81961f0 con 0x7faea81056b0 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faea80ff620 0x7faea819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faea80ff620 0x7faea819b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56024/0 (socket says 172.21.15.57:56024) 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea6ffd700 1 -- 172.21.15.57:0/196851667 learned_addr learned my addr 172.21.15.57:0/196851667 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea77fe700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faea81056b0 0x7faea8195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea67fc700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea80fffe0 0x7faea819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea6ffd700 1 -- 172.21.15.57:0/196851667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faea81056b0 msgr2=0x7faea8195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea6ffd700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faea81056b0 0x7faea8195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea6ffd700 1 -- 172.21.15.57:0/196851667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea80fffe0 msgr2=0x7faea819bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea6ffd700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea80fffe0 0x7faea819bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea6ffd700 1 -- 172.21.15.57:0/196851667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae900095f0 con 0x7faea80ff620 2023-10-08T13:51:25.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea67fc700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea80fffe0 0x7faea819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:25.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faea6ffd700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faea80ff620 0x7faea819b720 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fae9800d980 tx=0x7fae9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:25.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7fae9ffff700 1 -- 172.21.15.57:0/196851667 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98004020 con 0x7faea80ff620 2023-10-08T13:51:25.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7faead62f700 1 -- 172.21.15.57:0/196851667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea81964d0 con 0x7faea80ff620 2023-10-08T13:51:25.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.147+0000 7fae9ffff700 1 -- 172.21.15.57:0/196851667 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae98009730 con 0x7faea80ff620 2023-10-08T13:51:25.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.148+0000 7fae9ffff700 1 -- 172.21.15.57:0/196851667 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98003a30 con 0x7faea80ff620 2023-10-08T13:51:25.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.148+0000 7faead62f700 1 -- 172.21.15.57:0/196851667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea81a2710 con 0x7faea80ff620 2023-10-08T13:51:25.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.148+0000 7faead62f700 1 -- 172.21.15.57:0/196851667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea8107c30 con 0x7faea80ff620 2023-10-08T13:51:25.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.149+0000 7fae9ffff700 1 -- 172.21.15.57:0/196851667 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fae980098a0 con 0x7faea80ff620 2023-10-08T13:51:25.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.150+0000 7fae9ffff700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fae940a14d0 0x7fae940a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.150+0000 7faea67fc700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fae940a14d0 0x7fae940a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.150+0000 7fae9ffff700 1 -- 172.21.15.57:0/196851667 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fae980cceb0 con 0x7faea80ff620 2023-10-08T13:51:25.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.151+0000 7faea67fc700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fae940a14d0 0x7fae940a3990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b2e0 tx=0x7fae90005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:25.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.155+0000 7fae9ffff700 1 -- 172.21.15.57:0/196851667 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fae98096090 con 0x7faea80ff620 2023-10-08T13:51:25.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:24 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2156132711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:25.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:24 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3841856932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:25.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:24 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3872789766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:25.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:24 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2156132711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:25.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:24 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3841856932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:25.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:24 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3872789766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:25.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.359+0000 7faead62f700 1 -- 172.21.15.57:0/196851667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faea8196d10 con 0x7faea80ff620 2023-10-08T13:51:25.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.359+0000 7fae9ffff700 1 -- 172.21.15.57:0/196851667 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae980996b0 con 0x7faea80ff620 2023-10-08T13:51:25.360 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:51:25.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 -- 172.21.15.57:0/196851667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fae940a14d0 msgr2=0x7fae940a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fae940a14d0 0x7fae940a3990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b2e0 tx=0x7fae90005fd0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 -- 172.21.15.57:0/196851667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faea80ff620 msgr2=0x7faea819b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faea80ff620 0x7faea819b720 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fae9800d980 tx=0x7fae9800dc90 comp rx=0 tx=0).stop 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 -- 172.21.15.57:0/196851667 shutdown_connections 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faea81056b0 0x7faea8195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faea80ff620 0x7faea819b720 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fae940a14d0 0x7fae940a3990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 --2- 172.21.15.57:0/196851667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea80fffe0 0x7faea819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 -- 172.21.15.57:0/196851667 >> 172.21.15.57:0/196851667 conn(0x7faea8076780 msgr2=0x7faea80fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 -- 172.21.15.57:0/196851667 shutdown_connections 2023-10-08T13:51:25.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.361+0000 7faead62f700 1 -- 172.21.15.57:0/196851667 wait complete. 2023-10-08T13:51:25.375 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T13:51:25.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.503+0000 7faa4cf77700 1 -- 172.21.15.57:0/3663096697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa48105630 msgr2=0x7faa48105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.503+0000 7faa4cf77700 1 --2- 172.21.15.57:0/3663096697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa48105630 0x7faa48105a10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faa38009990 tx=0x7faa38009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.503+0000 7faa4cf77700 1 -- 172.21.15.57:0/3663096697 shutdown_connections 2023-10-08T13:51:25.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.503+0000 7faa4cf77700 1 --2- 172.21.15.57:0/3663096697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa48100000 0x7faa4810c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.503+0000 7faa4cf77700 1 --2- 172.21.15.57:0/3663096697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa48105630 0x7faa48105a10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.503+0000 7faa4cf77700 1 --2- 172.21.15.57:0/3663096697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa480ff640 0x7faa480ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.503+0000 7faa4cf77700 1 -- 172.21.15.57:0/3663096697 >> 172.21.15.57:0/3663096697 conn(0x7faa48076790 msgr2=0x7faa48076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:25.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.503+0000 7faa4cf77700 1 -- 172.21.15.57:0/3663096697 shutdown_connections 2023-10-08T13:51:25.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.503+0000 7faa4cf77700 1 -- 172.21.15.57:0/3663096697 wait complete. 2023-10-08T13:51:25.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4cf77700 1 Processor -- start 2023-10-08T13:51:25.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4cf77700 1 -- start start 2023-10-08T13:51:25.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4cf77700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa480ff640 0x7faa4819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4cf77700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa48100000 0x7faa4819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4cf77700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa48105630 0x7faa48195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4cf77700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa48195ee0 con 0x7faa480ff640 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4cf77700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa48196050 con 0x7faa48105630 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4cf77700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa48196210 con 0x7faa48100000 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa480ff640 0x7faa4819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa480ff640 0x7faa4819b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56038/0 (socket says 172.21.15.57:56038) 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.504+0000 7faa4659c700 1 -- 172.21.15.57:0/981478115 learned_addr learned my addr 172.21.15.57:0/981478115 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa45d9b700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa48100000 0x7faa4819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa46d9d700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa48105630 0x7faa48195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa4659c700 1 -- 172.21.15.57:0/981478115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa48100000 msgr2=0x7faa4819bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa4659c700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa48100000 0x7faa4819bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa4659c700 1 -- 172.21.15.57:0/981478115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa48105630 msgr2=0x7faa48195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa4659c700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa48105630 0x7faa48195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa4659c700 1 -- 172.21.15.57:0/981478115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380095f0 con 0x7faa480ff640 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa45d9b700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa48100000 0x7faa4819bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa4659c700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa480ff640 0x7faa4819b6f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7faa380098e0 tx=0x7faa3801a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa377fe700 1 -- 172.21.15.57:0/981478115 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3801f070 con 0x7faa480ff640 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa377fe700 1 -- 172.21.15.57:0/981478115 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa3800f460 con 0x7faa480ff640 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa4cf77700 1 -- 172.21.15.57:0/981478115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa48196490 con 0x7faa480ff640 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa377fe700 1 -- 172.21.15.57:0/981478115 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38025680 con 0x7faa480ff640 2023-10-08T13:51:25.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.505+0000 7faa4cf77700 1 -- 172.21.15.57:0/981478115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa48068c10 con 0x7faa480ff640 2023-10-08T13:51:25.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.506+0000 7faa4cf77700 1 -- 172.21.15.57:0/981478115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa48100d60 con 0x7faa480ff640 2023-10-08T13:51:25.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.507+0000 7faa377fe700 1 -- 172.21.15.57:0/981478115 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7faa38023070 con 0x7faa480ff640 2023-10-08T13:51:25.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.508+0000 7faa377fe700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa300a14d0 0x7faa300a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.508+0000 7faa45d9b700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa300a14d0 0x7faa300a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.508+0000 7faa377fe700 1 -- 172.21.15.57:0/981478115 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7faa380d0710 con 0x7faa480ff640 2023-10-08T13:51:25.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.508+0000 7faa45d9b700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa300a14d0 0x7faa300a3990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faa3c005950 tx=0x7faa3c0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:25.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.512+0000 7faa377fe700 1 -- 172.21.15.57:0/981478115 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faa38099980 con 0x7faa480ff640 2023-10-08T13:51:25.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.714+0000 7faa4cf77700 1 -- 172.21.15.57:0/981478115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa48196ce0 con 0x7faa480ff640 2023-10-08T13:51:25.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.714+0000 7faa377fe700 1 -- 172.21.15.57:0/981478115 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa3809cfa0 con 0x7faa480ff640 2023-10-08T13:51:25.716 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T13:51:25.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 -- 172.21.15.57:0/981478115 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa300a14d0 msgr2=0x7faa300a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa300a14d0 0x7faa300a3990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faa3c005950 tx=0x7faa3c0058e0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 -- 172.21.15.57:0/981478115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa480ff640 msgr2=0x7faa4819b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa480ff640 0x7faa4819b6f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7faa380098e0 tx=0x7faa3801a8d0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 -- 172.21.15.57:0/981478115 shutdown_connections 2023-10-08T13:51:25.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa300a14d0 0x7faa300a3990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa48105630 0x7faa48195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa480ff640 0x7faa4819b6f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 --2- 172.21.15.57:0/981478115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa48100000 0x7faa4819bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 -- 172.21.15.57:0/981478115 >> 172.21.15.57:0/981478115 conn(0x7faa48076790 msgr2=0x7faa480fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:25.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.716+0000 7faa4cf77700 1 -- 172.21.15.57:0/981478115 shutdown_connections 2023-10-08T13:51:25.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.717+0000 7faa4cf77700 1 -- 172.21.15.57:0/981478115 wait complete. 2023-10-08T13:51:25.730 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T13:51:25.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.858+0000 7f56ad27c700 1 -- 172.21.15.57:0/1764096694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56a8101100 msgr2=0x7f56a810d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.858+0000 7f56ad27c700 1 --2- 172.21.15.57:0/1764096694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56a8101100 0x7f56a810d610 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009990 tx=0x7f56a0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.858+0000 7f56ad27c700 1 -- 172.21.15.57:0/1764096694 shutdown_connections 2023-10-08T13:51:25.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.858+0000 7f56ad27c700 1 --2- 172.21.15.57:0/1764096694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56a8101100 0x7f56a810d610 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.858+0000 7f56ad27c700 1 --2- 172.21.15.57:0/1764096694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a8106750 0x7f56a8106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.858+0000 7f56ad27c700 1 --2- 172.21.15.57:0/1764096694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56a8100740 0x7f56a8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.858+0000 7f56ad27c700 1 -- 172.21.15.57:0/1764096694 >> 172.21.15.57:0/1764096694 conn(0x7f56a80fc230 msgr2=0x7f56a80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:25.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.858+0000 7f56ad27c700 1 -- 172.21.15.57:0/1764096694 shutdown_connections 2023-10-08T13:51:25.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.858+0000 7f56ad27c700 1 -- 172.21.15.57:0/1764096694 wait complete. 2023-10-08T13:51:25.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56ad27c700 1 Processor -- start 2023-10-08T13:51:25.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56ad27c700 1 -- start start 2023-10-08T13:51:25.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56ad27c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56a8100740 0x7f56a819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56ad27c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a8101100 0x7f56a819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56ad27c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56a8106750 0x7f56a8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56ad27c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81991a0 con 0x7f56a8106750 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56ad27c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a8199310 con 0x7f56a8100740 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56ad27c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81994d0 con 0x7f56a8101100 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56a8106750 0x7f56a8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56a8106750 0x7f56a8198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56056/0 (socket says 172.21.15.57:56056) 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a77fe700 1 -- 172.21.15.57:0/3358290093 learned_addr learned my addr 172.21.15.57:0/3358290093 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a67fc700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a8101100 0x7f56a819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a6ffd700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56a8100740 0x7f56a819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a77fe700 1 -- 172.21.15.57:0/3358290093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a8101100 msgr2=0x7f56a819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a77fe700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a8101100 0x7f56a819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a77fe700 1 -- 172.21.15.57:0/3358290093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56a8100740 msgr2=0x7f56a819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a77fe700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56a8100740 0x7f56a819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a77fe700 1 -- 172.21.15.57:0/3358290093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a00095f0 con 0x7f56a8106750 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.859+0000 7f56a6ffd700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56a8100740 0x7f56a819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.860+0000 7f56a77fe700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56a8106750 0x7f56a8198b40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f56a0004730 tx=0x7f56a0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.860+0000 7f568ffff700 1 -- 172.21.15.57:0/3358290093 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a001d070 con 0x7f56a8106750 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.860+0000 7f568ffff700 1 -- 172.21.15.57:0/3358290093 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56a0022470 con 0x7f56a8106750 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.860+0000 7f56ad27c700 1 -- 172.21.15.57:0/3358290093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a8199750 con 0x7f56a8106750 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.860+0000 7f568ffff700 1 -- 172.21.15.57:0/3358290093 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0017650 con 0x7f56a8106750 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.860+0000 7f56ad27c700 1 -- 172.21.15.57:0/3358290093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a81a5a50 con 0x7f56a8106750 2023-10-08T13:51:25.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.861+0000 7f56ad27c700 1 -- 172.21.15.57:0/3358290093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a8101e80 con 0x7f56a8106750 2023-10-08T13:51:25.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.862+0000 7f568ffff700 1 -- 172.21.15.57:0/3358290093 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f56a00225e0 con 0x7f56a8106750 2023-10-08T13:51:25.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.863+0000 7f568ffff700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56900a14d0 0x7f56900a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:25.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.863+0000 7f56a6ffd700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56900a14d0 0x7f56900a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:25.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.863+0000 7f568ffff700 1 -- 172.21.15.57:0/3358290093 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f56a00cec90 con 0x7f56a8106750 2023-10-08T13:51:25.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.863+0000 7f56a6ffd700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56900a14d0 0x7f56900a3990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5698009d50 tx=0x7f5698009040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:25.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:25.867+0000 7f568ffff700 1 -- 172.21.15.57:0/3358290093 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f56a0097f60 con 0x7f56a8106750 2023-10-08T13:51:25.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:25 smithi057 ceph-mon[99476]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 490 B/s wr, 10 op/s 2023-10-08T13:51:25.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:25 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3604557072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:25.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:25 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/196851667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:25.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:25 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/981478115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:26.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.069+0000 7f56ad27c700 1 -- 172.21.15.57:0/3358290093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f56a819a080 con 0x7f56a8106750 2023-10-08T13:51:26.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.069+0000 7f568ffff700 1 -- 172.21.15.57:0/3358290093 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f56a009b580 con 0x7f56a8106750 2023-10-08T13:51:26.071 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:51:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.071+0000 7f56ad27c700 1 -- 172.21.15.57:0/3358290093 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56900a14d0 msgr2=0x7f56900a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.071+0000 7f56ad27c700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56900a14d0 0x7f56900a3990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5698009d50 tx=0x7f5698009040 comp rx=0 tx=0).stop 2023-10-08T13:51:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.071+0000 7f56ad27c700 1 -- 172.21.15.57:0/3358290093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56a8106750 msgr2=0x7f56a8198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.071+0000 7f56ad27c700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56a8106750 0x7f56a8198b40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f56a0004730 tx=0x7f56a0004760 comp rx=0 tx=0).stop 2023-10-08T13:51:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.072+0000 7f56ad27c700 1 -- 172.21.15.57:0/3358290093 shutdown_connections 2023-10-08T13:51:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.072+0000 7f56ad27c700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56a8106750 0x7f56a8198b40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.072+0000 7f56ad27c700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f56900a14d0 0x7f56900a3990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.072+0000 7f56ad27c700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56a8100740 0x7f56a819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.072+0000 7f56ad27c700 1 --2- 172.21.15.57:0/3358290093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a8101100 0x7f56a819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.072+0000 7f56ad27c700 1 -- 172.21.15.57:0/3358290093 >> 172.21.15.57:0/3358290093 conn(0x7f56a80fc230 msgr2=0x7f56a810bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.072+0000 7f56ad27c700 1 -- 172.21.15.57:0/3358290093 shutdown_connections 2023-10-08T13:51:26.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.072+0000 7f56ad27c700 1 -- 172.21.15.57:0/3358290093 wait complete. 2023-10-08T13:51:26.085 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-08T13:51:26.085 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:51:26.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:25 smithi115 ceph-mon[104408]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 490 B/s wr, 10 op/s 2023-10-08T13:51:26.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:25 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3604557072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:26.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:25 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/196851667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:26.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:25 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/981478115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:26.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.211+0000 7f87e8e5c700 1 -- 172.21.15.57:0/1576194337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e40fff20 msgr2=0x7f87e410c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.211+0000 7f87e8e5c700 1 --2- 172.21.15.57:0/1576194337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e40fff20 0x7f87e410c3e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00b0d0 tx=0x7f87dc00b3e0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.211+0000 7f87e8e5c700 1 -- 172.21.15.57:0/1576194337 shutdown_connections 2023-10-08T13:51:26.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.211+0000 7f87e8e5c700 1 --2- 172.21.15.57:0/1576194337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e40fff20 0x7f87e410c3e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.211+0000 7f87e8e5c700 1 --2- 172.21.15.57:0/1576194337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e4105620 0x7f87e4105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.211+0000 7f87e8e5c700 1 --2- 172.21.15.57:0/1576194337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e40ff560 0x7f87e40ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.211+0000 7f87e8e5c700 1 -- 172.21.15.57:0/1576194337 >> 172.21.15.57:0/1576194337 conn(0x7f87e40fb070 msgr2=0x7f87e40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:26.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.211+0000 7f87e8e5c700 1 -- 172.21.15.57:0/1576194337 shutdown_connections 2023-10-08T13:51:26.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.211+0000 7f87e8e5c700 1 -- 172.21.15.57:0/1576194337 wait complete. 2023-10-08T13:51:26.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.211+0000 7f87e8e5c700 1 Processor -- start 2023-10-08T13:51:26.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e8e5c700 1 -- start start 2023-10-08T13:51:26.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e8e5c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e40ff560 0x7f87e419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:26.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e8e5c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e40fff20 0x7f87e419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:26.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e8e5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e4105620 0x7f87e4195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:26.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e8e5c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4195f60 con 0x7f87e40fff20 2023-10-08T13:51:26.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e8e5c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e41960d0 con 0x7f87e4105620 2023-10-08T13:51:26.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e8e5c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4196290 con 0x7f87e40ff560 2023-10-08T13:51:26.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e40fff20 0x7f87e419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:26.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e40fff20 0x7f87e419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56068/0 (socket says 172.21.15.57:56068) 2023-10-08T13:51:26.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e1d9b700 1 -- 172.21.15.57:0/3257285918 learned_addr learned my addr 172.21.15.57:0/3257285918 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:26.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e2d9d700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e4105620 0x7f87e4195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:26.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e259c700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e40ff560 0x7f87e419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:26.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e1d9b700 1 -- 172.21.15.57:0/3257285918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e40ff560 msgr2=0x7f87e419b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e1d9b700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e40ff560 0x7f87e419b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e1d9b700 1 -- 172.21.15.57:0/3257285918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e4105620 msgr2=0x7f87e4195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e1d9b700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e4105620 0x7f87e4195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e1d9b700 1 -- 172.21.15.57:0/3257285918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc009d00 con 0x7f87e40fff20 2023-10-08T13:51:26.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e259c700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e40ff560 0x7f87e419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:26.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87e1d9b700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e40fff20 0x7f87e419bd10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f87d8007b50 tx=0x7f87d8007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:26.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.212+0000 7f87d37fe700 1 -- 172.21.15.57:0/3257285918 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8011070 con 0x7f87e40fff20 2023-10-08T13:51:26.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.213+0000 7f87d37fe700 1 -- 172.21.15.57:0/3257285918 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87d800c2e0 con 0x7f87e40fff20 2023-10-08T13:51:26.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.213+0000 7f87e8e5c700 1 -- 172.21.15.57:0/3257285918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e4196570 con 0x7f87e40fff20 2023-10-08T13:51:26.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.213+0000 7f87d37fe700 1 -- 172.21.15.57:0/3257285918 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800e500 con 0x7f87e40fff20 2023-10-08T13:51:26.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.213+0000 7f87e8e5c700 1 -- 172.21.15.57:0/3257285918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e4068c10 con 0x7f87e40fff20 2023-10-08T13:51:26.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.214+0000 7f87e8e5c700 1 -- 172.21.15.57:0/3257285918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e4100c80 con 0x7f87e40fff20 2023-10-08T13:51:26.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.215+0000 7f87d37fe700 1 -- 172.21.15.57:0/3257285918 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f87d8004750 con 0x7f87e40fff20 2023-10-08T13:51:26.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.215+0000 7f87d37fe700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87cc0a14d0 0x7f87cc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:26.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.215+0000 7f87e259c700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87cc0a14d0 0x7f87cc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:26.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.215+0000 7f87d37fe700 1 -- 172.21.15.57:0/3257285918 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f87d80ce9e0 con 0x7f87e40fff20 2023-10-08T13:51:26.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.216+0000 7f87e259c700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87cc0a14d0 0x7f87cc0a3990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005fd0 tx=0x7f87d4005db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:26.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.219+0000 7f87d37fe700 1 -- 172.21.15.57:0/3257285918 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f87d8097cb0 con 0x7f87e40fff20 2023-10-08T13:51:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:25 smithi099 ceph-mon[103849]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 490 B/s wr, 10 op/s 2023-10-08T13:51:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:25 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3604557072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:25 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/196851667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:25 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/981478115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:26.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.471+0000 7f87e8e5c700 1 -- 172.21.15.57:0/3257285918 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87e4066bc0 con 0x7f87e40fff20 2023-10-08T13:51:26.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.472+0000 7f87d37fe700 1 -- 172.21.15.57:0/3257285918 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f87d809b2d0 con 0x7f87e40fff20 2023-10-08T13:51:26.473 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:26.473 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:50:07.719573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:51:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.474+0000 7f87e8e5c700 1 -- 172.21.15.57:0/3257285918 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87cc0a14d0 msgr2=0x7f87cc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.474+0000 7f87e8e5c700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87cc0a14d0 0x7f87cc0a3990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005fd0 tx=0x7f87d4005db0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.474+0000 7f87e8e5c700 1 -- 172.21.15.57:0/3257285918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e40fff20 msgr2=0x7f87e419bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.474+0000 7f87e8e5c700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e40fff20 0x7f87e419bd10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f87d8007b50 tx=0x7f87d8007e60 comp rx=0 tx=0).stop 2023-10-08T13:51:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.475+0000 7f87e8e5c700 1 -- 172.21.15.57:0/3257285918 shutdown_connections 2023-10-08T13:51:26.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.475+0000 7f87e8e5c700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87cc0a14d0 0x7f87cc0a3990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.475+0000 7f87e8e5c700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e4105620 0x7f87e4195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.475+0000 7f87e8e5c700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e40ff560 0x7f87e419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.475+0000 7f87e8e5c700 1 --2- 172.21.15.57:0/3257285918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e40fff20 0x7f87e419bd10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.475+0000 7f87e8e5c700 1 -- 172.21.15.57:0/3257285918 >> 172.21.15.57:0/3257285918 conn(0x7f87e40fb070 msgr2=0x7f87e40fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:26.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.475+0000 7f87e8e5c700 1 -- 172.21.15.57:0/3257285918 shutdown_connections 2023-10-08T13:51:26.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.475+0000 7f87e8e5c700 1 -- 172.21.15.57:0/3257285918 wait complete. 2023-10-08T13:51:26.478 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:51:26.489 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-08T13:51:26.489 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-08T13:51:26.489 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T13:51:26.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.614+0000 7f348b210700 1 -- 172.21.15.57:0/2363839275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34841057e0 msgr2=0x7f3484105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.614+0000 7f348b210700 1 --2- 172.21.15.57:0/2363839275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34841057e0 0x7f3484105bc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3478009990 tx=0x7f3478009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.614+0000 7f348b210700 1 -- 172.21.15.57:0/2363839275 shutdown_connections 2023-10-08T13:51:26.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.614+0000 7f348b210700 1 --2- 172.21.15.57:0/2363839275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3484100620 0x7f348410c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.614+0000 7f348b210700 1 --2- 172.21.15.57:0/2363839275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34841057e0 0x7f3484105bc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.614+0000 7f348b210700 1 --2- 172.21.15.57:0/2363839275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840ffc60 0x7f34841000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.614+0000 7f348b210700 1 -- 172.21.15.57:0/2363839275 >> 172.21.15.57:0/2363839275 conn(0x7f3484075a90 msgr2=0x7f34840fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:26.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.614+0000 7f348b210700 1 -- 172.21.15.57:0/2363839275 shutdown_connections 2023-10-08T13:51:26.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.614+0000 7f348b210700 1 -- 172.21.15.57:0/2363839275 wait complete. 2023-10-08T13:51:26.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f348b210700 1 Processor -- start 2023-10-08T13:51:26.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f348b210700 1 -- start start 2023-10-08T13:51:26.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f348b210700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840ffc60 0x7f348419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:26.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f348b210700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3484100620 0x7f348419beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:26.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f348b210700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34841057e0 0x7f3484195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:26.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f348b210700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3484196100 con 0x7f34841057e0 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f348b210700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3484196270 con 0x7f34840ffc60 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f348b210700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3484196430 con 0x7f3484100620 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f34897ad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34841057e0 0x7f3484195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f34897ad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34841057e0 0x7f3484195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56084/0 (socket says 172.21.15.57:56084) 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f34897ad700 1 -- 172.21.15.57:0/2655634994 learned_addr learned my addr 172.21.15.57:0/2655634994 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f3488fac700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840ffc60 0x7f348419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f3483fff700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3484100620 0x7f348419beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f34897ad700 1 -- 172.21.15.57:0/2655634994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3484100620 msgr2=0x7f348419beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f34897ad700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3484100620 0x7f348419beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f34897ad700 1 -- 172.21.15.57:0/2655634994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840ffc60 msgr2=0x7f348419b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f34897ad700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840ffc60 0x7f348419b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f34897ad700 1 -- 172.21.15.57:0/2655634994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780095f0 con 0x7f34841057e0 2023-10-08T13:51:26.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f3483fff700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3484100620 0x7f348419beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:26.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.615+0000 7f34897ad700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34841057e0 0x7f3484195aa0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f347c00db90 tx=0x7f347c00df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:26.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.616+0000 7f3481ffb700 1 -- 172.21.15.57:0/2655634994 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c00e800 con 0x7f34841057e0 2023-10-08T13:51:26.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.616+0000 7f3481ffb700 1 -- 172.21.15.57:0/2655634994 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f347c00bac0 con 0x7f34841057e0 2023-10-08T13:51:26.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.616+0000 7f3481ffb700 1 -- 172.21.15.57:0/2655634994 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0163f0 con 0x7f34841057e0 2023-10-08T13:51:26.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.616+0000 7f348b210700 1 -- 172.21.15.57:0/2655634994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34841966b0 con 0x7f34841057e0 2023-10-08T13:51:26.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.616+0000 7f348b210700 1 -- 172.21.15.57:0/2655634994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34840fed30 con 0x7f34841057e0 2023-10-08T13:51:26.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.617+0000 7f348b210700 1 -- 172.21.15.57:0/2655634994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34841968d0 con 0x7f34841057e0 2023-10-08T13:51:26.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.618+0000 7f3481ffb700 1 -- 172.21.15.57:0/2655634994 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f347c009580 con 0x7f34841057e0 2023-10-08T13:51:26.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.618+0000 7f3481ffb700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34740a14d0 0x7f34740a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:26.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.619+0000 7f3488fac700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34740a14d0 0x7f34740a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:26.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.619+0000 7f3481ffb700 1 -- 172.21.15.57:0/2655634994 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f347c006e40 con 0x7f34841057e0 2023-10-08T13:51:26.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.619+0000 7f3488fac700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34740a14d0 0x7f34740a3990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3478005850 tx=0x7f347801a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:26.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.623+0000 7f3481ffb700 1 -- 172.21.15.57:0/2655634994 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f347c050100 con 0x7f34841057e0 2023-10-08T13:51:26.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.821+0000 7f348b210700 1 -- 172.21.15.57:0/2655634994 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3484197000 con 0x7f34740a14d0 2023-10-08T13:51:26.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.837+0000 7f3481ffb700 1 -- 172.21.15.57:0/2655634994 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427464 (secure 0 0 0) 0x7f3484197000 con 0x7f34740a14d0 2023-10-08T13:51:26.843 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:26.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 -- 172.21.15.57:0/2655634994 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34740a14d0 msgr2=0x7f34740a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34740a14d0 0x7f34740a3990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3478005850 tx=0x7f347801a040 comp rx=0 tx=0).stop 2023-10-08T13:51:26.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 -- 172.21.15.57:0/2655634994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34841057e0 msgr2=0x7f3484195aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:26.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34841057e0 0x7f3484195aa0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f347c00db90 tx=0x7f347c00df50 comp rx=0 tx=0).stop 2023-10-08T13:51:26.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 -- 172.21.15.57:0/2655634994 shutdown_connections 2023-10-08T13:51:26.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34841057e0 0x7f3484195aa0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f34740a14d0 0x7f34740a3990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840ffc60 0x7f348419b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 --2- 172.21.15.57:0/2655634994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3484100620 0x7f348419beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:26.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 -- 172.21.15.57:0/2655634994 >> 172.21.15.57:0/2655634994 conn(0x7f3484075a90 msgr2=0x7f34840fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:26.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 -- 172.21.15.57:0/2655634994 shutdown_connections 2023-10-08T13:51:26.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:26.841+0000 7f348b210700 1 -- 172.21.15.57:0/2655634994 wait complete. 2023-10-08T13:51:26.914 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-08T13:51:26.914 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-10-08T13:51:26.130285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656625,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":929,"num_read_kb":590,"num_write":425,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338832,"kb_used_data":14556,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124602992,"statfs":{"total":1151940427776,"available":1151593463808,"internally_reserved":0,"allocated":14905344,"data_stored":8317584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002661"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808844+0000","last_change":"2023-10-08T13:49:44.786130+0000","last_active":"2023-10-08T13:51:15.808844+0000","last_peered":"2023-10-08T13:51:15.808844+0000","last_clean":"2023-10-08T13:51:15.808844+0000","last_became_active":"2023-10-08T13:49:44.785792+0000","last_became_peered":"2023-10-08T13:49:44.785792+0000","last_unstale":"2023-10-08T13:51:15.808844+0000","last_undegraded":"2023-10-08T13:51:15.808844+0000","last_fullsized":"2023-10-08T13:51:15.808844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:17.046800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808869+0000","last_change":"2023-10-08T13:49:44.777601+0000","last_active":"2023-10-08T13:51:15.808869+0000","last_peered":"2023-10-08T13:51:15.808869+0000","last_clean":"2023-10-08T13:51:15.808869+0000","last_became_active":"2023-10-08T13:49:44.777287+0000","last_became_peered":"2023-10-08T13:49:44.777287+0000","last_unstale":"2023-10-08T13:51:15.808869+0000","last_undegraded":"2023-10-08T13:51:15.808869+0000","last_fullsized":"2023-10-08T13:51:15.808869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:09:17.317629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808744+0000","last_change":"2023-10-08T13:49:44.787035+0000","last_active":"2023-10-08T13:51:15.808744+0000","last_peered":"2023-10-08T13:51:15.808744+0000","last_clean":"2023-10-08T13:51:15.808744+0000","last_became_active":"2023-10-08T13:49:44.785834+0000","last_became_peered":"2023-10-08T13:49:44.785834+0000","last_unstale":"2023-10-08T13:51:15.808744+0000","last_undegraded":"2023-10-08T13:51:15.808744+0000","last_fullsized":"2023-10-08T13:51:15.808744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:11.611404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783168+0000","last_change":"2023-10-08T13:49:44.781937+0000","last_active":"2023-10-08T13:51:15.783168+0000","last_peered":"2023-10-08T13:51:15.783168+0000","last_clean":"2023-10-08T13:51:15.783168+0000","last_became_active":"2023-10-08T13:49:44.780712+0000","last_became_peered":"2023-10-08T13:49:44.780712+0000","last_unstale":"2023-10-08T13:51:15.783168+0000","last_undegraded":"2023-10-08T13:51:15.783168+0000","last_fullsized":"2023-10-08T13:51:15.783168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:02.514007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783022+0000","last_change":"2023-10-08T13:49:44.788661+0000","last_active":"2023-10-08T13:51:15.783022+0000","last_peered":"2023-10-08T13:51:15.783022+0000","last_clean":"2023-10-08T13:51:15.783022+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:51:15.783022+0000","last_undegraded":"2023-10-08T13:51:15.783022+0000","last_fullsized":"2023-10-08T13:51:15.783022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:35.770287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809252+0000","last_change":"2023-10-08T13:49:44.778519+0000","last_active":"2023-10-08T13:51:15.809252+0000","last_peered":"2023-10-08T13:51:15.809252+0000","last_clean":"2023-10-08T13:51:15.809252+0000","last_became_active":"2023-10-08T13:49:44.778193+0000","last_became_peered":"2023-10-08T13:49:44.778193+0000","last_unstale":"2023-10-08T13:51:15.809252+0000","last_undegraded":"2023-10-08T13:51:15.809252+0000","last_fullsized":"2023-10-08T13:51:15.809252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:12.916019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809332+0000","last_change":"2023-10-08T13:49:44.791759+0000","last_active":"2023-10-08T13:51:15.809332+0000","last_peered":"2023-10-08T13:51:15.809332+0000","last_clean":"2023-10-08T13:51:15.809332+0000","last_became_active":"2023-10-08T13:49:44.791494+0000","last_became_peered":"2023-10-08T13:49:44.791494+0000","last_unstale":"2023-10-08T13:51:15.809332+0000","last_undegraded":"2023-10-08T13:51:15.809332+0000","last_fullsized":"2023-10-08T13:51:15.809332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:55.859609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787828+0000","last_change":"2023-10-08T13:49:44.786831+0000","last_active":"2023-10-08T13:51:15.787828+0000","last_peered":"2023-10-08T13:51:15.787828+0000","last_clean":"2023-10-08T13:51:15.787828+0000","last_became_active":"2023-10-08T13:49:44.786516+0000","last_became_peered":"2023-10-08T13:49:44.786516+0000","last_unstale":"2023-10-08T13:51:15.787828+0000","last_undegraded":"2023-10-08T13:51:15.787828+0000","last_fullsized":"2023-10-08T13:51:15.787828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:07.616821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787111+0000","last_change":"2023-10-08T13:49:44.789827+0000","last_active":"2023-10-08T13:51:15.787111+0000","last_peered":"2023-10-08T13:51:15.787111+0000","last_clean":"2023-10-08T13:51:15.787111+0000","last_became_active":"2023-10-08T13:49:44.786635+0000","last_became_peered":"2023-10-08T13:49:44.786635+0000","last_unstale":"2023-10-08T13:51:15.787111+0000","last_undegraded":"2023-10-08T13:51:15.787111+0000","last_fullsized":"2023-10-08T13:51:15.787111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:35.492437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809682+0000","last_change":"2023-10-08T13:49:44.784240+0000","last_active":"2023-10-08T13:51:15.809682+0000","last_peered":"2023-10-08T13:51:15.809682+0000","last_clean":"2023-10-08T13:51:15.809682+0000","last_became_active":"2023-10-08T13:49:44.783475+0000","last_became_peered":"2023-10-08T13:49:44.783475+0000","last_unstale":"2023-10-08T13:51:15.809682+0000","last_undegraded":"2023-10-08T13:51:15.809682+0000","last_fullsized":"2023-10-08T13:51:15.809682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:39.730679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808970+0000","last_change":"2023-10-08T13:49:44.790901+0000","last_active":"2023-10-08T13:51:15.808970+0000","last_peered":"2023-10-08T13:51:15.808970+0000","last_clean":"2023-10-08T13:51:15.808970+0000","last_became_active":"2023-10-08T13:49:44.790611+0000","last_became_peered":"2023-10-08T13:49:44.790611+0000","last_unstale":"2023-10-08T13:51:15.808970+0000","last_undegraded":"2023-10-08T13:51:15.808970+0000","last_fullsized":"2023-10-08T13:51:15.808970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:19:21.764105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783260+0000","last_change":"2023-10-08T13:49:44.775226+0000","last_active":"2023-10-08T13:51:15.783260+0000","last_peered":"2023-10-08T13:51:15.783260+0000","last_clean":"2023-10-08T13:51:15.783260+0000","last_became_active":"2023-10-08T13:49:44.774756+0000","last_became_peered":"2023-10-08T13:49:44.774756+0000","last_unstale":"2023-10-08T13:51:15.783260+0000","last_undegraded":"2023-10-08T13:51:15.783260+0000","last_fullsized":"2023-10-08T13:51:15.783260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:57:14.749066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809003+0000","last_change":"2023-10-08T13:49:44.781610+0000","last_active":"2023-10-08T13:51:15.809003+0000","last_peered":"2023-10-08T13:51:15.809003+0000","last_clean":"2023-10-08T13:51:15.809003+0000","last_became_active":"2023-10-08T13:49:44.781281+0000","last_became_peered":"2023-10-08T13:49:44.781281+0000","last_unstale":"2023-10-08T13:51:15.809003+0000","last_undegraded":"2023-10-08T13:51:15.809003+0000","last_fullsized":"2023-10-08T13:51:15.809003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:50.823909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808360+0000","last_change":"2023-10-08T13:49:44.775836+0000","last_active":"2023-10-08T13:51:15.808360+0000","last_peered":"2023-10-08T13:51:15.808360+0000","last_clean":"2023-10-08T13:51:15.808360+0000","last_became_active":"2023-10-08T13:49:44.775206+0000","last_became_peered":"2023-10-08T13:49:44.775206+0000","last_unstale":"2023-10-08T13:51:15.808360+0000","last_undegraded":"2023-10-08T13:51:15.808360+0000","last_fullsized":"2023-10-08T13:51:15.808360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:51.566800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808279+0000","last_change":"2023-10-08T13:49:44.778495+0000","last_active":"2023-10-08T13:51:15.808279+0000","last_peered":"2023-10-08T13:51:15.808279+0000","last_clean":"2023-10-08T13:51:15.808279+0000","last_became_active":"2023-10-08T13:49:44.778196+0000","last_became_peered":"2023-10-08T13:49:44.778196+0000","last_unstale":"2023-10-08T13:51:15.808279+0000","last_undegraded":"2023-10-08T13:51:15.808279+0000","last_fullsized":"2023-10-08T13:51:15.808279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:57:38.531853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786633+0000","last_change":"2023-10-08T13:49:44.792960+0000","last_active":"2023-10-08T13:51:15.786633+0000","last_peered":"2023-10-08T13:51:15.786633+0000","last_clean":"2023-10-08T13:51:15.786633+0000","last_became_active":"2023-10-08T13:49:44.792674+0000","last_became_peered":"2023-10-08T13:49:44.792674+0000","last_unstale":"2023-10-08T13:51:15.786633+0000","last_undegraded":"2023-10-08T13:51:15.786633+0000","last_fullsized":"2023-10-08T13:51:15.786633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:13.934954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808777+0000","last_change":"2023-10-08T13:49:44.785971+0000","last_active":"2023-10-08T13:51:15.808777+0000","last_peered":"2023-10-08T13:51:15.808777+0000","last_clean":"2023-10-08T13:51:15.808777+0000","last_became_active":"2023-10-08T13:49:44.785680+0000","last_became_peered":"2023-10-08T13:49:44.785680+0000","last_unstale":"2023-10-08T13:51:15.808777+0000","last_undegraded":"2023-10-08T13:51:15.808777+0000","last_fullsized":"2023-10-08T13:51:15.808777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:35.447519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786759+0000","last_change":"2023-10-08T13:49:44.787553+0000","last_active":"2023-10-08T13:51:15.786759+0000","last_peered":"2023-10-08T13:51:15.786759+0000","last_clean":"2023-10-08T13:51:15.786759+0000","last_became_active":"2023-10-08T13:49:44.787288+0000","last_became_peered":"2023-10-08T13:49:44.787288+0000","last_unstale":"2023-10-08T13:51:15.786759+0000","last_undegraded":"2023-10-08T13:51:15.786759+0000","last_fullsized":"2023-10-08T13:51:15.786759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:48.386120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788367+0000","last_change":"2023-10-08T13:49:44.789252+0000","last_active":"2023-10-08T13:51:15.788367+0000","last_peered":"2023-10-08T13:51:15.788367+0000","last_clean":"2023-10-08T13:51:15.788367+0000","last_became_active":"2023-10-08T13:49:44.788947+0000","last_became_peered":"2023-10-08T13:49:44.788947+0000","last_unstale":"2023-10-08T13:51:15.788367+0000","last_undegraded":"2023-10-08T13:51:15.788367+0000","last_fullsized":"2023-10-08T13:51:15.788367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:30.266913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787998+0000","last_change":"2023-10-08T13:49:49.163619+0000","last_active":"2023-10-08T13:51:15.787998+0000","last_peered":"2023-10-08T13:51:15.787998+0000","last_clean":"2023-10-08T13:51:15.787998+0000","last_became_active":"2023-10-08T13:49:49.163020+0000","last_became_peered":"2023-10-08T13:49:49.163020+0000","last_unstale":"2023-10-08T13:51:15.787998+0000","last_undegraded":"2023-10-08T13:51:15.787998+0000","last_fullsized":"2023-10-08T13:51:15.787998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:56.092404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":112,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:23.854771+0000","last_change":"2023-10-08T13:49:43.782863+0000","last_active":"2023-10-08T13:51:23.854771+0000","last_peered":"2023-10-08T13:51:23.854771+0000","last_clean":"2023-10-08T13:51:23.854771+0000","last_became_active":"2023-10-08T13:49:43.782477+0000","last_became_peered":"2023-10-08T13:49:43.782477+0000","last_unstale":"2023-10-08T13:51:23.854771+0000","last_undegraded":"2023-10-08T13:51:23.854771+0000","last_fullsized":"2023-10-08T13:51:23.854771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:10.134907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:16.876324+0000","last_change":"2023-10-08T13:51:16.875898+0000","last_active":"2023-10-08T13:51:16.876324+0000","last_peered":"2023-10-08T13:51:16.876324+0000","last_clean":"2023-10-08T13:51:16.876324+0000","last_became_active":"2023-10-08T13:51:16.875581+0000","last_became_peered":"2023-10-08T13:51:16.875581+0000","last_unstale":"2023-10-08T13:51:16.876324+0000","last_undegraded":"2023-10-08T13:51:16.876324+0000","last_fullsized":"2023-10-08T13:51:16.876324+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:27.797425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782603+0000","last_change":"2023-10-08T13:49:49.164248+0000","last_active":"2023-10-08T13:51:15.782603+0000","last_peered":"2023-10-08T13:51:15.782603+0000","last_clean":"2023-10-08T13:51:15.782603+0000","last_became_active":"2023-10-08T13:49:49.163624+0000","last_became_peered":"2023-10-08T13:49:49.163624+0000","last_unstale":"2023-10-08T13:51:15.782603+0000","last_undegraded":"2023-10-08T13:51:15.782603+0000","last_fullsized":"2023-10-08T13:51:15.782603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:04.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788426+0000","last_change":"2023-10-08T13:49:43.788026+0000","last_active":"2023-10-08T13:51:15.788426+0000","last_peered":"2023-10-08T13:51:15.788426+0000","last_clean":"2023-10-08T13:51:15.788426+0000","last_became_active":"2023-10-08T13:49:43.787639+0000","last_became_peered":"2023-10-08T13:49:43.787639+0000","last_unstale":"2023-10-08T13:51:15.788426+0000","last_undegraded":"2023-10-08T13:51:15.788426+0000","last_fullsized":"2023-10-08T13:51:15.788426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:25:48.367765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808934+0000","last_change":"2023-10-08T13:49:44.786322+0000","last_active":"2023-10-08T13:51:15.808934+0000","last_peered":"2023-10-08T13:51:15.808934+0000","last_clean":"2023-10-08T13:51:15.808934+0000","last_became_active":"2023-10-08T13:49:44.785953+0000","last_became_peered":"2023-10-08T13:49:44.785953+0000","last_unstale":"2023-10-08T13:51:15.808934+0000","last_undegraded":"2023-10-08T13:51:15.808934+0000","last_fullsized":"2023-10-08T13:51:15.808934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:39.866785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782827+0000","last_change":"2023-10-08T13:50:15.976961+0000","last_active":"2023-10-08T13:51:15.782827+0000","last_peered":"2023-10-08T13:51:15.782827+0000","last_clean":"2023-10-08T13:51:15.782827+0000","last_became_active":"2023-10-08T13:50:15.976492+0000","last_became_peered":"2023-10-08T13:50:15.976492+0000","last_unstale":"2023-10-08T13:51:15.782827+0000","last_undegraded":"2023-10-08T13:51:15.782827+0000","last_fullsized":"2023-10-08T13:51:15.782827+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:05:48.947259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783518+0000","last_change":"2023-10-08T13:49:43.776239+0000","last_active":"2023-10-08T13:51:15.783518+0000","last_peered":"2023-10-08T13:51:15.783518+0000","last_clean":"2023-10-08T13:51:15.783518+0000","last_became_active":"2023-10-08T13:49:43.775969+0000","last_became_peered":"2023-10-08T13:49:43.775969+0000","last_unstale":"2023-10-08T13:51:15.783518+0000","last_undegraded":"2023-10-08T13:51:15.783518+0000","last_fullsized":"2023-10-08T13:51:15.783518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:24.011655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786966+0000","last_change":"2023-10-08T13:49:44.794912+0000","last_active":"2023-10-08T13:51:15.786966+0000","last_peered":"2023-10-08T13:51:15.786966+0000","last_clean":"2023-10-08T13:51:15.786966+0000","last_became_active":"2023-10-08T13:49:44.794590+0000","last_became_peered":"2023-10-08T13:49:44.794590+0000","last_unstale":"2023-10-08T13:51:15.786966+0000","last_undegraded":"2023-10-08T13:51:15.786966+0000","last_fullsized":"2023-10-08T13:51:15.786966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:51.631386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782715+0000","last_change":"2023-10-08T13:49:49.152149+0000","last_active":"2023-10-08T13:51:15.782715+0000","last_peered":"2023-10-08T13:51:15.782715+0000","last_clean":"2023-10-08T13:51:15.782715+0000","last_became_active":"2023-10-08T13:49:49.151764+0000","last_became_peered":"2023-10-08T13:49:49.151764+0000","last_unstale":"2023-10-08T13:51:15.782715+0000","last_undegraded":"2023-10-08T13:51:15.782715+0000","last_fullsized":"2023-10-08T13:51:15.782715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:25:08.232556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808473+0000","last_change":"2023-10-08T13:50:15.800467+0000","last_active":"2023-10-08T13:51:15.808473+0000","last_peered":"2023-10-08T13:51:15.808473+0000","last_clean":"2023-10-08T13:51:15.808473+0000","last_became_active":"2023-10-08T13:50:15.788865+0000","last_became_peered":"2023-10-08T13:50:15.788865+0000","last_unstale":"2023-10-08T13:51:15.808473+0000","last_undegraded":"2023-10-08T13:51:15.808473+0000","last_fullsized":"2023-10-08T13:51:15.808473+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:11:01.887072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785234+0000","last_change":"2023-10-08T13:49:44.783915+0000","last_active":"2023-10-08T13:51:15.785234+0000","last_peered":"2023-10-08T13:51:15.785234+0000","last_clean":"2023-10-08T13:51:15.785234+0000","last_became_active":"2023-10-08T13:49:44.783628+0000","last_became_peered":"2023-10-08T13:49:44.783628+0000","last_unstale":"2023-10-08T13:51:15.785234+0000","last_undegraded":"2023-10-08T13:51:15.785234+0000","last_fullsized":"2023-10-08T13:51:15.785234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:43.759681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788199+0000","last_change":"2023-10-08T13:49:49.156241+0000","last_active":"2023-10-08T13:51:15.788199+0000","last_peered":"2023-10-08T13:51:15.788199+0000","last_clean":"2023-10-08T13:51:15.788199+0000","last_became_active":"2023-10-08T13:49:49.155652+0000","last_became_peered":"2023-10-08T13:49:49.155652+0000","last_unstale":"2023-10-08T13:51:15.788199+0000","last_undegraded":"2023-10-08T13:51:15.788199+0000","last_fullsized":"2023-10-08T13:51:15.788199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":119,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:23.885147+0000","last_change":"2023-10-08T13:50:15.795816+0000","last_active":"2023-10-08T13:51:23.885147+0000","last_peered":"2023-10-08T13:51:23.885147+0000","last_clean":"2023-10-08T13:51:23.885147+0000","last_became_active":"2023-10-08T13:50:15.789936+0000","last_became_peered":"2023-10-08T13:50:15.789936+0000","last_unstale":"2023-10-08T13:51:23.885147+0000","last_undegraded":"2023-10-08T13:51:23.885147+0000","last_fullsized":"2023-10-08T13:51:23.885147+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:21.496490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783042+0000","last_change":"2023-10-08T13:49:44.780935+0000","last_active":"2023-10-08T13:51:15.783042+0000","last_peered":"2023-10-08T13:51:15.783042+0000","last_clean":"2023-10-08T13:51:15.783042+0000","last_became_active":"2023-10-08T13:49:44.780613+0000","last_became_peered":"2023-10-08T13:49:44.780613+0000","last_unstale":"2023-10-08T13:51:15.783042+0000","last_undegraded":"2023-10-08T13:51:15.783042+0000","last_fullsized":"2023-10-08T13:51:15.783042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:57.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788141+0000","last_change":"2023-10-08T13:49:49.163652+0000","last_active":"2023-10-08T13:51:15.788141+0000","last_peered":"2023-10-08T13:51:15.788141+0000","last_clean":"2023-10-08T13:51:15.788141+0000","last_became_active":"2023-10-08T13:49:49.163140+0000","last_became_peered":"2023-10-08T13:49:49.163140+0000","last_unstale":"2023-10-08T13:51:15.788141+0000","last_undegraded":"2023-10-08T13:51:15.788141+0000","last_fullsized":"2023-10-08T13:51:15.788141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:48.733675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809171+0000","last_change":"2023-10-08T13:49:43.775799+0000","last_active":"2023-10-08T13:51:15.809171+0000","last_peered":"2023-10-08T13:51:15.809171+0000","last_clean":"2023-10-08T13:51:15.809171+0000","last_became_active":"2023-10-08T13:49:43.775470+0000","last_became_peered":"2023-10-08T13:49:43.775470+0000","last_unstale":"2023-10-08T13:51:15.809171+0000","last_undegraded":"2023-10-08T13:51:15.809171+0000","last_fullsized":"2023-10-08T13:51:15.809171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:50.404554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784980+0000","last_change":"2023-10-08T13:50:15.785798+0000","last_active":"2023-10-08T13:51:15.784980+0000","last_peered":"2023-10-08T13:51:15.784980+0000","last_clean":"2023-10-08T13:51:15.784980+0000","last_became_active":"2023-10-08T13:50:15.785343+0000","last_became_peered":"2023-10-08T13:50:15.785343+0000","last_unstale":"2023-10-08T13:51:15.784980+0000","last_undegraded":"2023-10-08T13:51:15.784980+0000","last_fullsized":"2023-10-08T13:51:15.784980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:36.717707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808771+0000","last_change":"2023-10-08T13:49:49.166702+0000","last_active":"2023-10-08T13:51:15.808771+0000","last_peered":"2023-10-08T13:51:15.808771+0000","last_clean":"2023-10-08T13:51:15.808771+0000","last_became_active":"2023-10-08T13:49:49.166256+0000","last_became_peered":"2023-10-08T13:49:49.166256+0000","last_unstale":"2023-10-08T13:51:15.808771+0000","last_undegraded":"2023-10-08T13:51:15.808771+0000","last_fullsized":"2023-10-08T13:51:15.808771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:18:57.565040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808332+0000","last_change":"2023-10-08T13:49:43.785085+0000","last_active":"2023-10-08T13:51:15.808332+0000","last_peered":"2023-10-08T13:51:15.808332+0000","last_clean":"2023-10-08T13:51:15.808332+0000","last_became_active":"2023-10-08T13:49:43.784746+0000","last_became_peered":"2023-10-08T13:49:43.784746+0000","last_unstale":"2023-10-08T13:51:15.808332+0000","last_undegraded":"2023-10-08T13:51:15.808332+0000","last_fullsized":"2023-10-08T13:51:15.808332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:13.471834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787752+0000","last_change":"2023-10-08T13:49:44.785670+0000","last_active":"2023-10-08T13:51:15.787752+0000","last_peered":"2023-10-08T13:51:15.787752+0000","last_clean":"2023-10-08T13:51:15.787752+0000","last_became_active":"2023-10-08T13:49:44.785221+0000","last_became_peered":"2023-10-08T13:49:44.785221+0000","last_unstale":"2023-10-08T13:51:15.787752+0000","last_undegraded":"2023-10-08T13:51:15.787752+0000","last_fullsized":"2023-10-08T13:51:15.787752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:25.415168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808870+0000","last_change":"2023-10-08T13:49:49.156294+0000","last_active":"2023-10-08T13:51:15.808870+0000","last_peered":"2023-10-08T13:51:15.808870+0000","last_clean":"2023-10-08T13:51:15.808870+0000","last_became_active":"2023-10-08T13:49:49.155821+0000","last_became_peered":"2023-10-08T13:49:49.155821+0000","last_unstale":"2023-10-08T13:51:15.808870+0000","last_undegraded":"2023-10-08T13:51:15.808870+0000","last_fullsized":"2023-10-08T13:51:15.808870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:59:43.831634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784681+0000","last_change":"2023-10-08T13:49:43.780052+0000","last_active":"2023-10-08T13:51:15.784681+0000","last_peered":"2023-10-08T13:51:15.784681+0000","last_clean":"2023-10-08T13:51:15.784681+0000","last_became_active":"2023-10-08T13:49:43.779731+0000","last_became_peered":"2023-10-08T13:49:43.779731+0000","last_unstale":"2023-10-08T13:51:15.784681+0000","last_undegraded":"2023-10-08T13:51:15.784681+0000","last_fullsized":"2023-10-08T13:51:15.784681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:38.800783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785843+0000","last_change":"2023-10-08T13:49:44.787880+0000","last_active":"2023-10-08T13:51:15.785843+0000","last_peered":"2023-10-08T13:51:15.785843+0000","last_clean":"2023-10-08T13:51:15.785843+0000","last_became_active":"2023-10-08T13:49:44.787591+0000","last_became_peered":"2023-10-08T13:49:44.787591+0000","last_unstale":"2023-10-08T13:51:15.785843+0000","last_undegraded":"2023-10-08T13:51:15.785843+0000","last_fullsized":"2023-10-08T13:51:15.785843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:53.078903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788057+0000","last_change":"2023-10-08T13:49:49.155004+0000","last_active":"2023-10-08T13:51:15.788057+0000","last_peered":"2023-10-08T13:51:15.788057+0000","last_clean":"2023-10-08T13:51:15.788057+0000","last_became_active":"2023-10-08T13:49:49.154503+0000","last_became_peered":"2023-10-08T13:49:49.154503+0000","last_unstale":"2023-10-08T13:51:15.788057+0000","last_undegraded":"2023-10-08T13:51:15.788057+0000","last_fullsized":"2023-10-08T13:51:15.788057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:21.565816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":174,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.851308+0000","last_change":"2023-10-08T13:49:43.775819+0000","last_active":"2023-10-08T13:51:21.851308+0000","last_peered":"2023-10-08T13:51:21.851308+0000","last_clean":"2023-10-08T13:51:21.851308+0000","last_became_active":"2023-10-08T13:49:43.775578+0000","last_became_peered":"2023-10-08T13:49:43.775578+0000","last_unstale":"2023-10-08T13:51:21.851308+0000","last_undegraded":"2023-10-08T13:51:21.851308+0000","last_fullsized":"2023-10-08T13:51:21.851308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:31.518191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787270+0000","last_change":"2023-10-08T13:49:44.785869+0000","last_active":"2023-10-08T13:51:15.787270+0000","last_peered":"2023-10-08T13:51:15.787270+0000","last_clean":"2023-10-08T13:51:15.787270+0000","last_became_active":"2023-10-08T13:49:44.785566+0000","last_became_peered":"2023-10-08T13:49:44.785566+0000","last_unstale":"2023-10-08T13:51:15.787270+0000","last_undegraded":"2023-10-08T13:51:15.787270+0000","last_fullsized":"2023-10-08T13:51:15.787270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:29.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785538+0000","last_change":"2023-10-08T13:49:49.162075+0000","last_active":"2023-10-08T13:51:15.785538+0000","last_peered":"2023-10-08T13:51:15.785538+0000","last_clean":"2023-10-08T13:51:15.785538+0000","last_became_active":"2023-10-08T13:49:49.161485+0000","last_became_peered":"2023-10-08T13:49:49.161485+0000","last_unstale":"2023-10-08T13:51:15.785538+0000","last_undegraded":"2023-10-08T13:51:15.785538+0000","last_fullsized":"2023-10-08T13:51:15.785538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:16.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808954+0000","last_change":"2023-10-08T13:49:43.776555+0000","last_active":"2023-10-08T13:51:15.808954+0000","last_peered":"2023-10-08T13:51:15.808954+0000","last_clean":"2023-10-08T13:51:15.808954+0000","last_became_active":"2023-10-08T13:49:43.776259+0000","last_became_peered":"2023-10-08T13:49:43.776259+0000","last_unstale":"2023-10-08T13:51:15.808954+0000","last_undegraded":"2023-10-08T13:51:15.808954+0000","last_fullsized":"2023-10-08T13:51:15.808954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:34.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809044+0000","last_change":"2023-10-08T13:49:44.779253+0000","last_active":"2023-10-08T13:51:15.809044+0000","last_peered":"2023-10-08T13:51:15.809044+0000","last_clean":"2023-10-08T13:51:15.809044+0000","last_became_active":"2023-10-08T13:49:44.778902+0000","last_became_peered":"2023-10-08T13:49:44.778902+0000","last_unstale":"2023-10-08T13:51:15.809044+0000","last_undegraded":"2023-10-08T13:51:15.809044+0000","last_fullsized":"2023-10-08T13:51:15.809044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:31:01.091424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"88'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782932+0000","last_change":"2023-10-08T13:49:49.164859+0000","last_active":"2023-10-08T13:51:15.782932+0000","last_peered":"2023-10-08T13:51:15.782932+0000","last_clean":"2023-10-08T13:51:15.782932+0000","last_became_active":"2023-10-08T13:49:49.164221+0000","last_became_peered":"2023-10-08T13:49:49.164221+0000","last_unstale":"2023-10-08T13:51:15.782932+0000","last_undegraded":"2023-10-08T13:51:15.782932+0000","last_fullsized":"2023-10-08T13:51:15.782932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:22.484934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785106+0000","last_change":"2023-10-08T13:49:43.788136+0000","last_active":"2023-10-08T13:51:15.785106+0000","last_peered":"2023-10-08T13:51:15.785106+0000","last_clean":"2023-10-08T13:51:15.785106+0000","last_became_active":"2023-10-08T13:49:43.787910+0000","last_became_peered":"2023-10-08T13:49:43.787910+0000","last_unstale":"2023-10-08T13:51:15.785106+0000","last_undegraded":"2023-10-08T13:51:15.785106+0000","last_fullsized":"2023-10-08T13:51:15.785106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:00:32.897972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787049+0000","last_change":"2023-10-08T13:49:44.786833+0000","last_active":"2023-10-08T13:51:15.787049+0000","last_peered":"2023-10-08T13:51:15.787049+0000","last_clean":"2023-10-08T13:51:15.787049+0000","last_became_active":"2023-10-08T13:49:44.786523+0000","last_became_peered":"2023-10-08T13:49:44.786523+0000","last_unstale":"2023-10-08T13:51:15.787049+0000","last_undegraded":"2023-10-08T13:51:15.787049+0000","last_fullsized":"2023-10-08T13:51:15.787049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:36:22.156252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785774+0000","last_change":"2023-10-08T13:50:15.977115+0000","last_active":"2023-10-08T13:51:15.785774+0000","last_peered":"2023-10-08T13:51:15.785774+0000","last_clean":"2023-10-08T13:51:15.785774+0000","last_became_active":"2023-10-08T13:50:15.976617+0000","last_became_peered":"2023-10-08T13:50:15.976617+0000","last_unstale":"2023-10-08T13:51:15.785774+0000","last_undegraded":"2023-10-08T13:51:15.785774+0000","last_fullsized":"2023-10-08T13:51:15.785774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:57.124054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'36","reported_seq":214,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.882495+0000","last_change":"2023-10-08T13:49:43.777736+0000","last_active":"2023-10-08T13:51:21.882495+0000","last_peered":"2023-10-08T13:51:21.882495+0000","last_clean":"2023-10-08T13:51:21.882495+0000","last_became_active":"2023-10-08T13:49:43.777431+0000","last_became_peered":"2023-10-08T13:49:43.777431+0000","last_unstale":"2023-10-08T13:51:21.882495+0000","last_undegraded":"2023-10-08T13:51:21.882495+0000","last_fullsized":"2023-10-08T13:51:21.882495+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:12.907196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59058,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":158,"num_read_kb":136,"num_write":38,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783122+0000","last_change":"2023-10-08T13:49:44.789122+0000","last_active":"2023-10-08T13:51:15.783122+0000","last_peered":"2023-10-08T13:51:15.783122+0000","last_clean":"2023-10-08T13:51:15.783122+0000","last_became_active":"2023-10-08T13:49:44.788831+0000","last_became_peered":"2023-10-08T13:49:44.788831+0000","last_unstale":"2023-10-08T13:51:15.783122+0000","last_undegraded":"2023-10-08T13:51:15.783122+0000","last_fullsized":"2023-10-08T13:51:15.783122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:45.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785476+0000","last_change":"2023-10-08T13:49:49.157319+0000","last_active":"2023-10-08T13:51:15.785476+0000","last_peered":"2023-10-08T13:51:15.785476+0000","last_clean":"2023-10-08T13:51:15.785476+0000","last_became_active":"2023-10-08T13:49:49.156897+0000","last_became_peered":"2023-10-08T13:49:49.156897+0000","last_unstale":"2023-10-08T13:51:15.785476+0000","last_undegraded":"2023-10-08T13:51:15.785476+0000","last_fullsized":"2023-10-08T13:51:15.785476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:25.460707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787914+0000","last_change":"2023-10-08T13:49:43.776699+0000","last_active":"2023-10-08T13:51:15.787914+0000","last_peered":"2023-10-08T13:51:15.787914+0000","last_clean":"2023-10-08T13:51:15.787914+0000","last_became_active":"2023-10-08T13:49:43.776293+0000","last_became_peered":"2023-10-08T13:49:43.776293+0000","last_unstale":"2023-10-08T13:51:15.787914+0000","last_undegraded":"2023-10-08T13:51:15.787914+0000","last_fullsized":"2023-10-08T13:51:15.787914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:19.056804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808734+0000","last_change":"2023-10-08T13:49:44.789915+0000","last_active":"2023-10-08T13:51:15.808734+0000","last_peered":"2023-10-08T13:51:15.808734+0000","last_clean":"2023-10-08T13:51:15.808734+0000","last_became_active":"2023-10-08T13:49:44.789617+0000","last_became_peered":"2023-10-08T13:49:44.789617+0000","last_unstale":"2023-10-08T13:51:15.808734+0000","last_undegraded":"2023-10-08T13:51:15.808734+0000","last_fullsized":"2023-10-08T13:51:15.808734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:06.286155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808527+0000","last_change":"2023-10-08T13:49:49.165946+0000","last_active":"2023-10-08T13:51:15.808527+0000","last_peered":"2023-10-08T13:51:15.808527+0000","last_clean":"2023-10-08T13:51:15.808527+0000","last_became_active":"2023-10-08T13:49:49.165581+0000","last_became_peered":"2023-10-08T13:49:49.165581+0000","last_unstale":"2023-10-08T13:51:15.808527+0000","last_undegraded":"2023-10-08T13:51:15.808527+0000","last_fullsized":"2023-10-08T13:51:15.808527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:13.772198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784836+0000","last_change":"2023-10-08T13:49:43.779161+0000","last_active":"2023-10-08T13:51:15.784836+0000","last_peered":"2023-10-08T13:51:15.784836+0000","last_clean":"2023-10-08T13:51:15.784836+0000","last_became_active":"2023-10-08T13:49:43.778862+0000","last_became_peered":"2023-10-08T13:49:43.778862+0000","last_unstale":"2023-10-08T13:51:15.784836+0000","last_undegraded":"2023-10-08T13:51:15.784836+0000","last_fullsized":"2023-10-08T13:51:15.784836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:57:44.075322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809115+0000","last_change":"2023-10-08T13:49:44.782072+0000","last_active":"2023-10-08T13:51:15.809115+0000","last_peered":"2023-10-08T13:51:15.809115+0000","last_clean":"2023-10-08T13:51:15.809115+0000","last_became_active":"2023-10-08T13:49:44.781756+0000","last_became_peered":"2023-10-08T13:49:44.781756+0000","last_unstale":"2023-10-08T13:51:15.809115+0000","last_undegraded":"2023-10-08T13:51:15.809115+0000","last_fullsized":"2023-10-08T13:51:15.809115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:32.877818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783652+0000","last_change":"2023-10-08T13:50:16.099755+0000","last_active":"2023-10-08T13:51:15.783652+0000","last_peered":"2023-10-08T13:51:15.783652+0000","last_clean":"2023-10-08T13:51:15.783652+0000","last_became_active":"2023-10-08T13:50:16.099267+0000","last_became_peered":"2023-10-08T13:50:16.099267+0000","last_unstale":"2023-10-08T13:51:15.783652+0000","last_undegraded":"2023-10-08T13:51:15.783652+0000","last_fullsized":"2023-10-08T13:51:15.783652+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:19.949246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.860207+0000","last_change":"2023-10-08T13:49:43.782822+0000","last_active":"2023-10-08T13:51:21.860207+0000","last_peered":"2023-10-08T13:51:21.860207+0000","last_clean":"2023-10-08T13:51:21.860207+0000","last_became_active":"2023-10-08T13:49:43.782524+0000","last_became_peered":"2023-10-08T13:49:43.782524+0000","last_unstale":"2023-10-08T13:51:21.860207+0000","last_undegraded":"2023-10-08T13:51:21.860207+0000","last_fullsized":"2023-10-08T13:51:21.860207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:11.847851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808786+0000","last_change":"2023-10-08T13:49:44.794296+0000","last_active":"2023-10-08T13:51:15.808786+0000","last_peered":"2023-10-08T13:51:15.808786+0000","last_clean":"2023-10-08T13:51:15.808786+0000","last_became_active":"2023-10-08T13:49:44.793987+0000","last_became_peered":"2023-10-08T13:49:44.793987+0000","last_unstale":"2023-10-08T13:51:15.808786+0000","last_undegraded":"2023-10-08T13:51:15.808786+0000","last_fullsized":"2023-10-08T13:51:15.808786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:41.562569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809550+0000","last_change":"2023-10-08T13:49:49.165016+0000","last_active":"2023-10-08T13:51:15.809550+0000","last_peered":"2023-10-08T13:51:15.809550+0000","last_clean":"2023-10-08T13:51:15.809550+0000","last_became_active":"2023-10-08T13:49:49.164531+0000","last_became_peered":"2023-10-08T13:49:49.164531+0000","last_unstale":"2023-10-08T13:51:15.809550+0000","last_undegraded":"2023-10-08T13:51:15.809550+0000","last_fullsized":"2023-10-08T13:51:15.809550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:45.014512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782774+0000","last_change":"2023-10-08T13:49:43.786929+0000","last_active":"2023-10-08T13:51:15.782774+0000","last_peered":"2023-10-08T13:51:15.782774+0000","last_clean":"2023-10-08T13:51:15.782774+0000","last_became_active":"2023-10-08T13:49:43.786641+0000","last_became_peered":"2023-10-08T13:49:43.786641+0000","last_unstale":"2023-10-08T13:51:15.782774+0000","last_undegraded":"2023-10-08T13:51:15.782774+0000","last_fullsized":"2023-10-08T13:51:15.782774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:30.054067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:16.851434+0000","last_change":"2023-10-08T13:51:16.850958+0000","last_active":"2023-10-08T13:51:16.851434+0000","last_peered":"2023-10-08T13:51:16.851434+0000","last_clean":"2023-10-08T13:51:16.851434+0000","last_became_active":"2023-10-08T13:51:16.850648+0000","last_became_peered":"2023-10-08T13:51:16.850648+0000","last_unstale":"2023-10-08T13:51:16.851434+0000","last_undegraded":"2023-10-08T13:51:16.851434+0000","last_fullsized":"2023-10-08T13:51:16.851434+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:30.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784760+0000","last_change":"2023-10-08T13:49:49.152226+0000","last_active":"2023-10-08T13:51:15.784760+0000","last_peered":"2023-10-08T13:51:15.784760+0000","last_clean":"2023-10-08T13:51:15.784760+0000","last_became_active":"2023-10-08T13:49:49.151772+0000","last_became_peered":"2023-10-08T13:49:49.151772+0000","last_unstale":"2023-10-08T13:51:15.784760+0000","last_undegraded":"2023-10-08T13:51:15.784760+0000","last_fullsized":"2023-10-08T13:51:15.784760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:53.856496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808693+0000","last_change":"2023-10-08T13:50:15.793598+0000","last_active":"2023-10-08T13:51:15.808693+0000","last_peered":"2023-10-08T13:51:15.808693+0000","last_clean":"2023-10-08T13:51:15.808693+0000","last_became_active":"2023-10-08T13:50:15.786421+0000","last_became_peered":"2023-10-08T13:50:15.786421+0000","last_unstale":"2023-10-08T13:51:15.808693+0000","last_undegraded":"2023-10-08T13:51:15.808693+0000","last_fullsized":"2023-10-08T13:51:15.808693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:09:37.586245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808647+0000","last_change":"2023-10-08T13:49:44.782805+0000","last_active":"2023-10-08T13:51:15.808647+0000","last_peered":"2023-10-08T13:51:15.808647+0000","last_clean":"2023-10-08T13:51:15.808647+0000","last_became_active":"2023-10-08T13:49:44.782485+0000","last_became_peered":"2023-10-08T13:49:44.782485+0000","last_unstale":"2023-10-08T13:51:15.808647+0000","last_undegraded":"2023-10-08T13:51:15.808647+0000","last_fullsized":"2023-10-08T13:51:15.808647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:19.805734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783005+0000","last_change":"2023-10-08T13:49:49.546226+0000","last_active":"2023-10-08T13:51:15.783005+0000","last_peered":"2023-10-08T13:51:15.783005+0000","last_clean":"2023-10-08T13:51:15.783005+0000","last_became_active":"2023-10-08T13:49:49.545626+0000","last_became_peered":"2023-10-08T13:49:49.545626+0000","last_unstale":"2023-10-08T13:51:15.783005+0000","last_undegraded":"2023-10-08T13:51:15.783005+0000","last_fullsized":"2023-10-08T13:51:15.783005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:37.184944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786007+0000","last_change":"2023-10-08T13:49:43.784705+0000","last_active":"2023-10-08T13:51:15.786007+0000","last_peered":"2023-10-08T13:51:15.786007+0000","last_clean":"2023-10-08T13:51:15.786007+0000","last_became_active":"2023-10-08T13:49:43.784391+0000","last_became_peered":"2023-10-08T13:49:43.784391+0000","last_unstale":"2023-10-08T13:51:15.786007+0000","last_undegraded":"2023-10-08T13:51:15.786007+0000","last_fullsized":"2023-10-08T13:51:15.786007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:45.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.810046+0000","last_change":"2023-10-08T13:49:44.789845+0000","last_active":"2023-10-08T13:51:15.810046+0000","last_peered":"2023-10-08T13:51:15.810046+0000","last_clean":"2023-10-08T13:51:15.810046+0000","last_became_active":"2023-10-08T13:49:44.789580+0000","last_became_peered":"2023-10-08T13:49:44.789580+0000","last_unstale":"2023-10-08T13:51:15.810046+0000","last_undegraded":"2023-10-08T13:51:15.810046+0000","last_fullsized":"2023-10-08T13:51:15.810046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:58.098757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809069+0000","last_change":"2023-10-08T13:49:49.153277+0000","last_active":"2023-10-08T13:51:15.809069+0000","last_peered":"2023-10-08T13:51:15.809069+0000","last_clean":"2023-10-08T13:51:15.809069+0000","last_became_active":"2023-10-08T13:49:49.152575+0000","last_became_peered":"2023-10-08T13:49:49.152575+0000","last_unstale":"2023-10-08T13:51:15.809069+0000","last_undegraded":"2023-10-08T13:51:15.809069+0000","last_fullsized":"2023-10-08T13:51:15.809069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:35.677823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.852851+0000","last_change":"2023-10-08T13:49:43.778442+0000","last_active":"2023-10-08T13:51:21.852851+0000","last_peered":"2023-10-08T13:51:21.852851+0000","last_clean":"2023-10-08T13:51:21.852851+0000","last_became_active":"2023-10-08T13:49:43.774799+0000","last_became_peered":"2023-10-08T13:49:43.774799+0000","last_unstale":"2023-10-08T13:51:21.852851+0000","last_undegraded":"2023-10-08T13:51:21.852851+0000","last_fullsized":"2023-10-08T13:51:21.852851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:15.464736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809560+0000","last_change":"2023-10-08T13:49:44.780097+0000","last_active":"2023-10-08T13:51:15.809560+0000","last_peered":"2023-10-08T13:51:15.809560+0000","last_clean":"2023-10-08T13:51:15.809560+0000","last_became_active":"2023-10-08T13:49:44.779774+0000","last_became_peered":"2023-10-08T13:49:44.779774+0000","last_unstale":"2023-10-08T13:51:15.809560+0000","last_undegraded":"2023-10-08T13:51:15.809560+0000","last_fullsized":"2023-10-08T13:51:15.809560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:19.132295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808399+0000","last_change":"2023-10-08T13:49:49.547129+0000","last_active":"2023-10-08T13:51:15.808399+0000","last_peered":"2023-10-08T13:51:15.808399+0000","last_clean":"2023-10-08T13:51:15.808399+0000","last_became_active":"2023-10-08T13:49:49.546694+0000","last_became_peered":"2023-10-08T13:49:49.546694+0000","last_unstale":"2023-10-08T13:51:15.808399+0000","last_undegraded":"2023-10-08T13:51:15.808399+0000","last_fullsized":"2023-10-08T13:51:15.808399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:45.524650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784878+0000","last_change":"2023-10-08T13:49:44.779875+0000","last_active":"2023-10-08T13:51:15.784878+0000","last_peered":"2023-10-08T13:51:15.784878+0000","last_clean":"2023-10-08T13:51:15.784878+0000","last_became_active":"2023-10-08T13:49:44.779570+0000","last_became_peered":"2023-10-08T13:49:44.779570+0000","last_unstale":"2023-10-08T13:51:15.784878+0000","last_undegraded":"2023-10-08T13:51:15.784878+0000","last_fullsized":"2023-10-08T13:51:15.784878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:02.981998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:23.862244+0000","last_change":"2023-10-08T13:49:43.787574+0000","last_active":"2023-10-08T13:51:23.862244+0000","last_peered":"2023-10-08T13:51:23.862244+0000","last_clean":"2023-10-08T13:51:23.862244+0000","last_became_active":"2023-10-08T13:49:43.787261+0000","last_became_peered":"2023-10-08T13:49:43.787261+0000","last_unstale":"2023-10-08T13:51:23.862244+0000","last_undegraded":"2023-10-08T13:51:23.862244+0000","last_fullsized":"2023-10-08T13:51:23.862244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:28.676135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809431+0000","last_change":"2023-10-08T13:49:49.163856+0000","last_active":"2023-10-08T13:51:15.809431+0000","last_peered":"2023-10-08T13:51:15.809431+0000","last_clean":"2023-10-08T13:51:15.809431+0000","last_became_active":"2023-10-08T13:49:49.163456+0000","last_became_peered":"2023-10-08T13:49:49.163456+0000","last_unstale":"2023-10-08T13:51:15.809431+0000","last_undegraded":"2023-10-08T13:51:15.809431+0000","last_fullsized":"2023-10-08T13:51:15.809431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:31.654309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788320+0000","last_change":"2023-10-08T13:49:44.792935+0000","last_active":"2023-10-08T13:51:15.788320+0000","last_peered":"2023-10-08T13:51:15.788320+0000","last_clean":"2023-10-08T13:51:15.788320+0000","last_became_active":"2023-10-08T13:49:44.792667+0000","last_became_peered":"2023-10-08T13:49:44.792667+0000","last_unstale":"2023-10-08T13:51:15.788320+0000","last_undegraded":"2023-10-08T13:51:15.788320+0000","last_fullsized":"2023-10-08T13:51:15.788320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:40.964452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783131+0000","last_change":"2023-10-08T13:49:43.781424+0000","last_active":"2023-10-08T13:51:15.783131+0000","last_peered":"2023-10-08T13:51:15.783131+0000","last_clean":"2023-10-08T13:51:15.783131+0000","last_became_active":"2023-10-08T13:49:43.780930+0000","last_became_peered":"2023-10-08T13:49:43.780930+0000","last_unstale":"2023-10-08T13:51:15.783131+0000","last_undegraded":"2023-10-08T13:51:15.783131+0000","last_fullsized":"2023-10-08T13:51:15.783131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:56:50.397289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808562+0000","last_change":"2023-10-08T13:48:45.192219+0000","last_active":"2023-10-08T13:51:15.808562+0000","last_peered":"2023-10-08T13:51:15.808562+0000","last_clean":"2023-10-08T13:51:15.808562+0000","last_became_active":"2023-10-08T13:48:45.191813+0000","last_became_peered":"2023-10-08T13:48:45.191813+0000","last_unstale":"2023-10-08T13:51:15.808562+0000","last_undegraded":"2023-10-08T13:51:15.808562+0000","last_fullsized":"2023-10-08T13:51:15.808562+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_clean_scrub_stamp":"2023-10-08T13:45:51.040695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.303937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785025+0000","last_change":"2023-10-08T13:49:49.546181+0000","last_active":"2023-10-08T13:51:15.785025+0000","last_peered":"2023-10-08T13:51:15.785025+0000","last_clean":"2023-10-08T13:51:15.785025+0000","last_became_active":"2023-10-08T13:49:49.545726+0000","last_became_peered":"2023-10-08T13:49:49.545726+0000","last_unstale":"2023-10-08T13:51:15.785025+0000","last_undegraded":"2023-10-08T13:51:15.785025+0000","last_fullsized":"2023-10-08T13:51:15.785025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:45.370909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809633+0000","last_change":"2023-10-08T13:49:44.785544+0000","last_active":"2023-10-08T13:51:15.809633+0000","last_peered":"2023-10-08T13:51:15.809633+0000","last_clean":"2023-10-08T13:51:15.809633+0000","last_became_active":"2023-10-08T13:49:44.785007+0000","last_became_peered":"2023-10-08T13:49:44.785007+0000","last_unstale":"2023-10-08T13:51:15.809633+0000","last_undegraded":"2023-10-08T13:51:15.809633+0000","last_fullsized":"2023-10-08T13:51:15.809633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:16:55.058693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808677+0000","last_change":"2023-10-08T13:49:43.779207+0000","last_active":"2023-10-08T13:51:15.808677+0000","last_peered":"2023-10-08T13:51:15.808677+0000","last_clean":"2023-10-08T13:51:15.808677+0000","last_became_active":"2023-10-08T13:49:43.778846+0000","last_became_peered":"2023-10-08T13:49:43.778846+0000","last_unstale":"2023-10-08T13:51:15.808677+0000","last_undegraded":"2023-10-08T13:51:15.808677+0000","last_fullsized":"2023-10-08T13:51:15.808677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:45.868187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808454+0000","last_change":"2023-10-08T13:49:49.162156+0000","last_active":"2023-10-08T13:51:15.808454+0000","last_peered":"2023-10-08T13:51:15.808454+0000","last_clean":"2023-10-08T13:51:15.808454+0000","last_became_active":"2023-10-08T13:49:49.161720+0000","last_became_peered":"2023-10-08T13:49:49.161720+0000","last_unstale":"2023-10-08T13:51:15.808454+0000","last_undegraded":"2023-10-08T13:51:15.808454+0000","last_fullsized":"2023-10-08T13:51:15.808454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:13.499915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783327+0000","last_change":"2023-10-08T13:49:43.782322+0000","last_active":"2023-10-08T13:51:15.783327+0000","last_peered":"2023-10-08T13:51:15.783327+0000","last_clean":"2023-10-08T13:51:15.783327+0000","last_became_active":"2023-10-08T13:49:43.781938+0000","last_became_peered":"2023-10-08T13:49:43.781938+0000","last_unstale":"2023-10-08T13:51:15.783327+0000","last_undegraded":"2023-10-08T13:51:15.783327+0000","last_fullsized":"2023-10-08T13:51:15.783327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:57.627507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809219+0000","last_change":"2023-10-08T13:49:44.778340+0000","last_active":"2023-10-08T13:51:15.809219+0000","last_peered":"2023-10-08T13:51:15.809219+0000","last_clean":"2023-10-08T13:51:15.809219+0000","last_became_active":"2023-10-08T13:49:44.778000+0000","last_became_peered":"2023-10-08T13:49:44.778000+0000","last_unstale":"2023-10-08T13:51:15.809219+0000","last_undegraded":"2023-10-08T13:51:15.809219+0000","last_fullsized":"2023-10-08T13:51:15.809219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:23.350449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787735+0000","last_change":"2023-10-08T13:49:49.545915+0000","last_active":"2023-10-08T13:51:15.787735+0000","last_peered":"2023-10-08T13:51:15.787735+0000","last_clean":"2023-10-08T13:51:15.787735+0000","last_became_active":"2023-10-08T13:49:49.545396+0000","last_became_peered":"2023-10-08T13:49:49.545396+0000","last_unstale":"2023-10-08T13:51:15.787735+0000","last_undegraded":"2023-10-08T13:51:15.787735+0000","last_fullsized":"2023-10-08T13:51:15.787735+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:07:03.850707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:16.798882+0000","last_change":"2023-10-08T13:49:43.776479+0000","last_active":"2023-10-08T13:51:16.798882+0000","last_peered":"2023-10-08T13:51:16.798882+0000","last_clean":"2023-10-08T13:51:16.798882+0000","last_became_active":"2023-10-08T13:49:43.776130+0000","last_became_peered":"2023-10-08T13:49:43.776130+0000","last_unstale":"2023-10-08T13:51:16.798882+0000","last_undegraded":"2023-10-08T13:51:16.798882+0000","last_fullsized":"2023-10-08T13:51:16.798882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.760660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788025+0000","last_change":"2023-10-08T13:49:44.793864+0000","last_active":"2023-10-08T13:51:15.788025+0000","last_peered":"2023-10-08T13:51:15.788025+0000","last_clean":"2023-10-08T13:51:15.788025+0000","last_became_active":"2023-10-08T13:49:44.793469+0000","last_became_peered":"2023-10-08T13:49:44.793469+0000","last_unstale":"2023-10-08T13:51:15.788025+0000","last_undegraded":"2023-10-08T13:51:15.788025+0000","last_fullsized":"2023-10-08T13:51:15.788025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:28.669711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787773+0000","last_change":"2023-10-08T13:49:49.150696+0000","last_active":"2023-10-08T13:51:15.787773+0000","last_peered":"2023-10-08T13:51:15.787773+0000","last_clean":"2023-10-08T13:51:15.787773+0000","last_became_active":"2023-10-08T13:49:49.150188+0000","last_became_peered":"2023-10-08T13:49:49.150188+0000","last_unstale":"2023-10-08T13:51:15.787773+0000","last_undegraded":"2023-10-08T13:51:15.787773+0000","last_fullsized":"2023-10-08T13:51:15.787773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:51:35.775240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783459+0000","last_change":"2023-10-08T13:49:43.786245+0000","last_active":"2023-10-08T13:51:15.783459+0000","last_peered":"2023-10-08T13:51:15.783459+0000","last_clean":"2023-10-08T13:51:15.783459+0000","last_became_active":"2023-10-08T13:49:43.785986+0000","last_became_peered":"2023-10-08T13:49:43.785986+0000","last_unstale":"2023-10-08T13:51:15.783459+0000","last_undegraded":"2023-10-08T13:51:15.783459+0000","last_fullsized":"2023-10-08T13:51:15.783459+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:53:59.643447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808953+0000","last_change":"2023-10-08T13:49:44.790494+0000","last_active":"2023-10-08T13:51:15.808953+0000","last_peered":"2023-10-08T13:51:15.808953+0000","last_clean":"2023-10-08T13:51:15.808953+0000","last_became_active":"2023-10-08T13:49:44.790091+0000","last_became_peered":"2023-10-08T13:49:44.790091+0000","last_unstale":"2023-10-08T13:51:15.808953+0000","last_undegraded":"2023-10-08T13:51:15.808953+0000","last_fullsized":"2023-10-08T13:51:15.808953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:49.309597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782982+0000","last_change":"2023-10-08T13:49:49.547271+0000","last_active":"2023-10-08T13:51:15.782982+0000","last_peered":"2023-10-08T13:51:15.782982+0000","last_clean":"2023-10-08T13:51:15.782982+0000","last_became_active":"2023-10-08T13:49:49.546603+0000","last_became_peered":"2023-10-08T13:49:49.546603+0000","last_unstale":"2023-10-08T13:51:15.782982+0000","last_undegraded":"2023-10-08T13:51:15.782982+0000","last_fullsized":"2023-10-08T13:51:15.782982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:29.908321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786664+0000","last_change":"2023-10-08T13:49:43.788638+0000","last_active":"2023-10-08T13:51:15.786664+0000","last_peered":"2023-10-08T13:51:15.786664+0000","last_clean":"2023-10-08T13:51:15.786664+0000","last_became_active":"2023-10-08T13:49:43.788303+0000","last_became_peered":"2023-10-08T13:49:43.788303+0000","last_unstale":"2023-10-08T13:51:15.786664+0000","last_undegraded":"2023-10-08T13:51:15.786664+0000","last_fullsized":"2023-10-08T13:51:15.786664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:52.637775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782787+0000","last_change":"2023-10-08T13:49:44.777546+0000","last_active":"2023-10-08T13:51:15.782787+0000","last_peered":"2023-10-08T13:51:15.782787+0000","last_clean":"2023-10-08T13:51:15.782787+0000","last_became_active":"2023-10-08T13:49:44.777261+0000","last_became_peered":"2023-10-08T13:49:44.777261+0000","last_unstale":"2023-10-08T13:51:15.782787+0000","last_undegraded":"2023-10-08T13:51:15.782787+0000","last_fullsized":"2023-10-08T13:51:15.782787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:26:36.368188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783728+0000","last_change":"2023-10-08T13:50:15.787317+0000","last_active":"2023-10-08T13:51:15.783728+0000","last_peered":"2023-10-08T13:51:15.783728+0000","last_clean":"2023-10-08T13:51:15.783728+0000","last_became_active":"2023-10-08T13:50:15.786816+0000","last_became_peered":"2023-10-08T13:50:15.786816+0000","last_unstale":"2023-10-08T13:51:15.783728+0000","last_undegraded":"2023-10-08T13:51:15.783728+0000","last_fullsized":"2023-10-08T13:51:15.783728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:53.766803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788167+0000","last_change":"2023-10-08T13:49:43.776693+0000","last_active":"2023-10-08T13:51:15.788167+0000","last_peered":"2023-10-08T13:51:15.788167+0000","last_clean":"2023-10-08T13:51:15.788167+0000","last_became_active":"2023-10-08T13:49:43.776367+0000","last_became_peered":"2023-10-08T13:49:43.776367+0000","last_unstale":"2023-10-08T13:51:15.788167+0000","last_undegraded":"2023-10-08T13:51:15.788167+0000","last_fullsized":"2023-10-08T13:51:15.788167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:17.415011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808348+0000","last_change":"2023-10-08T13:49:44.779472+0000","last_active":"2023-10-08T13:51:15.808348+0000","last_peered":"2023-10-08T13:51:15.808348+0000","last_clean":"2023-10-08T13:51:15.808348+0000","last_became_active":"2023-10-08T13:49:44.779172+0000","last_became_peered":"2023-10-08T13:49:44.779172+0000","last_unstale":"2023-10-08T13:51:15.808348+0000","last_undegraded":"2023-10-08T13:51:15.808348+0000","last_fullsized":"2023-10-08T13:51:15.808348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:17.314243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782946+0000","last_change":"2023-10-08T13:49:49.164535+0000","last_active":"2023-10-08T13:51:15.782946+0000","last_peered":"2023-10-08T13:51:15.782946+0000","last_clean":"2023-10-08T13:51:15.782946+0000","last_became_active":"2023-10-08T13:49:49.163467+0000","last_became_peered":"2023-10-08T13:49:49.163467+0000","last_unstale":"2023-10-08T13:51:15.782946+0000","last_undegraded":"2023-10-08T13:51:15.782946+0000","last_fullsized":"2023-10-08T13:51:15.782946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:55.058656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787896+0000","last_change":"2023-10-08T13:49:43.775311+0000","last_active":"2023-10-08T13:51:15.787896+0000","last_peered":"2023-10-08T13:51:15.787896+0000","last_clean":"2023-10-08T13:51:15.787896+0000","last_became_active":"2023-10-08T13:49:43.775005+0000","last_became_peered":"2023-10-08T13:49:43.775005+0000","last_unstale":"2023-10-08T13:51:15.787896+0000","last_undegraded":"2023-10-08T13:51:15.787896+0000","last_fullsized":"2023-10-08T13:51:15.787896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:33.713395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788118+0000","last_change":"2023-10-08T13:49:44.792051+0000","last_active":"2023-10-08T13:51:15.788118+0000","last_peered":"2023-10-08T13:51:15.788118+0000","last_clean":"2023-10-08T13:51:15.788118+0000","last_became_active":"2023-10-08T13:49:44.790045+0000","last_became_peered":"2023-10-08T13:49:44.790045+0000","last_unstale":"2023-10-08T13:51:15.788118+0000","last_undegraded":"2023-10-08T13:51:15.788118+0000","last_fullsized":"2023-10-08T13:51:15.788118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:28.699799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782910+0000","last_change":"2023-10-08T13:49:49.546219+0000","last_active":"2023-10-08T13:51:15.782910+0000","last_peered":"2023-10-08T13:51:15.782910+0000","last_clean":"2023-10-08T13:51:15.782910+0000","last_became_active":"2023-10-08T13:49:49.545783+0000","last_became_peered":"2023-10-08T13:49:49.545783+0000","last_unstale":"2023-10-08T13:51:15.782910+0000","last_undegraded":"2023-10-08T13:51:15.782910+0000","last_fullsized":"2023-10-08T13:51:15.782910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:00.497908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782922+0000","last_change":"2023-10-08T13:49:43.776252+0000","last_active":"2023-10-08T13:51:15.782922+0000","last_peered":"2023-10-08T13:51:15.782922+0000","last_clean":"2023-10-08T13:51:15.782922+0000","last_became_active":"2023-10-08T13:49:43.775935+0000","last_became_peered":"2023-10-08T13:49:43.775935+0000","last_unstale":"2023-10-08T13:51:15.782922+0000","last_undegraded":"2023-10-08T13:51:15.782922+0000","last_fullsized":"2023-10-08T13:51:15.782922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:58.949365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809345+0000","last_change":"2023-10-08T13:49:44.788628+0000","last_active":"2023-10-08T13:51:15.809345+0000","last_peered":"2023-10-08T13:51:15.809345+0000","last_clean":"2023-10-08T13:51:15.809345+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:51:15.809345+0000","last_undegraded":"2023-10-08T13:51:15.809345+0000","last_fullsized":"2023-10-08T13:51:15.809345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:46:42.311296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786697+0000","last_change":"2023-10-08T13:49:49.153641+0000","last_active":"2023-10-08T13:51:15.786697+0000","last_peered":"2023-10-08T13:51:15.786697+0000","last_clean":"2023-10-08T13:51:15.786697+0000","last_became_active":"2023-10-08T13:49:49.153172+0000","last_became_peered":"2023-10-08T13:49:49.153172+0000","last_unstale":"2023-10-08T13:51:15.786697+0000","last_undegraded":"2023-10-08T13:51:15.786697+0000","last_fullsized":"2023-10-08T13:51:15.786697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:40:51.726343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:22.853599+0000","last_change":"2023-10-08T13:49:43.782683+0000","last_active":"2023-10-08T13:51:22.853599+0000","last_peered":"2023-10-08T13:51:22.853599+0000","last_clean":"2023-10-08T13:51:22.853599+0000","last_became_active":"2023-10-08T13:49:43.782269+0000","last_became_peered":"2023-10-08T13:49:43.782269+0000","last_unstale":"2023-10-08T13:51:22.853599+0000","last_undegraded":"2023-10-08T13:51:22.853599+0000","last_fullsized":"2023-10-08T13:51:22.853599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:50.850581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809118+0000","last_change":"2023-10-08T13:49:44.787894+0000","last_active":"2023-10-08T13:51:15.809118+0000","last_peered":"2023-10-08T13:51:15.809118+0000","last_clean":"2023-10-08T13:51:15.809118+0000","last_became_active":"2023-10-08T13:49:44.787599+0000","last_became_peered":"2023-10-08T13:49:44.787599+0000","last_unstale":"2023-10-08T13:51:15.809118+0000","last_undegraded":"2023-10-08T13:51:15.809118+0000","last_fullsized":"2023-10-08T13:51:15.809118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:17:15.131051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808379+0000","last_change":"2023-10-08T13:49:49.164256+0000","last_active":"2023-10-08T13:51:15.808379+0000","last_peered":"2023-10-08T13:51:15.808379+0000","last_clean":"2023-10-08T13:51:15.808379+0000","last_became_active":"2023-10-08T13:49:49.163793+0000","last_became_peered":"2023-10-08T13:49:49.163793+0000","last_unstale":"2023-10-08T13:51:15.808379+0000","last_undegraded":"2023-10-08T13:51:15.808379+0000","last_fullsized":"2023-10-08T13:51:15.808379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:19.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785320+0000","last_change":"2023-10-08T13:49:43.778311+0000","last_active":"2023-10-08T13:51:15.785320+0000","last_peered":"2023-10-08T13:51:15.785320+0000","last_clean":"2023-10-08T13:51:15.785320+0000","last_became_active":"2023-10-08T13:49:43.777984+0000","last_became_peered":"2023-10-08T13:49:43.777984+0000","last_unstale":"2023-10-08T13:51:15.785320+0000","last_undegraded":"2023-10-08T13:51:15.785320+0000","last_fullsized":"2023-10-08T13:51:15.785320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:56.890729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808931+0000","last_change":"2023-10-08T13:49:44.793424+0000","last_active":"2023-10-08T13:51:15.808931+0000","last_peered":"2023-10-08T13:51:15.808931+0000","last_clean":"2023-10-08T13:51:15.808931+0000","last_became_active":"2023-10-08T13:49:44.792825+0000","last_became_peered":"2023-10-08T13:49:44.792825+0000","last_unstale":"2023-10-08T13:51:15.808931+0000","last_undegraded":"2023-10-08T13:51:15.808931+0000","last_fullsized":"2023-10-08T13:51:15.808931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:39.463676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784486+0000","last_change":"2023-10-08T13:49:49.151915+0000","last_active":"2023-10-08T13:51:15.784486+0000","last_peered":"2023-10-08T13:51:15.784486+0000","last_clean":"2023-10-08T13:51:15.784486+0000","last_became_active":"2023-10-08T13:49:49.151434+0000","last_became_peered":"2023-10-08T13:49:49.151434+0000","last_unstale":"2023-10-08T13:51:15.784486+0000","last_undegraded":"2023-10-08T13:51:15.784486+0000","last_fullsized":"2023-10-08T13:51:15.784486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:54:07.631498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809499+0000","last_change":"2023-10-08T13:50:15.787243+0000","last_active":"2023-10-08T13:51:15.809499+0000","last_peered":"2023-10-08T13:51:15.809499+0000","last_clean":"2023-10-08T13:51:15.809499+0000","last_became_active":"2023-10-08T13:50:15.782305+0000","last_became_peered":"2023-10-08T13:50:15.782305+0000","last_unstale":"2023-10-08T13:51:15.809499+0000","last_undegraded":"2023-10-08T13:51:15.809499+0000","last_fullsized":"2023-10-08T13:51:15.809499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:08.520010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809254+0000","last_change":"2023-10-08T13:49:44.790949+0000","last_active":"2023-10-08T13:51:15.809254+0000","last_peered":"2023-10-08T13:51:15.809254+0000","last_clean":"2023-10-08T13:51:15.809254+0000","last_became_active":"2023-10-08T13:49:44.790645+0000","last_became_peered":"2023-10-08T13:49:44.790645+0000","last_unstale":"2023-10-08T13:51:15.809254+0000","last_undegraded":"2023-10-08T13:51:15.809254+0000","last_fullsized":"2023-10-08T13:51:15.809254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:48.890730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808397+0000","last_change":"2023-10-08T13:49:44.781165+0000","last_active":"2023-10-08T13:51:15.808397+0000","last_peered":"2023-10-08T13:51:15.808397+0000","last_clean":"2023-10-08T13:51:15.808397+0000","last_became_active":"2023-10-08T13:49:44.780828+0000","last_became_peered":"2023-10-08T13:49:44.780828+0000","last_unstale":"2023-10-08T13:51:15.808397+0000","last_undegraded":"2023-10-08T13:51:15.808397+0000","last_fullsized":"2023-10-08T13:51:15.808397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:04.259482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788103+0000","last_change":"2023-10-08T13:49:44.781271+0000","last_active":"2023-10-08T13:51:15.788103+0000","last_peered":"2023-10-08T13:51:15.788103+0000","last_clean":"2023-10-08T13:51:15.788103+0000","last_became_active":"2023-10-08T13:49:44.780912+0000","last_became_peered":"2023-10-08T13:49:44.780912+0000","last_unstale":"2023-10-08T13:51:15.788103+0000","last_undegraded":"2023-10-08T13:51:15.788103+0000","last_fullsized":"2023-10-08T13:51:15.788103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:17.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787876+0000","last_change":"2023-10-08T13:49:44.777613+0000","last_active":"2023-10-08T13:51:15.787876+0000","last_peered":"2023-10-08T13:51:15.787876+0000","last_clean":"2023-10-08T13:51:15.787876+0000","last_became_active":"2023-10-08T13:49:44.777304+0000","last_became_peered":"2023-10-08T13:49:44.777304+0000","last_unstale":"2023-10-08T13:51:15.787876+0000","last_undegraded":"2023-10-08T13:51:15.787876+0000","last_fullsized":"2023-10-08T13:51:15.787876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:45.482935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808572+0000","last_change":"2023-10-08T13:49:44.782685+0000","last_active":"2023-10-08T13:51:15.808572+0000","last_peered":"2023-10-08T13:51:15.808572+0000","last_clean":"2023-10-08T13:51:15.808572+0000","last_became_active":"2023-10-08T13:49:44.781816+0000","last_became_peered":"2023-10-08T13:49:44.781816+0000","last_unstale":"2023-10-08T13:51:15.808572+0000","last_undegraded":"2023-10-08T13:51:15.808572+0000","last_fullsized":"2023-10-08T13:51:15.808572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:53.781237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809306+0000","last_change":"2023-10-08T13:49:44.783973+0000","last_active":"2023-10-08T13:51:15.809306+0000","last_peered":"2023-10-08T13:51:15.809306+0000","last_clean":"2023-10-08T13:51:15.809306+0000","last_became_active":"2023-10-08T13:49:44.783648+0000","last_became_peered":"2023-10-08T13:49:44.783648+0000","last_unstale":"2023-10-08T13:51:15.809306+0000","last_undegraded":"2023-10-08T13:51:15.809306+0000","last_fullsized":"2023-10-08T13:51:15.809306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:01.253610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808667+0000","last_change":"2023-10-08T13:49:44.781848+0000","last_active":"2023-10-08T13:51:15.808667+0000","last_peered":"2023-10-08T13:51:15.808667+0000","last_clean":"2023-10-08T13:51:15.808667+0000","last_became_active":"2023-10-08T13:49:44.781542+0000","last_became_peered":"2023-10-08T13:49:44.781542+0000","last_unstale":"2023-10-08T13:51:15.808667+0000","last_undegraded":"2023-10-08T13:51:15.808667+0000","last_fullsized":"2023-10-08T13:51:15.808667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:26:38.726298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788486+0000","last_change":"2023-10-08T13:49:44.786541+0000","last_active":"2023-10-08T13:51:15.788486+0000","last_peered":"2023-10-08T13:51:15.788486+0000","last_clean":"2023-10-08T13:51:15.788486+0000","last_became_active":"2023-10-08T13:49:44.786214+0000","last_became_peered":"2023-10-08T13:49:44.786214+0000","last_unstale":"2023-10-08T13:51:15.788486+0000","last_undegraded":"2023-10-08T13:51:15.788486+0000","last_fullsized":"2023-10-08T13:51:15.788486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:30:18.342774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788081+0000","last_change":"2023-10-08T13:49:44.788445+0000","last_active":"2023-10-08T13:51:15.788081+0000","last_peered":"2023-10-08T13:51:15.788081+0000","last_clean":"2023-10-08T13:51:15.788081+0000","last_became_active":"2023-10-08T13:49:44.788075+0000","last_became_peered":"2023-10-08T13:49:44.788075+0000","last_unstale":"2023-10-08T13:51:15.788081+0000","last_undegraded":"2023-10-08T13:51:15.788081+0000","last_fullsized":"2023-10-08T13:51:15.788081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:08:20.701231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784625+0000","last_change":"2023-10-08T13:49:44.783755+0000","last_active":"2023-10-08T13:51:15.784625+0000","last_peered":"2023-10-08T13:51:15.784625+0000","last_clean":"2023-10-08T13:51:15.784625+0000","last_became_active":"2023-10-08T13:49:44.783467+0000","last_became_peered":"2023-10-08T13:49:44.783467+0000","last_unstale":"2023-10-08T13:51:15.784625+0000","last_undegraded":"2023-10-08T13:51:15.784625+0000","last_fullsized":"2023-10-08T13:51:15.784625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:39.838940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788245+0000","last_change":"2023-10-08T13:49:44.779342+0000","last_active":"2023-10-08T13:51:15.788245+0000","last_peered":"2023-10-08T13:51:15.788245+0000","last_clean":"2023-10-08T13:51:15.788245+0000","last_became_active":"2023-10-08T13:49:44.778518+0000","last_became_peered":"2023-10-08T13:49:44.778518+0000","last_unstale":"2023-10-08T13:51:15.788245+0000","last_undegraded":"2023-10-08T13:51:15.788245+0000","last_fullsized":"2023-10-08T13:51:15.788245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:33.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786859+0000","last_change":"2023-10-08T13:49:44.792955+0000","last_active":"2023-10-08T13:51:15.786859+0000","last_peered":"2023-10-08T13:51:15.786859+0000","last_clean":"2023-10-08T13:51:15.786859+0000","last_became_active":"2023-10-08T13:49:44.792637+0000","last_became_peered":"2023-10-08T13:49:44.792637+0000","last_unstale":"2023-10-08T13:51:15.786859+0000","last_undegraded":"2023-10-08T13:51:15.786859+0000","last_fullsized":"2023-10-08T13:51:15.786859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:40.635913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809138+0000","last_change":"2023-10-08T13:49:44.786817+0000","last_active":"2023-10-08T13:51:15.809138+0000","last_peered":"2023-10-08T13:51:15.809138+0000","last_clean":"2023-10-08T13:51:15.809138+0000","last_became_active":"2023-10-08T13:49:44.786439+0000","last_became_peered":"2023-10-08T13:49:44.786439+0000","last_unstale":"2023-10-08T13:51:15.809138+0000","last_undegraded":"2023-10-08T13:51:15.809138+0000","last_fullsized":"2023-10-08T13:51:15.809138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:45.308600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808563+0000","last_change":"2023-10-08T13:50:15.788414+0000","last_active":"2023-10-08T13:51:15.808563+0000","last_peered":"2023-10-08T13:51:15.808563+0000","last_clean":"2023-10-08T13:51:15.808563+0000","last_became_active":"2023-10-08T13:50:15.788079+0000","last_became_peered":"2023-10-08T13:50:15.788079+0000","last_unstale":"2023-10-08T13:51:15.808563+0000","last_undegraded":"2023-10-08T13:51:15.808563+0000","last_fullsized":"2023-10-08T13:51:15.808563+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:43.723339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66011,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":793,"num_read_kb":471,"num_write":242,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":203412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":531579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64000000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69099999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54000000000000004}]},{"osd":1,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80400000000000005,"5min":1.012,"15min":1.012},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83399999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50600000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.36299999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.50900000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.80400000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.496},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.5,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88100000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63300000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82399999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56299999999999994}]},{"osd":6,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.77100000000000002,"15min":0.72899999999999998},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2759999999999998,"5min":9.2759999999999998,"15min":9.2759999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.78600000000000003,"15min":0.79400000000000004},"min":{"1min":0.55200000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.294,"5min":1.55,"15min":1.55},"last":0.76000000000000001}]},{"osd":7,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.54200000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92100000000000004,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.58199999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0089999999999999,"5min":1.214,"15min":1.214},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Oct 8 13:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.871}]},{"osd":9,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.64300000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.97099999999999997,"5min":1.355,"15min":1.355},"last":0.60499999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.68999999999999995,"5min":0.628,"15min":0.628},"max":{"1min":1.0449999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.94899999999999995,"15min":0.96899999999999997},"min":{"1min":0.68899999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.024,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.81399999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1753088,"data_stored":1178086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46600000000000003,"15min":0.46999999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58999999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.624,"15min":0.623},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61499999999999999,"15min":0.624},"min":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82299999999999995,"5min":0.871,"15min":0.871},"last":3.097},{"interface":"front","average":{"1min":0.628,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.438,"15min":0.438},"max":{"1min":0.83599999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57199999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.755,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.317,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73599999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83799999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82399999999999995,"5min":1.034,"15min":1.034},"last":0.67700000000000005}]},{"osd":6,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85299999999999998,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":3.069},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":1.083,"15min":1.083},"last":0.71299999999999997}]},{"osd":7,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55300000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.77300000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92200000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88300000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.749}]},{"osd":9,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.95199999999999996,"5min":1.05,"15min":1.05},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86799999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69699999999999995}]},{"osd":11,"last update":"Sun Oct 8 13:51:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.755,"15min":0.753},"min":{"1min":0.54400000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79200000000000004,"15min":0.78700000000000003},"min":{"1min":0.65200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93899999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.79500000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":535707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.443,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.627,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45900000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.39100000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":4.6029999999999998,"15min":4.6029999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.65800000000000003,"15min":0.67800000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71899999999999997,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.66100000000000003,"15min":0.67800000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67200000000000004,"5min":2.806,"15min":2.806},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.63100000000000001,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79400000000000004,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.52700000000000002}]},{"osd":3,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.47099999999999997,"5min":6.9969999999999999,"15min":6.9969999999999999},"last":0.496},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.60199999999999998,"15min":0.621},"min":{"1min":0.309,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56599999999999995,"5min":3.7589999999999999,"15min":3.7589999999999999},"last":0.373}]},{"osd":4,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77700000000000002,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.76000000000000001,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.51600000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":5.3390000000000004,"15min":5.3390000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82999999999999996,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.9570000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.345}]},{"osd":7,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.80000000000000004,"15min":0.78900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76800000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.82099999999999995,"15min":0.84699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.748,"5min":4.3680000000000003,"15min":4.3680000000000003},"last":0.69799999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84599999999999997,"5min":5.2930000000000001,"15min":5.2930000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.83299999999999996,"15min":0.81899999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83899999999999997,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":0.64100000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.85599999999999998,"15min":0.88300000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.88,"5min":2.113,"15min":2.113},"last":0.75},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89400000000000002,"5min":1.923,"15min":1.923},"last":0.66000000000000003}]},{"osd":11,"last update":"Sun Oct 8 13:50:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.623},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68899999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103212,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":527289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":0.54100000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82999999999999996,"5min":5.5860000000000003,"15min":5.5860000000000003},"last":0.379},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56499999999999995,"15min":0.53100000000000003},"min":{"1min":0.36599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.38900000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61199999999999999,"15min":0.57099999999999995},"min":{"1min":0.435,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96399999999999997,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79100000000000004,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.58699999999999997}]},{"osd":2,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47399999999999998,"15min":0.46200000000000002},"min":{"1min":0.34999999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.873,"5min":1.762,"15min":1.762},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.39300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84599999999999997,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61499999999999999,"15min":0.59999999999999998},"min":{"1min":0.46800000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89300000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.504}]},{"osd":4,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0289999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.67500000000000004,"15min":0.64700000000000002},"min":{"1min":0.51000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92400000000000004,"5min":2.109,"15min":2.109},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.373,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.89900000000000002,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.48899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.92500000000000004,"5min":1.476,"15min":1.476},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66700000000000004,"15min":0.64700000000000002},"min":{"1min":0.58299999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.69599999999999995},"min":{"1min":0.56100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90100000000000002,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97899999999999998,"5min":1.958,"15min":1.958},"last":0.436},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.58399999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95899999999999996,"5min":2.198,"15min":2.198},"last":0.67200000000000004}]},{"osd":9,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.69999999999999996},"min":{"1min":0.58599999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.93899999999999995,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.752,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1519999999999999,"5min":2.347,"15min":2.347},"last":0.47399999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.73699999999999999,"15min":0.69699999999999995},"min":{"1min":0.66000000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":4.2039999999999997,"5min":4.2039999999999997,"15min":4.2039999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.71699999999999997,"15min":0.68700000000000006},"min":{"1min":0.74299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.61299999999999999}]},{"osd":11,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56699999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364849,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1124530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.439}]},{"osd":1,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42199999999999999,"15min":0.41699999999999998},"min":{"1min":0.33400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64600000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57699999999999996}]},{"osd":2,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.435,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86599999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.48499999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68600000000000005}]},{"osd":3,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.60399999999999998}]},{"osd":4,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.50900000000000001},"min":{"1min":0.42399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73699999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.47999999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.52600000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.77600000000000002}]},{"osd":6,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61599999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:51:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.63500000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88600000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71999999999999997,"15min":0.71799999999999997},"min":{"1min":0.63300000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.79400000000000004}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71999999999999997,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64000000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88200000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.60399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91200000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.73599999999999999}]},{"osd":11,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.82299999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":527115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.39400000000000002,"15min":0.375},"min":{"1min":0.32300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"min":{"1min":0.309,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.441}]},{"osd":1,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80800000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.45300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81499999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.55000000000000004}]},{"osd":2,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79900000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.498},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85199999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.46899999999999997}]},{"osd":4,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82499999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74199999999999999,"5min":1.073,"15min":1.073},"last":0.59199999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.879,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":1.075,"15min":1.075},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.749,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67800000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:51:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89800000000000002,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.626},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.876,"5min":1.147,"15min":1.147},"last":0.66300000000000003}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.73299999999999998,"15min":0.747},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.96799999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":1.042,"15min":1.042},"last":0.74099999999999999}]},{"osd":10,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86899999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0469999999999999,"5min":1.159,"15min":1.159},"last":0.76000000000000001}]},{"osd":11,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58199999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542219,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":591921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.52400000000000002},"min":{"1min":0.38700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.90400000000000003,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:50:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.55900000000000005,"15min":0.50700000000000001},"min":{"1min":0.51900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.56599999999999995,"15min":0.52100000000000002},"min":{"1min":0.51300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54800000000000004}]},{"osd":3,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":3.7959999999999998,"15min":3.7959999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90600000000000003,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.629}]},{"osd":4,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.41899999999999998},"min":{"1min":0.40300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.75,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.504},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.44700000000000001},"min":{"1min":0.38100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48599999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.623,"15min":0.56599999999999995},"min":{"1min":0.51700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.65300000000000002,"15min":0.60699999999999998},"min":{"1min":0.56299999999999994,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91800000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72499999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.68700000000000006,"15min":0.76100000000000001},"min":{"1min":0.435,"5min":0.376,"15min":0.376},"max":{"1min":0.80000000000000004,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72499999999999998,"15min":0.75600000000000001},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85699999999999998,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.48199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60699999999999998,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56200000000000006}]},{"osd":8,"last update":"Sun Oct 8 13:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.71399999999999997,"15min":0.68799999999999994},"min":{"1min":0.57999999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98899999999999999,"5min":1.077,"15min":1.077},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.745,"15min":0.72599999999999998},"min":{"1min":0.69199999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.82499999999999996}]},{"osd":9,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72899999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92200000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79200000000000004,"15min":0.78800000000000003},"min":{"1min":0.55000000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.95499999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.88200000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.54000000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79700000000000004,"15min":0.78800000000000003},"min":{"1min":0.61299999999999999,"5min":0.498,"15min":0.498},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.78200000000000003}]},{"osd":11,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.86099999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705743,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":528571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 13:50:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54100000000000004,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72199999999999998,"5min":1.01,"15min":1.01},"last":0.51300000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71999999999999997,"5min":1.377,"15min":1.377},"last":0.498},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.54300000000000004},"min":{"1min":0.45500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68799999999999994,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52400000000000002}]},{"osd":3,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.39600000000000002}]},{"osd":4,"last update":"Sun Oct 8 13:50:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.81899999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71399999999999997,"5min":1.444,"15min":1.444},"last":0.64800000000000002}]},{"osd":5,"last update":"Sun Oct 8 13:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63500000000000001,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85799999999999998,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77400000000000002,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.36899999999999999}]},{"osd":7,"last update":"Sun Oct 8 13:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.73299999999999998}]},{"osd":8,"last update":"Sun Oct 8 13:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.73699999999999999,"15min":0.76800000000000002},"min":{"1min":0.41099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82899999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":1.327,"15min":1.327},"last":0.56499999999999995}]},{"osd":9,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.44600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90600000000000003,"5min":1.774,"15min":1.774},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.018,"5min":1.839,"15min":1.839},"last":0.67400000000000004}]},{"osd":10,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.59099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91100000000000003,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85699999999999998,"5min":1.411,"15min":1.411},"last":0.76600000000000001}]},{"osd":11,"last update":"Sun Oct 8 13:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.754,"15min":0.753},"min":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91600000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94399999999999995,"5min":1.208,"15min":1.208},"last":0.64100000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378694,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":590897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.94599999999999995,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81799999999999995,"5min":1.538,"15min":1.538},"last":0.45500000000000002}]},{"osd":1,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54800000000000004,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.55900000000000005},"min":{"1min":0.38900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73299999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.61599999999999999,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.497},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.55400000000000005}]},{"osd":4,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.621},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.47899999999999998,"15min":0.44},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":0.876,"15min":0.876},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.39200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.75}]},{"osd":6,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.70099999999999996}]},{"osd":7,"last update":"Sun Oct 8 13:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.622,"15min":0.60399999999999998},"min":{"1min":0.45100000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65100000000000002}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.56000000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.85099999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.68600000000000005,"15min":0.66900000000000004},"min":{"1min":0.53100000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.70399999999999996,"15min":0.68000000000000005},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.83899999999999997}]},{"osd":11,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79100000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":526823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.34799999999999998},"min":{"1min":0.28100000000000003,"5min":0.22,"15min":0.22},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47699999999999998,"15min":0.42099999999999999},"min":{"1min":0.32700000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65700000000000003,"5min":0.87,"15min":0.87},"last":0.48899999999999999}]},{"osd":1,"last update":"Sun Oct 8 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.53300000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93300000000000005,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.53100000000000003},"min":{"1min":0.40000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68899999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.60899999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61199999999999999,"15min":0.56599999999999995},"min":{"1min":0.47899999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.78500000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:50:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":2.077,"15min":2.077},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60299999999999998,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88300000000000001,"5min":2.129,"15min":2.129},"last":0.51000000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.60899999999999999},"min":{"1min":0.53700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76100000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65600000000000003,"15min":0.60799999999999998},"min":{"1min":0.432,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91000000000000003,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.75}]},{"osd":6,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.32400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80200000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.432},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.625},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53500000000000003}]},{"osd":7,"last update":"Sun Oct 8 13:50:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.76800000000000002,"15min":0.80100000000000005},"min":{"1min":0.48799999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.872,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.75600000000000001,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83199999999999996,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.57299999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:48:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.46800000000000003}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.069,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65400000000000003}]},{"osd":10,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.95799999999999996,"5min":1.028,"15min":1.028},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.82399999999999995,"15min":0.84299999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92000000000000004,"5min":1.147,"15min":1.147},"last":0.66800000000000004}]},{"osd":11,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.69899999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":531720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.51800000000000002,"15min":0.47999999999999998},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.35599999999999998},"min":{"1min":0.30599999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.36199999999999999},"min":{"1min":0.30399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.36799999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.54900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54900000000000004}]},{"osd":3,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.51500000000000001},"min":{"1min":0.40200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.497}]},{"osd":5,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.629,"15min":0.58699999999999997},"min":{"1min":0.502,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0009999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67000000000000004,"15min":0.63200000000000001},"min":{"1min":0.58299999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.71399999999999997}]},{"osd":6,"last update":"Sun Oct 8 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.66300000000000003,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.377,"15min":0.377},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.629}]},{"osd":7,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.38800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77100000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.59299999999999997,"15min":0.57299999999999995},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84599999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.43099999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73199999999999998,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96299999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.45900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97899999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.50900000000000001}]},{"osd":9,"last update":"Sun Oct 8 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.76300000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79900000000000004,"15min":0.80200000000000005},"min":{"1min":0.47199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94899999999999995,"5min":1.04,"15min":1.04},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59399999999999997}]},{"osd":11,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64500000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1683456,"data_stored":1123346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.50700000000000001,"15min":0.47899999999999998},"min":{"1min":0.40400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52300000000000002,"15min":0.47599999999999998},"min":{"1min":0.438,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.873,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47599999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.378},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.38900000000000001},"min":{"1min":0.30499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71099999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.58099999999999996,"15min":0.52000000000000002},"min":{"1min":0.46600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.52800000000000002}]},{"osd":4,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.56599999999999995},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.499}]},{"osd":6,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.54700000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.46999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56100000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88700000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.62}]},{"osd":9,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.66400000000000003,"15min":0.64300000000000002},"min":{"1min":0.57599999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73599999999999999,"15min":0.72099999999999997},"min":{"1min":0.629,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.752}]},{"osd":10,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72799999999999998,"15min":0.72599999999999998},"min":{"1min":0.59499999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.93999999999999995,"5min":0.997,"15min":0.997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.022,"5min":1.101,"15min":1.101},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T13:51:26.926 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T13:51:27.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.051+0000 7efdc18a7700 1 -- 172.21.15.57:0/2964383264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdbc105600 msgr2=0x7efdbc1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.051+0000 7efdc18a7700 1 --2- 172.21.15.57:0/2964383264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdbc105600 0x7efdbc1059e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efdac0099c0 tx=0x7efdac009cd0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.051+0000 7efdc18a7700 1 -- 172.21.15.57:0/2964383264 shutdown_connections 2023-10-08T13:51:27.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.051+0000 7efdc18a7700 1 --2- 172.21.15.57:0/2964383264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdbc0fff00 0x7efdbc10c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.051+0000 7efdc18a7700 1 --2- 172.21.15.57:0/2964383264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdbc105600 0x7efdbc1059e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.051+0000 7efdc18a7700 1 --2- 172.21.15.57:0/2964383264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc0ff540 0x7efdbc0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.051+0000 7efdc18a7700 1 -- 172.21.15.57:0/2964383264 >> 172.21.15.57:0/2964383264 conn(0x7efdbc0fb030 msgr2=0x7efdbc0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:27.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.051+0000 7efdc18a7700 1 -- 172.21.15.57:0/2964383264 shutdown_connections 2023-10-08T13:51:27.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.051+0000 7efdc18a7700 1 -- 172.21.15.57:0/2964383264 wait complete. 2023-10-08T13:51:27.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.052+0000 7efdc18a7700 1 Processor -- start 2023-10-08T13:51:27.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.052+0000 7efdc18a7700 1 -- start start 2023-10-08T13:51:27.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.052+0000 7efdc18a7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdbc0ff540 0x7efdbc10cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.052+0000 7efdc18a7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc0fff00 0x7efdbc10d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.052+0000 7efdc18a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdbc105600 0x7efdbc10d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.052+0000 7efdc18a7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc1a4e10 con 0x7efdbc105600 2023-10-08T13:51:27.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.052+0000 7efdc18a7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc1a4f80 con 0x7efdbc0ff540 2023-10-08T13:51:27.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.052+0000 7efdc18a7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc1a5140 con 0x7efdbc0fff00 2023-10-08T13:51:27.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdbb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdbc105600 0x7efdbc10d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdbb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdbc105600 0x7efdbc10d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56098/0 (socket says 172.21.15.57:56098) 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdbb7fe700 1 -- 172.21.15.57:0/2131647465 learned_addr learned my addr 172.21.15.57:0/2131647465 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdba7fc700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc0fff00 0x7efdbc10d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdbaffd700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdbc0ff540 0x7efdbc10cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdbb7fe700 1 -- 172.21.15.57:0/2131647465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc0fff00 msgr2=0x7efdbc10d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdbb7fe700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc0fff00 0x7efdbc10d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdbb7fe700 1 -- 172.21.15.57:0/2131647465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdbc0ff540 msgr2=0x7efdbc10cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdbb7fe700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdbc0ff540 0x7efdbc10cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdbb7fe700 1 -- 172.21.15.57:0/2131647465 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac0095f0 con 0x7efdbc105600 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdbb7fe700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdbc105600 0x7efdbc10d810 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7efdb4007ee0 tx=0x7efdb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdc08a5700 1 -- 172.21.15.57:0/2131647465 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb400dd20 con 0x7efdbc105600 2023-10-08T13:51:27.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdc08a5700 1 -- 172.21.15.57:0/2131647465 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb400f040 con 0x7efdbc105600 2023-10-08T13:51:27.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdc18a7700 1 -- 172.21.15.57:0/2131647465 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc1a5420 con 0x7efdbc105600 2023-10-08T13:51:27.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdc08a5700 1 -- 172.21.15.57:0/2131647465 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40048a0 con 0x7efdbc105600 2023-10-08T13:51:27.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.053+0000 7efdc18a7700 1 -- 172.21.15.57:0/2131647465 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc1a58e0 con 0x7efdbc105600 2023-10-08T13:51:27.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.054+0000 7efdc18a7700 1 -- 172.21.15.57:0/2131647465 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc107a80 con 0x7efdbc105600 2023-10-08T13:51:27.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.055+0000 7efdc08a5700 1 -- 172.21.15.57:0/2131647465 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7efdb4003a00 con 0x7efdbc105600 2023-10-08T13:51:27.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.056+0000 7efdc08a5700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efda40a14d0 0x7efda40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.056+0000 7efdc08a5700 1 -- 172.21.15.57:0/2131647465 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7efdb40cd250 con 0x7efdbc105600 2023-10-08T13:51:27.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.056+0000 7efdbaffd700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efda40a14d0 0x7efda40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.056+0000 7efdbaffd700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efda40a14d0 0x7efda40a3990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efdac009990 tx=0x7efdac003680 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:27.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.059+0000 7efdc08a5700 1 -- 172.21.15.57:0/2131647465 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efdb4096430 con 0x7efdbc105600 2023-10-08T13:51:27.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:26 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3358290093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:27.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:26 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3257285918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:27.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:26 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3358290093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:27.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:26 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3257285918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:27.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.263+0000 7efdc18a7700 1 -- 172.21.15.57:0/2131647465 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efdbc10e2e0 con 0x7efda40a14d0 2023-10-08T13:51:27.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:26 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3358290093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:27.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:26 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3257285918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:27.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.278+0000 7efdc08a5700 1 -- 172.21.15.57:0/2131647465 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427464 (secure 0 0 0) 0x7efdbc10e2e0 con 0x7efda40a14d0 2023-10-08T13:51:27.284 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:27.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.282+0000 7efdc18a7700 1 -- 172.21.15.57:0/2131647465 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efda40a14d0 msgr2=0x7efda40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.282+0000 7efdc18a7700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efda40a14d0 0x7efda40a3990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efdac009990 tx=0x7efdac003680 comp rx=0 tx=0).stop 2023-10-08T13:51:27.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.283+0000 7efdc18a7700 1 -- 172.21.15.57:0/2131647465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdbc105600 msgr2=0x7efdbc10d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.283+0000 7efdc18a7700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdbc105600 0x7efdbc10d810 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7efdb4007ee0 tx=0x7efdb400d3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.283+0000 7efdc18a7700 1 -- 172.21.15.57:0/2131647465 shutdown_connections 2023-10-08T13:51:27.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.283+0000 7efdc18a7700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdbc105600 0x7efdbc10d810 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.283+0000 7efdc18a7700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efda40a14d0 0x7efda40a3990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.283+0000 7efdc18a7700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdbc0ff540 0x7efdbc10cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.283+0000 7efdc18a7700 1 --2- 172.21.15.57:0/2131647465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc0fff00 0x7efdbc10d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.283+0000 7efdc18a7700 1 -- 172.21.15.57:0/2131647465 >> 172.21.15.57:0/2131647465 conn(0x7efdbc0fb030 msgr2=0x7efdbc0fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:27.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.283+0000 7efdc18a7700 1 -- 172.21.15.57:0/2131647465 shutdown_connections 2023-10-08T13:51:27.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.283+0000 7efdc18a7700 1 -- 172.21.15.57:0/2131647465 wait complete. 2023-10-08T13:51:27.289 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-08T13:51:27.296 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-10-08T13:51:26.130285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656625,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":929,"num_read_kb":590,"num_write":425,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338832,"kb_used_data":14556,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124602992,"statfs":{"total":1151940427776,"available":1151593463808,"internally_reserved":0,"allocated":14905344,"data_stored":8317584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002661"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808844+0000","last_change":"2023-10-08T13:49:44.786130+0000","last_active":"2023-10-08T13:51:15.808844+0000","last_peered":"2023-10-08T13:51:15.808844+0000","last_clean":"2023-10-08T13:51:15.808844+0000","last_became_active":"2023-10-08T13:49:44.785792+0000","last_became_peered":"2023-10-08T13:49:44.785792+0000","last_unstale":"2023-10-08T13:51:15.808844+0000","last_undegraded":"2023-10-08T13:51:15.808844+0000","last_fullsized":"2023-10-08T13:51:15.808844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:17.046800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808869+0000","last_change":"2023-10-08T13:49:44.777601+0000","last_active":"2023-10-08T13:51:15.808869+0000","last_peered":"2023-10-08T13:51:15.808869+0000","last_clean":"2023-10-08T13:51:15.808869+0000","last_became_active":"2023-10-08T13:49:44.777287+0000","last_became_peered":"2023-10-08T13:49:44.777287+0000","last_unstale":"2023-10-08T13:51:15.808869+0000","last_undegraded":"2023-10-08T13:51:15.808869+0000","last_fullsized":"2023-10-08T13:51:15.808869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:09:17.317629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808744+0000","last_change":"2023-10-08T13:49:44.787035+0000","last_active":"2023-10-08T13:51:15.808744+0000","last_peered":"2023-10-08T13:51:15.808744+0000","last_clean":"2023-10-08T13:51:15.808744+0000","last_became_active":"2023-10-08T13:49:44.785834+0000","last_became_peered":"2023-10-08T13:49:44.785834+0000","last_unstale":"2023-10-08T13:51:15.808744+0000","last_undegraded":"2023-10-08T13:51:15.808744+0000","last_fullsized":"2023-10-08T13:51:15.808744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:11.611404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783168+0000","last_change":"2023-10-08T13:49:44.781937+0000","last_active":"2023-10-08T13:51:15.783168+0000","last_peered":"2023-10-08T13:51:15.783168+0000","last_clean":"2023-10-08T13:51:15.783168+0000","last_became_active":"2023-10-08T13:49:44.780712+0000","last_became_peered":"2023-10-08T13:49:44.780712+0000","last_unstale":"2023-10-08T13:51:15.783168+0000","last_undegraded":"2023-10-08T13:51:15.783168+0000","last_fullsized":"2023-10-08T13:51:15.783168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:02.514007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783022+0000","last_change":"2023-10-08T13:49:44.788661+0000","last_active":"2023-10-08T13:51:15.783022+0000","last_peered":"2023-10-08T13:51:15.783022+0000","last_clean":"2023-10-08T13:51:15.783022+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:51:15.783022+0000","last_undegraded":"2023-10-08T13:51:15.783022+0000","last_fullsized":"2023-10-08T13:51:15.783022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:35.770287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809252+0000","last_change":"2023-10-08T13:49:44.778519+0000","last_active":"2023-10-08T13:51:15.809252+0000","last_peered":"2023-10-08T13:51:15.809252+0000","last_clean":"2023-10-08T13:51:15.809252+0000","last_became_active":"2023-10-08T13:49:44.778193+0000","last_became_peered":"2023-10-08T13:49:44.778193+0000","last_unstale":"2023-10-08T13:51:15.809252+0000","last_undegraded":"2023-10-08T13:51:15.809252+0000","last_fullsized":"2023-10-08T13:51:15.809252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:12.916019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809332+0000","last_change":"2023-10-08T13:49:44.791759+0000","last_active":"2023-10-08T13:51:15.809332+0000","last_peered":"2023-10-08T13:51:15.809332+0000","last_clean":"2023-10-08T13:51:15.809332+0000","last_became_active":"2023-10-08T13:49:44.791494+0000","last_became_peered":"2023-10-08T13:49:44.791494+0000","last_unstale":"2023-10-08T13:51:15.809332+0000","last_undegraded":"2023-10-08T13:51:15.809332+0000","last_fullsized":"2023-10-08T13:51:15.809332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:55.859609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787828+0000","last_change":"2023-10-08T13:49:44.786831+0000","last_active":"2023-10-08T13:51:15.787828+0000","last_peered":"2023-10-08T13:51:15.787828+0000","last_clean":"2023-10-08T13:51:15.787828+0000","last_became_active":"2023-10-08T13:49:44.786516+0000","last_became_peered":"2023-10-08T13:49:44.786516+0000","last_unstale":"2023-10-08T13:51:15.787828+0000","last_undegraded":"2023-10-08T13:51:15.787828+0000","last_fullsized":"2023-10-08T13:51:15.787828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:07.616821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787111+0000","last_change":"2023-10-08T13:49:44.789827+0000","last_active":"2023-10-08T13:51:15.787111+0000","last_peered":"2023-10-08T13:51:15.787111+0000","last_clean":"2023-10-08T13:51:15.787111+0000","last_became_active":"2023-10-08T13:49:44.786635+0000","last_became_peered":"2023-10-08T13:49:44.786635+0000","last_unstale":"2023-10-08T13:51:15.787111+0000","last_undegraded":"2023-10-08T13:51:15.787111+0000","last_fullsized":"2023-10-08T13:51:15.787111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:35.492437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809682+0000","last_change":"2023-10-08T13:49:44.784240+0000","last_active":"2023-10-08T13:51:15.809682+0000","last_peered":"2023-10-08T13:51:15.809682+0000","last_clean":"2023-10-08T13:51:15.809682+0000","last_became_active":"2023-10-08T13:49:44.783475+0000","last_became_peered":"2023-10-08T13:49:44.783475+0000","last_unstale":"2023-10-08T13:51:15.809682+0000","last_undegraded":"2023-10-08T13:51:15.809682+0000","last_fullsized":"2023-10-08T13:51:15.809682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:39.730679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808970+0000","last_change":"2023-10-08T13:49:44.790901+0000","last_active":"2023-10-08T13:51:15.808970+0000","last_peered":"2023-10-08T13:51:15.808970+0000","last_clean":"2023-10-08T13:51:15.808970+0000","last_became_active":"2023-10-08T13:49:44.790611+0000","last_became_peered":"2023-10-08T13:49:44.790611+0000","last_unstale":"2023-10-08T13:51:15.808970+0000","last_undegraded":"2023-10-08T13:51:15.808970+0000","last_fullsized":"2023-10-08T13:51:15.808970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:19:21.764105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783260+0000","last_change":"2023-10-08T13:49:44.775226+0000","last_active":"2023-10-08T13:51:15.783260+0000","last_peered":"2023-10-08T13:51:15.783260+0000","last_clean":"2023-10-08T13:51:15.783260+0000","last_became_active":"2023-10-08T13:49:44.774756+0000","last_became_peered":"2023-10-08T13:49:44.774756+0000","last_unstale":"2023-10-08T13:51:15.783260+0000","last_undegraded":"2023-10-08T13:51:15.783260+0000","last_fullsized":"2023-10-08T13:51:15.783260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:57:14.749066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809003+0000","last_change":"2023-10-08T13:49:44.781610+0000","last_active":"2023-10-08T13:51:15.809003+0000","last_peered":"2023-10-08T13:51:15.809003+0000","last_clean":"2023-10-08T13:51:15.809003+0000","last_became_active":"2023-10-08T13:49:44.781281+0000","last_became_peered":"2023-10-08T13:49:44.781281+0000","last_unstale":"2023-10-08T13:51:15.809003+0000","last_undegraded":"2023-10-08T13:51:15.809003+0000","last_fullsized":"2023-10-08T13:51:15.809003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:50.823909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808360+0000","last_change":"2023-10-08T13:49:44.775836+0000","last_active":"2023-10-08T13:51:15.808360+0000","last_peered":"2023-10-08T13:51:15.808360+0000","last_clean":"2023-10-08T13:51:15.808360+0000","last_became_active":"2023-10-08T13:49:44.775206+0000","last_became_peered":"2023-10-08T13:49:44.775206+0000","last_unstale":"2023-10-08T13:51:15.808360+0000","last_undegraded":"2023-10-08T13:51:15.808360+0000","last_fullsized":"2023-10-08T13:51:15.808360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:51.566800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808279+0000","last_change":"2023-10-08T13:49:44.778495+0000","last_active":"2023-10-08T13:51:15.808279+0000","last_peered":"2023-10-08T13:51:15.808279+0000","last_clean":"2023-10-08T13:51:15.808279+0000","last_became_active":"2023-10-08T13:49:44.778196+0000","last_became_peered":"2023-10-08T13:49:44.778196+0000","last_unstale":"2023-10-08T13:51:15.808279+0000","last_undegraded":"2023-10-08T13:51:15.808279+0000","last_fullsized":"2023-10-08T13:51:15.808279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:57:38.531853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786633+0000","last_change":"2023-10-08T13:49:44.792960+0000","last_active":"2023-10-08T13:51:15.786633+0000","last_peered":"2023-10-08T13:51:15.786633+0000","last_clean":"2023-10-08T13:51:15.786633+0000","last_became_active":"2023-10-08T13:49:44.792674+0000","last_became_peered":"2023-10-08T13:49:44.792674+0000","last_unstale":"2023-10-08T13:51:15.786633+0000","last_undegraded":"2023-10-08T13:51:15.786633+0000","last_fullsized":"2023-10-08T13:51:15.786633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:13.934954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808777+0000","last_change":"2023-10-08T13:49:44.785971+0000","last_active":"2023-10-08T13:51:15.808777+0000","last_peered":"2023-10-08T13:51:15.808777+0000","last_clean":"2023-10-08T13:51:15.808777+0000","last_became_active":"2023-10-08T13:49:44.785680+0000","last_became_peered":"2023-10-08T13:49:44.785680+0000","last_unstale":"2023-10-08T13:51:15.808777+0000","last_undegraded":"2023-10-08T13:51:15.808777+0000","last_fullsized":"2023-10-08T13:51:15.808777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:35.447519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786759+0000","last_change":"2023-10-08T13:49:44.787553+0000","last_active":"2023-10-08T13:51:15.786759+0000","last_peered":"2023-10-08T13:51:15.786759+0000","last_clean":"2023-10-08T13:51:15.786759+0000","last_became_active":"2023-10-08T13:49:44.787288+0000","last_became_peered":"2023-10-08T13:49:44.787288+0000","last_unstale":"2023-10-08T13:51:15.786759+0000","last_undegraded":"2023-10-08T13:51:15.786759+0000","last_fullsized":"2023-10-08T13:51:15.786759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:48.386120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788367+0000","last_change":"2023-10-08T13:49:44.789252+0000","last_active":"2023-10-08T13:51:15.788367+0000","last_peered":"2023-10-08T13:51:15.788367+0000","last_clean":"2023-10-08T13:51:15.788367+0000","last_became_active":"2023-10-08T13:49:44.788947+0000","last_became_peered":"2023-10-08T13:49:44.788947+0000","last_unstale":"2023-10-08T13:51:15.788367+0000","last_undegraded":"2023-10-08T13:51:15.788367+0000","last_fullsized":"2023-10-08T13:51:15.788367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:30.266913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787998+0000","last_change":"2023-10-08T13:49:49.163619+0000","last_active":"2023-10-08T13:51:15.787998+0000","last_peered":"2023-10-08T13:51:15.787998+0000","last_clean":"2023-10-08T13:51:15.787998+0000","last_became_active":"2023-10-08T13:49:49.163020+0000","last_became_peered":"2023-10-08T13:49:49.163020+0000","last_unstale":"2023-10-08T13:51:15.787998+0000","last_undegraded":"2023-10-08T13:51:15.787998+0000","last_fullsized":"2023-10-08T13:51:15.787998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:56.092404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":112,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:23.854771+0000","last_change":"2023-10-08T13:49:43.782863+0000","last_active":"2023-10-08T13:51:23.854771+0000","last_peered":"2023-10-08T13:51:23.854771+0000","last_clean":"2023-10-08T13:51:23.854771+0000","last_became_active":"2023-10-08T13:49:43.782477+0000","last_became_peered":"2023-10-08T13:49:43.782477+0000","last_unstale":"2023-10-08T13:51:23.854771+0000","last_undegraded":"2023-10-08T13:51:23.854771+0000","last_fullsized":"2023-10-08T13:51:23.854771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:10.134907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:16.876324+0000","last_change":"2023-10-08T13:51:16.875898+0000","last_active":"2023-10-08T13:51:16.876324+0000","last_peered":"2023-10-08T13:51:16.876324+0000","last_clean":"2023-10-08T13:51:16.876324+0000","last_became_active":"2023-10-08T13:51:16.875581+0000","last_became_peered":"2023-10-08T13:51:16.875581+0000","last_unstale":"2023-10-08T13:51:16.876324+0000","last_undegraded":"2023-10-08T13:51:16.876324+0000","last_fullsized":"2023-10-08T13:51:16.876324+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:27.797425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782603+0000","last_change":"2023-10-08T13:49:49.164248+0000","last_active":"2023-10-08T13:51:15.782603+0000","last_peered":"2023-10-08T13:51:15.782603+0000","last_clean":"2023-10-08T13:51:15.782603+0000","last_became_active":"2023-10-08T13:49:49.163624+0000","last_became_peered":"2023-10-08T13:49:49.163624+0000","last_unstale":"2023-10-08T13:51:15.782603+0000","last_undegraded":"2023-10-08T13:51:15.782603+0000","last_fullsized":"2023-10-08T13:51:15.782603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:04.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788426+0000","last_change":"2023-10-08T13:49:43.788026+0000","last_active":"2023-10-08T13:51:15.788426+0000","last_peered":"2023-10-08T13:51:15.788426+0000","last_clean":"2023-10-08T13:51:15.788426+0000","last_became_active":"2023-10-08T13:49:43.787639+0000","last_became_peered":"2023-10-08T13:49:43.787639+0000","last_unstale":"2023-10-08T13:51:15.788426+0000","last_undegraded":"2023-10-08T13:51:15.788426+0000","last_fullsized":"2023-10-08T13:51:15.788426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:25:48.367765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808934+0000","last_change":"2023-10-08T13:49:44.786322+0000","last_active":"2023-10-08T13:51:15.808934+0000","last_peered":"2023-10-08T13:51:15.808934+0000","last_clean":"2023-10-08T13:51:15.808934+0000","last_became_active":"2023-10-08T13:49:44.785953+0000","last_became_peered":"2023-10-08T13:49:44.785953+0000","last_unstale":"2023-10-08T13:51:15.808934+0000","last_undegraded":"2023-10-08T13:51:15.808934+0000","last_fullsized":"2023-10-08T13:51:15.808934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:39.866785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782827+0000","last_change":"2023-10-08T13:50:15.976961+0000","last_active":"2023-10-08T13:51:15.782827+0000","last_peered":"2023-10-08T13:51:15.782827+0000","last_clean":"2023-10-08T13:51:15.782827+0000","last_became_active":"2023-10-08T13:50:15.976492+0000","last_became_peered":"2023-10-08T13:50:15.976492+0000","last_unstale":"2023-10-08T13:51:15.782827+0000","last_undegraded":"2023-10-08T13:51:15.782827+0000","last_fullsized":"2023-10-08T13:51:15.782827+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:05:48.947259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783518+0000","last_change":"2023-10-08T13:49:43.776239+0000","last_active":"2023-10-08T13:51:15.783518+0000","last_peered":"2023-10-08T13:51:15.783518+0000","last_clean":"2023-10-08T13:51:15.783518+0000","last_became_active":"2023-10-08T13:49:43.775969+0000","last_became_peered":"2023-10-08T13:49:43.775969+0000","last_unstale":"2023-10-08T13:51:15.783518+0000","last_undegraded":"2023-10-08T13:51:15.783518+0000","last_fullsized":"2023-10-08T13:51:15.783518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:24.011655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786966+0000","last_change":"2023-10-08T13:49:44.794912+0000","last_active":"2023-10-08T13:51:15.786966+0000","last_peered":"2023-10-08T13:51:15.786966+0000","last_clean":"2023-10-08T13:51:15.786966+0000","last_became_active":"2023-10-08T13:49:44.794590+0000","last_became_peered":"2023-10-08T13:49:44.794590+0000","last_unstale":"2023-10-08T13:51:15.786966+0000","last_undegraded":"2023-10-08T13:51:15.786966+0000","last_fullsized":"2023-10-08T13:51:15.786966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:51.631386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782715+0000","last_change":"2023-10-08T13:49:49.152149+0000","last_active":"2023-10-08T13:51:15.782715+0000","last_peered":"2023-10-08T13:51:15.782715+0000","last_clean":"2023-10-08T13:51:15.782715+0000","last_became_active":"2023-10-08T13:49:49.151764+0000","last_became_peered":"2023-10-08T13:49:49.151764+0000","last_unstale":"2023-10-08T13:51:15.782715+0000","last_undegraded":"2023-10-08T13:51:15.782715+0000","last_fullsized":"2023-10-08T13:51:15.782715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:25:08.232556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808473+0000","last_change":"2023-10-08T13:50:15.800467+0000","last_active":"2023-10-08T13:51:15.808473+0000","last_peered":"2023-10-08T13:51:15.808473+0000","last_clean":"2023-10-08T13:51:15.808473+0000","last_became_active":"2023-10-08T13:50:15.788865+0000","last_became_peered":"2023-10-08T13:50:15.788865+0000","last_unstale":"2023-10-08T13:51:15.808473+0000","last_undegraded":"2023-10-08T13:51:15.808473+0000","last_fullsized":"2023-10-08T13:51:15.808473+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:11:01.887072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785234+0000","last_change":"2023-10-08T13:49:44.783915+0000","last_active":"2023-10-08T13:51:15.785234+0000","last_peered":"2023-10-08T13:51:15.785234+0000","last_clean":"2023-10-08T13:51:15.785234+0000","last_became_active":"2023-10-08T13:49:44.783628+0000","last_became_peered":"2023-10-08T13:49:44.783628+0000","last_unstale":"2023-10-08T13:51:15.785234+0000","last_undegraded":"2023-10-08T13:51:15.785234+0000","last_fullsized":"2023-10-08T13:51:15.785234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:43.759681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788199+0000","last_change":"2023-10-08T13:49:49.156241+0000","last_active":"2023-10-08T13:51:15.788199+0000","last_peered":"2023-10-08T13:51:15.788199+0000","last_clean":"2023-10-08T13:51:15.788199+0000","last_became_active":"2023-10-08T13:49:49.155652+0000","last_became_peered":"2023-10-08T13:49:49.155652+0000","last_unstale":"2023-10-08T13:51:15.788199+0000","last_undegraded":"2023-10-08T13:51:15.788199+0000","last_fullsized":"2023-10-08T13:51:15.788199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":119,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:23.885147+0000","last_change":"2023-10-08T13:50:15.795816+0000","last_active":"2023-10-08T13:51:23.885147+0000","last_peered":"2023-10-08T13:51:23.885147+0000","last_clean":"2023-10-08T13:51:23.885147+0000","last_became_active":"2023-10-08T13:50:15.789936+0000","last_became_peered":"2023-10-08T13:50:15.789936+0000","last_unstale":"2023-10-08T13:51:23.885147+0000","last_undegraded":"2023-10-08T13:51:23.885147+0000","last_fullsized":"2023-10-08T13:51:23.885147+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:21.496490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783042+0000","last_change":"2023-10-08T13:49:44.780935+0000","last_active":"2023-10-08T13:51:15.783042+0000","last_peered":"2023-10-08T13:51:15.783042+0000","last_clean":"2023-10-08T13:51:15.783042+0000","last_became_active":"2023-10-08T13:49:44.780613+0000","last_became_peered":"2023-10-08T13:49:44.780613+0000","last_unstale":"2023-10-08T13:51:15.783042+0000","last_undegraded":"2023-10-08T13:51:15.783042+0000","last_fullsized":"2023-10-08T13:51:15.783042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:57.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788141+0000","last_change":"2023-10-08T13:49:49.163652+0000","last_active":"2023-10-08T13:51:15.788141+0000","last_peered":"2023-10-08T13:51:15.788141+0000","last_clean":"2023-10-08T13:51:15.788141+0000","last_became_active":"2023-10-08T13:49:49.163140+0000","last_became_peered":"2023-10-08T13:49:49.163140+0000","last_unstale":"2023-10-08T13:51:15.788141+0000","last_undegraded":"2023-10-08T13:51:15.788141+0000","last_fullsized":"2023-10-08T13:51:15.788141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:48.733675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809171+0000","last_change":"2023-10-08T13:49:43.775799+0000","last_active":"2023-10-08T13:51:15.809171+0000","last_peered":"2023-10-08T13:51:15.809171+0000","last_clean":"2023-10-08T13:51:15.809171+0000","last_became_active":"2023-10-08T13:49:43.775470+0000","last_became_peered":"2023-10-08T13:49:43.775470+0000","last_unstale":"2023-10-08T13:51:15.809171+0000","last_undegraded":"2023-10-08T13:51:15.809171+0000","last_fullsized":"2023-10-08T13:51:15.809171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:50.404554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784980+0000","last_change":"2023-10-08T13:50:15.785798+0000","last_active":"2023-10-08T13:51:15.784980+0000","last_peered":"2023-10-08T13:51:15.784980+0000","last_clean":"2023-10-08T13:51:15.784980+0000","last_became_active":"2023-10-08T13:50:15.785343+0000","last_became_peered":"2023-10-08T13:50:15.785343+0000","last_unstale":"2023-10-08T13:51:15.784980+0000","last_undegraded":"2023-10-08T13:51:15.784980+0000","last_fullsized":"2023-10-08T13:51:15.784980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:36.717707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808771+0000","last_change":"2023-10-08T13:49:49.166702+0000","last_active":"2023-10-08T13:51:15.808771+0000","last_peered":"2023-10-08T13:51:15.808771+0000","last_clean":"2023-10-08T13:51:15.808771+0000","last_became_active":"2023-10-08T13:49:49.166256+0000","last_became_peered":"2023-10-08T13:49:49.166256+0000","last_unstale":"2023-10-08T13:51:15.808771+0000","last_undegraded":"2023-10-08T13:51:15.808771+0000","last_fullsized":"2023-10-08T13:51:15.808771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:18:57.565040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808332+0000","last_change":"2023-10-08T13:49:43.785085+0000","last_active":"2023-10-08T13:51:15.808332+0000","last_peered":"2023-10-08T13:51:15.808332+0000","last_clean":"2023-10-08T13:51:15.808332+0000","last_became_active":"2023-10-08T13:49:43.784746+0000","last_became_peered":"2023-10-08T13:49:43.784746+0000","last_unstale":"2023-10-08T13:51:15.808332+0000","last_undegraded":"2023-10-08T13:51:15.808332+0000","last_fullsized":"2023-10-08T13:51:15.808332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:13.471834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787752+0000","last_change":"2023-10-08T13:49:44.785670+0000","last_active":"2023-10-08T13:51:15.787752+0000","last_peered":"2023-10-08T13:51:15.787752+0000","last_clean":"2023-10-08T13:51:15.787752+0000","last_became_active":"2023-10-08T13:49:44.785221+0000","last_became_peered":"2023-10-08T13:49:44.785221+0000","last_unstale":"2023-10-08T13:51:15.787752+0000","last_undegraded":"2023-10-08T13:51:15.787752+0000","last_fullsized":"2023-10-08T13:51:15.787752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:25.415168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808870+0000","last_change":"2023-10-08T13:49:49.156294+0000","last_active":"2023-10-08T13:51:15.808870+0000","last_peered":"2023-10-08T13:51:15.808870+0000","last_clean":"2023-10-08T13:51:15.808870+0000","last_became_active":"2023-10-08T13:49:49.155821+0000","last_became_peered":"2023-10-08T13:49:49.155821+0000","last_unstale":"2023-10-08T13:51:15.808870+0000","last_undegraded":"2023-10-08T13:51:15.808870+0000","last_fullsized":"2023-10-08T13:51:15.808870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:59:43.831634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784681+0000","last_change":"2023-10-08T13:49:43.780052+0000","last_active":"2023-10-08T13:51:15.784681+0000","last_peered":"2023-10-08T13:51:15.784681+0000","last_clean":"2023-10-08T13:51:15.784681+0000","last_became_active":"2023-10-08T13:49:43.779731+0000","last_became_peered":"2023-10-08T13:49:43.779731+0000","last_unstale":"2023-10-08T13:51:15.784681+0000","last_undegraded":"2023-10-08T13:51:15.784681+0000","last_fullsized":"2023-10-08T13:51:15.784681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:38.800783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785843+0000","last_change":"2023-10-08T13:49:44.787880+0000","last_active":"2023-10-08T13:51:15.785843+0000","last_peered":"2023-10-08T13:51:15.785843+0000","last_clean":"2023-10-08T13:51:15.785843+0000","last_became_active":"2023-10-08T13:49:44.787591+0000","last_became_peered":"2023-10-08T13:49:44.787591+0000","last_unstale":"2023-10-08T13:51:15.785843+0000","last_undegraded":"2023-10-08T13:51:15.785843+0000","last_fullsized":"2023-10-08T13:51:15.785843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:53.078903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788057+0000","last_change":"2023-10-08T13:49:49.155004+0000","last_active":"2023-10-08T13:51:15.788057+0000","last_peered":"2023-10-08T13:51:15.788057+0000","last_clean":"2023-10-08T13:51:15.788057+0000","last_became_active":"2023-10-08T13:49:49.154503+0000","last_became_peered":"2023-10-08T13:49:49.154503+0000","last_unstale":"2023-10-08T13:51:15.788057+0000","last_undegraded":"2023-10-08T13:51:15.788057+0000","last_fullsized":"2023-10-08T13:51:15.788057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:21.565816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":174,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.851308+0000","last_change":"2023-10-08T13:49:43.775819+0000","last_active":"2023-10-08T13:51:21.851308+0000","last_peered":"2023-10-08T13:51:21.851308+0000","last_clean":"2023-10-08T13:51:21.851308+0000","last_became_active":"2023-10-08T13:49:43.775578+0000","last_became_peered":"2023-10-08T13:49:43.775578+0000","last_unstale":"2023-10-08T13:51:21.851308+0000","last_undegraded":"2023-10-08T13:51:21.851308+0000","last_fullsized":"2023-10-08T13:51:21.851308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:31.518191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787270+0000","last_change":"2023-10-08T13:49:44.785869+0000","last_active":"2023-10-08T13:51:15.787270+0000","last_peered":"2023-10-08T13:51:15.787270+0000","last_clean":"2023-10-08T13:51:15.787270+0000","last_became_active":"2023-10-08T13:49:44.785566+0000","last_became_peered":"2023-10-08T13:49:44.785566+0000","last_unstale":"2023-10-08T13:51:15.787270+0000","last_undegraded":"2023-10-08T13:51:15.787270+0000","last_fullsized":"2023-10-08T13:51:15.787270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:29.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785538+0000","last_change":"2023-10-08T13:49:49.162075+0000","last_active":"2023-10-08T13:51:15.785538+0000","last_peered":"2023-10-08T13:51:15.785538+0000","last_clean":"2023-10-08T13:51:15.785538+0000","last_became_active":"2023-10-08T13:49:49.161485+0000","last_became_peered":"2023-10-08T13:49:49.161485+0000","last_unstale":"2023-10-08T13:51:15.785538+0000","last_undegraded":"2023-10-08T13:51:15.785538+0000","last_fullsized":"2023-10-08T13:51:15.785538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:16.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808954+0000","last_change":"2023-10-08T13:49:43.776555+0000","last_active":"2023-10-08T13:51:15.808954+0000","last_peered":"2023-10-08T13:51:15.808954+0000","last_clean":"2023-10-08T13:51:15.808954+0000","last_became_active":"2023-10-08T13:49:43.776259+0000","last_became_peered":"2023-10-08T13:49:43.776259+0000","last_unstale":"2023-10-08T13:51:15.808954+0000","last_undegraded":"2023-10-08T13:51:15.808954+0000","last_fullsized":"2023-10-08T13:51:15.808954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:34.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809044+0000","last_change":"2023-10-08T13:49:44.779253+0000","last_active":"2023-10-08T13:51:15.809044+0000","last_peered":"2023-10-08T13:51:15.809044+0000","last_clean":"2023-10-08T13:51:15.809044+0000","last_became_active":"2023-10-08T13:49:44.778902+0000","last_became_peered":"2023-10-08T13:49:44.778902+0000","last_unstale":"2023-10-08T13:51:15.809044+0000","last_undegraded":"2023-10-08T13:51:15.809044+0000","last_fullsized":"2023-10-08T13:51:15.809044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:31:01.091424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"88'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782932+0000","last_change":"2023-10-08T13:49:49.164859+0000","last_active":"2023-10-08T13:51:15.782932+0000","last_peered":"2023-10-08T13:51:15.782932+0000","last_clean":"2023-10-08T13:51:15.782932+0000","last_became_active":"2023-10-08T13:49:49.164221+0000","last_became_peered":"2023-10-08T13:49:49.164221+0000","last_unstale":"2023-10-08T13:51:15.782932+0000","last_undegraded":"2023-10-08T13:51:15.782932+0000","last_fullsized":"2023-10-08T13:51:15.782932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:22.484934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785106+0000","last_change":"2023-10-08T13:49:43.788136+0000","last_active":"2023-10-08T13:51:15.785106+0000","last_peered":"2023-10-08T13:51:15.785106+0000","last_clean":"2023-10-08T13:51:15.785106+0000","last_became_active":"2023-10-08T13:49:43.787910+0000","last_became_peered":"2023-10-08T13:49:43.787910+0000","last_unstale":"2023-10-08T13:51:15.785106+0000","last_undegraded":"2023-10-08T13:51:15.785106+0000","last_fullsized":"2023-10-08T13:51:15.785106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:00:32.897972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787049+0000","last_change":"2023-10-08T13:49:44.786833+0000","last_active":"2023-10-08T13:51:15.787049+0000","last_peered":"2023-10-08T13:51:15.787049+0000","last_clean":"2023-10-08T13:51:15.787049+0000","last_became_active":"2023-10-08T13:49:44.786523+0000","last_became_peered":"2023-10-08T13:49:44.786523+0000","last_unstale":"2023-10-08T13:51:15.787049+0000","last_undegraded":"2023-10-08T13:51:15.787049+0000","last_fullsized":"2023-10-08T13:51:15.787049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:36:22.156252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785774+0000","last_change":"2023-10-08T13:50:15.977115+0000","last_active":"2023-10-08T13:51:15.785774+0000","last_peered":"2023-10-08T13:51:15.785774+0000","last_clean":"2023-10-08T13:51:15.785774+0000","last_became_active":"2023-10-08T13:50:15.976617+0000","last_became_peered":"2023-10-08T13:50:15.976617+0000","last_unstale":"2023-10-08T13:51:15.785774+0000","last_undegraded":"2023-10-08T13:51:15.785774+0000","last_fullsized":"2023-10-08T13:51:15.785774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:57.124054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'36","reported_seq":214,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.882495+0000","last_change":"2023-10-08T13:49:43.777736+0000","last_active":"2023-10-08T13:51:21.882495+0000","last_peered":"2023-10-08T13:51:21.882495+0000","last_clean":"2023-10-08T13:51:21.882495+0000","last_became_active":"2023-10-08T13:49:43.777431+0000","last_became_peered":"2023-10-08T13:49:43.777431+0000","last_unstale":"2023-10-08T13:51:21.882495+0000","last_undegraded":"2023-10-08T13:51:21.882495+0000","last_fullsized":"2023-10-08T13:51:21.882495+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:12.907196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59058,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":158,"num_read_kb":136,"num_write":38,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783122+0000","last_change":"2023-10-08T13:49:44.789122+0000","last_active":"2023-10-08T13:51:15.783122+0000","last_peered":"2023-10-08T13:51:15.783122+0000","last_clean":"2023-10-08T13:51:15.783122+0000","last_became_active":"2023-10-08T13:49:44.788831+0000","last_became_peered":"2023-10-08T13:49:44.788831+0000","last_unstale":"2023-10-08T13:51:15.783122+0000","last_undegraded":"2023-10-08T13:51:15.783122+0000","last_fullsized":"2023-10-08T13:51:15.783122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:45.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785476+0000","last_change":"2023-10-08T13:49:49.157319+0000","last_active":"2023-10-08T13:51:15.785476+0000","last_peered":"2023-10-08T13:51:15.785476+0000","last_clean":"2023-10-08T13:51:15.785476+0000","last_became_active":"2023-10-08T13:49:49.156897+0000","last_became_peered":"2023-10-08T13:49:49.156897+0000","last_unstale":"2023-10-08T13:51:15.785476+0000","last_undegraded":"2023-10-08T13:51:15.785476+0000","last_fullsized":"2023-10-08T13:51:15.785476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:25.460707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787914+0000","last_change":"2023-10-08T13:49:43.776699+0000","last_active":"2023-10-08T13:51:15.787914+0000","last_peered":"2023-10-08T13:51:15.787914+0000","last_clean":"2023-10-08T13:51:15.787914+0000","last_became_active":"2023-10-08T13:49:43.776293+0000","last_became_peered":"2023-10-08T13:49:43.776293+0000","last_unstale":"2023-10-08T13:51:15.787914+0000","last_undegraded":"2023-10-08T13:51:15.787914+0000","last_fullsized":"2023-10-08T13:51:15.787914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:19.056804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808734+0000","last_change":"2023-10-08T13:49:44.789915+0000","last_active":"2023-10-08T13:51:15.808734+0000","last_peered":"2023-10-08T13:51:15.808734+0000","last_clean":"2023-10-08T13:51:15.808734+0000","last_became_active":"2023-10-08T13:49:44.789617+0000","last_became_peered":"2023-10-08T13:49:44.789617+0000","last_unstale":"2023-10-08T13:51:15.808734+0000","last_undegraded":"2023-10-08T13:51:15.808734+0000","last_fullsized":"2023-10-08T13:51:15.808734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:06.286155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808527+0000","last_change":"2023-10-08T13:49:49.165946+0000","last_active":"2023-10-08T13:51:15.808527+0000","last_peered":"2023-10-08T13:51:15.808527+0000","last_clean":"2023-10-08T13:51:15.808527+0000","last_became_active":"2023-10-08T13:49:49.165581+0000","last_became_peered":"2023-10-08T13:49:49.165581+0000","last_unstale":"2023-10-08T13:51:15.808527+0000","last_undegraded":"2023-10-08T13:51:15.808527+0000","last_fullsized":"2023-10-08T13:51:15.808527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:13.772198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784836+0000","last_change":"2023-10-08T13:49:43.779161+0000","last_active":"2023-10-08T13:51:15.784836+0000","last_peered":"2023-10-08T13:51:15.784836+0000","last_clean":"2023-10-08T13:51:15.784836+0000","last_became_active":"2023-10-08T13:49:43.778862+0000","last_became_peered":"2023-10-08T13:49:43.778862+0000","last_unstale":"2023-10-08T13:51:15.784836+0000","last_undegraded":"2023-10-08T13:51:15.784836+0000","last_fullsized":"2023-10-08T13:51:15.784836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:57:44.075322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809115+0000","last_change":"2023-10-08T13:49:44.782072+0000","last_active":"2023-10-08T13:51:15.809115+0000","last_peered":"2023-10-08T13:51:15.809115+0000","last_clean":"2023-10-08T13:51:15.809115+0000","last_became_active":"2023-10-08T13:49:44.781756+0000","last_became_peered":"2023-10-08T13:49:44.781756+0000","last_unstale":"2023-10-08T13:51:15.809115+0000","last_undegraded":"2023-10-08T13:51:15.809115+0000","last_fullsized":"2023-10-08T13:51:15.809115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:32.877818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783652+0000","last_change":"2023-10-08T13:50:16.099755+0000","last_active":"2023-10-08T13:51:15.783652+0000","last_peered":"2023-10-08T13:51:15.783652+0000","last_clean":"2023-10-08T13:51:15.783652+0000","last_became_active":"2023-10-08T13:50:16.099267+0000","last_became_peered":"2023-10-08T13:50:16.099267+0000","last_unstale":"2023-10-08T13:51:15.783652+0000","last_undegraded":"2023-10-08T13:51:15.783652+0000","last_fullsized":"2023-10-08T13:51:15.783652+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:19.949246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.860207+0000","last_change":"2023-10-08T13:49:43.782822+0000","last_active":"2023-10-08T13:51:21.860207+0000","last_peered":"2023-10-08T13:51:21.860207+0000","last_clean":"2023-10-08T13:51:21.860207+0000","last_became_active":"2023-10-08T13:49:43.782524+0000","last_became_peered":"2023-10-08T13:49:43.782524+0000","last_unstale":"2023-10-08T13:51:21.860207+0000","last_undegraded":"2023-10-08T13:51:21.860207+0000","last_fullsized":"2023-10-08T13:51:21.860207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:11.847851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808786+0000","last_change":"2023-10-08T13:49:44.794296+0000","last_active":"2023-10-08T13:51:15.808786+0000","last_peered":"2023-10-08T13:51:15.808786+0000","last_clean":"2023-10-08T13:51:15.808786+0000","last_became_active":"2023-10-08T13:49:44.793987+0000","last_became_peered":"2023-10-08T13:49:44.793987+0000","last_unstale":"2023-10-08T13:51:15.808786+0000","last_undegraded":"2023-10-08T13:51:15.808786+0000","last_fullsized":"2023-10-08T13:51:15.808786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:41.562569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809550+0000","last_change":"2023-10-08T13:49:49.165016+0000","last_active":"2023-10-08T13:51:15.809550+0000","last_peered":"2023-10-08T13:51:15.809550+0000","last_clean":"2023-10-08T13:51:15.809550+0000","last_became_active":"2023-10-08T13:49:49.164531+0000","last_became_peered":"2023-10-08T13:49:49.164531+0000","last_unstale":"2023-10-08T13:51:15.809550+0000","last_undegraded":"2023-10-08T13:51:15.809550+0000","last_fullsized":"2023-10-08T13:51:15.809550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:45.014512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782774+0000","last_change":"2023-10-08T13:49:43.786929+0000","last_active":"2023-10-08T13:51:15.782774+0000","last_peered":"2023-10-08T13:51:15.782774+0000","last_clean":"2023-10-08T13:51:15.782774+0000","last_became_active":"2023-10-08T13:49:43.786641+0000","last_became_peered":"2023-10-08T13:49:43.786641+0000","last_unstale":"2023-10-08T13:51:15.782774+0000","last_undegraded":"2023-10-08T13:51:15.782774+0000","last_fullsized":"2023-10-08T13:51:15.782774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:30.054067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:16.851434+0000","last_change":"2023-10-08T13:51:16.850958+0000","last_active":"2023-10-08T13:51:16.851434+0000","last_peered":"2023-10-08T13:51:16.851434+0000","last_clean":"2023-10-08T13:51:16.851434+0000","last_became_active":"2023-10-08T13:51:16.850648+0000","last_became_peered":"2023-10-08T13:51:16.850648+0000","last_unstale":"2023-10-08T13:51:16.851434+0000","last_undegraded":"2023-10-08T13:51:16.851434+0000","last_fullsized":"2023-10-08T13:51:16.851434+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:30.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784760+0000","last_change":"2023-10-08T13:49:49.152226+0000","last_active":"2023-10-08T13:51:15.784760+0000","last_peered":"2023-10-08T13:51:15.784760+0000","last_clean":"2023-10-08T13:51:15.784760+0000","last_became_active":"2023-10-08T13:49:49.151772+0000","last_became_peered":"2023-10-08T13:49:49.151772+0000","last_unstale":"2023-10-08T13:51:15.784760+0000","last_undegraded":"2023-10-08T13:51:15.784760+0000","last_fullsized":"2023-10-08T13:51:15.784760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:53.856496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808693+0000","last_change":"2023-10-08T13:50:15.793598+0000","last_active":"2023-10-08T13:51:15.808693+0000","last_peered":"2023-10-08T13:51:15.808693+0000","last_clean":"2023-10-08T13:51:15.808693+0000","last_became_active":"2023-10-08T13:50:15.786421+0000","last_became_peered":"2023-10-08T13:50:15.786421+0000","last_unstale":"2023-10-08T13:51:15.808693+0000","last_undegraded":"2023-10-08T13:51:15.808693+0000","last_fullsized":"2023-10-08T13:51:15.808693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:09:37.586245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808647+0000","last_change":"2023-10-08T13:49:44.782805+0000","last_active":"2023-10-08T13:51:15.808647+0000","last_peered":"2023-10-08T13:51:15.808647+0000","last_clean":"2023-10-08T13:51:15.808647+0000","last_became_active":"2023-10-08T13:49:44.782485+0000","last_became_peered":"2023-10-08T13:49:44.782485+0000","last_unstale":"2023-10-08T13:51:15.808647+0000","last_undegraded":"2023-10-08T13:51:15.808647+0000","last_fullsized":"2023-10-08T13:51:15.808647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:19.805734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783005+0000","last_change":"2023-10-08T13:49:49.546226+0000","last_active":"2023-10-08T13:51:15.783005+0000","last_peered":"2023-10-08T13:51:15.783005+0000","last_clean":"2023-10-08T13:51:15.783005+0000","last_became_active":"2023-10-08T13:49:49.545626+0000","last_became_peered":"2023-10-08T13:49:49.545626+0000","last_unstale":"2023-10-08T13:51:15.783005+0000","last_undegraded":"2023-10-08T13:51:15.783005+0000","last_fullsized":"2023-10-08T13:51:15.783005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:37.184944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786007+0000","last_change":"2023-10-08T13:49:43.784705+0000","last_active":"2023-10-08T13:51:15.786007+0000","last_peered":"2023-10-08T13:51:15.786007+0000","last_clean":"2023-10-08T13:51:15.786007+0000","last_became_active":"2023-10-08T13:49:43.784391+0000","last_became_peered":"2023-10-08T13:49:43.784391+0000","last_unstale":"2023-10-08T13:51:15.786007+0000","last_undegraded":"2023-10-08T13:51:15.786007+0000","last_fullsized":"2023-10-08T13:51:15.786007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:45.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.810046+0000","last_change":"2023-10-08T13:49:44.789845+0000","last_active":"2023-10-08T13:51:15.810046+0000","last_peered":"2023-10-08T13:51:15.810046+0000","last_clean":"2023-10-08T13:51:15.810046+0000","last_became_active":"2023-10-08T13:49:44.789580+0000","last_became_peered":"2023-10-08T13:49:44.789580+0000","last_unstale":"2023-10-08T13:51:15.810046+0000","last_undegraded":"2023-10-08T13:51:15.810046+0000","last_fullsized":"2023-10-08T13:51:15.810046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:58.098757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809069+0000","last_change":"2023-10-08T13:49:49.153277+0000","last_active":"2023-10-08T13:51:15.809069+0000","last_peered":"2023-10-08T13:51:15.809069+0000","last_clean":"2023-10-08T13:51:15.809069+0000","last_became_active":"2023-10-08T13:49:49.152575+0000","last_became_peered":"2023-10-08T13:49:49.152575+0000","last_unstale":"2023-10-08T13:51:15.809069+0000","last_undegraded":"2023-10-08T13:51:15.809069+0000","last_fullsized":"2023-10-08T13:51:15.809069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:35.677823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.852851+0000","last_change":"2023-10-08T13:49:43.778442+0000","last_active":"2023-10-08T13:51:21.852851+0000","last_peered":"2023-10-08T13:51:21.852851+0000","last_clean":"2023-10-08T13:51:21.852851+0000","last_became_active":"2023-10-08T13:49:43.774799+0000","last_became_peered":"2023-10-08T13:49:43.774799+0000","last_unstale":"2023-10-08T13:51:21.852851+0000","last_undegraded":"2023-10-08T13:51:21.852851+0000","last_fullsized":"2023-10-08T13:51:21.852851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:15.464736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809560+0000","last_change":"2023-10-08T13:49:44.780097+0000","last_active":"2023-10-08T13:51:15.809560+0000","last_peered":"2023-10-08T13:51:15.809560+0000","last_clean":"2023-10-08T13:51:15.809560+0000","last_became_active":"2023-10-08T13:49:44.779774+0000","last_became_peered":"2023-10-08T13:49:44.779774+0000","last_unstale":"2023-10-08T13:51:15.809560+0000","last_undegraded":"2023-10-08T13:51:15.809560+0000","last_fullsized":"2023-10-08T13:51:15.809560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:19.132295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808399+0000","last_change":"2023-10-08T13:49:49.547129+0000","last_active":"2023-10-08T13:51:15.808399+0000","last_peered":"2023-10-08T13:51:15.808399+0000","last_clean":"2023-10-08T13:51:15.808399+0000","last_became_active":"2023-10-08T13:49:49.546694+0000","last_became_peered":"2023-10-08T13:49:49.546694+0000","last_unstale":"2023-10-08T13:51:15.808399+0000","last_undegraded":"2023-10-08T13:51:15.808399+0000","last_fullsized":"2023-10-08T13:51:15.808399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:45.524650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784878+0000","last_change":"2023-10-08T13:49:44.779875+0000","last_active":"2023-10-08T13:51:15.784878+0000","last_peered":"2023-10-08T13:51:15.784878+0000","last_clean":"2023-10-08T13:51:15.784878+0000","last_became_active":"2023-10-08T13:49:44.779570+0000","last_became_peered":"2023-10-08T13:49:44.779570+0000","last_unstale":"2023-10-08T13:51:15.784878+0000","last_undegraded":"2023-10-08T13:51:15.784878+0000","last_fullsized":"2023-10-08T13:51:15.784878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:02.981998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:23.862244+0000","last_change":"2023-10-08T13:49:43.787574+0000","last_active":"2023-10-08T13:51:23.862244+0000","last_peered":"2023-10-08T13:51:23.862244+0000","last_clean":"2023-10-08T13:51:23.862244+0000","last_became_active":"2023-10-08T13:49:43.787261+0000","last_became_peered":"2023-10-08T13:49:43.787261+0000","last_unstale":"2023-10-08T13:51:23.862244+0000","last_undegraded":"2023-10-08T13:51:23.862244+0000","last_fullsized":"2023-10-08T13:51:23.862244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:28.676135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809431+0000","last_change":"2023-10-08T13:49:49.163856+0000","last_active":"2023-10-08T13:51:15.809431+0000","last_peered":"2023-10-08T13:51:15.809431+0000","last_clean":"2023-10-08T13:51:15.809431+0000","last_became_active":"2023-10-08T13:49:49.163456+0000","last_became_peered":"2023-10-08T13:49:49.163456+0000","last_unstale":"2023-10-08T13:51:15.809431+0000","last_undegraded":"2023-10-08T13:51:15.809431+0000","last_fullsized":"2023-10-08T13:51:15.809431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:31.654309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788320+0000","last_change":"2023-10-08T13:49:44.792935+0000","last_active":"2023-10-08T13:51:15.788320+0000","last_peered":"2023-10-08T13:51:15.788320+0000","last_clean":"2023-10-08T13:51:15.788320+0000","last_became_active":"2023-10-08T13:49:44.792667+0000","last_became_peered":"2023-10-08T13:49:44.792667+0000","last_unstale":"2023-10-08T13:51:15.788320+0000","last_undegraded":"2023-10-08T13:51:15.788320+0000","last_fullsized":"2023-10-08T13:51:15.788320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:40.964452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783131+0000","last_change":"2023-10-08T13:49:43.781424+0000","last_active":"2023-10-08T13:51:15.783131+0000","last_peered":"2023-10-08T13:51:15.783131+0000","last_clean":"2023-10-08T13:51:15.783131+0000","last_became_active":"2023-10-08T13:49:43.780930+0000","last_became_peered":"2023-10-08T13:49:43.780930+0000","last_unstale":"2023-10-08T13:51:15.783131+0000","last_undegraded":"2023-10-08T13:51:15.783131+0000","last_fullsized":"2023-10-08T13:51:15.783131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:56:50.397289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808562+0000","last_change":"2023-10-08T13:48:45.192219+0000","last_active":"2023-10-08T13:51:15.808562+0000","last_peered":"2023-10-08T13:51:15.808562+0000","last_clean":"2023-10-08T13:51:15.808562+0000","last_became_active":"2023-10-08T13:48:45.191813+0000","last_became_peered":"2023-10-08T13:48:45.191813+0000","last_unstale":"2023-10-08T13:51:15.808562+0000","last_undegraded":"2023-10-08T13:51:15.808562+0000","last_fullsized":"2023-10-08T13:51:15.808562+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_clean_scrub_stamp":"2023-10-08T13:45:51.040695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.303937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785025+0000","last_change":"2023-10-08T13:49:49.546181+0000","last_active":"2023-10-08T13:51:15.785025+0000","last_peered":"2023-10-08T13:51:15.785025+0000","last_clean":"2023-10-08T13:51:15.785025+0000","last_became_active":"2023-10-08T13:49:49.545726+0000","last_became_peered":"2023-10-08T13:49:49.545726+0000","last_unstale":"2023-10-08T13:51:15.785025+0000","last_undegraded":"2023-10-08T13:51:15.785025+0000","last_fullsized":"2023-10-08T13:51:15.785025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:45.370909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809633+0000","last_change":"2023-10-08T13:49:44.785544+0000","last_active":"2023-10-08T13:51:15.809633+0000","last_peered":"2023-10-08T13:51:15.809633+0000","last_clean":"2023-10-08T13:51:15.809633+0000","last_became_active":"2023-10-08T13:49:44.785007+0000","last_became_peered":"2023-10-08T13:49:44.785007+0000","last_unstale":"2023-10-08T13:51:15.809633+0000","last_undegraded":"2023-10-08T13:51:15.809633+0000","last_fullsized":"2023-10-08T13:51:15.809633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:16:55.058693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808677+0000","last_change":"2023-10-08T13:49:43.779207+0000","last_active":"2023-10-08T13:51:15.808677+0000","last_peered":"2023-10-08T13:51:15.808677+0000","last_clean":"2023-10-08T13:51:15.808677+0000","last_became_active":"2023-10-08T13:49:43.778846+0000","last_became_peered":"2023-10-08T13:49:43.778846+0000","last_unstale":"2023-10-08T13:51:15.808677+0000","last_undegraded":"2023-10-08T13:51:15.808677+0000","last_fullsized":"2023-10-08T13:51:15.808677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:45.868187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808454+0000","last_change":"2023-10-08T13:49:49.162156+0000","last_active":"2023-10-08T13:51:15.808454+0000","last_peered":"2023-10-08T13:51:15.808454+0000","last_clean":"2023-10-08T13:51:15.808454+0000","last_became_active":"2023-10-08T13:49:49.161720+0000","last_became_peered":"2023-10-08T13:49:49.161720+0000","last_unstale":"2023-10-08T13:51:15.808454+0000","last_undegraded":"2023-10-08T13:51:15.808454+0000","last_fullsized":"2023-10-08T13:51:15.808454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:13.499915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783327+0000","last_change":"2023-10-08T13:49:43.782322+0000","last_active":"2023-10-08T13:51:15.783327+0000","last_peered":"2023-10-08T13:51:15.783327+0000","last_clean":"2023-10-08T13:51:15.783327+0000","last_became_active":"2023-10-08T13:49:43.781938+0000","last_became_peered":"2023-10-08T13:49:43.781938+0000","last_unstale":"2023-10-08T13:51:15.783327+0000","last_undegraded":"2023-10-08T13:51:15.783327+0000","last_fullsized":"2023-10-08T13:51:15.783327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:57.627507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809219+0000","last_change":"2023-10-08T13:49:44.778340+0000","last_active":"2023-10-08T13:51:15.809219+0000","last_peered":"2023-10-08T13:51:15.809219+0000","last_clean":"2023-10-08T13:51:15.809219+0000","last_became_active":"2023-10-08T13:49:44.778000+0000","last_became_peered":"2023-10-08T13:49:44.778000+0000","last_unstale":"2023-10-08T13:51:15.809219+0000","last_undegraded":"2023-10-08T13:51:15.809219+0000","last_fullsized":"2023-10-08T13:51:15.809219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:23.350449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787735+0000","last_change":"2023-10-08T13:49:49.545915+0000","last_active":"2023-10-08T13:51:15.787735+0000","last_peered":"2023-10-08T13:51:15.787735+0000","last_clean":"2023-10-08T13:51:15.787735+0000","last_became_active":"2023-10-08T13:49:49.545396+0000","last_became_peered":"2023-10-08T13:49:49.545396+0000","last_unstale":"2023-10-08T13:51:15.787735+0000","last_undegraded":"2023-10-08T13:51:15.787735+0000","last_fullsized":"2023-10-08T13:51:15.787735+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:07:03.850707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:16.798882+0000","last_change":"2023-10-08T13:49:43.776479+0000","last_active":"2023-10-08T13:51:16.798882+0000","last_peered":"2023-10-08T13:51:16.798882+0000","last_clean":"2023-10-08T13:51:16.798882+0000","last_became_active":"2023-10-08T13:49:43.776130+0000","last_became_peered":"2023-10-08T13:49:43.776130+0000","last_unstale":"2023-10-08T13:51:16.798882+0000","last_undegraded":"2023-10-08T13:51:16.798882+0000","last_fullsized":"2023-10-08T13:51:16.798882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.760660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788025+0000","last_change":"2023-10-08T13:49:44.793864+0000","last_active":"2023-10-08T13:51:15.788025+0000","last_peered":"2023-10-08T13:51:15.788025+0000","last_clean":"2023-10-08T13:51:15.788025+0000","last_became_active":"2023-10-08T13:49:44.793469+0000","last_became_peered":"2023-10-08T13:49:44.793469+0000","last_unstale":"2023-10-08T13:51:15.788025+0000","last_undegraded":"2023-10-08T13:51:15.788025+0000","last_fullsized":"2023-10-08T13:51:15.788025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:28.669711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787773+0000","last_change":"2023-10-08T13:49:49.150696+0000","last_active":"2023-10-08T13:51:15.787773+0000","last_peered":"2023-10-08T13:51:15.787773+0000","last_clean":"2023-10-08T13:51:15.787773+0000","last_became_active":"2023-10-08T13:49:49.150188+0000","last_became_peered":"2023-10-08T13:49:49.150188+0000","last_unstale":"2023-10-08T13:51:15.787773+0000","last_undegraded":"2023-10-08T13:51:15.787773+0000","last_fullsized":"2023-10-08T13:51:15.787773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:51:35.775240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783459+0000","last_change":"2023-10-08T13:49:43.786245+0000","last_active":"2023-10-08T13:51:15.783459+0000","last_peered":"2023-10-08T13:51:15.783459+0000","last_clean":"2023-10-08T13:51:15.783459+0000","last_became_active":"2023-10-08T13:49:43.785986+0000","last_became_peered":"2023-10-08T13:49:43.785986+0000","last_unstale":"2023-10-08T13:51:15.783459+0000","last_undegraded":"2023-10-08T13:51:15.783459+0000","last_fullsized":"2023-10-08T13:51:15.783459+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:53:59.643447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808953+0000","last_change":"2023-10-08T13:49:44.790494+0000","last_active":"2023-10-08T13:51:15.808953+0000","last_peered":"2023-10-08T13:51:15.808953+0000","last_clean":"2023-10-08T13:51:15.808953+0000","last_became_active":"2023-10-08T13:49:44.790091+0000","last_became_peered":"2023-10-08T13:49:44.790091+0000","last_unstale":"2023-10-08T13:51:15.808953+0000","last_undegraded":"2023-10-08T13:51:15.808953+0000","last_fullsized":"2023-10-08T13:51:15.808953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:49.309597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782982+0000","last_change":"2023-10-08T13:49:49.547271+0000","last_active":"2023-10-08T13:51:15.782982+0000","last_peered":"2023-10-08T13:51:15.782982+0000","last_clean":"2023-10-08T13:51:15.782982+0000","last_became_active":"2023-10-08T13:49:49.546603+0000","last_became_peered":"2023-10-08T13:49:49.546603+0000","last_unstale":"2023-10-08T13:51:15.782982+0000","last_undegraded":"2023-10-08T13:51:15.782982+0000","last_fullsized":"2023-10-08T13:51:15.782982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:29.908321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786664+0000","last_change":"2023-10-08T13:49:43.788638+0000","last_active":"2023-10-08T13:51:15.786664+0000","last_peered":"2023-10-08T13:51:15.786664+0000","last_clean":"2023-10-08T13:51:15.786664+0000","last_became_active":"2023-10-08T13:49:43.788303+0000","last_became_peered":"2023-10-08T13:49:43.788303+0000","last_unstale":"2023-10-08T13:51:15.786664+0000","last_undegraded":"2023-10-08T13:51:15.786664+0000","last_fullsized":"2023-10-08T13:51:15.786664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:52.637775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782787+0000","last_change":"2023-10-08T13:49:44.777546+0000","last_active":"2023-10-08T13:51:15.782787+0000","last_peered":"2023-10-08T13:51:15.782787+0000","last_clean":"2023-10-08T13:51:15.782787+0000","last_became_active":"2023-10-08T13:49:44.777261+0000","last_became_peered":"2023-10-08T13:49:44.777261+0000","last_unstale":"2023-10-08T13:51:15.782787+0000","last_undegraded":"2023-10-08T13:51:15.782787+0000","last_fullsized":"2023-10-08T13:51:15.782787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:26:36.368188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783728+0000","last_change":"2023-10-08T13:50:15.787317+0000","last_active":"2023-10-08T13:51:15.783728+0000","last_peered":"2023-10-08T13:51:15.783728+0000","last_clean":"2023-10-08T13:51:15.783728+0000","last_became_active":"2023-10-08T13:50:15.786816+0000","last_became_peered":"2023-10-08T13:50:15.786816+0000","last_unstale":"2023-10-08T13:51:15.783728+0000","last_undegraded":"2023-10-08T13:51:15.783728+0000","last_fullsized":"2023-10-08T13:51:15.783728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:53.766803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788167+0000","last_change":"2023-10-08T13:49:43.776693+0000","last_active":"2023-10-08T13:51:15.788167+0000","last_peered":"2023-10-08T13:51:15.788167+0000","last_clean":"2023-10-08T13:51:15.788167+0000","last_became_active":"2023-10-08T13:49:43.776367+0000","last_became_peered":"2023-10-08T13:49:43.776367+0000","last_unstale":"2023-10-08T13:51:15.788167+0000","last_undegraded":"2023-10-08T13:51:15.788167+0000","last_fullsized":"2023-10-08T13:51:15.788167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:17.415011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808348+0000","last_change":"2023-10-08T13:49:44.779472+0000","last_active":"2023-10-08T13:51:15.808348+0000","last_peered":"2023-10-08T13:51:15.808348+0000","last_clean":"2023-10-08T13:51:15.808348+0000","last_became_active":"2023-10-08T13:49:44.779172+0000","last_became_peered":"2023-10-08T13:49:44.779172+0000","last_unstale":"2023-10-08T13:51:15.808348+0000","last_undegraded":"2023-10-08T13:51:15.808348+0000","last_fullsized":"2023-10-08T13:51:15.808348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:17.314243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782946+0000","last_change":"2023-10-08T13:49:49.164535+0000","last_active":"2023-10-08T13:51:15.782946+0000","last_peered":"2023-10-08T13:51:15.782946+0000","last_clean":"2023-10-08T13:51:15.782946+0000","last_became_active":"2023-10-08T13:49:49.163467+0000","last_became_peered":"2023-10-08T13:49:49.163467+0000","last_unstale":"2023-10-08T13:51:15.782946+0000","last_undegraded":"2023-10-08T13:51:15.782946+0000","last_fullsized":"2023-10-08T13:51:15.782946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:55.058656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787896+0000","last_change":"2023-10-08T13:49:43.775311+0000","last_active":"2023-10-08T13:51:15.787896+0000","last_peered":"2023-10-08T13:51:15.787896+0000","last_clean":"2023-10-08T13:51:15.787896+0000","last_became_active":"2023-10-08T13:49:43.775005+0000","last_became_peered":"2023-10-08T13:49:43.775005+0000","last_unstale":"2023-10-08T13:51:15.787896+0000","last_undegraded":"2023-10-08T13:51:15.787896+0000","last_fullsized":"2023-10-08T13:51:15.787896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:33.713395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788118+0000","last_change":"2023-10-08T13:49:44.792051+0000","last_active":"2023-10-08T13:51:15.788118+0000","last_peered":"2023-10-08T13:51:15.788118+0000","last_clean":"2023-10-08T13:51:15.788118+0000","last_became_active":"2023-10-08T13:49:44.790045+0000","last_became_peered":"2023-10-08T13:49:44.790045+0000","last_unstale":"2023-10-08T13:51:15.788118+0000","last_undegraded":"2023-10-08T13:51:15.788118+0000","last_fullsized":"2023-10-08T13:51:15.788118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:28.699799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782910+0000","last_change":"2023-10-08T13:49:49.546219+0000","last_active":"2023-10-08T13:51:15.782910+0000","last_peered":"2023-10-08T13:51:15.782910+0000","last_clean":"2023-10-08T13:51:15.782910+0000","last_became_active":"2023-10-08T13:49:49.545783+0000","last_became_peered":"2023-10-08T13:49:49.545783+0000","last_unstale":"2023-10-08T13:51:15.782910+0000","last_undegraded":"2023-10-08T13:51:15.782910+0000","last_fullsized":"2023-10-08T13:51:15.782910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:00.497908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782922+0000","last_change":"2023-10-08T13:49:43.776252+0000","last_active":"2023-10-08T13:51:15.782922+0000","last_peered":"2023-10-08T13:51:15.782922+0000","last_clean":"2023-10-08T13:51:15.782922+0000","last_became_active":"2023-10-08T13:49:43.775935+0000","last_became_peered":"2023-10-08T13:49:43.775935+0000","last_unstale":"2023-10-08T13:51:15.782922+0000","last_undegraded":"2023-10-08T13:51:15.782922+0000","last_fullsized":"2023-10-08T13:51:15.782922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:58.949365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809345+0000","last_change":"2023-10-08T13:49:44.788628+0000","last_active":"2023-10-08T13:51:15.809345+0000","last_peered":"2023-10-08T13:51:15.809345+0000","last_clean":"2023-10-08T13:51:15.809345+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:51:15.809345+0000","last_undegraded":"2023-10-08T13:51:15.809345+0000","last_fullsized":"2023-10-08T13:51:15.809345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:46:42.311296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786697+0000","last_change":"2023-10-08T13:49:49.153641+0000","last_active":"2023-10-08T13:51:15.786697+0000","last_peered":"2023-10-08T13:51:15.786697+0000","last_clean":"2023-10-08T13:51:15.786697+0000","last_became_active":"2023-10-08T13:49:49.153172+0000","last_became_peered":"2023-10-08T13:49:49.153172+0000","last_unstale":"2023-10-08T13:51:15.786697+0000","last_undegraded":"2023-10-08T13:51:15.786697+0000","last_fullsized":"2023-10-08T13:51:15.786697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:40:51.726343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:22.853599+0000","last_change":"2023-10-08T13:49:43.782683+0000","last_active":"2023-10-08T13:51:22.853599+0000","last_peered":"2023-10-08T13:51:22.853599+0000","last_clean":"2023-10-08T13:51:22.853599+0000","last_became_active":"2023-10-08T13:49:43.782269+0000","last_became_peered":"2023-10-08T13:49:43.782269+0000","last_unstale":"2023-10-08T13:51:22.853599+0000","last_undegraded":"2023-10-08T13:51:22.853599+0000","last_fullsized":"2023-10-08T13:51:22.853599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:50.850581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809118+0000","last_change":"2023-10-08T13:49:44.787894+0000","last_active":"2023-10-08T13:51:15.809118+0000","last_peered":"2023-10-08T13:51:15.809118+0000","last_clean":"2023-10-08T13:51:15.809118+0000","last_became_active":"2023-10-08T13:49:44.787599+0000","last_became_peered":"2023-10-08T13:49:44.787599+0000","last_unstale":"2023-10-08T13:51:15.809118+0000","last_undegraded":"2023-10-08T13:51:15.809118+0000","last_fullsized":"2023-10-08T13:51:15.809118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:17:15.131051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808379+0000","last_change":"2023-10-08T13:49:49.164256+0000","last_active":"2023-10-08T13:51:15.808379+0000","last_peered":"2023-10-08T13:51:15.808379+0000","last_clean":"2023-10-08T13:51:15.808379+0000","last_became_active":"2023-10-08T13:49:49.163793+0000","last_became_peered":"2023-10-08T13:49:49.163793+0000","last_unstale":"2023-10-08T13:51:15.808379+0000","last_undegraded":"2023-10-08T13:51:15.808379+0000","last_fullsized":"2023-10-08T13:51:15.808379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:19.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785320+0000","last_change":"2023-10-08T13:49:43.778311+0000","last_active":"2023-10-08T13:51:15.785320+0000","last_peered":"2023-10-08T13:51:15.785320+0000","last_clean":"2023-10-08T13:51:15.785320+0000","last_became_active":"2023-10-08T13:49:43.777984+0000","last_became_peered":"2023-10-08T13:49:43.777984+0000","last_unstale":"2023-10-08T13:51:15.785320+0000","last_undegraded":"2023-10-08T13:51:15.785320+0000","last_fullsized":"2023-10-08T13:51:15.785320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:56.890729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808931+0000","last_change":"2023-10-08T13:49:44.793424+0000","last_active":"2023-10-08T13:51:15.808931+0000","last_peered":"2023-10-08T13:51:15.808931+0000","last_clean":"2023-10-08T13:51:15.808931+0000","last_became_active":"2023-10-08T13:49:44.792825+0000","last_became_peered":"2023-10-08T13:49:44.792825+0000","last_unstale":"2023-10-08T13:51:15.808931+0000","last_undegraded":"2023-10-08T13:51:15.808931+0000","last_fullsized":"2023-10-08T13:51:15.808931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:39.463676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784486+0000","last_change":"2023-10-08T13:49:49.151915+0000","last_active":"2023-10-08T13:51:15.784486+0000","last_peered":"2023-10-08T13:51:15.784486+0000","last_clean":"2023-10-08T13:51:15.784486+0000","last_became_active":"2023-10-08T13:49:49.151434+0000","last_became_peered":"2023-10-08T13:49:49.151434+0000","last_unstale":"2023-10-08T13:51:15.784486+0000","last_undegraded":"2023-10-08T13:51:15.784486+0000","last_fullsized":"2023-10-08T13:51:15.784486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:54:07.631498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809499+0000","last_change":"2023-10-08T13:50:15.787243+0000","last_active":"2023-10-08T13:51:15.809499+0000","last_peered":"2023-10-08T13:51:15.809499+0000","last_clean":"2023-10-08T13:51:15.809499+0000","last_became_active":"2023-10-08T13:50:15.782305+0000","last_became_peered":"2023-10-08T13:50:15.782305+0000","last_unstale":"2023-10-08T13:51:15.809499+0000","last_undegraded":"2023-10-08T13:51:15.809499+0000","last_fullsized":"2023-10-08T13:51:15.809499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:08.520010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809254+0000","last_change":"2023-10-08T13:49:44.790949+0000","last_active":"2023-10-08T13:51:15.809254+0000","last_peered":"2023-10-08T13:51:15.809254+0000","last_clean":"2023-10-08T13:51:15.809254+0000","last_became_active":"2023-10-08T13:49:44.790645+0000","last_became_peered":"2023-10-08T13:49:44.790645+0000","last_unstale":"2023-10-08T13:51:15.809254+0000","last_undegraded":"2023-10-08T13:51:15.809254+0000","last_fullsized":"2023-10-08T13:51:15.809254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:48.890730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808397+0000","last_change":"2023-10-08T13:49:44.781165+0000","last_active":"2023-10-08T13:51:15.808397+0000","last_peered":"2023-10-08T13:51:15.808397+0000","last_clean":"2023-10-08T13:51:15.808397+0000","last_became_active":"2023-10-08T13:49:44.780828+0000","last_became_peered":"2023-10-08T13:49:44.780828+0000","last_unstale":"2023-10-08T13:51:15.808397+0000","last_undegraded":"2023-10-08T13:51:15.808397+0000","last_fullsized":"2023-10-08T13:51:15.808397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:04.259482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788103+0000","last_change":"2023-10-08T13:49:44.781271+0000","last_active":"2023-10-08T13:51:15.788103+0000","last_peered":"2023-10-08T13:51:15.788103+0000","last_clean":"2023-10-08T13:51:15.788103+0000","last_became_active":"2023-10-08T13:49:44.780912+0000","last_became_peered":"2023-10-08T13:49:44.780912+0000","last_unstale":"2023-10-08T13:51:15.788103+0000","last_undegraded":"2023-10-08T13:51:15.788103+0000","last_fullsized":"2023-10-08T13:51:15.788103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:17.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787876+0000","last_change":"2023-10-08T13:49:44.777613+0000","last_active":"2023-10-08T13:51:15.787876+0000","last_peered":"2023-10-08T13:51:15.787876+0000","last_clean":"2023-10-08T13:51:15.787876+0000","last_became_active":"2023-10-08T13:49:44.777304+0000","last_became_peered":"2023-10-08T13:49:44.777304+0000","last_unstale":"2023-10-08T13:51:15.787876+0000","last_undegraded":"2023-10-08T13:51:15.787876+0000","last_fullsized":"2023-10-08T13:51:15.787876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:45.482935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808572+0000","last_change":"2023-10-08T13:49:44.782685+0000","last_active":"2023-10-08T13:51:15.808572+0000","last_peered":"2023-10-08T13:51:15.808572+0000","last_clean":"2023-10-08T13:51:15.808572+0000","last_became_active":"2023-10-08T13:49:44.781816+0000","last_became_peered":"2023-10-08T13:49:44.781816+0000","last_unstale":"2023-10-08T13:51:15.808572+0000","last_undegraded":"2023-10-08T13:51:15.808572+0000","last_fullsized":"2023-10-08T13:51:15.808572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:53.781237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809306+0000","last_change":"2023-10-08T13:49:44.783973+0000","last_active":"2023-10-08T13:51:15.809306+0000","last_peered":"2023-10-08T13:51:15.809306+0000","last_clean":"2023-10-08T13:51:15.809306+0000","last_became_active":"2023-10-08T13:49:44.783648+0000","last_became_peered":"2023-10-08T13:49:44.783648+0000","last_unstale":"2023-10-08T13:51:15.809306+0000","last_undegraded":"2023-10-08T13:51:15.809306+0000","last_fullsized":"2023-10-08T13:51:15.809306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:01.253610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808667+0000","last_change":"2023-10-08T13:49:44.781848+0000","last_active":"2023-10-08T13:51:15.808667+0000","last_peered":"2023-10-08T13:51:15.808667+0000","last_clean":"2023-10-08T13:51:15.808667+0000","last_became_active":"2023-10-08T13:49:44.781542+0000","last_became_peered":"2023-10-08T13:49:44.781542+0000","last_unstale":"2023-10-08T13:51:15.808667+0000","last_undegraded":"2023-10-08T13:51:15.808667+0000","last_fullsized":"2023-10-08T13:51:15.808667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:26:38.726298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788486+0000","last_change":"2023-10-08T13:49:44.786541+0000","last_active":"2023-10-08T13:51:15.788486+0000","last_peered":"2023-10-08T13:51:15.788486+0000","last_clean":"2023-10-08T13:51:15.788486+0000","last_became_active":"2023-10-08T13:49:44.786214+0000","last_became_peered":"2023-10-08T13:49:44.786214+0000","last_unstale":"2023-10-08T13:51:15.788486+0000","last_undegraded":"2023-10-08T13:51:15.788486+0000","last_fullsized":"2023-10-08T13:51:15.788486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:30:18.342774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788081+0000","last_change":"2023-10-08T13:49:44.788445+0000","last_active":"2023-10-08T13:51:15.788081+0000","last_peered":"2023-10-08T13:51:15.788081+0000","last_clean":"2023-10-08T13:51:15.788081+0000","last_became_active":"2023-10-08T13:49:44.788075+0000","last_became_peered":"2023-10-08T13:49:44.788075+0000","last_unstale":"2023-10-08T13:51:15.788081+0000","last_undegraded":"2023-10-08T13:51:15.788081+0000","last_fullsized":"2023-10-08T13:51:15.788081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:08:20.701231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784625+0000","last_change":"2023-10-08T13:49:44.783755+0000","last_active":"2023-10-08T13:51:15.784625+0000","last_peered":"2023-10-08T13:51:15.784625+0000","last_clean":"2023-10-08T13:51:15.784625+0000","last_became_active":"2023-10-08T13:49:44.783467+0000","last_became_peered":"2023-10-08T13:49:44.783467+0000","last_unstale":"2023-10-08T13:51:15.784625+0000","last_undegraded":"2023-10-08T13:51:15.784625+0000","last_fullsized":"2023-10-08T13:51:15.784625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:39.838940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788245+0000","last_change":"2023-10-08T13:49:44.779342+0000","last_active":"2023-10-08T13:51:15.788245+0000","last_peered":"2023-10-08T13:51:15.788245+0000","last_clean":"2023-10-08T13:51:15.788245+0000","last_became_active":"2023-10-08T13:49:44.778518+0000","last_became_peered":"2023-10-08T13:49:44.778518+0000","last_unstale":"2023-10-08T13:51:15.788245+0000","last_undegraded":"2023-10-08T13:51:15.788245+0000","last_fullsized":"2023-10-08T13:51:15.788245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:33.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786859+0000","last_change":"2023-10-08T13:49:44.792955+0000","last_active":"2023-10-08T13:51:15.786859+0000","last_peered":"2023-10-08T13:51:15.786859+0000","last_clean":"2023-10-08T13:51:15.786859+0000","last_became_active":"2023-10-08T13:49:44.792637+0000","last_became_peered":"2023-10-08T13:49:44.792637+0000","last_unstale":"2023-10-08T13:51:15.786859+0000","last_undegraded":"2023-10-08T13:51:15.786859+0000","last_fullsized":"2023-10-08T13:51:15.786859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:40.635913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809138+0000","last_change":"2023-10-08T13:49:44.786817+0000","last_active":"2023-10-08T13:51:15.809138+0000","last_peered":"2023-10-08T13:51:15.809138+0000","last_clean":"2023-10-08T13:51:15.809138+0000","last_became_active":"2023-10-08T13:49:44.786439+0000","last_became_peered":"2023-10-08T13:49:44.786439+0000","last_unstale":"2023-10-08T13:51:15.809138+0000","last_undegraded":"2023-10-08T13:51:15.809138+0000","last_fullsized":"2023-10-08T13:51:15.809138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:45.308600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808563+0000","last_change":"2023-10-08T13:50:15.788414+0000","last_active":"2023-10-08T13:51:15.808563+0000","last_peered":"2023-10-08T13:51:15.808563+0000","last_clean":"2023-10-08T13:51:15.808563+0000","last_became_active":"2023-10-08T13:50:15.788079+0000","last_became_peered":"2023-10-08T13:50:15.788079+0000","last_unstale":"2023-10-08T13:51:15.808563+0000","last_undegraded":"2023-10-08T13:51:15.808563+0000","last_fullsized":"2023-10-08T13:51:15.808563+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:43.723339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66011,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":793,"num_read_kb":471,"num_write":242,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":203412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":531579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64000000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69099999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54000000000000004}]},{"osd":1,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80400000000000005,"5min":1.012,"15min":1.012},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83399999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50600000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.36299999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.50900000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.80400000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.496},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.5,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88100000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63300000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82399999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56299999999999994}]},{"osd":6,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.77100000000000002,"15min":0.72899999999999998},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2759999999999998,"5min":9.2759999999999998,"15min":9.2759999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.78600000000000003,"15min":0.79400000000000004},"min":{"1min":0.55200000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.294,"5min":1.55,"15min":1.55},"last":0.76000000000000001}]},{"osd":7,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.54200000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92100000000000004,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.58199999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0089999999999999,"5min":1.214,"15min":1.214},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Oct 8 13:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.871}]},{"osd":9,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.64300000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.97099999999999997,"5min":1.355,"15min":1.355},"last":0.60499999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.68999999999999995,"5min":0.628,"15min":0.628},"max":{"1min":1.0449999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.94899999999999995,"15min":0.96899999999999997},"min":{"1min":0.68899999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.024,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.81399999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1753088,"data_stored":1178086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46600000000000003,"15min":0.46999999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58999999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.624,"15min":0.623},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61499999999999999,"15min":0.624},"min":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82299999999999995,"5min":0.871,"15min":0.871},"last":3.097},{"interface":"front","average":{"1min":0.628,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.438,"15min":0.438},"max":{"1min":0.83599999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57199999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.755,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.317,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73599999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83799999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82399999999999995,"5min":1.034,"15min":1.034},"last":0.67700000000000005}]},{"osd":6,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85299999999999998,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":3.069},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":1.083,"15min":1.083},"last":0.71299999999999997}]},{"osd":7,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55300000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.77300000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92200000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88300000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.749}]},{"osd":9,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.95199999999999996,"5min":1.05,"15min":1.05},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86799999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69699999999999995}]},{"osd":11,"last update":"Sun Oct 8 13:51:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.755,"15min":0.753},"min":{"1min":0.54400000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79200000000000004,"15min":0.78700000000000003},"min":{"1min":0.65200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93899999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.79500000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":535707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.443,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.627,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45900000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.39100000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":4.6029999999999998,"15min":4.6029999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.65800000000000003,"15min":0.67800000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71899999999999997,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.66100000000000003,"15min":0.67800000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67200000000000004,"5min":2.806,"15min":2.806},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.63100000000000001,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79400000000000004,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.52700000000000002}]},{"osd":3,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.47099999999999997,"5min":6.9969999999999999,"15min":6.9969999999999999},"last":0.496},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.60199999999999998,"15min":0.621},"min":{"1min":0.309,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56599999999999995,"5min":3.7589999999999999,"15min":3.7589999999999999},"last":0.373}]},{"osd":4,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77700000000000002,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.76000000000000001,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.51600000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":5.3390000000000004,"15min":5.3390000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82999999999999996,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.9570000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.345}]},{"osd":7,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.80000000000000004,"15min":0.78900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76800000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.82099999999999995,"15min":0.84699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.748,"5min":4.3680000000000003,"15min":4.3680000000000003},"last":0.69799999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84599999999999997,"5min":5.2930000000000001,"15min":5.2930000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.83299999999999996,"15min":0.81899999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83899999999999997,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":0.64100000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.85599999999999998,"15min":0.88300000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.88,"5min":2.113,"15min":2.113},"last":0.75},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89400000000000002,"5min":1.923,"15min":1.923},"last":0.66000000000000003}]},{"osd":11,"last update":"Sun Oct 8 13:50:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.623},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68899999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103212,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":527289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":0.54100000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82999999999999996,"5min":5.5860000000000003,"15min":5.5860000000000003},"last":0.379},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56499999999999995,"15min":0.53100000000000003},"min":{"1min":0.36599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.38900000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61199999999999999,"15min":0.57099999999999995},"min":{"1min":0.435,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96399999999999997,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79100000000000004,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.58699999999999997}]},{"osd":2,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47399999999999998,"15min":0.46200000000000002},"min":{"1min":0.34999999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.873,"5min":1.762,"15min":1.762},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.39300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84599999999999997,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61499999999999999,"15min":0.59999999999999998},"min":{"1min":0.46800000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89300000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.504}]},{"osd":4,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0289999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.67500000000000004,"15min":0.64700000000000002},"min":{"1min":0.51000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92400000000000004,"5min":2.109,"15min":2.109},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.373,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.89900000000000002,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.48899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.92500000000000004,"5min":1.476,"15min":1.476},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66700000000000004,"15min":0.64700000000000002},"min":{"1min":0.58299999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.69599999999999995},"min":{"1min":0.56100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90100000000000002,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97899999999999998,"5min":1.958,"15min":1.958},"last":0.436},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.58399999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95899999999999996,"5min":2.198,"15min":2.198},"last":0.67200000000000004}]},{"osd":9,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.69999999999999996},"min":{"1min":0.58599999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.93899999999999995,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.752,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1519999999999999,"5min":2.347,"15min":2.347},"last":0.47399999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.73699999999999999,"15min":0.69699999999999995},"min":{"1min":0.66000000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":4.2039999999999997,"5min":4.2039999999999997,"15min":4.2039999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.71699999999999997,"15min":0.68700000000000006},"min":{"1min":0.74299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.61299999999999999}]},{"osd":11,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56699999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364849,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1124530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.439}]},{"osd":1,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42199999999999999,"15min":0.41699999999999998},"min":{"1min":0.33400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64600000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57699999999999996}]},{"osd":2,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.435,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86599999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.48499999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68600000000000005}]},{"osd":3,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.60399999999999998}]},{"osd":4,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.50900000000000001},"min":{"1min":0.42399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73699999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.47999999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.52600000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.77600000000000002}]},{"osd":6,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61599999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:51:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.63500000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88600000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71999999999999997,"15min":0.71799999999999997},"min":{"1min":0.63300000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.79400000000000004}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71999999999999997,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64000000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88200000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.60399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91200000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.73599999999999999}]},{"osd":11,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.82299999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":527115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.39400000000000002,"15min":0.375},"min":{"1min":0.32300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"min":{"1min":0.309,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.441}]},{"osd":1,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80800000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.45300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81499999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.55000000000000004}]},{"osd":2,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79900000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.498},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85199999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.46899999999999997}]},{"osd":4,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82499999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74199999999999999,"5min":1.073,"15min":1.073},"last":0.59199999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.879,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":1.075,"15min":1.075},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.749,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67800000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:51:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89800000000000002,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.626},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.876,"5min":1.147,"15min":1.147},"last":0.66300000000000003}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.73299999999999998,"15min":0.747},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.96799999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":1.042,"15min":1.042},"last":0.74099999999999999}]},{"osd":10,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86899999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0469999999999999,"5min":1.159,"15min":1.159},"last":0.76000000000000001}]},{"osd":11,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58199999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542219,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":591921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.52400000000000002},"min":{"1min":0.38700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.90400000000000003,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:50:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.55900000000000005,"15min":0.50700000000000001},"min":{"1min":0.51900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.56599999999999995,"15min":0.52100000000000002},"min":{"1min":0.51300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54800000000000004}]},{"osd":3,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":3.7959999999999998,"15min":3.7959999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90600000000000003,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.629}]},{"osd":4,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.41899999999999998},"min":{"1min":0.40300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.75,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.504},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.44700000000000001},"min":{"1min":0.38100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48599999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.623,"15min":0.56599999999999995},"min":{"1min":0.51700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.65300000000000002,"15min":0.60699999999999998},"min":{"1min":0.56299999999999994,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91800000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72499999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.68700000000000006,"15min":0.76100000000000001},"min":{"1min":0.435,"5min":0.376,"15min":0.376},"max":{"1min":0.80000000000000004,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72499999999999998,"15min":0.75600000000000001},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85699999999999998,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.48199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60699999999999998,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56200000000000006}]},{"osd":8,"last update":"Sun Oct 8 13:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.71399999999999997,"15min":0.68799999999999994},"min":{"1min":0.57999999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98899999999999999,"5min":1.077,"15min":1.077},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.745,"15min":0.72599999999999998},"min":{"1min":0.69199999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.82499999999999996}]},{"osd":9,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72899999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92200000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79200000000000004,"15min":0.78800000000000003},"min":{"1min":0.55000000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.95499999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.88200000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.54000000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79700000000000004,"15min":0.78800000000000003},"min":{"1min":0.61299999999999999,"5min":0.498,"15min":0.498},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.78200000000000003}]},{"osd":11,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.86099999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705743,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":528571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 13:50:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54100000000000004,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72199999999999998,"5min":1.01,"15min":1.01},"last":0.51300000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71999999999999997,"5min":1.377,"15min":1.377},"last":0.498},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.54300000000000004},"min":{"1min":0.45500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68799999999999994,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52400000000000002}]},{"osd":3,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.39600000000000002}]},{"osd":4,"last update":"Sun Oct 8 13:50:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.81899999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71399999999999997,"5min":1.444,"15min":1.444},"last":0.64800000000000002}]},{"osd":5,"last update":"Sun Oct 8 13:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63500000000000001,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85799999999999998,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77400000000000002,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.36899999999999999}]},{"osd":7,"last update":"Sun Oct 8 13:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.73299999999999998}]},{"osd":8,"last update":"Sun Oct 8 13:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.73699999999999999,"15min":0.76800000000000002},"min":{"1min":0.41099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82899999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":1.327,"15min":1.327},"last":0.56499999999999995}]},{"osd":9,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.44600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90600000000000003,"5min":1.774,"15min":1.774},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.018,"5min":1.839,"15min":1.839},"last":0.67400000000000004}]},{"osd":10,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.59099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91100000000000003,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85699999999999998,"5min":1.411,"15min":1.411},"last":0.76600000000000001}]},{"osd":11,"last update":"Sun Oct 8 13:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.754,"15min":0.753},"min":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91600000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94399999999999995,"5min":1.208,"15min":1.208},"last":0.64100000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378694,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":590897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.94599999999999995,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81799999999999995,"5min":1.538,"15min":1.538},"last":0.45500000000000002}]},{"osd":1,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54800000000000004,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.55900000000000005},"min":{"1min":0.38900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73299999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.61599999999999999,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.497},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.55400000000000005}]},{"osd":4,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.621},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.47899999999999998,"15min":0.44},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":0.876,"15min":0.876},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.39200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.75}]},{"osd":6,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.70099999999999996}]},{"osd":7,"last update":"Sun Oct 8 13:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.622,"15min":0.60399999999999998},"min":{"1min":0.45100000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65100000000000002}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.56000000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.85099999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.68600000000000005,"15min":0.66900000000000004},"min":{"1min":0.53100000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.70399999999999996,"15min":0.68000000000000005},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.83899999999999997}]},{"osd":11,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79100000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":526823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.34799999999999998},"min":{"1min":0.28100000000000003,"5min":0.22,"15min":0.22},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47699999999999998,"15min":0.42099999999999999},"min":{"1min":0.32700000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65700000000000003,"5min":0.87,"15min":0.87},"last":0.48899999999999999}]},{"osd":1,"last update":"Sun Oct 8 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.53300000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93300000000000005,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.53100000000000003},"min":{"1min":0.40000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68899999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.60899999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61199999999999999,"15min":0.56599999999999995},"min":{"1min":0.47899999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.78500000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:50:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":2.077,"15min":2.077},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60299999999999998,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88300000000000001,"5min":2.129,"15min":2.129},"last":0.51000000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.60899999999999999},"min":{"1min":0.53700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76100000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65600000000000003,"15min":0.60799999999999998},"min":{"1min":0.432,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91000000000000003,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.75}]},{"osd":6,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.32400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80200000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.432},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.625},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53500000000000003}]},{"osd":7,"last update":"Sun Oct 8 13:50:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.76800000000000002,"15min":0.80100000000000005},"min":{"1min":0.48799999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.872,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.75600000000000001,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83199999999999996,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.57299999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:48:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.46800000000000003}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.069,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65400000000000003}]},{"osd":10,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.95799999999999996,"5min":1.028,"15min":1.028},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.82399999999999995,"15min":0.84299999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92000000000000004,"5min":1.147,"15min":1.147},"last":0.66800000000000004}]},{"osd":11,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.69899999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":531720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.51800000000000002,"15min":0.47999999999999998},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.35599999999999998},"min":{"1min":0.30599999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.36199999999999999},"min":{"1min":0.30399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.36799999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.54900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54900000000000004}]},{"osd":3,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.51500000000000001},"min":{"1min":0.40200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.497}]},{"osd":5,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.629,"15min":0.58699999999999997},"min":{"1min":0.502,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0009999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67000000000000004,"15min":0.63200000000000001},"min":{"1min":0.58299999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.71399999999999997}]},{"osd":6,"last update":"Sun Oct 8 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.66300000000000003,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.377,"15min":0.377},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.629}]},{"osd":7,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.38800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77100000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.59299999999999997,"15min":0.57299999999999995},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84599999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.43099999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73199999999999998,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96299999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.45900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97899999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.50900000000000001}]},{"osd":9,"last update":"Sun Oct 8 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.76300000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79900000000000004,"15min":0.80200000000000005},"min":{"1min":0.47199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94899999999999995,"5min":1.04,"15min":1.04},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59399999999999997}]},{"osd":11,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64500000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1683456,"data_stored":1123346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.50700000000000001,"15min":0.47899999999999998},"min":{"1min":0.40400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52300000000000002,"15min":0.47599999999999998},"min":{"1min":0.438,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.873,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47599999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.378},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.38900000000000001},"min":{"1min":0.30499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71099999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.58099999999999996,"15min":0.52000000000000002},"min":{"1min":0.46600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.52800000000000002}]},{"osd":4,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.56599999999999995},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.499}]},{"osd":6,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.54700000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.46999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56100000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88700000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.62}]},{"osd":9,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.66400000000000003,"15min":0.64300000000000002},"min":{"1min":0.57599999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73599999999999999,"15min":0.72099999999999997},"min":{"1min":0.629,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.752}]},{"osd":10,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72799999999999998,"15min":0.72599999999999998},"min":{"1min":0.59499999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.93999999999999995,"5min":0.997,"15min":0.997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.022,"5min":1.101,"15min":1.101},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T13:51:27.305 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-08T13:51:27.305 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T13:51:27.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.461+0000 7fcb649ae700 1 -- 172.21.15.57:0/2156106713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb60105600 msgr2=0x7fcb601059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.461+0000 7fcb649ae700 1 --2- 172.21.15.57:0/2156106713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb60105600 0x7fcb601059e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcb500099c0 tx=0x7fcb50009cd0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.462+0000 7fcb649ae700 1 -- 172.21.15.57:0/2156106713 shutdown_connections 2023-10-08T13:51:27.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.462+0000 7fcb649ae700 1 --2- 172.21.15.57:0/2156106713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb600fff00 0x7fcb6010c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.462+0000 7fcb649ae700 1 --2- 172.21.15.57:0/2156106713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb60105600 0x7fcb601059e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.462+0000 7fcb649ae700 1 --2- 172.21.15.57:0/2156106713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb600ff540 0x7fcb600ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.462+0000 7fcb649ae700 1 -- 172.21.15.57:0/2156106713 >> 172.21.15.57:0/2156106713 conn(0x7fcb600fb030 msgr2=0x7fcb600fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:27.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.462+0000 7fcb649ae700 1 -- 172.21.15.57:0/2156106713 shutdown_connections 2023-10-08T13:51:27.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.462+0000 7fcb649ae700 1 -- 172.21.15.57:0/2156106713 wait complete. 2023-10-08T13:51:27.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb649ae700 1 Processor -- start 2023-10-08T13:51:27.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb649ae700 1 -- start start 2023-10-08T13:51:27.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb649ae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb600ff540 0x7fcb60197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb649ae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb600fff00 0x7fcb60197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb649ae700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb60105600 0x7fcb601a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb649ae700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601986d0 con 0x7fcb60105600 2023-10-08T13:51:27.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb649ae700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601a2ee0 con 0x7fcb600ff540 2023-10-08T13:51:27.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb649ae700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601a3020 con 0x7fcb600fff00 2023-10-08T13:51:27.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb5ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb60105600 0x7fcb601a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb5ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb60105600 0x7fcb601a29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56112/0 (socket says 172.21.15.57:56112) 2023-10-08T13:51:27.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb5ed9d700 1 -- 172.21.15.57:0/111730126 learned_addr learned my addr 172.21.15.57:0/111730126 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb5dd9b700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb600fff00 0x7fcb60197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb5e59c700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb600ff540 0x7fcb60197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb5ed9d700 1 -- 172.21.15.57:0/111730126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb600fff00 msgr2=0x7fcb60197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb5ed9d700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb600fff00 0x7fcb60197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb5ed9d700 1 -- 172.21.15.57:0/111730126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb600ff540 msgr2=0x7fcb60197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb5ed9d700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb600ff540 0x7fcb60197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.463+0000 7fcb5ed9d700 1 -- 172.21.15.57:0/111730126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb500095f0 con 0x7fcb60105600 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.464+0000 7fcb5ed9d700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb60105600 0x7fcb601a29a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcb58007ee0 tx=0x7fcb5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.464+0000 7fcb577fe700 1 -- 172.21.15.57:0/111730126 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58017070 con 0x7fcb60105600 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.464+0000 7fcb649ae700 1 -- 172.21.15.57:0/111730126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb601a32b0 con 0x7fcb60105600 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.464+0000 7fcb577fe700 1 -- 172.21.15.57:0/111730126 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb5800f040 con 0x7fcb60105600 2023-10-08T13:51:27.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.464+0000 7fcb577fe700 1 -- 172.21.15.57:0/111730126 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58005740 con 0x7fcb60105600 2023-10-08T13:51:27.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.464+0000 7fcb649ae700 1 -- 172.21.15.57:0/111730126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb601a3770 con 0x7fcb60105600 2023-10-08T13:51:27.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.465+0000 7fcb649ae700 1 -- 172.21.15.57:0/111730126 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb60107a80 con 0x7fcb60105600 2023-10-08T13:51:27.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.466+0000 7fcb577fe700 1 -- 172.21.15.57:0/111730126 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fcb5800dc60 con 0x7fcb60105600 2023-10-08T13:51:27.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.467+0000 7fcb577fe700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb4c0a14d0 0x7fcb4c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.467+0000 7fcb5e59c700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb4c0a14d0 0x7fcb4c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.467+0000 7fcb577fe700 1 -- 172.21.15.57:0/111730126 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcb580cc320 con 0x7fcb60105600 2023-10-08T13:51:27.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.467+0000 7fcb5e59c700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb4c0a14d0 0x7fcb4c0a3990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb50005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:27.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.471+0000 7fcb577fe700 1 -- 172.21.15.57:0/111730126 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcb58095560 con 0x7fcb60105600 2023-10-08T13:51:27.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.673+0000 7fcb649ae700 1 -- 172.21.15.57:0/111730126 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb60067e60 con 0x7fcb4c0a14d0 2023-10-08T13:51:27.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.688+0000 7fcb577fe700 1 -- 172.21.15.57:0/111730126 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427464 (secure 0 0 0) 0x7fcb60067e60 con 0x7fcb4c0a14d0 2023-10-08T13:51:27.694 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:27.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 -- 172.21.15.57:0/111730126 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb4c0a14d0 msgr2=0x7fcb4c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb4c0a14d0 0x7fcb4c0a3990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb50005fd0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 -- 172.21.15.57:0/111730126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb60105600 msgr2=0x7fcb601a29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb60105600 0x7fcb601a29a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcb58007ee0 tx=0x7fcb5800d3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 -- 172.21.15.57:0/111730126 shutdown_connections 2023-10-08T13:51:27.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb60105600 0x7fcb601a29a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcb4c0a14d0 0x7fcb4c0a3990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb600ff540 0x7fcb60197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 --2- 172.21.15.57:0/111730126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb600fff00 0x7fcb60197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 -- 172.21.15.57:0/111730126 >> 172.21.15.57:0/111730126 conn(0x7fcb600fb030 msgr2=0x7fcb600fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 -- 172.21.15.57:0/111730126 shutdown_connections 2023-10-08T13:51:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.692+0000 7fcb649ae700 1 -- 172.21.15.57:0/111730126 wait complete. 2023-10-08T13:51:27.698 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-08T13:51:27.705 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-10-08T13:51:26.130285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656625,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":929,"num_read_kb":590,"num_write":425,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181962,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338832,"kb_used_data":14556,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124602992,"statfs":{"total":1151940427776,"available":1151593463808,"internally_reserved":0,"allocated":14905344,"data_stored":8317584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002661"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808844+0000","last_change":"2023-10-08T13:49:44.786130+0000","last_active":"2023-10-08T13:51:15.808844+0000","last_peered":"2023-10-08T13:51:15.808844+0000","last_clean":"2023-10-08T13:51:15.808844+0000","last_became_active":"2023-10-08T13:49:44.785792+0000","last_became_peered":"2023-10-08T13:49:44.785792+0000","last_unstale":"2023-10-08T13:51:15.808844+0000","last_undegraded":"2023-10-08T13:51:15.808844+0000","last_fullsized":"2023-10-08T13:51:15.808844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:17.046800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808869+0000","last_change":"2023-10-08T13:49:44.777601+0000","last_active":"2023-10-08T13:51:15.808869+0000","last_peered":"2023-10-08T13:51:15.808869+0000","last_clean":"2023-10-08T13:51:15.808869+0000","last_became_active":"2023-10-08T13:49:44.777287+0000","last_became_peered":"2023-10-08T13:49:44.777287+0000","last_unstale":"2023-10-08T13:51:15.808869+0000","last_undegraded":"2023-10-08T13:51:15.808869+0000","last_fullsized":"2023-10-08T13:51:15.808869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:09:17.317629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808744+0000","last_change":"2023-10-08T13:49:44.787035+0000","last_active":"2023-10-08T13:51:15.808744+0000","last_peered":"2023-10-08T13:51:15.808744+0000","last_clean":"2023-10-08T13:51:15.808744+0000","last_became_active":"2023-10-08T13:49:44.785834+0000","last_became_peered":"2023-10-08T13:49:44.785834+0000","last_unstale":"2023-10-08T13:51:15.808744+0000","last_undegraded":"2023-10-08T13:51:15.808744+0000","last_fullsized":"2023-10-08T13:51:15.808744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:11.611404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783168+0000","last_change":"2023-10-08T13:49:44.781937+0000","last_active":"2023-10-08T13:51:15.783168+0000","last_peered":"2023-10-08T13:51:15.783168+0000","last_clean":"2023-10-08T13:51:15.783168+0000","last_became_active":"2023-10-08T13:49:44.780712+0000","last_became_peered":"2023-10-08T13:49:44.780712+0000","last_unstale":"2023-10-08T13:51:15.783168+0000","last_undegraded":"2023-10-08T13:51:15.783168+0000","last_fullsized":"2023-10-08T13:51:15.783168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:02.514007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783022+0000","last_change":"2023-10-08T13:49:44.788661+0000","last_active":"2023-10-08T13:51:15.783022+0000","last_peered":"2023-10-08T13:51:15.783022+0000","last_clean":"2023-10-08T13:51:15.783022+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:51:15.783022+0000","last_undegraded":"2023-10-08T13:51:15.783022+0000","last_fullsized":"2023-10-08T13:51:15.783022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:35.770287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809252+0000","last_change":"2023-10-08T13:49:44.778519+0000","last_active":"2023-10-08T13:51:15.809252+0000","last_peered":"2023-10-08T13:51:15.809252+0000","last_clean":"2023-10-08T13:51:15.809252+0000","last_became_active":"2023-10-08T13:49:44.778193+0000","last_became_peered":"2023-10-08T13:49:44.778193+0000","last_unstale":"2023-10-08T13:51:15.809252+0000","last_undegraded":"2023-10-08T13:51:15.809252+0000","last_fullsized":"2023-10-08T13:51:15.809252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:12.916019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809332+0000","last_change":"2023-10-08T13:49:44.791759+0000","last_active":"2023-10-08T13:51:15.809332+0000","last_peered":"2023-10-08T13:51:15.809332+0000","last_clean":"2023-10-08T13:51:15.809332+0000","last_became_active":"2023-10-08T13:49:44.791494+0000","last_became_peered":"2023-10-08T13:49:44.791494+0000","last_unstale":"2023-10-08T13:51:15.809332+0000","last_undegraded":"2023-10-08T13:51:15.809332+0000","last_fullsized":"2023-10-08T13:51:15.809332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:55.859609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787828+0000","last_change":"2023-10-08T13:49:44.786831+0000","last_active":"2023-10-08T13:51:15.787828+0000","last_peered":"2023-10-08T13:51:15.787828+0000","last_clean":"2023-10-08T13:51:15.787828+0000","last_became_active":"2023-10-08T13:49:44.786516+0000","last_became_peered":"2023-10-08T13:49:44.786516+0000","last_unstale":"2023-10-08T13:51:15.787828+0000","last_undegraded":"2023-10-08T13:51:15.787828+0000","last_fullsized":"2023-10-08T13:51:15.787828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:07.616821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787111+0000","last_change":"2023-10-08T13:49:44.789827+0000","last_active":"2023-10-08T13:51:15.787111+0000","last_peered":"2023-10-08T13:51:15.787111+0000","last_clean":"2023-10-08T13:51:15.787111+0000","last_became_active":"2023-10-08T13:49:44.786635+0000","last_became_peered":"2023-10-08T13:49:44.786635+0000","last_unstale":"2023-10-08T13:51:15.787111+0000","last_undegraded":"2023-10-08T13:51:15.787111+0000","last_fullsized":"2023-10-08T13:51:15.787111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:35.492437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809682+0000","last_change":"2023-10-08T13:49:44.784240+0000","last_active":"2023-10-08T13:51:15.809682+0000","last_peered":"2023-10-08T13:51:15.809682+0000","last_clean":"2023-10-08T13:51:15.809682+0000","last_became_active":"2023-10-08T13:49:44.783475+0000","last_became_peered":"2023-10-08T13:49:44.783475+0000","last_unstale":"2023-10-08T13:51:15.809682+0000","last_undegraded":"2023-10-08T13:51:15.809682+0000","last_fullsized":"2023-10-08T13:51:15.809682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:39.730679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808970+0000","last_change":"2023-10-08T13:49:44.790901+0000","last_active":"2023-10-08T13:51:15.808970+0000","last_peered":"2023-10-08T13:51:15.808970+0000","last_clean":"2023-10-08T13:51:15.808970+0000","last_became_active":"2023-10-08T13:49:44.790611+0000","last_became_peered":"2023-10-08T13:49:44.790611+0000","last_unstale":"2023-10-08T13:51:15.808970+0000","last_undegraded":"2023-10-08T13:51:15.808970+0000","last_fullsized":"2023-10-08T13:51:15.808970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:19:21.764105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783260+0000","last_change":"2023-10-08T13:49:44.775226+0000","last_active":"2023-10-08T13:51:15.783260+0000","last_peered":"2023-10-08T13:51:15.783260+0000","last_clean":"2023-10-08T13:51:15.783260+0000","last_became_active":"2023-10-08T13:49:44.774756+0000","last_became_peered":"2023-10-08T13:49:44.774756+0000","last_unstale":"2023-10-08T13:51:15.783260+0000","last_undegraded":"2023-10-08T13:51:15.783260+0000","last_fullsized":"2023-10-08T13:51:15.783260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:57:14.749066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809003+0000","last_change":"2023-10-08T13:49:44.781610+0000","last_active":"2023-10-08T13:51:15.809003+0000","last_peered":"2023-10-08T13:51:15.809003+0000","last_clean":"2023-10-08T13:51:15.809003+0000","last_became_active":"2023-10-08T13:49:44.781281+0000","last_became_peered":"2023-10-08T13:49:44.781281+0000","last_unstale":"2023-10-08T13:51:15.809003+0000","last_undegraded":"2023-10-08T13:51:15.809003+0000","last_fullsized":"2023-10-08T13:51:15.809003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:50.823909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808360+0000","last_change":"2023-10-08T13:49:44.775836+0000","last_active":"2023-10-08T13:51:15.808360+0000","last_peered":"2023-10-08T13:51:15.808360+0000","last_clean":"2023-10-08T13:51:15.808360+0000","last_became_active":"2023-10-08T13:49:44.775206+0000","last_became_peered":"2023-10-08T13:49:44.775206+0000","last_unstale":"2023-10-08T13:51:15.808360+0000","last_undegraded":"2023-10-08T13:51:15.808360+0000","last_fullsized":"2023-10-08T13:51:15.808360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:51.566800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808279+0000","last_change":"2023-10-08T13:49:44.778495+0000","last_active":"2023-10-08T13:51:15.808279+0000","last_peered":"2023-10-08T13:51:15.808279+0000","last_clean":"2023-10-08T13:51:15.808279+0000","last_became_active":"2023-10-08T13:49:44.778196+0000","last_became_peered":"2023-10-08T13:49:44.778196+0000","last_unstale":"2023-10-08T13:51:15.808279+0000","last_undegraded":"2023-10-08T13:51:15.808279+0000","last_fullsized":"2023-10-08T13:51:15.808279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:57:38.531853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786633+0000","last_change":"2023-10-08T13:49:44.792960+0000","last_active":"2023-10-08T13:51:15.786633+0000","last_peered":"2023-10-08T13:51:15.786633+0000","last_clean":"2023-10-08T13:51:15.786633+0000","last_became_active":"2023-10-08T13:49:44.792674+0000","last_became_peered":"2023-10-08T13:49:44.792674+0000","last_unstale":"2023-10-08T13:51:15.786633+0000","last_undegraded":"2023-10-08T13:51:15.786633+0000","last_fullsized":"2023-10-08T13:51:15.786633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:13.934954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808777+0000","last_change":"2023-10-08T13:49:44.785971+0000","last_active":"2023-10-08T13:51:15.808777+0000","last_peered":"2023-10-08T13:51:15.808777+0000","last_clean":"2023-10-08T13:51:15.808777+0000","last_became_active":"2023-10-08T13:49:44.785680+0000","last_became_peered":"2023-10-08T13:49:44.785680+0000","last_unstale":"2023-10-08T13:51:15.808777+0000","last_undegraded":"2023-10-08T13:51:15.808777+0000","last_fullsized":"2023-10-08T13:51:15.808777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:35.447519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786759+0000","last_change":"2023-10-08T13:49:44.787553+0000","last_active":"2023-10-08T13:51:15.786759+0000","last_peered":"2023-10-08T13:51:15.786759+0000","last_clean":"2023-10-08T13:51:15.786759+0000","last_became_active":"2023-10-08T13:49:44.787288+0000","last_became_peered":"2023-10-08T13:49:44.787288+0000","last_unstale":"2023-10-08T13:51:15.786759+0000","last_undegraded":"2023-10-08T13:51:15.786759+0000","last_fullsized":"2023-10-08T13:51:15.786759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:48.386120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788367+0000","last_change":"2023-10-08T13:49:44.789252+0000","last_active":"2023-10-08T13:51:15.788367+0000","last_peered":"2023-10-08T13:51:15.788367+0000","last_clean":"2023-10-08T13:51:15.788367+0000","last_became_active":"2023-10-08T13:49:44.788947+0000","last_became_peered":"2023-10-08T13:49:44.788947+0000","last_unstale":"2023-10-08T13:51:15.788367+0000","last_undegraded":"2023-10-08T13:51:15.788367+0000","last_fullsized":"2023-10-08T13:51:15.788367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:30.266913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787998+0000","last_change":"2023-10-08T13:49:49.163619+0000","last_active":"2023-10-08T13:51:15.787998+0000","last_peered":"2023-10-08T13:51:15.787998+0000","last_clean":"2023-10-08T13:51:15.787998+0000","last_became_active":"2023-10-08T13:49:49.163020+0000","last_became_peered":"2023-10-08T13:49:49.163020+0000","last_unstale":"2023-10-08T13:51:15.787998+0000","last_undegraded":"2023-10-08T13:51:15.787998+0000","last_fullsized":"2023-10-08T13:51:15.787998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:56.092404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":112,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:23.854771+0000","last_change":"2023-10-08T13:49:43.782863+0000","last_active":"2023-10-08T13:51:23.854771+0000","last_peered":"2023-10-08T13:51:23.854771+0000","last_clean":"2023-10-08T13:51:23.854771+0000","last_became_active":"2023-10-08T13:49:43.782477+0000","last_became_peered":"2023-10-08T13:49:43.782477+0000","last_unstale":"2023-10-08T13:51:23.854771+0000","last_undegraded":"2023-10-08T13:51:23.854771+0000","last_fullsized":"2023-10-08T13:51:23.854771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:10.134907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:16.876324+0000","last_change":"2023-10-08T13:51:16.875898+0000","last_active":"2023-10-08T13:51:16.876324+0000","last_peered":"2023-10-08T13:51:16.876324+0000","last_clean":"2023-10-08T13:51:16.876324+0000","last_became_active":"2023-10-08T13:51:16.875581+0000","last_became_peered":"2023-10-08T13:51:16.875581+0000","last_unstale":"2023-10-08T13:51:16.876324+0000","last_undegraded":"2023-10-08T13:51:16.876324+0000","last_fullsized":"2023-10-08T13:51:16.876324+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:27.797425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782603+0000","last_change":"2023-10-08T13:49:49.164248+0000","last_active":"2023-10-08T13:51:15.782603+0000","last_peered":"2023-10-08T13:51:15.782603+0000","last_clean":"2023-10-08T13:51:15.782603+0000","last_became_active":"2023-10-08T13:49:49.163624+0000","last_became_peered":"2023-10-08T13:49:49.163624+0000","last_unstale":"2023-10-08T13:51:15.782603+0000","last_undegraded":"2023-10-08T13:51:15.782603+0000","last_fullsized":"2023-10-08T13:51:15.782603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:04.560541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788426+0000","last_change":"2023-10-08T13:49:43.788026+0000","last_active":"2023-10-08T13:51:15.788426+0000","last_peered":"2023-10-08T13:51:15.788426+0000","last_clean":"2023-10-08T13:51:15.788426+0000","last_became_active":"2023-10-08T13:49:43.787639+0000","last_became_peered":"2023-10-08T13:49:43.787639+0000","last_unstale":"2023-10-08T13:51:15.788426+0000","last_undegraded":"2023-10-08T13:51:15.788426+0000","last_fullsized":"2023-10-08T13:51:15.788426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:25:48.367765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808934+0000","last_change":"2023-10-08T13:49:44.786322+0000","last_active":"2023-10-08T13:51:15.808934+0000","last_peered":"2023-10-08T13:51:15.808934+0000","last_clean":"2023-10-08T13:51:15.808934+0000","last_became_active":"2023-10-08T13:49:44.785953+0000","last_became_peered":"2023-10-08T13:49:44.785953+0000","last_unstale":"2023-10-08T13:51:15.808934+0000","last_undegraded":"2023-10-08T13:51:15.808934+0000","last_fullsized":"2023-10-08T13:51:15.808934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:39.866785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782827+0000","last_change":"2023-10-08T13:50:15.976961+0000","last_active":"2023-10-08T13:51:15.782827+0000","last_peered":"2023-10-08T13:51:15.782827+0000","last_clean":"2023-10-08T13:51:15.782827+0000","last_became_active":"2023-10-08T13:50:15.976492+0000","last_became_peered":"2023-10-08T13:50:15.976492+0000","last_unstale":"2023-10-08T13:51:15.782827+0000","last_undegraded":"2023-10-08T13:51:15.782827+0000","last_fullsized":"2023-10-08T13:51:15.782827+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:05:48.947259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783518+0000","last_change":"2023-10-08T13:49:43.776239+0000","last_active":"2023-10-08T13:51:15.783518+0000","last_peered":"2023-10-08T13:51:15.783518+0000","last_clean":"2023-10-08T13:51:15.783518+0000","last_became_active":"2023-10-08T13:49:43.775969+0000","last_became_peered":"2023-10-08T13:49:43.775969+0000","last_unstale":"2023-10-08T13:51:15.783518+0000","last_undegraded":"2023-10-08T13:51:15.783518+0000","last_fullsized":"2023-10-08T13:51:15.783518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:24.011655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786966+0000","last_change":"2023-10-08T13:49:44.794912+0000","last_active":"2023-10-08T13:51:15.786966+0000","last_peered":"2023-10-08T13:51:15.786966+0000","last_clean":"2023-10-08T13:51:15.786966+0000","last_became_active":"2023-10-08T13:49:44.794590+0000","last_became_peered":"2023-10-08T13:49:44.794590+0000","last_unstale":"2023-10-08T13:51:15.786966+0000","last_undegraded":"2023-10-08T13:51:15.786966+0000","last_fullsized":"2023-10-08T13:51:15.786966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:51.631386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782715+0000","last_change":"2023-10-08T13:49:49.152149+0000","last_active":"2023-10-08T13:51:15.782715+0000","last_peered":"2023-10-08T13:51:15.782715+0000","last_clean":"2023-10-08T13:51:15.782715+0000","last_became_active":"2023-10-08T13:49:49.151764+0000","last_became_peered":"2023-10-08T13:49:49.151764+0000","last_unstale":"2023-10-08T13:51:15.782715+0000","last_undegraded":"2023-10-08T13:51:15.782715+0000","last_fullsized":"2023-10-08T13:51:15.782715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:25:08.232556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808473+0000","last_change":"2023-10-08T13:50:15.800467+0000","last_active":"2023-10-08T13:51:15.808473+0000","last_peered":"2023-10-08T13:51:15.808473+0000","last_clean":"2023-10-08T13:51:15.808473+0000","last_became_active":"2023-10-08T13:50:15.788865+0000","last_became_peered":"2023-10-08T13:50:15.788865+0000","last_unstale":"2023-10-08T13:51:15.808473+0000","last_undegraded":"2023-10-08T13:51:15.808473+0000","last_fullsized":"2023-10-08T13:51:15.808473+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:11:01.887072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785234+0000","last_change":"2023-10-08T13:49:44.783915+0000","last_active":"2023-10-08T13:51:15.785234+0000","last_peered":"2023-10-08T13:51:15.785234+0000","last_clean":"2023-10-08T13:51:15.785234+0000","last_became_active":"2023-10-08T13:49:44.783628+0000","last_became_peered":"2023-10-08T13:49:44.783628+0000","last_unstale":"2023-10-08T13:51:15.785234+0000","last_undegraded":"2023-10-08T13:51:15.785234+0000","last_fullsized":"2023-10-08T13:51:15.785234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:43.759681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788199+0000","last_change":"2023-10-08T13:49:49.156241+0000","last_active":"2023-10-08T13:51:15.788199+0000","last_peered":"2023-10-08T13:51:15.788199+0000","last_clean":"2023-10-08T13:51:15.788199+0000","last_became_active":"2023-10-08T13:49:49.155652+0000","last_became_peered":"2023-10-08T13:49:49.155652+0000","last_unstale":"2023-10-08T13:51:15.788199+0000","last_undegraded":"2023-10-08T13:51:15.788199+0000","last_fullsized":"2023-10-08T13:51:15.788199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.772697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":119,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:23.885147+0000","last_change":"2023-10-08T13:50:15.795816+0000","last_active":"2023-10-08T13:51:23.885147+0000","last_peered":"2023-10-08T13:51:23.885147+0000","last_clean":"2023-10-08T13:51:23.885147+0000","last_became_active":"2023-10-08T13:50:15.789936+0000","last_became_peered":"2023-10-08T13:50:15.789936+0000","last_unstale":"2023-10-08T13:51:23.885147+0000","last_undegraded":"2023-10-08T13:51:23.885147+0000","last_fullsized":"2023-10-08T13:51:23.885147+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:21.496490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783042+0000","last_change":"2023-10-08T13:49:44.780935+0000","last_active":"2023-10-08T13:51:15.783042+0000","last_peered":"2023-10-08T13:51:15.783042+0000","last_clean":"2023-10-08T13:51:15.783042+0000","last_became_active":"2023-10-08T13:49:44.780613+0000","last_became_peered":"2023-10-08T13:49:44.780613+0000","last_unstale":"2023-10-08T13:51:15.783042+0000","last_undegraded":"2023-10-08T13:51:15.783042+0000","last_fullsized":"2023-10-08T13:51:15.783042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:57.649759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788141+0000","last_change":"2023-10-08T13:49:49.163652+0000","last_active":"2023-10-08T13:51:15.788141+0000","last_peered":"2023-10-08T13:51:15.788141+0000","last_clean":"2023-10-08T13:51:15.788141+0000","last_became_active":"2023-10-08T13:49:49.163140+0000","last_became_peered":"2023-10-08T13:49:49.163140+0000","last_unstale":"2023-10-08T13:51:15.788141+0000","last_undegraded":"2023-10-08T13:51:15.788141+0000","last_fullsized":"2023-10-08T13:51:15.788141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:48.733675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809171+0000","last_change":"2023-10-08T13:49:43.775799+0000","last_active":"2023-10-08T13:51:15.809171+0000","last_peered":"2023-10-08T13:51:15.809171+0000","last_clean":"2023-10-08T13:51:15.809171+0000","last_became_active":"2023-10-08T13:49:43.775470+0000","last_became_peered":"2023-10-08T13:49:43.775470+0000","last_unstale":"2023-10-08T13:51:15.809171+0000","last_undegraded":"2023-10-08T13:51:15.809171+0000","last_fullsized":"2023-10-08T13:51:15.809171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:50.404554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784980+0000","last_change":"2023-10-08T13:50:15.785798+0000","last_active":"2023-10-08T13:51:15.784980+0000","last_peered":"2023-10-08T13:51:15.784980+0000","last_clean":"2023-10-08T13:51:15.784980+0000","last_became_active":"2023-10-08T13:50:15.785343+0000","last_became_peered":"2023-10-08T13:50:15.785343+0000","last_unstale":"2023-10-08T13:51:15.784980+0000","last_undegraded":"2023-10-08T13:51:15.784980+0000","last_fullsized":"2023-10-08T13:51:15.784980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:36.717707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808771+0000","last_change":"2023-10-08T13:49:49.166702+0000","last_active":"2023-10-08T13:51:15.808771+0000","last_peered":"2023-10-08T13:51:15.808771+0000","last_clean":"2023-10-08T13:51:15.808771+0000","last_became_active":"2023-10-08T13:49:49.166256+0000","last_became_peered":"2023-10-08T13:49:49.166256+0000","last_unstale":"2023-10-08T13:51:15.808771+0000","last_undegraded":"2023-10-08T13:51:15.808771+0000","last_fullsized":"2023-10-08T13:51:15.808771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:18:57.565040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808332+0000","last_change":"2023-10-08T13:49:43.785085+0000","last_active":"2023-10-08T13:51:15.808332+0000","last_peered":"2023-10-08T13:51:15.808332+0000","last_clean":"2023-10-08T13:51:15.808332+0000","last_became_active":"2023-10-08T13:49:43.784746+0000","last_became_peered":"2023-10-08T13:49:43.784746+0000","last_unstale":"2023-10-08T13:51:15.808332+0000","last_undegraded":"2023-10-08T13:51:15.808332+0000","last_fullsized":"2023-10-08T13:51:15.808332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:13.471834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787752+0000","last_change":"2023-10-08T13:49:44.785670+0000","last_active":"2023-10-08T13:51:15.787752+0000","last_peered":"2023-10-08T13:51:15.787752+0000","last_clean":"2023-10-08T13:51:15.787752+0000","last_became_active":"2023-10-08T13:49:44.785221+0000","last_became_peered":"2023-10-08T13:49:44.785221+0000","last_unstale":"2023-10-08T13:51:15.787752+0000","last_undegraded":"2023-10-08T13:51:15.787752+0000","last_fullsized":"2023-10-08T13:51:15.787752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:25.415168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808870+0000","last_change":"2023-10-08T13:49:49.156294+0000","last_active":"2023-10-08T13:51:15.808870+0000","last_peered":"2023-10-08T13:51:15.808870+0000","last_clean":"2023-10-08T13:51:15.808870+0000","last_became_active":"2023-10-08T13:49:49.155821+0000","last_became_peered":"2023-10-08T13:49:49.155821+0000","last_unstale":"2023-10-08T13:51:15.808870+0000","last_undegraded":"2023-10-08T13:51:15.808870+0000","last_fullsized":"2023-10-08T13:51:15.808870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:59:43.831634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784681+0000","last_change":"2023-10-08T13:49:43.780052+0000","last_active":"2023-10-08T13:51:15.784681+0000","last_peered":"2023-10-08T13:51:15.784681+0000","last_clean":"2023-10-08T13:51:15.784681+0000","last_became_active":"2023-10-08T13:49:43.779731+0000","last_became_peered":"2023-10-08T13:49:43.779731+0000","last_unstale":"2023-10-08T13:51:15.784681+0000","last_undegraded":"2023-10-08T13:51:15.784681+0000","last_fullsized":"2023-10-08T13:51:15.784681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:38.800783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785843+0000","last_change":"2023-10-08T13:49:44.787880+0000","last_active":"2023-10-08T13:51:15.785843+0000","last_peered":"2023-10-08T13:51:15.785843+0000","last_clean":"2023-10-08T13:51:15.785843+0000","last_became_active":"2023-10-08T13:49:44.787591+0000","last_became_peered":"2023-10-08T13:49:44.787591+0000","last_unstale":"2023-10-08T13:51:15.785843+0000","last_undegraded":"2023-10-08T13:51:15.785843+0000","last_fullsized":"2023-10-08T13:51:15.785843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:53.078903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788057+0000","last_change":"2023-10-08T13:49:49.155004+0000","last_active":"2023-10-08T13:51:15.788057+0000","last_peered":"2023-10-08T13:51:15.788057+0000","last_clean":"2023-10-08T13:51:15.788057+0000","last_became_active":"2023-10-08T13:49:49.154503+0000","last_became_peered":"2023-10-08T13:49:49.154503+0000","last_unstale":"2023-10-08T13:51:15.788057+0000","last_undegraded":"2023-10-08T13:51:15.788057+0000","last_fullsized":"2023-10-08T13:51:15.788057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:21.565816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":174,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.851308+0000","last_change":"2023-10-08T13:49:43.775819+0000","last_active":"2023-10-08T13:51:21.851308+0000","last_peered":"2023-10-08T13:51:21.851308+0000","last_clean":"2023-10-08T13:51:21.851308+0000","last_became_active":"2023-10-08T13:49:43.775578+0000","last_became_peered":"2023-10-08T13:49:43.775578+0000","last_unstale":"2023-10-08T13:51:21.851308+0000","last_undegraded":"2023-10-08T13:51:21.851308+0000","last_fullsized":"2023-10-08T13:51:21.851308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:31.518191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787270+0000","last_change":"2023-10-08T13:49:44.785869+0000","last_active":"2023-10-08T13:51:15.787270+0000","last_peered":"2023-10-08T13:51:15.787270+0000","last_clean":"2023-10-08T13:51:15.787270+0000","last_became_active":"2023-10-08T13:49:44.785566+0000","last_became_peered":"2023-10-08T13:49:44.785566+0000","last_unstale":"2023-10-08T13:51:15.787270+0000","last_undegraded":"2023-10-08T13:51:15.787270+0000","last_fullsized":"2023-10-08T13:51:15.787270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:29.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785538+0000","last_change":"2023-10-08T13:49:49.162075+0000","last_active":"2023-10-08T13:51:15.785538+0000","last_peered":"2023-10-08T13:51:15.785538+0000","last_clean":"2023-10-08T13:51:15.785538+0000","last_became_active":"2023-10-08T13:49:49.161485+0000","last_became_peered":"2023-10-08T13:49:49.161485+0000","last_unstale":"2023-10-08T13:51:15.785538+0000","last_undegraded":"2023-10-08T13:51:15.785538+0000","last_fullsized":"2023-10-08T13:51:15.785538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:16.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808954+0000","last_change":"2023-10-08T13:49:43.776555+0000","last_active":"2023-10-08T13:51:15.808954+0000","last_peered":"2023-10-08T13:51:15.808954+0000","last_clean":"2023-10-08T13:51:15.808954+0000","last_became_active":"2023-10-08T13:49:43.776259+0000","last_became_peered":"2023-10-08T13:49:43.776259+0000","last_unstale":"2023-10-08T13:51:15.808954+0000","last_undegraded":"2023-10-08T13:51:15.808954+0000","last_fullsized":"2023-10-08T13:51:15.808954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:34.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809044+0000","last_change":"2023-10-08T13:49:44.779253+0000","last_active":"2023-10-08T13:51:15.809044+0000","last_peered":"2023-10-08T13:51:15.809044+0000","last_clean":"2023-10-08T13:51:15.809044+0000","last_became_active":"2023-10-08T13:49:44.778902+0000","last_became_peered":"2023-10-08T13:49:44.778902+0000","last_unstale":"2023-10-08T13:51:15.809044+0000","last_undegraded":"2023-10-08T13:51:15.809044+0000","last_fullsized":"2023-10-08T13:51:15.809044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:31:01.091424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"88'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782932+0000","last_change":"2023-10-08T13:49:49.164859+0000","last_active":"2023-10-08T13:51:15.782932+0000","last_peered":"2023-10-08T13:51:15.782932+0000","last_clean":"2023-10-08T13:51:15.782932+0000","last_became_active":"2023-10-08T13:49:49.164221+0000","last_became_peered":"2023-10-08T13:49:49.164221+0000","last_unstale":"2023-10-08T13:51:15.782932+0000","last_undegraded":"2023-10-08T13:51:15.782932+0000","last_fullsized":"2023-10-08T13:51:15.782932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:22.484934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785106+0000","last_change":"2023-10-08T13:49:43.788136+0000","last_active":"2023-10-08T13:51:15.785106+0000","last_peered":"2023-10-08T13:51:15.785106+0000","last_clean":"2023-10-08T13:51:15.785106+0000","last_became_active":"2023-10-08T13:49:43.787910+0000","last_became_peered":"2023-10-08T13:49:43.787910+0000","last_unstale":"2023-10-08T13:51:15.785106+0000","last_undegraded":"2023-10-08T13:51:15.785106+0000","last_fullsized":"2023-10-08T13:51:15.785106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:00:32.897972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787049+0000","last_change":"2023-10-08T13:49:44.786833+0000","last_active":"2023-10-08T13:51:15.787049+0000","last_peered":"2023-10-08T13:51:15.787049+0000","last_clean":"2023-10-08T13:51:15.787049+0000","last_became_active":"2023-10-08T13:49:44.786523+0000","last_became_peered":"2023-10-08T13:49:44.786523+0000","last_unstale":"2023-10-08T13:51:15.787049+0000","last_undegraded":"2023-10-08T13:51:15.787049+0000","last_fullsized":"2023-10-08T13:51:15.787049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:36:22.156252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785774+0000","last_change":"2023-10-08T13:50:15.977115+0000","last_active":"2023-10-08T13:51:15.785774+0000","last_peered":"2023-10-08T13:51:15.785774+0000","last_clean":"2023-10-08T13:51:15.785774+0000","last_became_active":"2023-10-08T13:50:15.976617+0000","last_became_peered":"2023-10-08T13:50:15.976617+0000","last_unstale":"2023-10-08T13:51:15.785774+0000","last_undegraded":"2023-10-08T13:51:15.785774+0000","last_fullsized":"2023-10-08T13:51:15.785774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:57.124054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'36","reported_seq":214,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.882495+0000","last_change":"2023-10-08T13:49:43.777736+0000","last_active":"2023-10-08T13:51:21.882495+0000","last_peered":"2023-10-08T13:51:21.882495+0000","last_clean":"2023-10-08T13:51:21.882495+0000","last_became_active":"2023-10-08T13:49:43.777431+0000","last_became_peered":"2023-10-08T13:49:43.777431+0000","last_unstale":"2023-10-08T13:51:21.882495+0000","last_undegraded":"2023-10-08T13:51:21.882495+0000","last_fullsized":"2023-10-08T13:51:21.882495+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:12.907196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59058,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":158,"num_read_kb":136,"num_write":38,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783122+0000","last_change":"2023-10-08T13:49:44.789122+0000","last_active":"2023-10-08T13:51:15.783122+0000","last_peered":"2023-10-08T13:51:15.783122+0000","last_clean":"2023-10-08T13:51:15.783122+0000","last_became_active":"2023-10-08T13:49:44.788831+0000","last_became_peered":"2023-10-08T13:49:44.788831+0000","last_unstale":"2023-10-08T13:51:15.783122+0000","last_undegraded":"2023-10-08T13:51:15.783122+0000","last_fullsized":"2023-10-08T13:51:15.783122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:45.871900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785476+0000","last_change":"2023-10-08T13:49:49.157319+0000","last_active":"2023-10-08T13:51:15.785476+0000","last_peered":"2023-10-08T13:51:15.785476+0000","last_clean":"2023-10-08T13:51:15.785476+0000","last_became_active":"2023-10-08T13:49:49.156897+0000","last_became_peered":"2023-10-08T13:49:49.156897+0000","last_unstale":"2023-10-08T13:51:15.785476+0000","last_undegraded":"2023-10-08T13:51:15.785476+0000","last_fullsized":"2023-10-08T13:51:15.785476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:25.460707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787914+0000","last_change":"2023-10-08T13:49:43.776699+0000","last_active":"2023-10-08T13:51:15.787914+0000","last_peered":"2023-10-08T13:51:15.787914+0000","last_clean":"2023-10-08T13:51:15.787914+0000","last_became_active":"2023-10-08T13:49:43.776293+0000","last_became_peered":"2023-10-08T13:49:43.776293+0000","last_unstale":"2023-10-08T13:51:15.787914+0000","last_undegraded":"2023-10-08T13:51:15.787914+0000","last_fullsized":"2023-10-08T13:51:15.787914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:19.056804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808734+0000","last_change":"2023-10-08T13:49:44.789915+0000","last_active":"2023-10-08T13:51:15.808734+0000","last_peered":"2023-10-08T13:51:15.808734+0000","last_clean":"2023-10-08T13:51:15.808734+0000","last_became_active":"2023-10-08T13:49:44.789617+0000","last_became_peered":"2023-10-08T13:49:44.789617+0000","last_unstale":"2023-10-08T13:51:15.808734+0000","last_undegraded":"2023-10-08T13:51:15.808734+0000","last_fullsized":"2023-10-08T13:51:15.808734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:06.286155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808527+0000","last_change":"2023-10-08T13:49:49.165946+0000","last_active":"2023-10-08T13:51:15.808527+0000","last_peered":"2023-10-08T13:51:15.808527+0000","last_clean":"2023-10-08T13:51:15.808527+0000","last_became_active":"2023-10-08T13:49:49.165581+0000","last_became_peered":"2023-10-08T13:49:49.165581+0000","last_unstale":"2023-10-08T13:51:15.808527+0000","last_undegraded":"2023-10-08T13:51:15.808527+0000","last_fullsized":"2023-10-08T13:51:15.808527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:13.772198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784836+0000","last_change":"2023-10-08T13:49:43.779161+0000","last_active":"2023-10-08T13:51:15.784836+0000","last_peered":"2023-10-08T13:51:15.784836+0000","last_clean":"2023-10-08T13:51:15.784836+0000","last_became_active":"2023-10-08T13:49:43.778862+0000","last_became_peered":"2023-10-08T13:49:43.778862+0000","last_unstale":"2023-10-08T13:51:15.784836+0000","last_undegraded":"2023-10-08T13:51:15.784836+0000","last_fullsized":"2023-10-08T13:51:15.784836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:57:44.075322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809115+0000","last_change":"2023-10-08T13:49:44.782072+0000","last_active":"2023-10-08T13:51:15.809115+0000","last_peered":"2023-10-08T13:51:15.809115+0000","last_clean":"2023-10-08T13:51:15.809115+0000","last_became_active":"2023-10-08T13:49:44.781756+0000","last_became_peered":"2023-10-08T13:49:44.781756+0000","last_unstale":"2023-10-08T13:51:15.809115+0000","last_undegraded":"2023-10-08T13:51:15.809115+0000","last_fullsized":"2023-10-08T13:51:15.809115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:32.877818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783652+0000","last_change":"2023-10-08T13:50:16.099755+0000","last_active":"2023-10-08T13:51:15.783652+0000","last_peered":"2023-10-08T13:51:15.783652+0000","last_clean":"2023-10-08T13:51:15.783652+0000","last_became_active":"2023-10-08T13:50:16.099267+0000","last_became_peered":"2023-10-08T13:50:16.099267+0000","last_unstale":"2023-10-08T13:51:15.783652+0000","last_undegraded":"2023-10-08T13:51:15.783652+0000","last_fullsized":"2023-10-08T13:51:15.783652+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:19.949246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.860207+0000","last_change":"2023-10-08T13:49:43.782822+0000","last_active":"2023-10-08T13:51:21.860207+0000","last_peered":"2023-10-08T13:51:21.860207+0000","last_clean":"2023-10-08T13:51:21.860207+0000","last_became_active":"2023-10-08T13:49:43.782524+0000","last_became_peered":"2023-10-08T13:49:43.782524+0000","last_unstale":"2023-10-08T13:51:21.860207+0000","last_undegraded":"2023-10-08T13:51:21.860207+0000","last_fullsized":"2023-10-08T13:51:21.860207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:11.847851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808786+0000","last_change":"2023-10-08T13:49:44.794296+0000","last_active":"2023-10-08T13:51:15.808786+0000","last_peered":"2023-10-08T13:51:15.808786+0000","last_clean":"2023-10-08T13:51:15.808786+0000","last_became_active":"2023-10-08T13:49:44.793987+0000","last_became_peered":"2023-10-08T13:49:44.793987+0000","last_unstale":"2023-10-08T13:51:15.808786+0000","last_undegraded":"2023-10-08T13:51:15.808786+0000","last_fullsized":"2023-10-08T13:51:15.808786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:41.562569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809550+0000","last_change":"2023-10-08T13:49:49.165016+0000","last_active":"2023-10-08T13:51:15.809550+0000","last_peered":"2023-10-08T13:51:15.809550+0000","last_clean":"2023-10-08T13:51:15.809550+0000","last_became_active":"2023-10-08T13:49:49.164531+0000","last_became_peered":"2023-10-08T13:49:49.164531+0000","last_unstale":"2023-10-08T13:51:15.809550+0000","last_undegraded":"2023-10-08T13:51:15.809550+0000","last_fullsized":"2023-10-08T13:51:15.809550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:45.014512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782774+0000","last_change":"2023-10-08T13:49:43.786929+0000","last_active":"2023-10-08T13:51:15.782774+0000","last_peered":"2023-10-08T13:51:15.782774+0000","last_clean":"2023-10-08T13:51:15.782774+0000","last_became_active":"2023-10-08T13:49:43.786641+0000","last_became_peered":"2023-10-08T13:49:43.786641+0000","last_unstale":"2023-10-08T13:51:15.782774+0000","last_undegraded":"2023-10-08T13:51:15.782774+0000","last_fullsized":"2023-10-08T13:51:15.782774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:30.054067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:16.851434+0000","last_change":"2023-10-08T13:51:16.850958+0000","last_active":"2023-10-08T13:51:16.851434+0000","last_peered":"2023-10-08T13:51:16.851434+0000","last_clean":"2023-10-08T13:51:16.851434+0000","last_became_active":"2023-10-08T13:51:16.850648+0000","last_became_peered":"2023-10-08T13:51:16.850648+0000","last_unstale":"2023-10-08T13:51:16.851434+0000","last_undegraded":"2023-10-08T13:51:16.851434+0000","last_fullsized":"2023-10-08T13:51:16.851434+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:30.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784760+0000","last_change":"2023-10-08T13:49:49.152226+0000","last_active":"2023-10-08T13:51:15.784760+0000","last_peered":"2023-10-08T13:51:15.784760+0000","last_clean":"2023-10-08T13:51:15.784760+0000","last_became_active":"2023-10-08T13:49:49.151772+0000","last_became_peered":"2023-10-08T13:49:49.151772+0000","last_unstale":"2023-10-08T13:51:15.784760+0000","last_undegraded":"2023-10-08T13:51:15.784760+0000","last_fullsized":"2023-10-08T13:51:15.784760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:53.856496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808693+0000","last_change":"2023-10-08T13:50:15.793598+0000","last_active":"2023-10-08T13:51:15.808693+0000","last_peered":"2023-10-08T13:51:15.808693+0000","last_clean":"2023-10-08T13:51:15.808693+0000","last_became_active":"2023-10-08T13:50:15.786421+0000","last_became_peered":"2023-10-08T13:50:15.786421+0000","last_unstale":"2023-10-08T13:51:15.808693+0000","last_undegraded":"2023-10-08T13:51:15.808693+0000","last_fullsized":"2023-10-08T13:51:15.808693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:09:37.586245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808647+0000","last_change":"2023-10-08T13:49:44.782805+0000","last_active":"2023-10-08T13:51:15.808647+0000","last_peered":"2023-10-08T13:51:15.808647+0000","last_clean":"2023-10-08T13:51:15.808647+0000","last_became_active":"2023-10-08T13:49:44.782485+0000","last_became_peered":"2023-10-08T13:49:44.782485+0000","last_unstale":"2023-10-08T13:51:15.808647+0000","last_undegraded":"2023-10-08T13:51:15.808647+0000","last_fullsized":"2023-10-08T13:51:15.808647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:25:19.805734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783005+0000","last_change":"2023-10-08T13:49:49.546226+0000","last_active":"2023-10-08T13:51:15.783005+0000","last_peered":"2023-10-08T13:51:15.783005+0000","last_clean":"2023-10-08T13:51:15.783005+0000","last_became_active":"2023-10-08T13:49:49.545626+0000","last_became_peered":"2023-10-08T13:49:49.545626+0000","last_unstale":"2023-10-08T13:51:15.783005+0000","last_undegraded":"2023-10-08T13:51:15.783005+0000","last_fullsized":"2023-10-08T13:51:15.783005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:37.184944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786007+0000","last_change":"2023-10-08T13:49:43.784705+0000","last_active":"2023-10-08T13:51:15.786007+0000","last_peered":"2023-10-08T13:51:15.786007+0000","last_clean":"2023-10-08T13:51:15.786007+0000","last_became_active":"2023-10-08T13:49:43.784391+0000","last_became_peered":"2023-10-08T13:49:43.784391+0000","last_unstale":"2023-10-08T13:51:15.786007+0000","last_undegraded":"2023-10-08T13:51:15.786007+0000","last_fullsized":"2023-10-08T13:51:15.786007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:45.154880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.810046+0000","last_change":"2023-10-08T13:49:44.789845+0000","last_active":"2023-10-08T13:51:15.810046+0000","last_peered":"2023-10-08T13:51:15.810046+0000","last_clean":"2023-10-08T13:51:15.810046+0000","last_became_active":"2023-10-08T13:49:44.789580+0000","last_became_peered":"2023-10-08T13:49:44.789580+0000","last_unstale":"2023-10-08T13:51:15.810046+0000","last_undegraded":"2023-10-08T13:51:15.810046+0000","last_fullsized":"2023-10-08T13:51:15.810046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:58.098757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809069+0000","last_change":"2023-10-08T13:49:49.153277+0000","last_active":"2023-10-08T13:51:15.809069+0000","last_peered":"2023-10-08T13:51:15.809069+0000","last_clean":"2023-10-08T13:51:15.809069+0000","last_became_active":"2023-10-08T13:49:49.152575+0000","last_became_peered":"2023-10-08T13:49:49.152575+0000","last_unstale":"2023-10-08T13:51:15.809069+0000","last_undegraded":"2023-10-08T13:51:15.809069+0000","last_fullsized":"2023-10-08T13:51:15.809069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:35.677823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:21.852851+0000","last_change":"2023-10-08T13:49:43.778442+0000","last_active":"2023-10-08T13:51:21.852851+0000","last_peered":"2023-10-08T13:51:21.852851+0000","last_clean":"2023-10-08T13:51:21.852851+0000","last_became_active":"2023-10-08T13:49:43.774799+0000","last_became_peered":"2023-10-08T13:49:43.774799+0000","last_unstale":"2023-10-08T13:51:21.852851+0000","last_undegraded":"2023-10-08T13:51:21.852851+0000","last_fullsized":"2023-10-08T13:51:21.852851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:43:15.464736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809560+0000","last_change":"2023-10-08T13:49:44.780097+0000","last_active":"2023-10-08T13:51:15.809560+0000","last_peered":"2023-10-08T13:51:15.809560+0000","last_clean":"2023-10-08T13:51:15.809560+0000","last_became_active":"2023-10-08T13:49:44.779774+0000","last_became_peered":"2023-10-08T13:49:44.779774+0000","last_unstale":"2023-10-08T13:51:15.809560+0000","last_undegraded":"2023-10-08T13:51:15.809560+0000","last_fullsized":"2023-10-08T13:51:15.809560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:19.132295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808399+0000","last_change":"2023-10-08T13:49:49.547129+0000","last_active":"2023-10-08T13:51:15.808399+0000","last_peered":"2023-10-08T13:51:15.808399+0000","last_clean":"2023-10-08T13:51:15.808399+0000","last_became_active":"2023-10-08T13:49:49.546694+0000","last_became_peered":"2023-10-08T13:49:49.546694+0000","last_unstale":"2023-10-08T13:51:15.808399+0000","last_undegraded":"2023-10-08T13:51:15.808399+0000","last_fullsized":"2023-10-08T13:51:15.808399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:45.524650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784878+0000","last_change":"2023-10-08T13:49:44.779875+0000","last_active":"2023-10-08T13:51:15.784878+0000","last_peered":"2023-10-08T13:51:15.784878+0000","last_clean":"2023-10-08T13:51:15.784878+0000","last_became_active":"2023-10-08T13:49:44.779570+0000","last_became_peered":"2023-10-08T13:49:44.779570+0000","last_unstale":"2023-10-08T13:51:15.784878+0000","last_undegraded":"2023-10-08T13:51:15.784878+0000","last_fullsized":"2023-10-08T13:51:15.784878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:02.981998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:23.862244+0000","last_change":"2023-10-08T13:49:43.787574+0000","last_active":"2023-10-08T13:51:23.862244+0000","last_peered":"2023-10-08T13:51:23.862244+0000","last_clean":"2023-10-08T13:51:23.862244+0000","last_became_active":"2023-10-08T13:49:43.787261+0000","last_became_peered":"2023-10-08T13:49:43.787261+0000","last_unstale":"2023-10-08T13:51:23.862244+0000","last_undegraded":"2023-10-08T13:51:23.862244+0000","last_fullsized":"2023-10-08T13:51:23.862244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:28.676135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809431+0000","last_change":"2023-10-08T13:49:49.163856+0000","last_active":"2023-10-08T13:51:15.809431+0000","last_peered":"2023-10-08T13:51:15.809431+0000","last_clean":"2023-10-08T13:51:15.809431+0000","last_became_active":"2023-10-08T13:49:49.163456+0000","last_became_peered":"2023-10-08T13:49:49.163456+0000","last_unstale":"2023-10-08T13:51:15.809431+0000","last_undegraded":"2023-10-08T13:51:15.809431+0000","last_fullsized":"2023-10-08T13:51:15.809431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:31.654309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788320+0000","last_change":"2023-10-08T13:49:44.792935+0000","last_active":"2023-10-08T13:51:15.788320+0000","last_peered":"2023-10-08T13:51:15.788320+0000","last_clean":"2023-10-08T13:51:15.788320+0000","last_became_active":"2023-10-08T13:49:44.792667+0000","last_became_peered":"2023-10-08T13:49:44.792667+0000","last_unstale":"2023-10-08T13:51:15.788320+0000","last_undegraded":"2023-10-08T13:51:15.788320+0000","last_fullsized":"2023-10-08T13:51:15.788320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:40.964452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783131+0000","last_change":"2023-10-08T13:49:43.781424+0000","last_active":"2023-10-08T13:51:15.783131+0000","last_peered":"2023-10-08T13:51:15.783131+0000","last_clean":"2023-10-08T13:51:15.783131+0000","last_became_active":"2023-10-08T13:49:43.780930+0000","last_became_peered":"2023-10-08T13:49:43.780930+0000","last_unstale":"2023-10-08T13:51:15.783131+0000","last_undegraded":"2023-10-08T13:51:15.783131+0000","last_fullsized":"2023-10-08T13:51:15.783131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:56:50.397289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808562+0000","last_change":"2023-10-08T13:48:45.192219+0000","last_active":"2023-10-08T13:51:15.808562+0000","last_peered":"2023-10-08T13:51:15.808562+0000","last_clean":"2023-10-08T13:51:15.808562+0000","last_became_active":"2023-10-08T13:48:45.191813+0000","last_became_peered":"2023-10-08T13:48:45.191813+0000","last_unstale":"2023-10-08T13:51:15.808562+0000","last_undegraded":"2023-10-08T13:51:15.808562+0000","last_fullsized":"2023-10-08T13:51:15.808562+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:45:51.040695+0000","last_clean_scrub_stamp":"2023-10-08T13:45:51.040695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.303937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785025+0000","last_change":"2023-10-08T13:49:49.546181+0000","last_active":"2023-10-08T13:51:15.785025+0000","last_peered":"2023-10-08T13:51:15.785025+0000","last_clean":"2023-10-08T13:51:15.785025+0000","last_became_active":"2023-10-08T13:49:49.545726+0000","last_became_peered":"2023-10-08T13:49:49.545726+0000","last_unstale":"2023-10-08T13:51:15.785025+0000","last_undegraded":"2023-10-08T13:51:15.785025+0000","last_fullsized":"2023-10-08T13:51:15.785025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:45.370909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809633+0000","last_change":"2023-10-08T13:49:44.785544+0000","last_active":"2023-10-08T13:51:15.809633+0000","last_peered":"2023-10-08T13:51:15.809633+0000","last_clean":"2023-10-08T13:51:15.809633+0000","last_became_active":"2023-10-08T13:49:44.785007+0000","last_became_peered":"2023-10-08T13:49:44.785007+0000","last_unstale":"2023-10-08T13:51:15.809633+0000","last_undegraded":"2023-10-08T13:51:15.809633+0000","last_fullsized":"2023-10-08T13:51:15.809633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:16:55.058693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808677+0000","last_change":"2023-10-08T13:49:43.779207+0000","last_active":"2023-10-08T13:51:15.808677+0000","last_peered":"2023-10-08T13:51:15.808677+0000","last_clean":"2023-10-08T13:51:15.808677+0000","last_became_active":"2023-10-08T13:49:43.778846+0000","last_became_peered":"2023-10-08T13:49:43.778846+0000","last_unstale":"2023-10-08T13:51:15.808677+0000","last_undegraded":"2023-10-08T13:51:15.808677+0000","last_fullsized":"2023-10-08T13:51:15.808677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:50:45.868187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808454+0000","last_change":"2023-10-08T13:49:49.162156+0000","last_active":"2023-10-08T13:51:15.808454+0000","last_peered":"2023-10-08T13:51:15.808454+0000","last_clean":"2023-10-08T13:51:15.808454+0000","last_became_active":"2023-10-08T13:49:49.161720+0000","last_became_peered":"2023-10-08T13:49:49.161720+0000","last_unstale":"2023-10-08T13:51:15.808454+0000","last_undegraded":"2023-10-08T13:51:15.808454+0000","last_fullsized":"2023-10-08T13:51:15.808454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:13.499915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783327+0000","last_change":"2023-10-08T13:49:43.782322+0000","last_active":"2023-10-08T13:51:15.783327+0000","last_peered":"2023-10-08T13:51:15.783327+0000","last_clean":"2023-10-08T13:51:15.783327+0000","last_became_active":"2023-10-08T13:49:43.781938+0000","last_became_peered":"2023-10-08T13:49:43.781938+0000","last_unstale":"2023-10-08T13:51:15.783327+0000","last_undegraded":"2023-10-08T13:51:15.783327+0000","last_fullsized":"2023-10-08T13:51:15.783327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:57.627507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809219+0000","last_change":"2023-10-08T13:49:44.778340+0000","last_active":"2023-10-08T13:51:15.809219+0000","last_peered":"2023-10-08T13:51:15.809219+0000","last_clean":"2023-10-08T13:51:15.809219+0000","last_became_active":"2023-10-08T13:49:44.778000+0000","last_became_peered":"2023-10-08T13:49:44.778000+0000","last_unstale":"2023-10-08T13:51:15.809219+0000","last_undegraded":"2023-10-08T13:51:15.809219+0000","last_fullsized":"2023-10-08T13:51:15.809219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:23.350449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787735+0000","last_change":"2023-10-08T13:49:49.545915+0000","last_active":"2023-10-08T13:51:15.787735+0000","last_peered":"2023-10-08T13:51:15.787735+0000","last_clean":"2023-10-08T13:51:15.787735+0000","last_became_active":"2023-10-08T13:49:49.545396+0000","last_became_peered":"2023-10-08T13:49:49.545396+0000","last_unstale":"2023-10-08T13:51:15.787735+0000","last_undegraded":"2023-10-08T13:51:15.787735+0000","last_fullsized":"2023-10-08T13:51:15.787735+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:07:03.850707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:16.798882+0000","last_change":"2023-10-08T13:49:43.776479+0000","last_active":"2023-10-08T13:51:16.798882+0000","last_peered":"2023-10-08T13:51:16.798882+0000","last_clean":"2023-10-08T13:51:16.798882+0000","last_became_active":"2023-10-08T13:49:43.776130+0000","last_became_peered":"2023-10-08T13:49:43.776130+0000","last_unstale":"2023-10-08T13:51:16.798882+0000","last_undegraded":"2023-10-08T13:51:16.798882+0000","last_fullsized":"2023-10-08T13:51:16.798882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:36.760660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788025+0000","last_change":"2023-10-08T13:49:44.793864+0000","last_active":"2023-10-08T13:51:15.788025+0000","last_peered":"2023-10-08T13:51:15.788025+0000","last_clean":"2023-10-08T13:51:15.788025+0000","last_became_active":"2023-10-08T13:49:44.793469+0000","last_became_peered":"2023-10-08T13:49:44.793469+0000","last_unstale":"2023-10-08T13:51:15.788025+0000","last_undegraded":"2023-10-08T13:51:15.788025+0000","last_fullsized":"2023-10-08T13:51:15.788025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:28.669711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787773+0000","last_change":"2023-10-08T13:49:49.150696+0000","last_active":"2023-10-08T13:51:15.787773+0000","last_peered":"2023-10-08T13:51:15.787773+0000","last_clean":"2023-10-08T13:51:15.787773+0000","last_became_active":"2023-10-08T13:49:49.150188+0000","last_became_peered":"2023-10-08T13:49:49.150188+0000","last_unstale":"2023-10-08T13:51:15.787773+0000","last_undegraded":"2023-10-08T13:51:15.787773+0000","last_fullsized":"2023-10-08T13:51:15.787773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:51:35.775240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783459+0000","last_change":"2023-10-08T13:49:43.786245+0000","last_active":"2023-10-08T13:51:15.783459+0000","last_peered":"2023-10-08T13:51:15.783459+0000","last_clean":"2023-10-08T13:51:15.783459+0000","last_became_active":"2023-10-08T13:49:43.785986+0000","last_became_peered":"2023-10-08T13:49:43.785986+0000","last_unstale":"2023-10-08T13:51:15.783459+0000","last_undegraded":"2023-10-08T13:51:15.783459+0000","last_fullsized":"2023-10-08T13:51:15.783459+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:53:59.643447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808953+0000","last_change":"2023-10-08T13:49:44.790494+0000","last_active":"2023-10-08T13:51:15.808953+0000","last_peered":"2023-10-08T13:51:15.808953+0000","last_clean":"2023-10-08T13:51:15.808953+0000","last_became_active":"2023-10-08T13:49:44.790091+0000","last_became_peered":"2023-10-08T13:49:44.790091+0000","last_unstale":"2023-10-08T13:51:15.808953+0000","last_undegraded":"2023-10-08T13:51:15.808953+0000","last_fullsized":"2023-10-08T13:51:15.808953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:49.309597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782982+0000","last_change":"2023-10-08T13:49:49.547271+0000","last_active":"2023-10-08T13:51:15.782982+0000","last_peered":"2023-10-08T13:51:15.782982+0000","last_clean":"2023-10-08T13:51:15.782982+0000","last_became_active":"2023-10-08T13:49:49.546603+0000","last_became_peered":"2023-10-08T13:49:49.546603+0000","last_unstale":"2023-10-08T13:51:15.782982+0000","last_undegraded":"2023-10-08T13:51:15.782982+0000","last_fullsized":"2023-10-08T13:51:15.782982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:29.908321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786664+0000","last_change":"2023-10-08T13:49:43.788638+0000","last_active":"2023-10-08T13:51:15.786664+0000","last_peered":"2023-10-08T13:51:15.786664+0000","last_clean":"2023-10-08T13:51:15.786664+0000","last_became_active":"2023-10-08T13:49:43.788303+0000","last_became_peered":"2023-10-08T13:49:43.788303+0000","last_unstale":"2023-10-08T13:51:15.786664+0000","last_undegraded":"2023-10-08T13:51:15.786664+0000","last_fullsized":"2023-10-08T13:51:15.786664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:52.637775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782787+0000","last_change":"2023-10-08T13:49:44.777546+0000","last_active":"2023-10-08T13:51:15.782787+0000","last_peered":"2023-10-08T13:51:15.782787+0000","last_clean":"2023-10-08T13:51:15.782787+0000","last_became_active":"2023-10-08T13:49:44.777261+0000","last_became_peered":"2023-10-08T13:49:44.777261+0000","last_unstale":"2023-10-08T13:51:15.782787+0000","last_undegraded":"2023-10-08T13:51:15.782787+0000","last_fullsized":"2023-10-08T13:51:15.782787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:26:36.368188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.783728+0000","last_change":"2023-10-08T13:50:15.787317+0000","last_active":"2023-10-08T13:51:15.783728+0000","last_peered":"2023-10-08T13:51:15.783728+0000","last_clean":"2023-10-08T13:51:15.783728+0000","last_became_active":"2023-10-08T13:50:15.786816+0000","last_became_peered":"2023-10-08T13:50:15.786816+0000","last_unstale":"2023-10-08T13:51:15.783728+0000","last_undegraded":"2023-10-08T13:51:15.783728+0000","last_fullsized":"2023-10-08T13:51:15.783728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:53.766803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788167+0000","last_change":"2023-10-08T13:49:43.776693+0000","last_active":"2023-10-08T13:51:15.788167+0000","last_peered":"2023-10-08T13:51:15.788167+0000","last_clean":"2023-10-08T13:51:15.788167+0000","last_became_active":"2023-10-08T13:49:43.776367+0000","last_became_peered":"2023-10-08T13:49:43.776367+0000","last_unstale":"2023-10-08T13:51:15.788167+0000","last_undegraded":"2023-10-08T13:51:15.788167+0000","last_fullsized":"2023-10-08T13:51:15.788167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:17.415011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808348+0000","last_change":"2023-10-08T13:49:44.779472+0000","last_active":"2023-10-08T13:51:15.808348+0000","last_peered":"2023-10-08T13:51:15.808348+0000","last_clean":"2023-10-08T13:51:15.808348+0000","last_became_active":"2023-10-08T13:49:44.779172+0000","last_became_peered":"2023-10-08T13:49:44.779172+0000","last_unstale":"2023-10-08T13:51:15.808348+0000","last_undegraded":"2023-10-08T13:51:15.808348+0000","last_fullsized":"2023-10-08T13:51:15.808348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:17.314243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782946+0000","last_change":"2023-10-08T13:49:49.164535+0000","last_active":"2023-10-08T13:51:15.782946+0000","last_peered":"2023-10-08T13:51:15.782946+0000","last_clean":"2023-10-08T13:51:15.782946+0000","last_became_active":"2023-10-08T13:49:49.163467+0000","last_became_peered":"2023-10-08T13:49:49.163467+0000","last_unstale":"2023-10-08T13:51:15.782946+0000","last_undegraded":"2023-10-08T13:51:15.782946+0000","last_fullsized":"2023-10-08T13:51:15.782946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:55.058656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787896+0000","last_change":"2023-10-08T13:49:43.775311+0000","last_active":"2023-10-08T13:51:15.787896+0000","last_peered":"2023-10-08T13:51:15.787896+0000","last_clean":"2023-10-08T13:51:15.787896+0000","last_became_active":"2023-10-08T13:49:43.775005+0000","last_became_peered":"2023-10-08T13:49:43.775005+0000","last_unstale":"2023-10-08T13:51:15.787896+0000","last_undegraded":"2023-10-08T13:51:15.787896+0000","last_fullsized":"2023-10-08T13:51:15.787896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:33.713395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788118+0000","last_change":"2023-10-08T13:49:44.792051+0000","last_active":"2023-10-08T13:51:15.788118+0000","last_peered":"2023-10-08T13:51:15.788118+0000","last_clean":"2023-10-08T13:51:15.788118+0000","last_became_active":"2023-10-08T13:49:44.790045+0000","last_became_peered":"2023-10-08T13:49:44.790045+0000","last_unstale":"2023-10-08T13:51:15.788118+0000","last_undegraded":"2023-10-08T13:51:15.788118+0000","last_fullsized":"2023-10-08T13:51:15.788118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:28.699799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782910+0000","last_change":"2023-10-08T13:49:49.546219+0000","last_active":"2023-10-08T13:51:15.782910+0000","last_peered":"2023-10-08T13:51:15.782910+0000","last_clean":"2023-10-08T13:51:15.782910+0000","last_became_active":"2023-10-08T13:49:49.545783+0000","last_became_peered":"2023-10-08T13:49:49.545783+0000","last_unstale":"2023-10-08T13:51:15.782910+0000","last_undegraded":"2023-10-08T13:51:15.782910+0000","last_fullsized":"2023-10-08T13:51:15.782910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:00.497908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.782922+0000","last_change":"2023-10-08T13:49:43.776252+0000","last_active":"2023-10-08T13:51:15.782922+0000","last_peered":"2023-10-08T13:51:15.782922+0000","last_clean":"2023-10-08T13:51:15.782922+0000","last_became_active":"2023-10-08T13:49:43.775935+0000","last_became_peered":"2023-10-08T13:49:43.775935+0000","last_unstale":"2023-10-08T13:51:15.782922+0000","last_undegraded":"2023-10-08T13:51:15.782922+0000","last_fullsized":"2023-10-08T13:51:15.782922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:58.949365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809345+0000","last_change":"2023-10-08T13:49:44.788628+0000","last_active":"2023-10-08T13:51:15.809345+0000","last_peered":"2023-10-08T13:51:15.809345+0000","last_clean":"2023-10-08T13:51:15.809345+0000","last_became_active":"2023-10-08T13:49:44.788305+0000","last_became_peered":"2023-10-08T13:49:44.788305+0000","last_unstale":"2023-10-08T13:51:15.809345+0000","last_undegraded":"2023-10-08T13:51:15.809345+0000","last_fullsized":"2023-10-08T13:51:15.809345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:46:42.311296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786697+0000","last_change":"2023-10-08T13:49:49.153641+0000","last_active":"2023-10-08T13:51:15.786697+0000","last_peered":"2023-10-08T13:51:15.786697+0000","last_clean":"2023-10-08T13:51:15.786697+0000","last_became_active":"2023-10-08T13:49:49.153172+0000","last_became_peered":"2023-10-08T13:49:49.153172+0000","last_unstale":"2023-10-08T13:51:15.786697+0000","last_undegraded":"2023-10-08T13:51:15.786697+0000","last_fullsized":"2023-10-08T13:51:15.786697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:40:51.726343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-08T13:51:22.853599+0000","last_change":"2023-10-08T13:49:43.782683+0000","last_active":"2023-10-08T13:51:22.853599+0000","last_peered":"2023-10-08T13:51:22.853599+0000","last_clean":"2023-10-08T13:51:22.853599+0000","last_became_active":"2023-10-08T13:49:43.782269+0000","last_became_peered":"2023-10-08T13:49:43.782269+0000","last_unstale":"2023-10-08T13:51:22.853599+0000","last_undegraded":"2023-10-08T13:51:22.853599+0000","last_fullsized":"2023-10-08T13:51:22.853599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:50.850581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809118+0000","last_change":"2023-10-08T13:49:44.787894+0000","last_active":"2023-10-08T13:51:15.809118+0000","last_peered":"2023-10-08T13:51:15.809118+0000","last_clean":"2023-10-08T13:51:15.809118+0000","last_became_active":"2023-10-08T13:49:44.787599+0000","last_became_peered":"2023-10-08T13:49:44.787599+0000","last_unstale":"2023-10-08T13:51:15.809118+0000","last_undegraded":"2023-10-08T13:51:15.809118+0000","last_fullsized":"2023-10-08T13:51:15.809118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:17:15.131051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808379+0000","last_change":"2023-10-08T13:49:49.164256+0000","last_active":"2023-10-08T13:51:15.808379+0000","last_peered":"2023-10-08T13:51:15.808379+0000","last_clean":"2023-10-08T13:51:15.808379+0000","last_became_active":"2023-10-08T13:49:49.163793+0000","last_became_peered":"2023-10-08T13:49:49.163793+0000","last_unstale":"2023-10-08T13:51:15.808379+0000","last_undegraded":"2023-10-08T13:51:15.808379+0000","last_fullsized":"2023-10-08T13:51:15.808379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:19.556886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.785320+0000","last_change":"2023-10-08T13:49:43.778311+0000","last_active":"2023-10-08T13:51:15.785320+0000","last_peered":"2023-10-08T13:51:15.785320+0000","last_clean":"2023-10-08T13:51:15.785320+0000","last_became_active":"2023-10-08T13:49:43.777984+0000","last_became_peered":"2023-10-08T13:49:43.777984+0000","last_unstale":"2023-10-08T13:51:15.785320+0000","last_undegraded":"2023-10-08T13:51:15.785320+0000","last_fullsized":"2023-10-08T13:51:15.785320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:56.890729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808931+0000","last_change":"2023-10-08T13:49:44.793424+0000","last_active":"2023-10-08T13:51:15.808931+0000","last_peered":"2023-10-08T13:51:15.808931+0000","last_clean":"2023-10-08T13:51:15.808931+0000","last_became_active":"2023-10-08T13:49:44.792825+0000","last_became_peered":"2023-10-08T13:49:44.792825+0000","last_unstale":"2023-10-08T13:51:15.808931+0000","last_undegraded":"2023-10-08T13:51:15.808931+0000","last_fullsized":"2023-10-08T13:51:15.808931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:39.463676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784486+0000","last_change":"2023-10-08T13:49:49.151915+0000","last_active":"2023-10-08T13:51:15.784486+0000","last_peered":"2023-10-08T13:51:15.784486+0000","last_clean":"2023-10-08T13:51:15.784486+0000","last_became_active":"2023-10-08T13:49:49.151434+0000","last_became_peered":"2023-10-08T13:49:49.151434+0000","last_unstale":"2023-10-08T13:51:15.784486+0000","last_undegraded":"2023-10-08T13:51:15.784486+0000","last_fullsized":"2023-10-08T13:51:15.784486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:48.129338+0000","last_clean_scrub_stamp":"2023-10-08T13:49:48.129338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:54:07.631498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809499+0000","last_change":"2023-10-08T13:50:15.787243+0000","last_active":"2023-10-08T13:51:15.809499+0000","last_peered":"2023-10-08T13:51:15.809499+0000","last_clean":"2023-10-08T13:51:15.809499+0000","last_became_active":"2023-10-08T13:50:15.782305+0000","last_became_peered":"2023-10-08T13:50:15.782305+0000","last_unstale":"2023-10-08T13:51:15.809499+0000","last_undegraded":"2023-10-08T13:51:15.809499+0000","last_fullsized":"2023-10-08T13:51:15.809499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:42.746495+0000","last_clean_scrub_stamp":"2023-10-08T13:49:42.746495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:08.520010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809254+0000","last_change":"2023-10-08T13:49:44.790949+0000","last_active":"2023-10-08T13:51:15.809254+0000","last_peered":"2023-10-08T13:51:15.809254+0000","last_clean":"2023-10-08T13:51:15.809254+0000","last_became_active":"2023-10-08T13:49:44.790645+0000","last_became_peered":"2023-10-08T13:49:44.790645+0000","last_unstale":"2023-10-08T13:51:15.809254+0000","last_undegraded":"2023-10-08T13:51:15.809254+0000","last_fullsized":"2023-10-08T13:51:15.809254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:48.890730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808397+0000","last_change":"2023-10-08T13:49:44.781165+0000","last_active":"2023-10-08T13:51:15.808397+0000","last_peered":"2023-10-08T13:51:15.808397+0000","last_clean":"2023-10-08T13:51:15.808397+0000","last_became_active":"2023-10-08T13:49:44.780828+0000","last_became_peered":"2023-10-08T13:49:44.780828+0000","last_unstale":"2023-10-08T13:51:15.808397+0000","last_undegraded":"2023-10-08T13:51:15.808397+0000","last_fullsized":"2023-10-08T13:51:15.808397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:04.259482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788103+0000","last_change":"2023-10-08T13:49:44.781271+0000","last_active":"2023-10-08T13:51:15.788103+0000","last_peered":"2023-10-08T13:51:15.788103+0000","last_clean":"2023-10-08T13:51:15.788103+0000","last_became_active":"2023-10-08T13:49:44.780912+0000","last_became_peered":"2023-10-08T13:49:44.780912+0000","last_unstale":"2023-10-08T13:51:15.788103+0000","last_undegraded":"2023-10-08T13:51:15.788103+0000","last_fullsized":"2023-10-08T13:51:15.788103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:17.875747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.787876+0000","last_change":"2023-10-08T13:49:44.777613+0000","last_active":"2023-10-08T13:51:15.787876+0000","last_peered":"2023-10-08T13:51:15.787876+0000","last_clean":"2023-10-08T13:51:15.787876+0000","last_became_active":"2023-10-08T13:49:44.777304+0000","last_became_peered":"2023-10-08T13:49:44.777304+0000","last_unstale":"2023-10-08T13:51:15.787876+0000","last_undegraded":"2023-10-08T13:51:15.787876+0000","last_fullsized":"2023-10-08T13:51:15.787876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:45.482935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808572+0000","last_change":"2023-10-08T13:49:44.782685+0000","last_active":"2023-10-08T13:51:15.808572+0000","last_peered":"2023-10-08T13:51:15.808572+0000","last_clean":"2023-10-08T13:51:15.808572+0000","last_became_active":"2023-10-08T13:49:44.781816+0000","last_became_peered":"2023-10-08T13:49:44.781816+0000","last_unstale":"2023-10-08T13:51:15.808572+0000","last_undegraded":"2023-10-08T13:51:15.808572+0000","last_fullsized":"2023-10-08T13:51:15.808572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:53.781237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809306+0000","last_change":"2023-10-08T13:49:44.783973+0000","last_active":"2023-10-08T13:51:15.809306+0000","last_peered":"2023-10-08T13:51:15.809306+0000","last_clean":"2023-10-08T13:51:15.809306+0000","last_became_active":"2023-10-08T13:49:44.783648+0000","last_became_peered":"2023-10-08T13:49:44.783648+0000","last_unstale":"2023-10-08T13:51:15.809306+0000","last_undegraded":"2023-10-08T13:51:15.809306+0000","last_fullsized":"2023-10-08T13:51:15.809306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:01.253610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808667+0000","last_change":"2023-10-08T13:49:44.781848+0000","last_active":"2023-10-08T13:51:15.808667+0000","last_peered":"2023-10-08T13:51:15.808667+0000","last_clean":"2023-10-08T13:51:15.808667+0000","last_became_active":"2023-10-08T13:49:44.781542+0000","last_became_peered":"2023-10-08T13:49:44.781542+0000","last_unstale":"2023-10-08T13:51:15.808667+0000","last_undegraded":"2023-10-08T13:51:15.808667+0000","last_fullsized":"2023-10-08T13:51:15.808667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:26:38.726298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788486+0000","last_change":"2023-10-08T13:49:44.786541+0000","last_active":"2023-10-08T13:51:15.788486+0000","last_peered":"2023-10-08T13:51:15.788486+0000","last_clean":"2023-10-08T13:51:15.788486+0000","last_became_active":"2023-10-08T13:49:44.786214+0000","last_became_peered":"2023-10-08T13:49:44.786214+0000","last_unstale":"2023-10-08T13:51:15.788486+0000","last_undegraded":"2023-10-08T13:51:15.788486+0000","last_fullsized":"2023-10-08T13:51:15.788486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:30:18.342774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788081+0000","last_change":"2023-10-08T13:49:44.788445+0000","last_active":"2023-10-08T13:51:15.788081+0000","last_peered":"2023-10-08T13:51:15.788081+0000","last_clean":"2023-10-08T13:51:15.788081+0000","last_became_active":"2023-10-08T13:49:44.788075+0000","last_became_peered":"2023-10-08T13:49:44.788075+0000","last_unstale":"2023-10-08T13:51:15.788081+0000","last_undegraded":"2023-10-08T13:51:15.788081+0000","last_fullsized":"2023-10-08T13:51:15.788081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:08:20.701231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.784625+0000","last_change":"2023-10-08T13:49:44.783755+0000","last_active":"2023-10-08T13:51:15.784625+0000","last_peered":"2023-10-08T13:51:15.784625+0000","last_clean":"2023-10-08T13:51:15.784625+0000","last_became_active":"2023-10-08T13:49:44.783467+0000","last_became_peered":"2023-10-08T13:49:44.783467+0000","last_unstale":"2023-10-08T13:51:15.784625+0000","last_undegraded":"2023-10-08T13:51:15.784625+0000","last_fullsized":"2023-10-08T13:51:15.784625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:39.838940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.788245+0000","last_change":"2023-10-08T13:49:44.779342+0000","last_active":"2023-10-08T13:51:15.788245+0000","last_peered":"2023-10-08T13:51:15.788245+0000","last_clean":"2023-10-08T13:51:15.788245+0000","last_became_active":"2023-10-08T13:49:44.778518+0000","last_became_peered":"2023-10-08T13:49:44.778518+0000","last_unstale":"2023-10-08T13:51:15.788245+0000","last_undegraded":"2023-10-08T13:51:15.788245+0000","last_fullsized":"2023-10-08T13:51:15.788245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:33.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.786859+0000","last_change":"2023-10-08T13:49:44.792955+0000","last_active":"2023-10-08T13:51:15.786859+0000","last_peered":"2023-10-08T13:51:15.786859+0000","last_clean":"2023-10-08T13:51:15.786859+0000","last_became_active":"2023-10-08T13:49:44.792637+0000","last_became_peered":"2023-10-08T13:49:44.792637+0000","last_unstale":"2023-10-08T13:51:15.786859+0000","last_undegraded":"2023-10-08T13:51:15.786859+0000","last_fullsized":"2023-10-08T13:51:15.786859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:40.635913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.809138+0000","last_change":"2023-10-08T13:49:44.786817+0000","last_active":"2023-10-08T13:51:15.809138+0000","last_peered":"2023-10-08T13:51:15.809138+0000","last_clean":"2023-10-08T13:51:15.809138+0000","last_became_active":"2023-10-08T13:49:44.786439+0000","last_became_peered":"2023-10-08T13:49:44.786439+0000","last_unstale":"2023-10-08T13:51:15.809138+0000","last_undegraded":"2023-10-08T13:51:15.809138+0000","last_fullsized":"2023-10-08T13:51:15.809138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:45.308600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-08T13:51:15.808563+0000","last_change":"2023-10-08T13:50:15.788414+0000","last_active":"2023-10-08T13:51:15.808563+0000","last_peered":"2023-10-08T13:51:15.808563+0000","last_clean":"2023-10-08T13:51:15.808563+0000","last_became_active":"2023-10-08T13:50:15.788079+0000","last_became_peered":"2023-10-08T13:50:15.788079+0000","last_unstale":"2023-10-08T13:51:15.808563+0000","last_undegraded":"2023-10-08T13:51:15.808563+0000","last_fullsized":"2023-10-08T13:51:15.808563+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T13:49:43.753004+0000","last_clean_scrub_stamp":"2023-10-08T13:49:43.753004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:43.723339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66011,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":793,"num_read_kb":471,"num_write":242,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1226,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":203412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":531579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64000000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69099999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54000000000000004}]},{"osd":1,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80400000000000005,"5min":1.012,"15min":1.012},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83399999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50600000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.36299999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.50900000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.80400000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.496},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.5,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88100000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63300000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82399999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56299999999999994}]},{"osd":6,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.77100000000000002,"15min":0.72899999999999998},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2759999999999998,"5min":9.2759999999999998,"15min":9.2759999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.78600000000000003,"15min":0.79400000000000004},"min":{"1min":0.55200000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.294,"5min":1.55,"15min":1.55},"last":0.76000000000000001}]},{"osd":7,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.54200000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92100000000000004,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.58199999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0089999999999999,"5min":1.214,"15min":1.214},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Oct 8 13:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.871}]},{"osd":9,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.64300000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.97099999999999997,"5min":1.355,"15min":1.355},"last":0.60499999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.68999999999999995,"5min":0.628,"15min":0.628},"max":{"1min":1.0449999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.94899999999999995,"15min":0.96899999999999997},"min":{"1min":0.68899999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.024,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.81399999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1753088,"data_stored":1178086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46600000000000003,"15min":0.46999999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.58999999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.624,"15min":0.623},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61499999999999999,"15min":0.624},"min":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82299999999999995,"5min":0.871,"15min":0.871},"last":3.097},{"interface":"front","average":{"1min":0.628,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.438,"15min":0.438},"max":{"1min":0.83599999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57199999999999995}]},{"osd":4,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.35199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.755,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.317,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73599999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83799999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.48499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82399999999999995,"5min":1.034,"15min":1.034},"last":0.67700000000000005}]},{"osd":6,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.495,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85299999999999998,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":3.069},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":1.083,"15min":1.083},"last":0.71299999999999997}]},{"osd":7,"last update":"Sun Oct 8 13:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.55300000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.77300000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92200000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88300000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.749}]},{"osd":9,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.95199999999999996,"5min":1.05,"15min":1.05},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86799999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69699999999999995}]},{"osd":11,"last update":"Sun Oct 8 13:51:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.755,"15min":0.753},"min":{"1min":0.54400000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79200000000000004,"15min":0.78700000000000003},"min":{"1min":0.65200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.93899999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.79500000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":535707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.443,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.627,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45900000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.39100000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":4.6029999999999998,"15min":4.6029999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.65800000000000003,"15min":0.67800000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71899999999999997,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.66100000000000003,"15min":0.67800000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67200000000000004,"5min":2.806,"15min":2.806},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.63100000000000001,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79400000000000004,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.52700000000000002}]},{"osd":3,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.47099999999999997,"5min":6.9969999999999999,"15min":6.9969999999999999},"last":0.496},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.60199999999999998,"15min":0.621},"min":{"1min":0.309,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56599999999999995,"5min":3.7589999999999999,"15min":3.7589999999999999},"last":0.373}]},{"osd":4,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77700000000000002,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.76000000000000001,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.51600000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":5.3390000000000004,"15min":5.3390000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82999999999999996,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:49:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.9570000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.345}]},{"osd":7,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.80000000000000004,"15min":0.78900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76800000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.82099999999999995,"15min":0.84699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.748,"5min":4.3680000000000003,"15min":4.3680000000000003},"last":0.69799999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84599999999999997,"5min":5.2930000000000001,"15min":5.2930000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.83299999999999996,"15min":0.81899999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.83899999999999997,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":0.64100000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.85599999999999998,"15min":0.88300000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.88,"5min":2.113,"15min":2.113},"last":0.75},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89400000000000002,"5min":1.923,"15min":1.923},"last":0.66000000000000003}]},{"osd":11,"last update":"Sun Oct 8 13:50:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.623},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68899999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103212,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":527289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":0.54100000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82999999999999996,"5min":5.5860000000000003,"15min":5.5860000000000003},"last":0.379},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56499999999999995,"15min":0.53100000000000003},"min":{"1min":0.36599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.38900000000000001}]},{"osd":1,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61199999999999999,"15min":0.57099999999999995},"min":{"1min":0.435,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96399999999999997,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79100000000000004,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.58699999999999997}]},{"osd":2,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47399999999999998,"15min":0.46200000000000002},"min":{"1min":0.34999999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.873,"5min":1.762,"15min":1.762},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.39300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84599999999999997,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61499999999999999,"15min":0.59999999999999998},"min":{"1min":0.46800000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89300000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.504}]},{"osd":4,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0289999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.67500000000000004,"15min":0.64700000000000002},"min":{"1min":0.51000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92400000000000004,"5min":2.109,"15min":2.109},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.373,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.89900000000000002,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.48899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.92500000000000004,"5min":1.476,"15min":1.476},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66700000000000004,"15min":0.64700000000000002},"min":{"1min":0.58299999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.69599999999999995},"min":{"1min":0.56100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90100000000000002,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97899999999999998,"5min":1.958,"15min":1.958},"last":0.436},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.58399999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95899999999999996,"5min":2.198,"15min":2.198},"last":0.67200000000000004}]},{"osd":9,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.69999999999999996},"min":{"1min":0.58599999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.93899999999999995,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.752,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1519999999999999,"5min":2.347,"15min":2.347},"last":0.47399999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.73699999999999999,"15min":0.69699999999999995},"min":{"1min":0.66000000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":4.2039999999999997,"5min":4.2039999999999997,"15min":4.2039999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.71699999999999997,"15min":0.68700000000000006},"min":{"1min":0.74299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.61299999999999999}]},{"osd":11,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56699999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364849,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1124530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.439}]},{"osd":1,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42199999999999999,"15min":0.41699999999999998},"min":{"1min":0.33400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64600000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.57699999999999996}]},{"osd":2,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58799999999999997,"15min":0.57399999999999995},"min":{"1min":0.435,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86599999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.48499999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68600000000000005}]},{"osd":3,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.60399999999999998}]},{"osd":4,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52900000000000003,"15min":0.50900000000000001},"min":{"1min":0.42399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73699999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.47999999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.52600000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.77600000000000002}]},{"osd":6,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61599999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:51:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.63500000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88600000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71999999999999997,"15min":0.71799999999999997},"min":{"1min":0.63300000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93500000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.79400000000000004}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71999999999999997,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64000000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88200000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.60399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91200000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.73599999999999999}]},{"osd":11,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.82299999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":527115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.39400000000000002,"15min":0.375},"min":{"1min":0.32300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"min":{"1min":0.309,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.441}]},{"osd":1,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80800000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.54600000000000004},"min":{"1min":0.45300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81499999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.55000000000000004}]},{"osd":2,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79900000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.498},"min":{"1min":0.32500000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85199999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.46899999999999997}]},{"osd":4,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82499999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74199999999999999,"5min":1.073,"15min":1.073},"last":0.59199999999999997}]},{"osd":5,"last update":"Sun Oct 8 13:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82999999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.879,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":1.075,"15min":1.075},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.749,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67800000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:51:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89800000000000002,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.626},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.75900000000000001,"15min":0.78100000000000003},"min":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.876,"5min":1.147,"15min":1.147},"last":0.66300000000000003}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.73299999999999998,"15min":0.747},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.96799999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":1.042,"15min":1.042},"last":0.74099999999999999}]},{"osd":10,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86899999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.59899999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0469999999999999,"5min":1.159,"15min":1.159},"last":0.76000000000000001}]},{"osd":11,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58199999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542219,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":591921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.52400000000000002},"min":{"1min":0.38700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.90400000000000003,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:50:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.55900000000000005,"15min":0.50700000000000001},"min":{"1min":0.51900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.56599999999999995,"15min":0.52100000000000002},"min":{"1min":0.51300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54800000000000004}]},{"osd":3,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":3.7959999999999998,"15min":3.7959999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90600000000000003,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.629}]},{"osd":4,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.41899999999999998},"min":{"1min":0.40300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.75,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.504},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.44700000000000001},"min":{"1min":0.38100000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48599999999999999}]},{"osd":5,"last update":"Sun Oct 8 13:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.623,"15min":0.56599999999999995},"min":{"1min":0.51700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.65300000000000002,"15min":0.60699999999999998},"min":{"1min":0.56299999999999994,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91800000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72499999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.68700000000000006,"15min":0.76100000000000001},"min":{"1min":0.435,"5min":0.376,"15min":0.376},"max":{"1min":0.80000000000000004,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72499999999999998,"15min":0.75600000000000001},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85699999999999998,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.48199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60699999999999998,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56200000000000006}]},{"osd":8,"last update":"Sun Oct 8 13:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.71399999999999997,"15min":0.68799999999999994},"min":{"1min":0.57999999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98899999999999999,"5min":1.077,"15min":1.077},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.745,"15min":0.72599999999999998},"min":{"1min":0.69199999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.82499999999999996}]},{"osd":9,"last update":"Sun Oct 8 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72899999999999998,"15min":0.73199999999999998},"min":{"1min":0.501,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92200000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79200000000000004,"15min":0.78800000000000003},"min":{"1min":0.55000000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.95499999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.88200000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.54000000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79700000000000004,"15min":0.78800000000000003},"min":{"1min":0.61299999999999999,"5min":0.498,"15min":0.498},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.78200000000000003}]},{"osd":11,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.86099999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705743,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":528571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 13:50:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54100000000000004,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72199999999999998,"5min":1.01,"15min":1.01},"last":0.51300000000000001}]},{"osd":2,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71999999999999997,"5min":1.377,"15min":1.377},"last":0.498},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.54300000000000004},"min":{"1min":0.45500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68799999999999994,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52400000000000002}]},{"osd":3,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.39600000000000002}]},{"osd":4,"last update":"Sun Oct 8 13:50:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.81899999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71399999999999997,"5min":1.444,"15min":1.444},"last":0.64800000000000002}]},{"osd":5,"last update":"Sun Oct 8 13:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63500000000000001,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85799999999999998,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":0.59899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77400000000000002,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.36899999999999999}]},{"osd":7,"last update":"Sun Oct 8 13:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.73299999999999998}]},{"osd":8,"last update":"Sun Oct 8 13:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.73699999999999999,"15min":0.76800000000000002},"min":{"1min":0.41099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82899999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83899999999999997,"5min":1.327,"15min":1.327},"last":0.56499999999999995}]},{"osd":9,"last update":"Sun Oct 8 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.44600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90600000000000003,"5min":1.774,"15min":1.774},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.018,"5min":1.839,"15min":1.839},"last":0.67400000000000004}]},{"osd":10,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.59099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91100000000000003,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85699999999999998,"5min":1.411,"15min":1.411},"last":0.76600000000000001}]},{"osd":11,"last update":"Sun Oct 8 13:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.754,"15min":0.753},"min":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91600000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94399999999999995,"5min":1.208,"15min":1.208},"last":0.64100000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378694,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":590897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.94599999999999995,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81799999999999995,"5min":1.538,"15min":1.538},"last":0.45500000000000002}]},{"osd":1,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54800000000000004,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.55900000000000005},"min":{"1min":0.38900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73299999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.61599999999999999,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.497},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.55400000000000005}]},{"osd":4,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.621},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.47899999999999998,"15min":0.44},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":0.876,"15min":0.876},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.52700000000000002,"15min":0.495},"min":{"1min":0.39200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.75}]},{"osd":6,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.70099999999999996}]},{"osd":7,"last update":"Sun Oct 8 13:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Sun Oct 8 13:50:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.622,"15min":0.60399999999999998},"min":{"1min":0.45100000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65100000000000002}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.56000000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.85099999999999998}]},{"osd":10,"last update":"Sun Oct 8 13:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.68600000000000005,"15min":0.66900000000000004},"min":{"1min":0.53100000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.70399999999999996,"15min":0.68000000000000005},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.83899999999999997}]},{"osd":11,"last update":"Sun Oct 8 13:50:43 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79100000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":526823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.34799999999999998},"min":{"1min":0.28100000000000003,"5min":0.22,"15min":0.22},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47699999999999998,"15min":0.42099999999999999},"min":{"1min":0.32700000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65700000000000003,"5min":0.87,"15min":0.87},"last":0.48899999999999999}]},{"osd":1,"last update":"Sun Oct 8 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.53300000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93300000000000005,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.53100000000000003},"min":{"1min":0.40000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68899999999999995,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.60899999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61199999999999999,"15min":0.56599999999999995},"min":{"1min":0.47899999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.78500000000000003}]},{"osd":4,"last update":"Sun Oct 8 13:50:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":2.077,"15min":2.077},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60299999999999998,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88300000000000001,"5min":2.129,"15min":2.129},"last":0.51000000000000001}]},{"osd":5,"last update":"Sun Oct 8 13:51:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.60899999999999999},"min":{"1min":0.53700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76100000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65600000000000003,"15min":0.60799999999999998},"min":{"1min":0.432,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91000000000000003,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.75}]},{"osd":6,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.32400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80200000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.432},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.625},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53500000000000003}]},{"osd":7,"last update":"Sun Oct 8 13:50:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.76800000000000002,"15min":0.80100000000000005},"min":{"1min":0.48799999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.872,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.75600000000000001,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83199999999999996,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.57299999999999995}]},{"osd":8,"last update":"Sun Oct 8 13:48:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.46800000000000003}]},{"osd":9,"last update":"Sun Oct 8 13:50:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.069,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91700000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65400000000000003}]},{"osd":10,"last update":"Sun Oct 8 13:50:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.95799999999999996,"5min":1.028,"15min":1.028},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.82399999999999995,"15min":0.84299999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92000000000000004,"5min":1.147,"15min":1.147},"last":0.66800000000000004}]},{"osd":11,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.69899999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":531720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:49:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.51800000000000002,"15min":0.47999999999999998},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52100000000000002,"15min":0.504},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":1,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.35599999999999998},"min":{"1min":0.30599999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.36199999999999999},"min":{"1min":0.30399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.36799999999999999}]},{"osd":2,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.54900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54900000000000004}]},{"osd":3,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.51500000000000001},"min":{"1min":0.40200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.497}]},{"osd":5,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.629,"15min":0.58699999999999997},"min":{"1min":0.502,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0009999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67000000000000004,"15min":0.63200000000000001},"min":{"1min":0.58299999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.71399999999999997}]},{"osd":6,"last update":"Sun Oct 8 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.66300000000000003,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.377,"15min":0.377},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.629}]},{"osd":7,"last update":"Sun Oct 8 13:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.38800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77100000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.59299999999999997,"15min":0.57299999999999995},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84599999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.43099999999999999}]},{"osd":8,"last update":"Sun Oct 8 13:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73199999999999998,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96299999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.45900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97899999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.50900000000000001}]},{"osd":9,"last update":"Sun Oct 8 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.76300000000000001}]},{"osd":10,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79900000000000004,"15min":0.80200000000000005},"min":{"1min":0.47199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94899999999999995,"5min":1.04,"15min":1.04},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59399999999999997}]},{"osd":11,"last update":"Sun Oct 8 13:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64500000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1683456,"data_stored":1123346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.50700000000000001,"15min":0.47899999999999998},"min":{"1min":0.40400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52300000000000002,"15min":0.47599999999999998},"min":{"1min":0.438,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.873,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47599999999999998}]},{"osd":2,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.378},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.38900000000000001},"min":{"1min":0.30499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71099999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.58099999999999996,"15min":0.52000000000000002},"min":{"1min":0.46600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.52800000000000002}]},{"osd":4,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.56599999999999995},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.499}]},{"osd":6,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.54700000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.46999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56100000000000005}]},{"osd":8,"last update":"Sun Oct 8 13:51:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88700000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.62}]},{"osd":9,"last update":"Sun Oct 8 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.66400000000000003,"15min":0.64300000000000002},"min":{"1min":0.57599999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73599999999999999,"15min":0.72099999999999997},"min":{"1min":0.629,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.752}]},{"osd":10,"last update":"Sun Oct 8 13:50:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72799999999999998,"15min":0.72599999999999998},"min":{"1min":0.59499999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.93999999999999995,"5min":0.997,"15min":0.997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.022,"5min":1.101,"15min":1.101},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T13:51:27.714 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:51:27.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.873+0000 7f4d6ce00700 1 -- 172.21.15.57:0/3610526967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d68100620 msgr2=0x7f4d6810c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.873+0000 7f4d6ce00700 1 --2- 172.21.15.57:0/3610526967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d68100620 0x7f4d6810c610 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000b0a0 tx=0x7f4d6000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.873+0000 7f4d6ce00700 1 -- 172.21.15.57:0/3610526967 shutdown_connections 2023-10-08T13:51:27.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.873+0000 7f4d6ce00700 1 --2- 172.21.15.57:0/3610526967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d68100620 0x7f4d6810c610 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.873+0000 7f4d6ce00700 1 --2- 172.21.15.57:0/3610526967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d681057e0 0x7f4d68105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.873+0000 7f4d6ce00700 1 --2- 172.21.15.57:0/3610526967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d680ffc60 0x7f4d681000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.873+0000 7f4d6ce00700 1 -- 172.21.15.57:0/3610526967 >> 172.21.15.57:0/3610526967 conn(0x7f4d68075a90 msgr2=0x7f4d680fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:27.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.874+0000 7f4d6ce00700 1 -- 172.21.15.57:0/3610526967 shutdown_connections 2023-10-08T13:51:27.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.874+0000 7f4d6ce00700 1 -- 172.21.15.57:0/3610526967 wait complete. 2023-10-08T13:51:27.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.874+0000 7f4d6ce00700 1 Processor -- start 2023-10-08T13:51:27.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.874+0000 7f4d6ce00700 1 -- start start 2023-10-08T13:51:27.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6ce00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d680ffc60 0x7f4d6819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6ce00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d68100620 0x7f4d6819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d681057e0 0x7f4d68194810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6ce00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d68194e80 con 0x7f4d680ffc60 2023-10-08T13:51:27.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6ce00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d68194ff0 con 0x7f4d681057e0 2023-10-08T13:51:27.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6ce00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681951b0 con 0x7f4d68100620 2023-10-08T13:51:27.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d680ffc60 0x7f4d6819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d680ffc60 0x7f4d6819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56122/0 (socket says 172.21.15.57:56122) 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6659c700 1 -- 172.21.15.57:0/174714124 learned_addr learned my addr 172.21.15.57:0/174714124 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d66d9d700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d681057e0 0x7f4d68194810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d65d9b700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d68100620 0x7f4d6819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6659c700 1 -- 172.21.15.57:0/174714124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d68100620 msgr2=0x7f4d6819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6659c700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d68100620 0x7f4d6819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6659c700 1 -- 172.21.15.57:0/174714124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d681057e0 msgr2=0x7f4d68194810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6659c700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d681057e0 0x7f4d68194810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6659c700 1 -- 172.21.15.57:0/174714124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d60009d00 con 0x7f4d680ffc60 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6659c700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d680ffc60 0x7f4d6819a6e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009de0 tx=0x7f4d5800f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d577fe700 1 -- 172.21.15.57:0/174714124 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800b220 con 0x7f4d680ffc60 2023-10-08T13:51:27.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d577fe700 1 -- 172.21.15.57:0/174714124 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d58011040 con 0x7f4d680ffc60 2023-10-08T13:51:27.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d6ce00700 1 -- 172.21.15.57:0/174714124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d68195490 con 0x7f4d680ffc60 2023-10-08T13:51:27.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.875+0000 7f4d577fe700 1 -- 172.21.15.57:0/174714124 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800adb0 con 0x7f4d680ffc60 2023-10-08T13:51:27.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.876+0000 7f4d6ce00700 1 -- 172.21.15.57:0/174714124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d681a16d0 con 0x7f4d680ffc60 2023-10-08T13:51:27.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.876+0000 7f4d6ce00700 1 -- 172.21.15.57:0/174714124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d68107d40 con 0x7f4d680ffc60 2023-10-08T13:51:27.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.877+0000 7f4d577fe700 1 -- 172.21.15.57:0/174714124 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f4d5800fb20 con 0x7f4d680ffc60 2023-10-08T13:51:27.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.878+0000 7f4d577fe700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4d500a14d0 0x7f4d500a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:27.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.878+0000 7f4d577fe700 1 -- 172.21.15.57:0/174714124 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4d580ce9f0 con 0x7f4d680ffc60 2023-10-08T13:51:27.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.878+0000 7f4d65d9b700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4d500a14d0 0x7f4d500a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:27.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.879+0000 7f4d65d9b700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4d500a14d0 0x7f4d500a3990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4d68195c30 tx=0x7f4d5c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:27.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:27.883+0000 7f4d577fe700 1 -- 172.21.15.57:0/174714124 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4d58051100 con 0x7f4d680ffc60 2023-10-08T13:51:27.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:27 smithi057 ceph-mon[99476]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-08T13:51:27.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:27 smithi057 ceph-mon[99476]: from='client.15738 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:51:28.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.085+0000 7f4d6ce00700 1 -- 172.21.15.57:0/174714124 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d68100f10 con 0x7f4d680ffc60 2023-10-08T13:51:28.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.087+0000 7f4d577fe700 1 -- 172.21.15.57:0/174714124 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24847 (secure 0 0 0) 0x7f4d58097c30 con 0x7f4d680ffc60 2023-10-08T13:51:28.088 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:28.088 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:51:16.778201+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:51:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 -- 172.21.15.57:0/174714124 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4d500a14d0 msgr2=0x7f4d500a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4d500a14d0 0x7f4d500a3990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4d68195c30 tx=0x7f4d5c00b410 comp rx=0 tx=0).stop 2023-10-08T13:51:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 -- 172.21.15.57:0/174714124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d680ffc60 msgr2=0x7f4d6819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d680ffc60 0x7f4d6819a6e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009de0 tx=0x7f4d5800f3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 -- 172.21.15.57:0/174714124 shutdown_connections 2023-10-08T13:51:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4d500a14d0 0x7f4d500a3990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d681057e0 0x7f4d68194810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d680ffc60 0x7f4d6819a6e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 --2- 172.21.15.57:0/174714124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d68100620 0x7f4d6819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 -- 172.21.15.57:0/174714124 >> 172.21.15.57:0/174714124 conn(0x7f4d68075a90 msgr2=0x7f4d680fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 -- 172.21.15.57:0/174714124 shutdown_connections 2023-10-08T13:51:28.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.089+0000 7f4d6ce00700 1 -- 172.21.15.57:0/174714124 wait complete. 2023-10-08T13:51:28.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T13:51:28.103 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T13:51:28.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:27 smithi115 ceph-mon[104408]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-08T13:51:28.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:27 smithi115 ceph-mon[104408]: from='client.15738 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:51:28.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.231+0000 7fecf56a7700 1 -- 172.21.15.57:0/1517882113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf0105650 msgr2=0x7fecf0105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.231+0000 7fecf56a7700 1 --2- 172.21.15.57:0/1517882113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf0105650 0x7fecf0105a30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fece4009990 tx=0x7fece4009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.231+0000 7fecf56a7700 1 -- 172.21.15.57:0/1517882113 shutdown_connections 2023-10-08T13:51:28.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.231+0000 7fecf56a7700 1 --2- 172.21.15.57:0/1517882113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecf0100000 0x7fecf010c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.231+0000 7fecf56a7700 1 --2- 172.21.15.57:0/1517882113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf0105650 0x7fecf0105a30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.231+0000 7fecf56a7700 1 --2- 172.21.15.57:0/1517882113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecf00ff640 0x7fecf00ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.231+0000 7fecf56a7700 1 -- 172.21.15.57:0/1517882113 >> 172.21.15.57:0/1517882113 conn(0x7fecf00767c0 msgr2=0x7fecf0076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:28.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.231+0000 7fecf56a7700 1 -- 172.21.15.57:0/1517882113 shutdown_connections 2023-10-08T13:51:28.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.231+0000 7fecf56a7700 1 -- 172.21.15.57:0/1517882113 wait complete. 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecf56a7700 1 Processor -- start 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecf56a7700 1 -- start start 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecf56a7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecf00ff640 0x7fecf019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecf56a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf0100000 0x7fecf019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecf56a7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecf0105650 0x7fecf0196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecf56a7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf0197080 con 0x7fecf0100000 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecf56a7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf01971f0 con 0x7fecf0105650 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecf56a7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf01973b0 con 0x7fecf00ff640 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecee7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf0100000 0x7fecf019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecee7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf0100000 0x7fecf019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56138/0 (socket says 172.21.15.57:56138) 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecee7fc700 1 -- 172.21.15.57:0/1573426266 learned_addr learned my addr 172.21.15.57:0/1573426266 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:28.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7feceeffd700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecf00ff640 0x7fecf019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecef7fe700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecf0105650 0x7fecf0196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecee7fc700 1 -- 172.21.15.57:0/1573426266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecf00ff640 msgr2=0x7fecf019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecee7fc700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecf00ff640 0x7fecf019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecee7fc700 1 -- 172.21.15.57:0/1573426266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecf0105650 msgr2=0x7fecf0196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecee7fc700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecf0105650 0x7fecf0196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.232+0000 7fecee7fc700 1 -- 172.21.15.57:0/1573426266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece40095f0 con 0x7fecf0100000 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.233+0000 7fecee7fc700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf0100000 0x7fecf019cde0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fece000e9a0 tx=0x7fece000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.233+0000 7fecd7fff700 1 -- 172.21.15.57:0/1573426266 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0009790 con 0x7fecf0100000 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.233+0000 7fecf56a7700 1 -- 172.21.15.57:0/1573426266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf0197690 con 0x7fecf0100000 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.233+0000 7fecd7fff700 1 -- 172.21.15.57:0/1573426266 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece0010430 con 0x7fecf0100000 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.233+0000 7fecd7fff700 1 -- 172.21.15.57:0/1573426266 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0018680 con 0x7fecf0100000 2023-10-08T13:51:28.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.233+0000 7fecf56a7700 1 -- 172.21.15.57:0/1573426266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf01a3890 con 0x7fecf0100000 2023-10-08T13:51:28.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.234+0000 7fecf56a7700 1 -- 172.21.15.57:0/1573426266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf0108c50 con 0x7fecf0100000 2023-10-08T13:51:28.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.235+0000 7fecd7fff700 1 -- 172.21.15.57:0/1573426266 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fece00105a0 con 0x7fecf0100000 2023-10-08T13:51:28.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.236+0000 7fecd7fff700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fecd80a14d0 0x7fecd80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.236+0000 7feceeffd700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fecd80a14d0 0x7fecd80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.236+0000 7fecd7fff700 1 -- 172.21.15.57:0/1573426266 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fece0014070 con 0x7fecf0100000 2023-10-08T13:51:28.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.236+0000 7feceeffd700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fecd80a14d0 0x7fecd80a3990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fece400b2e0 tx=0x7fece4005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:28.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.239+0000 7fecd7fff700 1 -- 172.21.15.57:0/1573426266 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fece0096760 con 0x7fecf0100000 2023-10-08T13:51:28.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:27 smithi099 ceph-mon[103849]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-08T13:51:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:27 smithi099 ceph-mon[103849]: from='client.15738 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:51:28.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.441+0000 7fecf56a7700 1 -- 172.21.15.57:0/1573426266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecf0197e90 con 0x7fecf0100000 2023-10-08T13:51:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.441+0000 7fecd7fff700 1 -- 172.21.15.57:0/1573426266 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fece00260a0 con 0x7fecf0100000 2023-10-08T13:51:28.442 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T13:51:28.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 -- 172.21.15.57:0/1573426266 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fecd80a14d0 msgr2=0x7fecd80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fecd80a14d0 0x7fecd80a3990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fece400b2e0 tx=0x7fece4005960 comp rx=0 tx=0).stop 2023-10-08T13:51:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 -- 172.21.15.57:0/1573426266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf0100000 msgr2=0x7fecf019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf0100000 0x7fecf019cde0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fece000e9a0 tx=0x7fece000ecb0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 -- 172.21.15.57:0/1573426266 shutdown_connections 2023-10-08T13:51:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fecd80a14d0 0x7fecd80a3990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecf0105650 0x7fecf0196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecf00ff640 0x7fecf019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 --2- 172.21.15.57:0/1573426266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fecf0100000 0x7fecf019cde0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 -- 172.21.15.57:0/1573426266 >> 172.21.15.57:0/1573426266 conn(0x7fecf00767c0 msgr2=0x7fecf0068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 -- 172.21.15.57:0/1573426266 shutdown_connections 2023-10-08T13:51:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.443+0000 7fecf56a7700 1 -- 172.21.15.57:0/1573426266 wait complete. 2023-10-08T13:51:28.457 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T13:51:28.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.585+0000 7f589f8bd700 1 -- 172.21.15.57:0/136837566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5898100000 msgr2=0x7f589810c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.585+0000 7f589f8bd700 1 --2- 172.21.15.57:0/136837566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5898100000 0x7f589810c4f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f589000b0a0 tx=0x7f589000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.585+0000 7f589f8bd700 1 -- 172.21.15.57:0/136837566 shutdown_connections 2023-10-08T13:51:28.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.585+0000 7f589f8bd700 1 --2- 172.21.15.57:0/136837566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5898100000 0x7f589810c4f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.585+0000 7f589f8bd700 1 --2- 172.21.15.57:0/136837566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898105630 0x7f5898105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.585+0000 7f589f8bd700 1 --2- 172.21.15.57:0/136837566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58980ff640 0x7f58980ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.585+0000 7f589f8bd700 1 -- 172.21.15.57:0/136837566 >> 172.21.15.57:0/136837566 conn(0x7f5898076790 msgr2=0x7f5898076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:28.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.585+0000 7f589f8bd700 1 -- 172.21.15.57:0/136837566 shutdown_connections 2023-10-08T13:51:28.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.585+0000 7f589f8bd700 1 -- 172.21.15.57:0/136837566 wait complete. 2023-10-08T13:51:28.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.585+0000 7f589f8bd700 1 Processor -- start 2023-10-08T13:51:28.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589f8bd700 1 -- start start 2023-10-08T13:51:28.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589f8bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58980ff640 0x7f5898106590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589f8bd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5898100000 0x7f5898106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589f8bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898105630 0x7f5898107010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589f8bd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981a4e70 con 0x7f5898100000 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589f8bd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981a4fe0 con 0x7f58980ff640 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589f8bd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981a51a0 con 0x7f5898105630 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589ce58700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5898100000 0x7f5898106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589ce58700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5898100000 0x7f5898106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56152/0 (socket says 172.21.15.57:56152) 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589ce58700 1 -- 172.21.15.57:0/789817684 learned_addr learned my addr 172.21.15.57:0/789817684 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589d659700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58980ff640 0x7f5898106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589de5a700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898105630 0x7f5898107010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589ce58700 1 -- 172.21.15.57:0/789817684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898105630 msgr2=0x7f5898107010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589ce58700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898105630 0x7f5898107010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589ce58700 1 -- 172.21.15.57:0/789817684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58980ff640 msgr2=0x7f5898106590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589ce58700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58980ff640 0x7f5898106590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589ce58700 1 -- 172.21.15.57:0/789817684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5890009d00 con 0x7f5898100000 2023-10-08T13:51:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.586+0000 7f589ce58700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5898100000 0x7f5898106ad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f588800d950 tx=0x7f588800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.587+0000 7f588e7fc700 1 -- 172.21.15.57:0/789817684 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588800f890 con 0x7f5898100000 2023-10-08T13:51:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.587+0000 7f589f8bd700 1 -- 172.21.15.57:0/789817684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58981a5480 con 0x7f5898100000 2023-10-08T13:51:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.587+0000 7f588e7fc700 1 -- 172.21.15.57:0/789817684 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5888010460 con 0x7f5898100000 2023-10-08T13:51:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.587+0000 7f589f8bd700 1 -- 172.21.15.57:0/789817684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58981a59e0 con 0x7f5898100000 2023-10-08T13:51:28.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.587+0000 7f588e7fc700 1 -- 172.21.15.57:0/789817684 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588801b650 con 0x7f5898100000 2023-10-08T13:51:28.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.588+0000 7f589f8bd700 1 -- 172.21.15.57:0/789817684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5898100d60 con 0x7f5898100000 2023-10-08T13:51:28.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.589+0000 7f588e7fc700 1 -- 172.21.15.57:0/789817684 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f58880108d0 con 0x7f5898100000 2023-10-08T13:51:28.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.589+0000 7f588e7fc700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f58840a14d0 0x7f58840a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.589+0000 7f588e7fc700 1 -- 172.21.15.57:0/789817684 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f58880cd290 con 0x7f5898100000 2023-10-08T13:51:28.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.589+0000 7f589d659700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f58840a14d0 0x7f58840a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.590+0000 7f589d659700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f58840a14d0 0x7f58840a3990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f589400abd0 tx=0x7f589400a380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:28.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.594+0000 7f588e7fc700 1 -- 172.21.15.57:0/789817684 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f58880964d0 con 0x7f5898100000 2023-10-08T13:51:28.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.802+0000 7f589f8bd700 1 -- 172.21.15.57:0/789817684 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5898107a50 con 0x7f5898100000 2023-10-08T13:51:28.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.802+0000 7f588e7fc700 1 -- 172.21.15.57:0/789817684 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5888099af0 con 0x7f5898100000 2023-10-08T13:51:28.803 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:51:28.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 -- 172.21.15.57:0/789817684 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f58840a14d0 msgr2=0x7f58840a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f58840a14d0 0x7f58840a3990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f589400abd0 tx=0x7f589400a380 comp rx=0 tx=0).stop 2023-10-08T13:51:28.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 -- 172.21.15.57:0/789817684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5898100000 msgr2=0x7f5898106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5898100000 0x7f5898106ad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f588800d950 tx=0x7f588800dc60 comp rx=0 tx=0).stop 2023-10-08T13:51:28.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 -- 172.21.15.57:0/789817684 shutdown_connections 2023-10-08T13:51:28.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5898105630 0x7f5898107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f58840a14d0 0x7f58840a3990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58980ff640 0x7f5898106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 --2- 172.21.15.57:0/789817684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5898100000 0x7f5898106ad0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 -- 172.21.15.57:0/789817684 >> 172.21.15.57:0/789817684 conn(0x7f5898076790 msgr2=0x7f58980fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:28.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 -- 172.21.15.57:0/789817684 shutdown_connections 2023-10-08T13:51:28.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.804+0000 7f589f8bd700 1 -- 172.21.15.57:0/789817684 wait complete. 2023-10-08T13:51:28.818 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T13:51:28.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.947+0000 7f3ff1693700 1 -- 172.21.15.57:0/4123287527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3fec106760 msgr2=0x7f3fec106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.947+0000 7f3ff1693700 1 --2- 172.21.15.57:0/4123287527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3fec106760 0x7f3fec106b40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009990 tx=0x7f3fdc009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.948+0000 7f3ff1693700 1 -- 172.21.15.57:0/4123287527 shutdown_connections 2023-10-08T13:51:28.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.948+0000 7f3ff1693700 1 --2- 172.21.15.57:0/4123287527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fec101110 0x7f3fec10d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.948+0000 7f3ff1693700 1 --2- 172.21.15.57:0/4123287527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3fec106760 0x7f3fec106b40 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.948+0000 7f3ff1693700 1 --2- 172.21.15.57:0/4123287527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fec100750 0x7f3fec100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.948+0000 7f3ff1693700 1 -- 172.21.15.57:0/4123287527 >> 172.21.15.57:0/4123287527 conn(0x7f3fec0fc220 msgr2=0x7f3fec0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:28.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.948+0000 7f3ff1693700 1 -- 172.21.15.57:0/4123287527 shutdown_connections 2023-10-08T13:51:28.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.948+0000 7f3ff1693700 1 -- 172.21.15.57:0/4123287527 wait complete. 2023-10-08T13:51:28.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.948+0000 7f3ff1693700 1 Processor -- start 2023-10-08T13:51:28.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3ff1693700 1 -- start start 2023-10-08T13:51:28.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3ff1693700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fec100750 0x7f3fec19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3ff1693700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3fec101110 0x7f3fec19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3ff1693700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fec106760 0x7f3fec1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3ff1693700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec197030 con 0x7f3fec101110 2023-10-08T13:51:28.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3ff1693700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec1971a0 con 0x7f3fec100750 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3ff1693700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec197360 con 0x7f3fec106760 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3fea7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3fec101110 0x7f3fec19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3fea7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3fec101110 0x7f3fec19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56166/0 (socket says 172.21.15.57:56166) 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3fea7fc700 1 -- 172.21.15.57:0/2211775494 learned_addr learned my addr 172.21.15.57:0/2211775494 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3feb7fe700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fec106760 0x7f3fec1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3feaffd700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fec100750 0x7f3fec19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3fea7fc700 1 -- 172.21.15.57:0/2211775494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fec106760 msgr2=0x7f3fec1969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3fea7fc700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fec106760 0x7f3fec1969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3fea7fc700 1 -- 172.21.15.57:0/2211775494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fec100750 msgr2=0x7f3fec19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3fea7fc700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fec100750 0x7f3fec19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3fea7fc700 1 -- 172.21.15.57:0/2211775494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc0095f0 con 0x7f3fec101110 2023-10-08T13:51:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.949+0000 7f3feaffd700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fec100750 0x7f3fec19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.950+0000 7f3fea7fc700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3fec101110 0x7f3fec19cde0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000d950 tx=0x7f3fe000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.950+0000 7f3fd3fff700 1 -- 172.21.15.57:0/2211775494 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0009940 con 0x7f3fec101110 2023-10-08T13:51:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.950+0000 7f3ff1693700 1 -- 172.21.15.57:0/2211775494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec197640 con 0x7f3fec101110 2023-10-08T13:51:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.950+0000 7f3fd3fff700 1 -- 172.21.15.57:0/2211775494 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe0010460 con 0x7f3fec101110 2023-10-08T13:51:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.950+0000 7f3fd3fff700 1 -- 172.21.15.57:0/2211775494 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000f620 con 0x7f3fec101110 2023-10-08T13:51:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.950+0000 7f3ff1693700 1 -- 172.21.15.57:0/2211775494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec1a3890 con 0x7f3fec101110 2023-10-08T13:51:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.951+0000 7f3ff1693700 1 -- 172.21.15.57:0/2211775494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec108ce0 con 0x7f3fec101110 2023-10-08T13:51:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.952+0000 7f3fd3fff700 1 -- 172.21.15.57:0/2211775494 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f3fe00108d0 con 0x7f3fec101110 2023-10-08T13:51:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.952+0000 7f3fd3fff700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3fd40a14d0 0x7f3fd40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.953+0000 7f3fd3fff700 1 -- 172.21.15.57:0/2211775494 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3fe00ccff0 con 0x7f3fec101110 2023-10-08T13:51:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.953+0000 7f3feaffd700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3fd40a14d0 0x7f3fd40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.953+0000 7f3feaffd700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3fd40a14d0 0x7f3fd40a3990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc006010 tx=0x7f3fdc01a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:28.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:28.957+0000 7f3fd3fff700 1 -- 172.21.15.57:0/2211775494 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3fe0096230 con 0x7f3fec101110 2023-10-08T13:51:29.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.159+0000 7f3ff1693700 1 -- 172.21.15.57:0/2211775494 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3fec197e90 con 0x7f3fec101110 2023-10-08T13:51:29.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.159+0000 7f3fd3fff700 1 -- 172.21.15.57:0/2211775494 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3fe0099850 con 0x7f3fec101110 2023-10-08T13:51:29.161 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T13:51:29.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 -- 172.21.15.57:0/2211775494 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3fd40a14d0 msgr2=0x7f3fd40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3fd40a14d0 0x7f3fd40a3990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc006010 tx=0x7f3fdc01a040 comp rx=0 tx=0).stop 2023-10-08T13:51:29.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 -- 172.21.15.57:0/2211775494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3fec101110 msgr2=0x7f3fec19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3fec101110 0x7f3fec19cde0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000d950 tx=0x7f3fe000dc60 comp rx=0 tx=0).stop 2023-10-08T13:51:29.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 -- 172.21.15.57:0/2211775494 shutdown_connections 2023-10-08T13:51:29.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fec106760 0x7f3fec1969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3fd40a14d0 0x7f3fd40a3990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fec100750 0x7f3fec19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 --2- 172.21.15.57:0/2211775494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3fec101110 0x7f3fec19cde0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 -- 172.21.15.57:0/2211775494 >> 172.21.15.57:0/2211775494 conn(0x7f3fec0fc220 msgr2=0x7f3fec0fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:29.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 -- 172.21.15.57:0/2211775494 shutdown_connections 2023-10-08T13:51:29.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.162+0000 7f3ff1693700 1 -- 172.21.15.57:0/2211775494 wait complete. 2023-10-08T13:51:29.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:28 smithi057 ceph-mon[99476]: from='client.15744 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:51:29.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:28 smithi057 ceph-mon[99476]: from='client.15750 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:51:29.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:28 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/174714124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:29.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:28 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1573426266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:29.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:28 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/789817684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:29.175 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T13:51:29.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:28 smithi115 ceph-mon[104408]: from='client.15744 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:51:29.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:28 smithi115 ceph-mon[104408]: from='client.15750 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:51:29.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:28 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/174714124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:29.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:28 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1573426266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:29.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:28 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/789817684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:28 smithi099 ceph-mon[103849]: from='client.15744 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:51:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:28 smithi099 ceph-mon[103849]: from='client.15750 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T13:51:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:28 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/174714124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:28 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1573426266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T13:51:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:28 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/789817684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T13:51:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.303+0000 7fdcab726700 1 -- 172.21.15.57:0/2700999753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdca41010e0 msgr2=0x7fdca410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.303+0000 7fdcab726700 1 --2- 172.21.15.57:0/2700999753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdca41010e0 0x7fdca410d5f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00b0a0 tx=0x7fdc9c00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.303+0000 7fdcab726700 1 -- 172.21.15.57:0/2700999753 shutdown_connections 2023-10-08T13:51:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.303+0000 7fdcab726700 1 --2- 172.21.15.57:0/2700999753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdca41010e0 0x7fdca410d5f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.303+0000 7fdcab726700 1 --2- 172.21.15.57:0/2700999753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdca4106730 0x7fdca4106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.303+0000 7fdcab726700 1 --2- 172.21.15.57:0/2700999753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca4100720 0x7fdca4100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.303+0000 7fdcab726700 1 -- 172.21.15.57:0/2700999753 >> 172.21.15.57:0/2700999753 conn(0x7fdca40fc230 msgr2=0x7fdca40fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.303+0000 7fdcab726700 1 -- 172.21.15.57:0/2700999753 shutdown_connections 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.303+0000 7fdcab726700 1 -- 172.21.15.57:0/2700999753 wait complete. 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdcab726700 1 Processor -- start 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdcab726700 1 -- start start 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdcab726700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdca4100720 0x7fdca419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdcab726700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca41010e0 0x7fdca419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdcab726700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdca4106730 0x7fdca4196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdcab726700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca4197100 con 0x7fdca4100720 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdcab726700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca4197270 con 0x7fdca4106730 2023-10-08T13:51:29.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdcab726700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca4197430 con 0x7fdca41010e0 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdca94c2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdca4100720 0x7fdca419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdca94c2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdca4100720 0x7fdca419c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56178/0 (socket says 172.21.15.57:56178) 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdca94c2700 1 -- 172.21.15.57:0/2749187818 learned_addr learned my addr 172.21.15.57:0/2749187818 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdca8cc1700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca41010e0 0x7fdca419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdca9cc3700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdca4106730 0x7fdca4196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdca94c2700 1 -- 172.21.15.57:0/2749187818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca41010e0 msgr2=0x7fdca419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdca94c2700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca41010e0 0x7fdca419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdca94c2700 1 -- 172.21.15.57:0/2749187818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdca4106730 msgr2=0x7fdca4196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.304+0000 7fdca94c2700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdca4106730 0x7fdca4196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.305+0000 7fdca94c2700 1 -- 172.21.15.57:0/2749187818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c009d00 con 0x7fdca4100720 2023-10-08T13:51:29.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.305+0000 7fdca94c2700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdca4100720 0x7fdca419c890 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdca000d980 tx=0x7fdca000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:29.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.305+0000 7fdc967fc700 1 -- 172.21.15.57:0/2749187818 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0004220 con 0x7fdca4100720 2023-10-08T13:51:29.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.305+0000 7fdc967fc700 1 -- 172.21.15.57:0/2749187818 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdca0007640 con 0x7fdca4100720 2023-10-08T13:51:29.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.305+0000 7fdcab726700 1 -- 172.21.15.57:0/2749187818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca4197710 con 0x7fdca4100720 2023-10-08T13:51:29.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.305+0000 7fdc967fc700 1 -- 172.21.15.57:0/2749187818 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0005610 con 0x7fdca4100720 2023-10-08T13:51:29.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.305+0000 7fdcab726700 1 -- 172.21.15.57:0/2749187818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca41a3860 con 0x7fdca4100720 2023-10-08T13:51:29.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.306+0000 7fdcab726700 1 -- 172.21.15.57:0/2749187818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca4108d40 con 0x7fdca4100720 2023-10-08T13:51:29.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.307+0000 7fdc967fc700 1 -- 172.21.15.57:0/2749187818 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fdca0004750 con 0x7fdca4100720 2023-10-08T13:51:29.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.308+0000 7fdc967fc700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdc900a1550 0x7fdc900a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:29.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.308+0000 7fdca8cc1700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdc900a1550 0x7fdc900a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:29.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.308+0000 7fdc967fc700 1 -- 172.21.15.57:0/2749187818 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdca00cd430 con 0x7fdca4100720 2023-10-08T13:51:29.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.308+0000 7fdca8cc1700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdc900a1550 0x7fdc900a3a10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdca41980a0 tx=0x7fdc98009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:29.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.311+0000 7fdc967fc700 1 -- 172.21.15.57:0/2749187818 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdca0096610 con 0x7fdca4100720 2023-10-08T13:51:29.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.512+0000 7fdcab726700 1 -- 172.21.15.57:0/2749187818 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdca4101e60 con 0x7fdca4100720 2023-10-08T13:51:29.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.512+0000 7fdc967fc700 1 -- 172.21.15.57:0/2749187818 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdca00160a0 con 0x7fdca4100720 2023-10-08T13:51:29.513 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T13:51:29.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.514+0000 7fdcab726700 1 -- 172.21.15.57:0/2749187818 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdc900a1550 msgr2=0x7fdc900a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.514+0000 7fdcab726700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdc900a1550 0x7fdc900a3a10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdca41980a0 tx=0x7fdc98009260 comp rx=0 tx=0).stop 2023-10-08T13:51:29.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.514+0000 7fdcab726700 1 -- 172.21.15.57:0/2749187818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdca4100720 msgr2=0x7fdca419c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.515+0000 7fdcab726700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdca4100720 0x7fdca419c890 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdca000d980 tx=0x7fdca000dc90 comp rx=0 tx=0).stop 2023-10-08T13:51:29.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.515+0000 7fdcab726700 1 -- 172.21.15.57:0/2749187818 shutdown_connections 2023-10-08T13:51:29.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.515+0000 7fdcab726700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdc900a1550 0x7fdc900a3a10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.515+0000 7fdcab726700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdca4106730 0x7fdca4196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.515+0000 7fdcab726700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdca4100720 0x7fdca419c890 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.515+0000 7fdcab726700 1 --2- 172.21.15.57:0/2749187818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca41010e0 0x7fdca419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.515+0000 7fdcab726700 1 -- 172.21.15.57:0/2749187818 >> 172.21.15.57:0/2749187818 conn(0x7fdca40fc230 msgr2=0x7fdca40fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:29.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.515+0000 7fdcab726700 1 -- 172.21.15.57:0/2749187818 shutdown_connections 2023-10-08T13:51:29.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.515+0000 7fdcab726700 1 -- 172.21.15.57:0/2749187818 wait complete. 2023-10-08T13:51:29.528 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:51:29.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.652+0000 7f33f3b42700 1 -- 172.21.15.57:0/4016381540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33ec105600 msgr2=0x7f33ec1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.652+0000 7f33f3b42700 1 --2- 172.21.15.57:0/4016381540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33ec105600 0x7f33ec1059e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f33e80099c0 tx=0x7f33e8009cd0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.652+0000 7f33f3b42700 1 -- 172.21.15.57:0/4016381540 shutdown_connections 2023-10-08T13:51:29.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.652+0000 7f33f3b42700 1 --2- 172.21.15.57:0/4016381540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec0fff00 0x7f33ec10c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.652+0000 7f33f3b42700 1 --2- 172.21.15.57:0/4016381540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33ec105600 0x7f33ec1059e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.652+0000 7f33f3b42700 1 --2- 172.21.15.57:0/4016381540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ec0ff540 0x7f33ec0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.652+0000 7f33f3b42700 1 -- 172.21.15.57:0/4016381540 >> 172.21.15.57:0/4016381540 conn(0x7f33ec0fb030 msgr2=0x7f33ec0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.652+0000 7f33f3b42700 1 -- 172.21.15.57:0/4016381540 shutdown_connections 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.652+0000 7f33f3b42700 1 -- 172.21.15.57:0/4016381540 wait complete. 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.652+0000 7f33f3b42700 1 Processor -- start 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f3b42700 1 -- start start 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f3b42700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33ec0ff540 0x7f33ec19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f3b42700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ec0fff00 0x7f33ec19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f3b42700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec105600 0x7f33ec195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f3b42700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec195ed0 con 0x7f33ec0ff540 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f3b42700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec196040 con 0x7f33ec105600 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f3b42700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec196200 con 0x7f33ec0fff00 2023-10-08T13:51:29.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f18de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33ec0ff540 0x7f33ec19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f18de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33ec0ff540 0x7f33ec19b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56192/0 (socket says 172.21.15.57:56192) 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f18de700 1 -- 172.21.15.57:0/3317950619 learned_addr learned my addr 172.21.15.57:0/3317950619 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f10dd700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ec0fff00 0x7f33ec19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f20df700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec105600 0x7f33ec195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f18de700 1 -- 172.21.15.57:0/3317950619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ec0fff00 msgr2=0x7f33ec19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f18de700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ec0fff00 0x7f33ec19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f18de700 1 -- 172.21.15.57:0/3317950619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec105600 msgr2=0x7f33ec195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f18de700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec105600 0x7f33ec195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f18de700 1 -- 172.21.15.57:0/3317950619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e80095f0 con 0x7f33ec0ff540 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.653+0000 7f33f18de700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33ec0ff540 0x7f33ec19b680 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f33e800b580 tx=0x7f33e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:29.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.654+0000 7f33deffd700 1 -- 172.21.15.57:0/3317950619 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e801d070 con 0x7f33ec0ff540 2023-10-08T13:51:29.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.654+0000 7f33f3b42700 1 -- 172.21.15.57:0/3317950619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ec196480 con 0x7f33ec0ff540 2023-10-08T13:51:29.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.654+0000 7f33deffd700 1 -- 172.21.15.57:0/3317950619 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33e800fb30 con 0x7f33ec0ff540 2023-10-08T13:51:29.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.654+0000 7f33deffd700 1 -- 172.21.15.57:0/3317950619 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8022c80 con 0x7f33ec0ff540 2023-10-08T13:51:29.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.654+0000 7f33f3b42700 1 -- 172.21.15.57:0/3317950619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ec1a26b0 con 0x7f33ec0ff540 2023-10-08T13:51:29.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.654+0000 7f33f3b42700 1 -- 172.21.15.57:0/3317950619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ec107a80 con 0x7f33ec0ff540 2023-10-08T13:51:29.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.656+0000 7f33deffd700 1 -- 172.21.15.57:0/3317950619 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f33e800fca0 con 0x7f33ec0ff540 2023-10-08T13:51:29.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.656+0000 7f33deffd700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33d80a14d0 0x7f33d80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:29.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.656+0000 7f33f10dd700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33d80a14d0 0x7f33d80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:29.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.656+0000 7f33deffd700 1 -- 172.21.15.57:0/3317950619 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f33e80ce3b0 con 0x7f33ec0ff540 2023-10-08T13:51:29.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.657+0000 7f33f10dd700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33d80a14d0 0x7f33d80a3990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f33ec196f00 tx=0x7f33e0009190 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:29.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.660+0000 7f33deffd700 1 -- 172.21.15.57:0/3317950619 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f33e8097680 con 0x7f33ec0ff540 2023-10-08T13:51:29.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.909+0000 7f33f3b42700 1 -- 172.21.15.57:0/3317950619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33ec067450 con 0x7f33ec0ff540 2023-10-08T13:51:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.910+0000 7f33deffd700 1 -- 172.21.15.57:0/3317950619 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f33e809aca0 con 0x7f33ec0ff540 2023-10-08T13:51:29.911 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:29.911 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:50:07.719573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:51:29.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 -- 172.21.15.57:0/3317950619 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33d80a14d0 msgr2=0x7f33d80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33d80a14d0 0x7f33d80a3990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f33ec196f00 tx=0x7f33e0009190 comp rx=0 tx=0).stop 2023-10-08T13:51:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 -- 172.21.15.57:0/3317950619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33ec0ff540 msgr2=0x7f33ec19b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33ec0ff540 0x7f33ec19b680 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f33e800b580 tx=0x7f33e8004730 comp rx=0 tx=0).stop 2023-10-08T13:51:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 -- 172.21.15.57:0/3317950619 shutdown_connections 2023-10-08T13:51:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33d80a14d0 0x7f33d80a3990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec105600 0x7f33ec195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33ec0ff540 0x7f33ec19b680 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 --2- 172.21.15.57:0/3317950619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ec0fff00 0x7f33ec19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 -- 172.21.15.57:0/3317950619 >> 172.21.15.57:0/3317950619 conn(0x7f33ec0fb030 msgr2=0x7f33ec0fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.912+0000 7f33f3b42700 1 -- 172.21.15.57:0/3317950619 shutdown_connections 2023-10-08T13:51:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:29.913+0000 7f33f3b42700 1 -- 172.21.15.57:0/3317950619 wait complete. 2023-10-08T13:51:29.915 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:51:29.926 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T13:51:30.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.051+0000 7f2be4a38700 1 -- 172.21.15.57:0/1881441212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2be00fef10 msgr2=0x7f2be00ff2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.051+0000 7f2be4a38700 1 --2- 172.21.15.57:0/1881441212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2be00fef10 0x7f2be00ff2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009990 tx=0x7f2bd4009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.051+0000 7f2be4a38700 1 -- 172.21.15.57:0/1881441212 shutdown_connections 2023-10-08T13:51:30.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.051+0000 7f2be4a38700 1 --2- 172.21.15.57:0/1881441212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104030 0x7f2be010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.051+0000 7f2be4a38700 1 --2- 172.21.15.57:0/1881441212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2be00fef10 0x7f2be00ff2f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.051+0000 7f2be4a38700 1 --2- 172.21.15.57:0/1881441212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be00ff8c0 0x7f2be0103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.051+0000 7f2be4a38700 1 -- 172.21.15.57:0/1881441212 >> 172.21.15.57:0/1881441212 conn(0x7f2be0075a90 msgr2=0x7f2be00fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.051+0000 7f2be4a38700 1 -- 172.21.15.57:0/1881441212 shutdown_connections 2023-10-08T13:51:30.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.051+0000 7f2be4a38700 1 -- 172.21.15.57:0/1881441212 wait complete. 2023-10-08T13:51:30.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.052+0000 7f2be4a38700 1 Processor -- start 2023-10-08T13:51:30.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.052+0000 7f2be4a38700 1 -- start start 2023-10-08T13:51:30.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.052+0000 7f2be4a38700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be00fef10 0x7f2be00faf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2be4a38700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104030 0x7f2be00fb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2be4a38700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2be00fb9a0 0x7f2be01a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2be4a38700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01a5310 con 0x7f2be00fb9a0 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2be4a38700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01a5480 con 0x7f2be0104030 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2be4a38700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01a5640 con 0x7f2be00fef10 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bded9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2be00fb9a0 0x7f2be01a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bded9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2be00fb9a0 0x7f2be01a4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56210/0 (socket says 172.21.15.57:56210) 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bded9d700 1 -- 172.21.15.57:0/1922076379 learned_addr learned my addr 172.21.15.57:0/1922076379 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bddd9b700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104030 0x7f2be00fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bde59c700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be00fef10 0x7f2be00faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bded9d700 1 -- 172.21.15.57:0/1922076379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be00fef10 msgr2=0x7f2be00faf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bded9d700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be00fef10 0x7f2be00faf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bded9d700 1 -- 172.21.15.57:0/1922076379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104030 msgr2=0x7f2be00fb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bded9d700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104030 0x7f2be00fb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bded9d700 1 -- 172.21.15.57:0/1922076379 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd40095f0 con 0x7f2be00fb9a0 2023-10-08T13:51:30.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bded9d700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2be00fb9a0 0x7f2be01a4d40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800ef30 tx=0x7f2bd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bcf7fe700 1 -- 172.21.15.57:0/1922076379 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8018070 con 0x7f2be00fb9a0 2023-10-08T13:51:30.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bcf7fe700 1 -- 172.21.15.57:0/1922076379 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd800f040 con 0x7f2be00fb9a0 2023-10-08T13:51:30.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2be4a38700 1 -- 172.21.15.57:0/1922076379 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be01a58c0 con 0x7f2be00fb9a0 2023-10-08T13:51:30.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2bcf7fe700 1 -- 172.21.15.57:0/1922076379 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800cd00 con 0x7f2be00fb9a0 2023-10-08T13:51:30.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.053+0000 7f2be4a38700 1 -- 172.21.15.57:0/1922076379 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be01a5dd0 con 0x7f2be00fb9a0 2023-10-08T13:51:30.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.054+0000 7f2be4a38700 1 -- 172.21.15.57:0/1922076379 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be0109220 con 0x7f2be00fb9a0 2023-10-08T13:51:30.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.055+0000 7f2bcf7fe700 1 -- 172.21.15.57:0/1922076379 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f2bd8003a00 con 0x7f2be00fb9a0 2023-10-08T13:51:30.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.056+0000 7f2bcf7fe700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2bc80a14d0 0x7f2bc80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.056+0000 7f2bcf7fe700 1 -- 172.21.15.57:0/1922076379 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f2bd80ccbf0 con 0x7f2be00fb9a0 2023-10-08T13:51:30.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.056+0000 7f2bde59c700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2bc80a14d0 0x7f2bc80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.057+0000 7f2bde59c700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2bc80a14d0 0x7f2bc80a3990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4006010 tx=0x7f2bd400b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.061+0000 7f2bcf7fe700 1 -- 172.21.15.57:0/1922076379 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2bd8095ec0 con 0x7f2be00fb9a0 2023-10-08T13:51:30.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:29 smithi115 ceph-mon[104408]: pgmap v297: 129 pgs: 129 active+clean; 646 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 993 B/s wr, 11 op/s 2023-10-08T13:51:30.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:29 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2211775494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:30.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:29 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2749187818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:30.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:29 smithi057 ceph-mon[99476]: pgmap v297: 129 pgs: 129 active+clean; 646 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 993 B/s wr, 11 op/s 2023-10-08T13:51:30.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:29 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2211775494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:30.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:29 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2749187818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.266+0000 7f2be4a38700 1 -- 172.21.15.57:0/1922076379 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2be01003e0 con 0x7f2be00fb9a0 2023-10-08T13:51:30.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.268+0000 7f2bcf7fe700 1 -- 172.21.15.57:0/1922076379 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24847 (secure 0 0 0) 0x7f2bd80994e0 con 0x7f2be00fb9a0 2023-10-08T13:51:30.269 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:51:30.269 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T13:51:16.778201+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T13:51:30.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 -- 172.21.15.57:0/1922076379 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2bc80a14d0 msgr2=0x7f2bc80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2bc80a14d0 0x7f2bc80a3990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4006010 tx=0x7f2bd400b260 comp rx=0 tx=0).stop 2023-10-08T13:51:30.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 -- 172.21.15.57:0/1922076379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2be00fb9a0 msgr2=0x7f2be01a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2be00fb9a0 0x7f2be01a4d40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800ef30 tx=0x7f2bd800c3a0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 -- 172.21.15.57:0/1922076379 shutdown_connections 2023-10-08T13:51:30.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2be00fb9a0 0x7f2be01a4d40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be00fef10 0x7f2be00faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2bc80a14d0 0x7f2bc80a3990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 --2- 172.21.15.57:0/1922076379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104030 0x7f2be00fb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 -- 172.21.15.57:0/1922076379 >> 172.21.15.57:0/1922076379 conn(0x7f2be0075a90 msgr2=0x7f2be00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 -- 172.21.15.57:0/1922076379 shutdown_connections 2023-10-08T13:51:30.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.270+0000 7f2be4a38700 1 -- 172.21.15.57:0/1922076379 wait complete. 2023-10-08T13:51:30.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:29 smithi099 ceph-mon[103849]: pgmap v297: 129 pgs: 129 active+clean; 646 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 993 B/s wr, 11 op/s 2023-10-08T13:51:30.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:29 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2211775494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T13:51:30.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:29 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2749187818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T13:51:30.284 DEBUG:tasks.fwd_scrub:Yielding 2023-10-08T13:51:30.284 INFO:teuthology.run_tasks:Running task exec... 2023-10-08T13:51:30.295 INFO:teuthology.task.exec:Executing custom commands... 2023-10-08T13:51:30.295 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T13:51:30.295 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-10-08T13:51:30.296 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-08T13:51:30.296 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-08T13:51:30.296 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-08T13:51:30.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.425+0000 7fbeabed6700 1 -- 172.21.15.57:0/1820742407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbea4105790 msgr2=0x7fbea4105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.425+0000 7fbeabed6700 1 --2- 172.21.15.57:0/1820742407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbea4105790 0x7fbea4105b70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009990 tx=0x7fbea0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.425+0000 7fbeabed6700 1 -- 172.21.15.57:0/1820742407 shutdown_connections 2023-10-08T13:51:30.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.425+0000 7fbeabed6700 1 --2- 172.21.15.57:0/1820742407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea41004b0 0x7fbea410c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.425+0000 7fbeabed6700 1 --2- 172.21.15.57:0/1820742407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbea4105790 0x7fbea4105b70 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.425+0000 7fbeabed6700 1 --2- 172.21.15.57:0/1820742407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbea40ffaf0 0x7fbea40fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.425+0000 7fbeabed6700 1 -- 172.21.15.57:0/1820742407 >> 172.21.15.57:0/1820742407 conn(0x7fbea4076460 msgr2=0x7fbea4076870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.425+0000 7fbeabed6700 1 -- 172.21.15.57:0/1820742407 shutdown_connections 2023-10-08T13:51:30.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.425+0000 7fbeabed6700 1 -- 172.21.15.57:0/1820742407 wait complete. 2023-10-08T13:51:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.425+0000 7fbeabed6700 1 Processor -- start 2023-10-08T13:51:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbeabed6700 1 -- start start 2023-10-08T13:51:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbeabed6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbea40ffaf0 0x7fbea410ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbeabed6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea41004b0 0x7fbea410d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbeabed6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbea4105790 0x7fbea410d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbeabed6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea41a4ed0 con 0x7fbea40ffaf0 2023-10-08T13:51:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbeabed6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea41a5040 con 0x7fbea41004b0 2023-10-08T13:51:30.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbeabed6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea41a5200 con 0x7fbea4105790 2023-10-08T13:51:30.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbea9c72700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbea40ffaf0 0x7fbea410ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbea9c72700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbea40ffaf0 0x7fbea410ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56224/0 (socket says 172.21.15.57:56224) 2023-10-08T13:51:30.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbea9c72700 1 -- 172.21.15.57:0/3359061030 learned_addr learned my addr 172.21.15.57:0/3359061030 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7f69088c1700 1 -- 172.21.15.57:0/423243461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6904100000 msgr2=0x7f690410c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7f69088c1700 1 --2- 172.21.15.57:0/423243461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6904100000 0x7f690410c510 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00b0a0 tx=0x7f68fc00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbeaa473700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbea4105790 0x7fbea410d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbea9471700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea41004b0 0x7fbea410d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7f69088c1700 1 -- 172.21.15.57:0/423243461 shutdown_connections 2023-10-08T13:51:30.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7f69088c1700 1 --2- 172.21.15.57:0/423243461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6904100000 0x7f690410c510 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7f69088c1700 1 --2- 172.21.15.57:0/423243461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6904105650 0x7f6904105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7f69088c1700 1 --2- 172.21.15.57:0/423243461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69040ff640 0x7f69040ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7f69088c1700 1 -- 172.21.15.57:0/423243461 >> 172.21.15.57:0/423243461 conn(0x7f69040767c0 msgr2=0x7f6904076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7f69088c1700 1 -- 172.21.15.57:0/423243461 shutdown_connections 2023-10-08T13:51:30.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7f69088c1700 1 -- 172.21.15.57:0/423243461 wait complete. 2023-10-08T13:51:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbea9c72700 1 -- 172.21.15.57:0/3359061030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbea4105790 msgr2=0x7fbea410d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbea9c72700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbea4105790 0x7fbea410d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbea9c72700 1 -- 172.21.15.57:0/3359061030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea41004b0 msgr2=0x7fbea410d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbea9c72700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea41004b0 0x7fbea410d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.426+0000 7fbea9c72700 1 -- 172.21.15.57:0/3359061030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00095f0 con 0x7fbea40ffaf0 2023-10-08T13:51:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.427+0000 7fbea9c72700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbea40ffaf0 0x7fbea410ce30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbea0005910 tx=0x7fbea0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.427+0000 7fbe9affd700 1 -- 172.21.15.57:0/3359061030 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea001d070 con 0x7fbea40ffaf0 2023-10-08T13:51:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.427+0000 7fbeabed6700 1 -- 172.21.15.57:0/3359061030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea41a5480 con 0x7fbea40ffaf0 2023-10-08T13:51:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.427+0000 7fbe9affd700 1 -- 172.21.15.57:0/3359061030 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbea000fb30 con 0x7fbea40ffaf0 2023-10-08T13:51:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.427+0000 7fbe9affd700 1 -- 172.21.15.57:0/3359061030 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0022c50 con 0x7fbea40ffaf0 2023-10-08T13:51:30.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.427+0000 7fbeabed6700 1 -- 172.21.15.57:0/3359061030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea41a58a0 con 0x7fbea40ffaf0 2023-10-08T13:51:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.427+0000 7f69088c1700 1 Processor -- start 2023-10-08T13:51:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.427+0000 7f69088c1700 1 -- start start 2023-10-08T13:51:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f69088c1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69040ff640 0x7f690419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f69088c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6904100000 0x7f690419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f69088c1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6904105650 0x7f69041968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f69088c1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69041970c0 con 0x7f6904105650 2023-10-08T13:51:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f69088c1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6904197230 con 0x7f6904100000 2023-10-08T13:51:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f69088c1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69041973f0 con 0x7f69040ff640 2023-10-08T13:51:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f690259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69040ff640 0x7f690419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f6902d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6904105650 0x7f69041968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f690259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69040ff640 0x7f690419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:57042/0 (socket says 172.21.15.57:57042) 2023-10-08T13:51:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f690259c700 1 -- 172.21.15.57:0/4065136707 learned_addr learned my addr 172.21.15.57:0/4065136707 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f6901d9b700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6904100000 0x7f690419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f6902d9d700 1 -- 172.21.15.57:0/4065136707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69040ff640 msgr2=0x7f690419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f6902d9d700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69040ff640 0x7f690419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f6902d9d700 1 -- 172.21.15.57:0/4065136707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6904100000 msgr2=0x7f690419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f6902d9d700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6904100000 0x7f690419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f6902d9d700 1 -- 172.21.15.57:0/4065136707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc009d00 con 0x7f6904105650 2023-10-08T13:51:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7fbeabed6700 1 -- 172.21.15.57:0/3359061030 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea4107b10 con 0x7fbea40ffaf0 2023-10-08T13:51:30.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f690259c700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69040ff640 0x7f690419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:51:30.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f6901d9b700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6904100000 0x7f690419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:51:30.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f6902d9d700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6904105650 0x7f69041968f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00b0a0 tx=0x7f68fc004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.428+0000 7f68f37fe700 1 -- 172.21.15.57:0/4065136707 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00e040 con 0x7f6904105650 2023-10-08T13:51:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.429+0000 7f68f37fe700 1 -- 172.21.15.57:0/4065136707 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68fc004e50 con 0x7f6904105650 2023-10-08T13:51:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.429+0000 7f68f37fe700 1 -- 172.21.15.57:0/4065136707 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc005040 con 0x7f6904105650 2023-10-08T13:51:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.429+0000 7f69088c1700 1 -- 172.21.15.57:0/4065136707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6904197670 con 0x7f6904105650 2023-10-08T13:51:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.429+0000 7f69088c1700 1 -- 172.21.15.57:0/4065136707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69041a37d0 con 0x7f6904105650 2023-10-08T13:51:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.430+0000 7fbe9affd700 1 -- 172.21.15.57:0/3359061030 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fbea000fca0 con 0x7fbea40ffaf0 2023-10-08T13:51:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.430+0000 7fbe9affd700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbe900a14d0 0x7fbe900a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.430+0000 7fbe9affd700 1 -- 172.21.15.57:0/3359061030 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fbea00ce580 con 0x7fbea40ffaf0 2023-10-08T13:51:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.430+0000 7fbea9471700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbe900a14d0 0x7fbe900a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.431+0000 7fbea9471700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbe900a14d0 0x7fbe900a3990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbea410e560 tx=0x7fbe9400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.433+0000 7f6914aaab80 1 Processor -- start 2023-10-08T13:51:30.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f68f37fe700 1 -- 172.21.15.57:0/4065136707 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f68fc01d470 con 0x7f6904105650 2023-10-08T13:51:30.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f6914aaab80 1 -- start start 2023-10-08T13:51:30.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ac32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ac3820 0x55ee96ad25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 0x55ee96ad4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f6914aaab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee96a2c4e0 con 0x55ee96ac2f00 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f6914aaab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee96a1dca0 con 0x55ee96ac3820 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f6914aaab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee96a1b5c0 con 0x55ee96ad2ae0 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f6901d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ac32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f6901d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ac32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56238/0 (socket says 172.21.15.57:56238) 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f6901d9b700 1 -- 172.21.15.57:0/2293615555 learned_addr learned my addr 172.21.15.57:0/2293615555 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.434+0000 7f690259c700 1 --2- 172.21.15.57:0/2293615555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ac3820 0x55ee96ad25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f6901d9b700 1 --2- 172.21.15.57:0/2293615555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 0x55ee96ad4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f68f37fe700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68ec0a1550 0x7f68ec0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f6901d9b700 1 -- 172.21.15.57:0/2293615555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 msgr2=0x55ee96ad4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f6901d9b700 1 --2- 172.21.15.57:0/2293615555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 0x55ee96ad4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f6901d9b700 1 -- 172.21.15.57:0/2293615555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ac3820 msgr2=0x55ee96ad25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f6901d9b700 1 --2- 172.21.15.57:0/2293615555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ac3820 0x55ee96ad25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f6901d9b700 1 -- 172.21.15.57:0/2293615555 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee96841e60 con 0x55ee96ac2f00 2023-10-08T13:51:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f6902d9d700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68ec0a1550 0x7f68ec0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f6901d9b700 1 --2- 172.21.15.57:0/2293615555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ac32e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f68f80139f0 tx=0x7f68f8009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0aa53ba3a28945 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f6902d9d700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68ec0a1550 0x7f68ec0a3a10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f68fc007bb0 tx=0x7f68fc06e360 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f68f37fe700 1 -- 172.21.15.57:0/4065136707 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f68fc09baf0 con 0x7f6904105650 2023-10-08T13:51:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.435+0000 7f68f0ff9700 1 -- 172.21.15.57:0/2293615555 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8004030 con 0x55ee96ac2f00 2023-10-08T13:51:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f68f0ff9700 1 -- 172.21.15.57:0/2293615555 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68f800f850 con 0x55ee96ac2f00 2023-10-08T13:51:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f68f0ff9700 1 -- 172.21.15.57:0/2293615555 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8003a20 con 0x55ee96ac2f00 2023-10-08T13:51:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7fbe9affd700 1 -- 172.21.15.57:0/3359061030 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbea0097970 con 0x7fbea40ffaf0 2023-10-08T13:51:30.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f6914aaab80 1 -- 172.21.15.57:0/2293615555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 msgr2=0x55ee96ac32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f6914aaab80 1 --2- 172.21.15.57:0/2293615555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ac32e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f68f80139f0 tx=0x7f68f8009c90 comp rx=0 tx=0).stop 2023-10-08T13:51:30.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f6914aaab80 1 -- 172.21.15.57:0/2293615555 shutdown_connections 2023-10-08T13:51:30.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f6914aaab80 1 --2- 172.21.15.57:0/2293615555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 0x55ee96ad4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f6914aaab80 1 --2- 172.21.15.57:0/2293615555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ac32e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f6914aaab80 1 --2- 172.21.15.57:0/2293615555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ac3820 0x55ee96ad25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f6914aaab80 1 -- 172.21.15.57:0/2293615555 >> 172.21.15.57:0/2293615555 conn(0x55ee96a382e0 msgr2=0x55ee96abf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f6914aaab80 1 -- 172.21.15.57:0/2293615555 shutdown_connections 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.436+0000 7f6914aaab80 1 -- 172.21.15.57:0/2293615555 wait complete. 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6914aaab80 1 -- start start 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6914aaab80 1 Processor -- start 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ad24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ac3820 0x55ee96acf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 0x55ee96acfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6914aaab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee96a16eb0 con 0x55ee96ac2f00 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6914aaab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee969b3790 con 0x55ee96ad2ae0 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6914aaab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee969b3430 con 0x55ee96ac3820 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6901d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ad24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6914aaab80 10 client.0 _ll_register_callbacks cb 0x55ee96810880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6901d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ad24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56246/0 (socket says 172.21.15.57:56246) 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.437+0000 7f6901d9b700 1 -- 172.21.15.57:0/821353187 learned_addr learned my addr 172.21.15.57:0/821353187 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f690259c700 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ac3820 0x55ee96acf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f6901d9b700 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 0x55ee96acfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f6901d9b700 1 -- 172.21.15.57:0/821353187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ac3820 msgr2=0x55ee96acf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f6901d9b700 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ac3820 0x55ee96acf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f6901d9b700 1 -- 172.21.15.57:0/821353187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 msgr2=0x55ee96acfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f6901d9b700 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 0x55ee96acfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f6901d9b700 1 -- 172.21.15.57:0/821353187 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f8013600 con 0x55ee96ac2f00 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f6901d9b700 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ad24a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f68f8010010 tx=0x7f68f8005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae6cff1e32255cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f68e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f68e9ffb700 1 -- 172.21.15.57:0/821353187 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8003880 con 0x55ee96ac2f00 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f68e9ffb700 1 -- 172.21.15.57:0/821353187 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f68f8005690 con 0x55ee96ac2f00 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f6914aaab80 1 -- 172.21.15.57:0/821353187 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee969def30 con 0x55ee96ac2f00 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f68e9ffb700 1 -- 172.21.15.57:0/821353187 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68f80039e0 con 0x55ee96ac2f00 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f6914aaab80 10 client.15810 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f6914aaab80 1 -- 172.21.15.57:0/821353187 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee9682ace0 con 0x55ee96ac2f00 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.438+0000 7f68e9ffb700 1 -- 172.21.15.57:0/821353187 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee969def30 con 0x55ee96ac2f00 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.439+0000 7f68e9ffb700 1 -- 172.21.15.57:0/821353187 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f68f802b020 con 0x55ee96ac2f00 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.439+0000 7f6914aaab80 10 client.15810 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.439+0000 7f6914aaab80 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:51:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.439+0000 7f6914aaab80 20 client.15810 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.439+0000 7f6914aaab80 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ee96ae23b0 0x55ee96ae47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.439+0000 7f6914aaab80 4 client.15810 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.439+0000 7f6914aaab80 1 -- 172.21.15.57:0/821353187 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee96a0fe90 con 0x55ee96ae23b0 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.439+0000 7f690259c700 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ee96ae23b0 0x55ee96ae47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.439+0000 7f68e9ffb700 10 client.15810 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.439+0000 7f690259c700 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ee96ae23b0 0x55ee96ae47a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff837c9424fb2f9 server_cookie=14f908f7ff93d6da in_seq=0 out_seq=0 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f68e9ffb700 1 -- 172.21.15.57:0/821353187 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ee96a0fe90 con 0x55ee96ae23b0 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f68e9ffb700 10 client.15810 handle_command_reply: tid=0 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f6914aaab80 1 client.15810 shutdown 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f6914aaab80 1 -- 172.21.15.57:0/821353187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 msgr2=0x55ee96ad24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f6914aaab80 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ad24a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f68f8010010 tx=0x7f68f8005e20 comp rx=0 tx=0).stop 2023-10-08T13:51:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f6914aaab80 1 -- 172.21.15.57:0/821353187 shutdown_connections 2023-10-08T13:51:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f6914aaab80 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ee96ae23b0 0x55ee96ae47a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f6914aaab80 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 0x55ee96acfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f6914aaab80 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac2f00 0x55ee96ad24a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f6914aaab80 1 --2- 172.21.15.57:0/821353187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ac3820 0x55ee96acf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f6914aaab80 1 -- 172.21.15.57:0/821353187 >> 172.21.15.57:0/821353187 conn(0x55ee96a382e0 msgr2=0x55ee96abef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.440+0000 7f68e9ffb700 0 client.15810 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.441+0000 7f6914aaab80 1 -- 172.21.15.57:0/821353187 shutdown_connections 2023-10-08T13:51:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.441+0000 7f6914aaab80 1 -- 172.21.15.57:0/821353187 wait complete. 2023-10-08T13:51:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.441+0000 7f6914aaab80 20 client.15810 trim_cache size 0 max 16384 2023-10-08T13:51:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6914aaab80 1 Processor -- start 2023-10-08T13:51:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6914aaab80 1 -- start start 2023-10-08T13:51:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ac2f00 0x55ee96acf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 0x55ee96ade490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6914aaab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee96ad5390 con 0x55ee96ac3820 2023-10-08T13:51:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6914aaab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee968979a0 con 0x55ee96ac2f00 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6914aaab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee96ac4150 con 0x55ee96ad2ae0 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f690259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f690259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56256/0 (socket says 172.21.15.57:56256) 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f690259c700 1 -- 172.21.15.57:0/4124409571 learned_addr learned my addr 172.21.15.57:0/4124409571 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6901d9b700 1 --2- 172.21.15.57:0/4124409571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ac2f00 0x55ee96acf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f690259c700 1 -- 172.21.15.57:0/4124409571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 msgr2=0x55ee96ade490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6901d9b700 1 --2- 172.21.15.57:0/4124409571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 0x55ee96ade490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f690259c700 1 --2- 172.21.15.57:0/4124409571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 0x55ee96ade490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f690259c700 1 -- 172.21.15.57:0/4124409571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ac2f00 msgr2=0x55ee96acf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f690259c700 1 --2- 172.21.15.57:0/4124409571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ac2f00 0x55ee96acf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f690259c700 1 -- 172.21.15.57:0/4124409571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee96abb320 con 0x55ee96ac3820 2023-10-08T13:51:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.458+0000 7f6901d9b700 1 --2- 172.21.15.57:0/4124409571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 0x55ee96ade490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f690259c700 1 --2- 172.21.15.57:0/4124409571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad9620 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f68f4004660 tx=0x7f68f4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef52bcf584a00716 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f68f17fa700 1 -- 172.21.15.57:0/4124409571 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f401e270 con 0x55ee96ac3820 2023-10-08T13:51:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f68f17fa700 1 -- 172.21.15.57:0/4124409571 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68f4017040 con 0x55ee96ac3820 2023-10-08T13:51:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f68f17fa700 1 -- 172.21.15.57:0/4124409571 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4010af0 con 0x55ee96ac3820 2023-10-08T13:51:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f6914aaab80 1 -- 172.21.15.57:0/4124409571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 msgr2=0x55ee96ad9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f6914aaab80 1 --2- 172.21.15.57:0/4124409571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad9620 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f68f4004660 tx=0x7f68f4009260 comp rx=0 tx=0).stop 2023-10-08T13:51:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f6914aaab80 1 -- 172.21.15.57:0/4124409571 shutdown_connections 2023-10-08T13:51:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f6914aaab80 1 --2- 172.21.15.57:0/4124409571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ad2ae0 0x55ee96ade490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f6914aaab80 1 --2- 172.21.15.57:0/4124409571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ac2f00 0x55ee96acf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f6914aaab80 1 --2- 172.21.15.57:0/4124409571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad9620 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.459+0000 7f6914aaab80 1 -- 172.21.15.57:0/4124409571 >> 172.21.15.57:0/4124409571 conn(0x55ee96a382e0 msgr2=0x55ee96addd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 1 -- 172.21.15.57:0/4124409571 shutdown_connections 2023-10-08T13:51:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 1 -- 172.21.15.57:0/4124409571 wait complete. 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 1 -- start start 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 1 Processor -- start 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ac2f00 0x55ee96ad6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 0x55ee96b361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee96ad7090 con 0x55ee96ac3820 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee96b366f0 con 0x55ee96ad2ae0 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee96b36830 con 0x55ee96ac2f00 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6914aaab80 10 client.0 _ll_register_callbacks cb 0x7f68f80138d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f690259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f690259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56262/0 (socket says 172.21.15.57:56262) 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f690259c700 1 -- 172.21.15.57:0/3109717873 learned_addr learned my addr 172.21.15.57:0/3109717873 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.460+0000 7f6901d9b700 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 0x55ee96b361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f6901d9b700 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ac2f00 0x55ee96ad6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f690259c700 1 -- 172.21.15.57:0/3109717873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ac2f00 msgr2=0x55ee96ad6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f690259c700 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ac2f00 0x55ee96ad6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f690259c700 1 -- 172.21.15.57:0/3109717873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 msgr2=0x55ee96b361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f690259c700 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 0x55ee96b361b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f690259c700 1 -- 172.21.15.57:0/3109717873 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f4004340 con 0x55ee96ac3820 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f6901d9b700 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 0x55ee96b361b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f690259c700 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad69b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005e90 tx=0x7f68f401eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da4c5ed22eb46b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f68e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f68e9ffb700 1 -- 172.21.15.57:0/3109717873 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4010820 con 0x55ee96ac3820 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f6914aaab80 1 -- 172.21.15.57:0/3109717873 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68f800f850 con 0x55ee96ac3820 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f68e9ffb700 1 -- 172.21.15.57:0/3109717873 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f68f4010980 con 0x55ee96ac3820 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f68e9ffb700 1 -- 172.21.15.57:0/3109717873 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68f4010b40 con 0x55ee96ac3820 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f6914aaab80 10 client.15816 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f6914aaab80 1 -- 172.21.15.57:0/3109717873 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee968682d0 con 0x55ee96ac3820 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f68e9ffb700 1 -- 172.21.15.57:0/3109717873 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68f800f850 con 0x55ee96ac3820 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.461+0000 7f68e9ffb700 1 -- 172.21.15.57:0/3109717873 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f68f40036c0 con 0x55ee96ac3820 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.462+0000 7f6914aaab80 10 client.15816 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.462+0000 7f6914aaab80 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.462+0000 7f6914aaab80 20 client.15816 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.462+0000 7f6914aaab80 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ee96b37f50 0x55ee96b3a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.462+0000 7f6914aaab80 4 client.15816 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.462+0000 7f6914aaab80 1 -- 172.21.15.57:0/3109717873 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ee96a0fbb0 con 0x55ee96b37f50 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.462+0000 7f6901d9b700 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ee96b37f50 0x55ee96b3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.462+0000 7f68e9ffb700 10 client.15816 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.462+0000 7f6901d9b700 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ee96b37f50 0x55ee96b3a370 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d79c288b3be646e server_cookie=309d23da76c5e1d3 in_seq=0 out_seq=0 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f68e9ffb700 1 -- 172.21.15.57:0/3109717873 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ee96a0fbb0 con 0x55ee96b37f50 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f68e9ffb700 10 client.15816 handle_command_reply: tid=0 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f6914aaab80 1 client.15816 shutdown 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f6914aaab80 1 -- 172.21.15.57:0/3109717873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 msgr2=0x55ee96ad69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f6914aaab80 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad69b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005e90 tx=0x7f68f401eef0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f6914aaab80 1 -- 172.21.15.57:0/3109717873 shutdown_connections 2023-10-08T13:51:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f6914aaab80 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ee96b37f50 0x55ee96b3a370 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f6914aaab80 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee96ad2ae0 0x55ee96b361b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f6914aaab80 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ee96ac2f00 0x55ee96ad6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f6914aaab80 1 --2- 172.21.15.57:0/3109717873 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ee96ac3820 0x55ee96ad69b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.463+0000 7f6914aaab80 1 -- 172.21.15.57:0/3109717873 >> 172.21.15.57:0/3109717873 conn(0x55ee96a382e0 msgr2=0x55ee96ade610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f68e9ffb700 0 client.15816 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f6914aaab80 1 -- 172.21.15.57:0/3109717873 shutdown_connections 2023-10-08T13:51:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f6914aaab80 1 -- 172.21.15.57:0/3109717873 wait complete. 2023-10-08T13:51:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f6914aaab80 20 client.15816 trim_cache size 0 max 16384 2023-10-08T13:51:30.472 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:51:30.472 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-10-08T13:51:30.473 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "ff3c039d-d4ec-408c-9cea-4b6d8dc84e7d", 2023-10-08T13:51:30.473 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-10-08T13:51:30.473 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 -- 172.21.15.57:0/4065136707 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68ec0a1550 msgr2=0x7f68ec0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68ec0a1550 0x7f68ec0a3a10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f68fc007bb0 tx=0x7f68fc06e360 comp rx=0 tx=0).stop 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 -- 172.21.15.57:0/4065136707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6904105650 msgr2=0x7f69041968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6904105650 0x7f69041968f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00b0a0 tx=0x7f68fc004ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 -- 172.21.15.57:0/4065136707 shutdown_connections 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6904105650 0x7f69041968f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69040ff640 0x7f690419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f68ec0a1550 0x7f68ec0a3a10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 --2- 172.21.15.57:0/4065136707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6904100000 0x7f690419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 -- 172.21.15.57:0/4065136707 >> 172.21.15.57:0/4065136707 conn(0x7f69040767c0 msgr2=0x7f69041033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 -- 172.21.15.57:0/4065136707 shutdown_connections 2023-10-08T13:51:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.464+0000 7f69088c1700 1 -- 172.21.15.57:0/4065136707 wait complete. 2023-10-08T13:51:30.478 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:51:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.607+0000 7ff4a476d700 1 -- 172.21.15.57:0/506020512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c100150 msgr2=0x7ff49c100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.607+0000 7ff4a476d700 1 --2- 172.21.15.57:0/506020512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c100150 0x7ff49c100530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.607+0000 7ff4a476d700 1 -- 172.21.15.57:0/506020512 shutdown_connections 2023-10-08T13:51:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.607+0000 7ff4a476d700 1 --2- 172.21.15.57:0/506020512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff49c10d190 0x7ff49c10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.607+0000 7ff4a476d700 1 --2- 172.21.15.57:0/506020512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c100150 0x7ff49c100530 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.607+0000 7ff4a476d700 1 --2- 172.21.15.57:0/506020512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49c100a70 0x7ff49c10cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.607+0000 7ff4a476d700 1 -- 172.21.15.57:0/506020512 >> 172.21.15.57:0/506020512 conn(0x7ff49c076780 msgr2=0x7ff49c076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.607+0000 7ff4a476d700 1 -- 172.21.15.57:0/506020512 shutdown_connections 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.607+0000 7ff4a476d700 1 -- 172.21.15.57:0/506020512 wait complete. 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a476d700 1 Processor -- start 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a476d700 1 -- start start 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a476d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c100150 0x7ff49c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a476d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49c100a70 0x7ff49c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a476d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff49c10d190 0x7ff49c19d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a476d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19ddd0 con 0x7ff49c10d190 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a476d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19df40 con 0x7ff49c100a70 2023-10-08T13:51:30.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a476d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19e100 con 0x7ff49c100150 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a2d0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff49c10d190 0x7ff49c19d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a2d0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff49c10d190 0x7ff49c19d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56278/0 (socket says 172.21.15.57:56278) 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/539985529 learned_addr learned my addr 172.21.15.57:0/539985529 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a2509700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c100150 0x7ff49c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49c100a70 0x7ff49c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/539985529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c100150 msgr2=0x7ff49c198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c100150 0x7ff49c198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/539985529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49c100a70 msgr2=0x7ff49c1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49c100a70 0x7ff49c1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/539985529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980095f0 con 0x7ff49c10d190 2023-10-08T13:51:30.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.608+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49c100a70 0x7ff49c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:30.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.609+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff49c10d190 0x7ff49c19d790 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff494009fd0 tx=0x7ff49400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.609+0000 7ff4937fe700 1 -- 172.21.15.57:0/539985529 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494018070 con 0x7ff49c10d190 2023-10-08T13:51:30.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.609+0000 7ff4937fe700 1 -- 172.21.15.57:0/539985529 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49400f040 con 0x7ff49c10d190 2023-10-08T13:51:30.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.609+0000 7ff4937fe700 1 -- 172.21.15.57:0/539985529 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400cdb0 con 0x7ff49c10d190 2023-10-08T13:51:30.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.609+0000 7ff4a476d700 1 -- 172.21.15.57:0/539985529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c19e380 con 0x7ff49c10d190 2023-10-08T13:51:30.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.609+0000 7ff4a476d700 1 -- 172.21.15.57:0/539985529 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c1a5c20 con 0x7ff49c10d190 2023-10-08T13:51:30.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.612+0000 7ff4937fe700 1 -- 172.21.15.57:0/539985529 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ff494004ad0 con 0x7ff49c10d190 2023-10-08T13:51:30.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4937fe700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff4880a14d0 0x7ff4880a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4a2509700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff4880a14d0 0x7ff4880a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4937fe700 1 -- 172.21.15.57:0/539985529 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff4940ccce0 con 0x7ff49c10d190 2023-10-08T13:51:30.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4b0956b80 1 Processor -- start 2023-10-08T13:51:30.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4b0956b80 1 -- start start 2023-10-08T13:51:30.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de08e80 0x557b9de09260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de097a0 0x557b9de14110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de14650 0x557b9de16a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4a2509700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff4880a14d0 0x7ff4880a3990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff49800b2e0 tx=0x7ff498005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9dd72460 con 0x557b9de097a0 2023-10-08T13:51:30.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9dd63c20 con 0x557b9de08e80 2023-10-08T13:51:30.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.613+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9dd61540 con 0x557b9de14650 2023-10-08T13:51:30.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a1d08700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de097a0 0x557b9de14110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a1d08700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de097a0 0x557b9de14110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56282/0 (socket says 172.21.15.57:56282) 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a1d08700 1 -- 172.21.15.57:0/2826004177 learned_addr learned my addr 172.21.15.57:0/2826004177 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a2509700 1 --2- 172.21.15.57:0/2826004177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de08e80 0x557b9de09260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/2826004177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de14650 0x557b9de16a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a1d08700 1 -- 172.21.15.57:0/2826004177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de14650 msgr2=0x557b9de16a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/2826004177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de14650 0x557b9de16a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a1d08700 1 -- 172.21.15.57:0/2826004177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de08e80 msgr2=0x557b9de09260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/2826004177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de08e80 0x557b9de09260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a1d08700 1 -- 172.21.15.57:0/2826004177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b9db87de0 con 0x557b9de097a0 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/2826004177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de097a0 0x557b9de14110 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009d80 tx=0x7ff48c0116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56902df64d33cc25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff490ff9700 1 -- 172.21.15.57:0/2826004177 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c011e20 con 0x557b9de097a0 2023-10-08T13:51:30.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff490ff9700 1 -- 172.21.15.57:0/2826004177 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff48c012910 con 0x557b9de097a0 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.614+0000 7ff490ff9700 1 -- 172.21.15.57:0/2826004177 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00f800 con 0x557b9de097a0 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.615+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2826004177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de097a0 msgr2=0x557b9de14110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.615+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2826004177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de097a0 0x557b9de14110 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009d80 tx=0x7ff48c0116c0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.615+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2826004177 shutdown_connections 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.615+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2826004177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de14650 0x557b9de16a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.615+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2826004177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de08e80 0x557b9de09260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.615+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2826004177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de097a0 0x557b9de14110 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.615+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2826004177 >> 172.21.15.57:0/2826004177 conn(0x557b9dd7e260 msgr2=0x557b9de052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.615+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2826004177 shutdown_connections 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2826004177 wait complete. 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4b0956b80 1 -- start start 2023-10-08T13:51:30.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4b0956b80 1 Processor -- start 2023-10-08T13:51:30.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de08e80 0x557b9de13fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de097a0 0x557b9de11330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de11910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9dd5ce30 con 0x557b9de14650 2023-10-08T13:51:30.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9dcf9710 con 0x557b9de08e80 2023-10-08T13:51:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9dcf93b0 con 0x557b9de097a0 2023-10-08T13:51:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4b0956b80 10 client.0 _ll_register_callbacks cb 0x557b9db56800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4a2d0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de11910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4a2d0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de11910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56292/0 (socket says 172.21.15.57:56292) 2023-10-08T13:51:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/2952652939 learned_addr learned my addr 172.21.15.57:0/2952652939 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.616+0000 7ff4a2509700 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de08e80 0x557b9de13fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de097a0 0x557b9de11330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/2952652939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de097a0 msgr2=0x557b9de11330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de097a0 0x557b9de11330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/2952652939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de08e80 msgr2=0x557b9de13fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de08e80 0x557b9de13fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/2952652939 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c0099b0 con 0x557b9de14650 2023-10-08T13:51:30.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de097a0 0x557b9de11330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:30.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de11910 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff4940060b0 tx=0x7ff49409a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406b7f30a129de32 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff485ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:30.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff485ffb700 1 -- 172.21.15.57:0/2952652939 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940b2420 con 0x557b9de14650 2023-10-08T13:51:30.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff485ffb700 1 -- 172.21.15.57:0/2952652939 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff49409a310 con 0x557b9de14650 2023-10-08T13:51:30.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2952652939 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b9dd24eb0 con 0x557b9de14650 2023-10-08T13:51:30.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff485ffb700 1 -- 172.21.15.57:0/2952652939 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff494099c10 con 0x557b9de14650 2023-10-08T13:51:30.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4b0956b80 10 client.15828 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:30.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2952652939 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557b9db70c60 con 0x557b9de14650 2023-10-08T13:51:30.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.617+0000 7ff485ffb700 1 -- 172.21.15.57:0/2952652939 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff494063030 con 0x557b9de14650 2023-10-08T13:51:30.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.618+0000 7ff485ffb700 1 -- 172.21.15.57:0/2952652939 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff4940a1a80 con 0x557b9de14650 2023-10-08T13:51:30.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.618+0000 7ff4b0956b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:30.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.618+0000 7ff4b0956b80 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:51:30.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.618+0000 7ff4b0956b80 20 client.15828 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:30.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.618+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557b9de240c0 0x557b9de264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.618+0000 7ff4b0956b80 4 client.15828 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.618+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2952652939 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b9dd55e10 con 0x557b9de240c0 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.618+0000 7ff4a2509700 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557b9de240c0 0x557b9de264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.618+0000 7ff485ffb700 10 client.15828 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.618+0000 7ff4a2509700 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557b9de240c0 0x557b9de264b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf5b0ff6e9898bf server_cookie=5222e8801b120fd9 in_seq=0 out_seq=0 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff485ffb700 1 -- 172.21.15.57:0/2952652939 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557b9dd55e10 con 0x557b9de240c0 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff485ffb700 10 client.15828 handle_command_reply: tid=0 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 client.15828 shutdown 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2952652939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 msgr2=0x557b9de11910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de11910 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff4940060b0 tx=0x7ff49409a150 comp rx=0 tx=0).stop 2023-10-08T13:51:30.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2952652939 shutdown_connections 2023-10-08T13:51:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557b9de240c0 0x557b9de264b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de11910 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de08e80 0x557b9de13fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2952652939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de097a0 0x557b9de11330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2952652939 >> 172.21.15.57:0/2952652939 conn(0x557b9dd7e260 msgr2=0x557b9de04ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff485ffb700 0 client.15828 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2952652939 shutdown_connections 2023-10-08T13:51:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.619+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2952652939 wait complete. 2023-10-08T13:51:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.620+0000 7ff4b0956b80 20 client.15828 trim_cache size 0 max 16384 2023-10-08T13:51:30.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.636+0000 7ff4b0956b80 1 Processor -- start 2023-10-08T13:51:30.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.636+0000 7ff4b0956b80 1 -- start start 2023-10-08T13:51:30.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de08e80 0x557b9de11880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 0x557b9de0bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de14650 0x557b9de1f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9dbdd920 con 0x557b9de08e80 2023-10-08T13:51:30.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9de01ae0 con 0x557b9de097a0 2023-10-08T13:51:30.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9de0a0d0 con 0x557b9de14650 2023-10-08T13:51:30.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a2509700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de08e80 0x557b9de11880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a2509700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de08e80 0x557b9de11880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56296/0 (socket says 172.21.15.57:56296) 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a2509700 1 -- 172.21.15.57:0/2191034256 learned_addr learned my addr 172.21.15.57:0/2191034256 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/2191034256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de14650 0x557b9de1f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/2191034256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 0x557b9de0bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a2509700 1 -- 172.21.15.57:0/2191034256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de14650 msgr2=0x557b9de1f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a2509700 1 --2- 172.21.15.57:0/2191034256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de14650 0x557b9de1f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a2509700 1 -- 172.21.15.57:0/2191034256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 msgr2=0x557b9de0bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a2509700 1 --2- 172.21.15.57:0/2191034256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 0x557b9de0bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a2509700 1 -- 172.21.15.57:0/2191034256 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b9de014f0 con 0x557b9de08e80 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/2191034256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 0x557b9de0bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.637+0000 7ff4a2509700 1 --2- 172.21.15.57:0/2191034256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de08e80 0x557b9de11880 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4fcec0f9dfddb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4917fa700 1 -- 172.21.15.57:0/2191034256 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498036de0 con 0x557b9de08e80 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4917fa700 1 -- 172.21.15.57:0/2191034256 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff498036070 con 0x557b9de08e80 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4917fa700 1 -- 172.21.15.57:0/2191034256 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498004b30 con 0x557b9de08e80 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2191034256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de08e80 msgr2=0x557b9de11880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2191034256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de08e80 0x557b9de11880 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498023000 comp rx=0 tx=0).stop 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2191034256 shutdown_connections 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2191034256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de14650 0x557b9de1f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2191034256 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de08e80 0x557b9de11880 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/2191034256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 0x557b9de0bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2191034256 >> 172.21.15.57:0/2191034256 conn(0x557b9dd7e260 msgr2=0x557b9de0fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2191034256 shutdown_connections 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.638+0000 7ff4b0956b80 1 -- 172.21.15.57:0/2191034256 wait complete. 2023-10-08T13:51:30.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4b0956b80 1 -- start start 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4b0956b80 1 Processor -- start 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de08e80 0x557b9de1aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 0x557b9de19340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4b0956b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de19880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9de1b780 con 0x557b9de14650 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9de19dc0 con 0x557b9de097a0 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4b0956b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9de19f80 con 0x557b9de08e80 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4b0956b80 10 client.0 _ll_register_callbacks cb 0x557b9db56800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a2d0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de19880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a2d0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de19880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56306/0 (socket says 172.21.15.57:56306) 2023-10-08T13:51:30.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/1408744250 learned_addr learned my addr 172.21.15.57:0/1408744250 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a2509700 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de08e80 0x557b9de1aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 0x557b9de19340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/1408744250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de08e80 msgr2=0x557b9de1aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de08e80 0x557b9de1aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/1408744250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 msgr2=0x557b9de19340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 0x557b9de19340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a2d0a700 1 -- 172.21.15.57:0/1408744250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c110180 con 0x557b9de14650 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.639+0000 7ff4a1d08700 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 0x557b9de19340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff4a2d0a700 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de19880 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff49409a270 tx=0x7ff494099030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e507f31d45ebea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff485ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:30.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff485ffb700 1 -- 172.21.15.57:0/1408744250 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494095c00 con 0x557b9de14650 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff485ffb700 1 -- 172.21.15.57:0/1408744250 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7ff494095d60 con 0x557b9de14650 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff4b0956b80 1 -- 172.21.15.57:0/1408744250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff48c012910 con 0x557b9de14650 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff485ffb700 1 -- 172.21.15.57:0/1408744250 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff49409a310 con 0x557b9de14650 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff4b0956b80 10 client.15834 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff4b0956b80 1 -- 172.21.15.57:0/1408744250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557b9dbae250 con 0x557b9de14650 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff485ffb700 1 -- 172.21.15.57:0/1408744250 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff48c012910 con 0x557b9de14650 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff485ffb700 1 -- 172.21.15.57:0/1408744250 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff494003a00 con 0x557b9de14650 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff4b0956b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff4b0956b80 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff4b0956b80 20 client.15834 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557b9de78d60 0x557b9de7b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff4b0956b80 4 client.15834 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.640+0000 7ff4b0956b80 1 -- 172.21.15.57:0/1408744250 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557b9dd55b30 con 0x557b9de78d60 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4a2509700 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557b9de78d60 0x557b9de7b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4a2509700 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557b9de78d60 0x557b9de7b150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92282320c9bf322 server_cookie=6cc7966193a921c in_seq=0 out_seq=0 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff485ffb700 10 client.15834 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff485ffb700 1 -- 172.21.15.57:0/1408744250 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557b9dd55b30 con 0x557b9de78d60 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff485ffb700 10 client.15834 handle_command_reply: tid=0 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4b0956b80 1 client.15834 shutdown 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4b0956b80 1 -- 172.21.15.57:0/1408744250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 msgr2=0x557b9de19880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de19880 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff49409a270 tx=0x7ff494099030 comp rx=0 tx=0).stop 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4b0956b80 1 -- 172.21.15.57:0/1408744250 shutdown_connections 2023-10-08T13:51:30.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557b9de78d60 0x557b9de7b150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557b9de14650 0x557b9de19880 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b9de08e80 0x557b9de1aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4b0956b80 1 --2- 172.21.15.57:0/1408744250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557b9de097a0 0x557b9de19340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4b0956b80 1 -- 172.21.15.57:0/1408744250 >> 172.21.15.57:0/1408744250 conn(0x557b9dd7e260 msgr2=0x557b9de1f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff485ffb700 0 client.15834 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:30.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.641+0000 7ff4b0956b80 1 -- 172.21.15.57:0/1408744250 shutdown_connections 2023-10-08T13:51:30.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4b0956b80 1 -- 172.21.15.57:0/1408744250 wait complete. 2023-10-08T13:51:30.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4b0956b80 20 client.15834 trim_cache size 0 max 16384 2023-10-08T13:51:30.651 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:51:30.652 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T13:51:30.652 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:51:30.652 INFO:teuthology.orchestra.run.smithi057.stdout: "ff3c039d-d4ec-408c-9cea-4b6d8dc84e7d": { 2023-10-08T13:51:30.652 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:51:30.652 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "ff3c039d-d4ec-408c-9cea-4b6d8dc84e7d", 2023-10-08T13:51:30.652 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:51:30.652 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:51:30.652 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:51:30.652 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:51:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 -- 172.21.15.57:0/539985529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff4880a14d0 msgr2=0x7ff4880a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff4880a14d0 0x7ff4880a3990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff49800b2e0 tx=0x7ff498005960 comp rx=0 tx=0).stop 2023-10-08T13:51:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 -- 172.21.15.57:0/539985529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff49c10d190 msgr2=0x7ff49c19d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff49c10d190 0x7ff49c19d790 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff494009fd0 tx=0x7ff49400ef90 comp rx=0 tx=0).stop 2023-10-08T13:51:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 -- 172.21.15.57:0/539985529 shutdown_connections 2023-10-08T13:51:30.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff49c10d190 0x7ff49c19d790 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c100150 0x7ff49c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff4880a14d0 0x7ff4880a3990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 --2- 172.21.15.57:0/539985529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49c100a70 0x7ff49c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 -- 172.21.15.57:0/539985529 >> 172.21.15.57:0/539985529 conn(0x7ff49c076780 msgr2=0x7ff49c10e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 -- 172.21.15.57:0/539985529 shutdown_connections 2023-10-08T13:51:30.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.642+0000 7ff4a476d700 1 -- 172.21.15.57:0/539985529 wait complete. 2023-10-08T13:51:30.659 INFO:tasks.cephfs.filesystem:scrub status for tag:ff3c039d-d4ec-408c-9cea-4b6d8dc84e7d - {'path': '/', 'tag': 'ff3c039d-d4ec-408c-9cea-4b6d8dc84e7d', 'options': 'recursive,force'} 2023-10-08T13:51:30.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.687+0000 7fbeabed6700 1 -- 172.21.15.57:0/3359061030 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fbea410e310 con 0x7fbe900a14d0 2023-10-08T13:51:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.798+0000 7fbe9affd700 1 -- 172.21.15.57:0/3359061030 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbea410e310 con 0x7fbe900a14d0 2023-10-08T13:51:30.799 INFO:teuthology.orchestra.run.smithi057.stdout:Schedule set for path / 2023-10-08T13:51:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.800+0000 7fbeabed6700 1 -- 172.21.15.57:0/3359061030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbe900a14d0 msgr2=0x7fbe900a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.800+0000 7fbeabed6700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbe900a14d0 0x7fbe900a3990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbea410e560 tx=0x7fbe9400b410 comp rx=0 tx=0).stop 2023-10-08T13:51:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.800+0000 7fbeabed6700 1 -- 172.21.15.57:0/3359061030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbea40ffaf0 msgr2=0x7fbea410ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.801+0000 7fbeabed6700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbea40ffaf0 0x7fbea410ce30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbea0005910 tx=0x7fbea0004730 comp rx=0 tx=0).stop 2023-10-08T13:51:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.801+0000 7fbeabed6700 1 -- 172.21.15.57:0/3359061030 shutdown_connections 2023-10-08T13:51:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.801+0000 7fbeabed6700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbea4105790 0x7fbea410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.801+0000 7fbeabed6700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbea40ffaf0 0x7fbea410ce30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.801+0000 7fbeabed6700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbe900a14d0 0x7fbe900a3990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.801+0000 7fbeabed6700 1 --2- 172.21.15.57:0/3359061030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea41004b0 0x7fbea410d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.801+0000 7fbeabed6700 1 -- 172.21.15.57:0/3359061030 >> 172.21.15.57:0/3359061030 conn(0x7fbea4076460 msgr2=0x7fbea40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.801+0000 7fbeabed6700 1 -- 172.21.15.57:0/3359061030 shutdown_connections 2023-10-08T13:51:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.801+0000 7fbeabed6700 1 -- 172.21.15.57:0/3359061030 wait complete. 2023-10-08T13:51:30.814 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-10-08T13:51:30.982 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:30 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3317950619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:30.982 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:30 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1922076379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.983+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1557321575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a08100730 msgr2=0x7f7a0810c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.983+0000 7f7a0e1fa700 1 --2- 172.21.15.57:0/1557321575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a08100730 0x7f7a0810c440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000b0a0 tx=0x7f7a0000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.983+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1557321575 shutdown_connections 2023-10-08T13:51:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.983+0000 7f7a0e1fa700 1 --2- 172.21.15.57:0/1557321575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a08100730 0x7f7a0810c440 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.983+0000 7f7a0e1fa700 1 --2- 172.21.15.57:0/1557321575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a081045b0 0x7f7a08104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.983+0000 7f7a0e1fa700 1 --2- 172.21.15.57:0/1557321575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a080ffd70 0x7f7a081001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.983+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1557321575 >> 172.21.15.57:0/1557321575 conn(0x7f7a08075bf0 msgr2=0x7f7a080fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.983+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1557321575 shutdown_connections 2023-10-08T13:51:30.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.983+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1557321575 wait complete. 2023-10-08T13:51:30.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.984+0000 7f7a0e1fa700 1 Processor -- start 2023-10-08T13:51:30.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.984+0000 7f7a0e1fa700 1 -- start start 2023-10-08T13:51:30.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a0e1fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a080ffd70 0x7f7a080fb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a0e1fa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a08100730 0x7f7a080fb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a0e1fa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a081045b0 0x7f7a080fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a0e1fa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a081a4fd0 con 0x7f7a080ffd70 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a0e1fa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a081a5140 con 0x7f7a081045b0 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a0e1fa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a081a5300 con 0x7f7a08100730 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a077fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a080ffd70 0x7f7a080fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a06ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a08100730 0x7f7a080fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a077fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a080ffd70 0x7f7a080fb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56338/0 (socket says 172.21.15.57:56338) 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a077fe700 1 -- 172.21.15.57:0/1371605759 learned_addr learned my addr 172.21.15.57:0/1371605759 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a07fff700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a081045b0 0x7f7a080fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a077fe700 1 -- 172.21.15.57:0/1371605759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a08100730 msgr2=0x7f7a080fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a077fe700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a08100730 0x7f7a080fb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a077fe700 1 -- 172.21.15.57:0/1371605759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a081045b0 msgr2=0x7f7a080fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:30.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a077fe700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a081045b0 0x7f7a080fbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:30.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a077fe700 1 -- 172.21.15.57:0/1371605759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc009520 con 0x7f7a080ffd70 2023-10-08T13:51:30.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a06ffd700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a08100730 0x7f7a080fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:51:30.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a077fe700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a080ffd70 0x7f7a080fb030 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f79f800eb30 tx=0x7f79f800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.985+0000 7f7a04ff9700 1 -- 172.21.15.57:0/1371605759 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8009880 con 0x7f7a080ffd70 2023-10-08T13:51:30.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.986+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1371605759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f8009520 con 0x7f7a080ffd70 2023-10-08T13:51:30.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.986+0000 7f7a04ff9700 1 -- 172.21.15.57:0/1371605759 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79f80044e0 con 0x7f7a080ffd70 2023-10-08T13:51:30.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.986+0000 7f7a04ff9700 1 -- 172.21.15.57:0/1371605759 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8003d80 con 0x7f7a080ffd70 2023-10-08T13:51:30.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.986+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1371605759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a00009d00 con 0x7f7a080ffd70 2023-10-08T13:51:30.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.987+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1371605759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a08100d80 con 0x7f7a080ffd70 2023-10-08T13:51:30.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.988+0000 7f7a04ff9700 1 -- 172.21.15.57:0/1371605759 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f79f8004000 con 0x7f7a080ffd70 2023-10-08T13:51:30.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.988+0000 7f7a04ff9700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79f00a14d0 0x7f79f00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:30.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.988+0000 7f7a04ff9700 1 -- 172.21.15.57:0/1371605759 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f79f8014070 con 0x7f7a080ffd70 2023-10-08T13:51:30.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.989+0000 7f7a06ffd700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79f00a14d0 0x7f79f00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:30.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.989+0000 7f7a06ffd700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79f00a14d0 0x7f79f00a3990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009fd0 tx=0x7f79fc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:30.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:30.992+0000 7f7a04ff9700 1 -- 172.21.15.57:0/1371605759 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f79f8096a30 con 0x7f7a080ffd70 2023-10-08T13:51:31.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:30 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3317950619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:31.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:30 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1922076379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:31.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.219+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1371605759 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a0810f530 con 0x7f79f00a14d0 2023-10-08T13:51:31.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:30 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3317950619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:51:31.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:30 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1922076379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T13:51:31.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.277+0000 7f7a04ff9700 1 -- 172.21.15.57:0/1371605759 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f7a0810f530 con 0x7f79f00a14d0 2023-10-08T13:51:31.279 INFO:teuthology.orchestra.run.smithi057.stdout:Retention added to path / 2023-10-08T13:51:31.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1371605759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79f00a14d0 msgr2=0x7f79f00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:31.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79f00a14d0 0x7f79f00a3990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009fd0 tx=0x7f79fc009260 comp rx=0 tx=0).stop 2023-10-08T13:51:31.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1371605759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a080ffd70 msgr2=0x7f7a080fb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:31.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a080ffd70 0x7f7a080fb030 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f79f800eb30 tx=0x7f79f800ee40 comp rx=0 tx=0).stop 2023-10-08T13:51:31.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1371605759 shutdown_connections 2023-10-08T13:51:31.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79f00a14d0 0x7f79f00a3990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a081045b0 0x7f7a080fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a080ffd70 0x7f7a080fb030 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 --2- 172.21.15.57:0/1371605759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a08100730 0x7f7a080fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1371605759 >> 172.21.15.57:0/1371605759 conn(0x7f7a08075bf0 msgr2=0x7f7a080ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:31.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1371605759 shutdown_connections 2023-10-08T13:51:31.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.280+0000 7f7a0e1fa700 1 -- 172.21.15.57:0/1371605759 wait complete. 2023-10-08T13:51:31.293 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-08T13:51:31.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.419+0000 7fb09cb5f700 1 -- 172.21.15.57:0/1429159183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0980ff5d0 msgr2=0x7fb0980ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:31.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.419+0000 7fb09cb5f700 1 --2- 172.21.15.57:0/1429159183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0980ff5d0 0x7fb0980ffa50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb088009990 tx=0x7fb088009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.419+0000 7fb09cb5f700 1 -- 172.21.15.57:0/1429159183 shutdown_connections 2023-10-08T13:51:31.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.419+0000 7fb09cb5f700 1 --2- 172.21.15.57:0/1429159183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0980fff90 0x7fb09810c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.419+0000 7fb09cb5f700 1 --2- 172.21.15.57:0/1429159183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0981045e0 0x7fb0981049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.419+0000 7fb09cb5f700 1 --2- 172.21.15.57:0/1429159183 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0980ff5d0 0x7fb0980ffa50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.419+0000 7fb09cb5f700 1 -- 172.21.15.57:0/1429159183 >> 172.21.15.57:0/1429159183 conn(0x7fb098076790 msgr2=0x7fb098076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:31.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.419+0000 7fb09cb5f700 1 -- 172.21.15.57:0/1429159183 shutdown_connections 2023-10-08T13:51:31.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.419+0000 7fb09cb5f700 1 -- 172.21.15.57:0/1429159183 wait complete. 2023-10-08T13:51:31.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb09cb5f700 1 Processor -- start 2023-10-08T13:51:31.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb09cb5f700 1 -- start start 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb09cb5f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0980ff5d0 0x7fb09819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb09cb5f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0980fff90 0x7fb09819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb09cb5f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0981045e0 0x7fb098196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb09cb5f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098196fe0 con 0x7fb0981045e0 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb09cb5f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098197150 con 0x7fb0980ff5d0 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb09cb5f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098197310 con 0x7fb0980fff90 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb096d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0981045e0 0x7fb098196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb096d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0981045e0 0x7fb098196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56356/0 (socket says 172.21.15.57:56356) 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb096d9d700 1 -- 172.21.15.57:0/3052584496 learned_addr learned my addr 172.21.15.57:0/3052584496 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb09659c700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0980ff5d0 0x7fb09819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:31.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb095d9b700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0980fff90 0x7fb09819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb096d9d700 1 -- 172.21.15.57:0/3052584496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0980fff90 msgr2=0x7fb09819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb096d9d700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0980fff90 0x7fb09819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb096d9d700 1 -- 172.21.15.57:0/3052584496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0980ff5d0 msgr2=0x7fb09819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb096d9d700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0980ff5d0 0x7fb09819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb096d9d700 1 -- 172.21.15.57:0/3052584496 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0880095f0 con 0x7fb0981045e0 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.420+0000 7fb095d9b700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0980fff90 0x7fb09819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.421+0000 7fb096d9d700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0981045e0 0x7fb098196980 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb09000ada0 tx=0x7fb09000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.421+0000 7fb0877fe700 1 -- 172.21.15.57:0/3052584496 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09000f070 con 0x7fb0981045e0 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.421+0000 7fb0877fe700 1 -- 172.21.15.57:0/3052584496 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb090007b30 con 0x7fb0981045e0 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.421+0000 7fb09cb5f700 1 -- 172.21.15.57:0/3052584496 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0981975f0 con 0x7fb0981045e0 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.421+0000 7fb0877fe700 1 -- 172.21.15.57:0/3052584496 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0900053e0 con 0x7fb0981045e0 2023-10-08T13:51:31.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.421+0000 7fb09cb5f700 1 -- 172.21.15.57:0/3052584496 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0981a37f0 con 0x7fb0981045e0 2023-10-08T13:51:31.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.422+0000 7fb09cb5f700 1 -- 172.21.15.57:0/3052584496 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb098100cd0 con 0x7fb0981045e0 2023-10-08T13:51:31.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.423+0000 7fb0877fe700 1 -- 172.21.15.57:0/3052584496 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb09000c980 con 0x7fb0981045e0 2023-10-08T13:51:31.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.423+0000 7fb0877fe700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0800a14d0 0x7fb0800a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:31.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.424+0000 7fb0877fe700 1 -- 172.21.15.57:0/3052584496 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb0900cc810 con 0x7fb0981045e0 2023-10-08T13:51:31.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.424+0000 7fb09659c700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0800a14d0 0x7fb0800a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:31.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.424+0000 7fb09659c700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0800a14d0 0x7fb0800a3990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb08c005950 tx=0x7fb08c00a300 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:31.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.428+0000 7fb0877fe700 1 -- 172.21.15.57:0/3052584496 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb090095a80 con 0x7fb0981045e0 2023-10-08T13:51:31.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.655+0000 7fb09cb5f700 1 -- 172.21.15.57:0/3052584496 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fb09810f530 con 0x7fb0800a14d0 2023-10-08T13:51:31.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.664+0000 7fb0877fe700 1 -- 172.21.15.57:0/3052584496 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7fb09810f530 con 0x7fb0800a14d0 2023-10-08T13:51:31.665 INFO:teuthology.orchestra.run.smithi057.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T13:51:30", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-10-08T13:51:31.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 -- 172.21.15.57:0/3052584496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0800a14d0 msgr2=0x7fb0800a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:31.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0800a14d0 0x7fb0800a3990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb08c005950 tx=0x7fb08c00a300 comp rx=0 tx=0).stop 2023-10-08T13:51:31.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 -- 172.21.15.57:0/3052584496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0981045e0 msgr2=0x7fb098196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:31.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0981045e0 0x7fb098196980 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb09000ada0 tx=0x7fb09000c330 comp rx=0 tx=0).stop 2023-10-08T13:51:31.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 -- 172.21.15.57:0/3052584496 shutdown_connections 2023-10-08T13:51:31.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0981045e0 0x7fb098196980 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0800a14d0 0x7fb0800a3990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0980ff5d0 0x7fb09819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 --2- 172.21.15.57:0/3052584496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0980fff90 0x7fb09819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 -- 172.21.15.57:0/3052584496 >> 172.21.15.57:0/3052584496 conn(0x7fb098076790 msgr2=0x7fb0981034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:31.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 -- 172.21.15.57:0/3052584496 shutdown_connections 2023-10-08T13:51:31.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.666+0000 7fb09cb5f700 1 -- 172.21.15.57:0/3052584496 wait complete. 2023-10-08T13:51:31.679 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-08T13:51:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 -- 172.21.15.57:0/2816726258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91ec0fff90 msgr2=0x7f91ec10c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 --2- 172.21.15.57:0/2816726258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91ec0fff90 0x7f91ec10c490 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f91e400b0a0 tx=0x7f91e400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 -- 172.21.15.57:0/2816726258 shutdown_connections 2023-10-08T13:51:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 --2- 172.21.15.57:0/2816726258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91ec0fff90 0x7f91ec10c490 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 --2- 172.21.15.57:0/2816726258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec1045e0 0x7f91ec1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 --2- 172.21.15.57:0/2816726258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91ec0ff5d0 0x7f91ec0ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 -- 172.21.15.57:0/2816726258 >> 172.21.15.57:0/2816726258 conn(0x7f91ec076790 msgr2=0x7f91ec076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 -- 172.21.15.57:0/2816726258 shutdown_connections 2023-10-08T13:51:31.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 -- 172.21.15.57:0/2816726258 wait complete. 2023-10-08T13:51:31.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 Processor -- start 2023-10-08T13:51:31.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.806+0000 7f91f3b02700 1 -- start start 2023-10-08T13:51:31.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f3b02700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91ec0ff5d0 0x7f91ec19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:31.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f3b02700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91ec0fff90 0x7f91ec19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:31.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f3b02700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec1045e0 0x7f91ec196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:31.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f3b02700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec197080 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f3b02700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1971f0 con 0x7f91ec1045e0 2023-10-08T13:51:31.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f3b02700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1973b0 con 0x7f91ec0fff90 2023-10-08T13:51:31.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f189e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91ec0ff5d0 0x7f91ec19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:31.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f189e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91ec0ff5d0 0x7f91ec19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56366/0 (socket says 172.21.15.57:56366) 2023-10-08T13:51:31.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f189e700 1 -- 172.21.15.57:0/3102006470 learned_addr learned my addr 172.21.15.57:0/3102006470 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:31.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f109d700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91ec0fff90 0x7f91ec19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:31.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f209f700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec1045e0 0x7f91ec196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:31.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f189e700 1 -- 172.21.15.57:0/3102006470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91ec0fff90 msgr2=0x7f91ec19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:31.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f189e700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91ec0fff90 0x7f91ec19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f189e700 1 -- 172.21.15.57:0/3102006470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec1045e0 msgr2=0x7f91ec196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:31.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f189e700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec1045e0 0x7f91ec196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:31.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f189e700 1 -- 172.21.15.57:0/3102006470 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e4009d00 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f189e700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91ec0ff5d0 0x7f91ec19c8e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f91e800e970 tx=0x7f91e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:31.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91e2ffd700 1 -- 172.21.15.57:0/3102006470 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8009790 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91e2ffd700 1 -- 172.21.15.57:0/3102006470 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91e800eec0 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.807+0000 7f91f3b02700 1 -- 172.21.15.57:0/3102006470 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ec197690 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.808+0000 7f91e2ffd700 1 -- 172.21.15.57:0/3102006470 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8003cf0 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.808+0000 7f91f3b02700 1 -- 172.21.15.57:0/3102006470 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ec1a38d0 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.808+0000 7f91f3b02700 1 -- 172.21.15.57:0/3102006470 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ec107be0 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.810+0000 7f91e2ffd700 1 -- 172.21.15.57:0/3102006470 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f91e800ccd0 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.810+0000 7f91e2ffd700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91d80a1550 0x7f91d80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:31.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.810+0000 7f91e2ffd700 1 -- 172.21.15.57:0/3102006470 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f91e8014070 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.810+0000 7f91f109d700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91d80a1550 0x7f91d80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:31.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.811+0000 7f91f109d700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91d80a1550 0x7f91d80a3a10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91ec076b90 tx=0x7f91dc00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:31.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:31.814+0000 7f91e2ffd700 1 -- 172.21.15.57:0/3102006470 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f91e8096770 con 0x7f91ec0ff5d0 2023-10-08T13:51:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:31 smithi057 ceph-mon[99476]: pgmap v298: 129 pgs: 129 active+clean; 646 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2023-10-08T13:51:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:31 smithi057 ceph-mon[99476]: scrub queued for path: / 2023-10-08T13:51:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:31 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:51:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:31 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:51:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:31 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:51:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:31 smithi057 ceph-mon[99476]: from='client.15801 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:32.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.041+0000 7f91f3b02700 1 -- 172.21.15.57:0/3102006470 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f91ec197ed0 con 0x7f91d80a1550 2023-10-08T13:51:32.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.049+0000 7f91e2ffd700 1 -- 172.21.15.57:0/3102006470 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f91ec197ed0 con 0x7f91d80a1550 2023-10-08T13:51:32.050 INFO:teuthology.orchestra.run.smithi057.stdout:/ 1m 6m3h 2023-10-08T13:51:32.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 -- 172.21.15.57:0/3102006470 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91d80a1550 msgr2=0x7f91d80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:32.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91d80a1550 0x7f91d80a3a10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91ec076b90 tx=0x7f91dc00b410 comp rx=0 tx=0).stop 2023-10-08T13:51:32.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 -- 172.21.15.57:0/3102006470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91ec0ff5d0 msgr2=0x7f91ec19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91ec0ff5d0 0x7f91ec19c8e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f91e800e970 tx=0x7f91e800ec80 comp rx=0 tx=0).stop 2023-10-08T13:51:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 -- 172.21.15.57:0/3102006470 shutdown_connections 2023-10-08T13:51:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f91d80a1550 0x7f91d80a3a10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91ec1045e0 0x7f91ec196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91ec0ff5d0 0x7f91ec19c8e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 --2- 172.21.15.57:0/3102006470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91ec0fff90 0x7f91ec19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 -- 172.21.15.57:0/3102006470 >> 172.21.15.57:0/3102006470 conn(0x7f91ec076790 msgr2=0x7f91ec0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 -- 172.21.15.57:0/3102006470 shutdown_connections 2023-10-08T13:51:32.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:51:32.051+0000 7f91f3b02700 1 -- 172.21.15.57:0/3102006470 wait complete. 2023-10-08T13:51:32.064 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-10-08T13:51:32.092 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-10-08T13:51:32.102 INFO:teuthology.run_tasks:Running task background_exec... 2023-10-08T13:51:32.226 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T13:51:32.226 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-10-08T13:51:32.227 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:31 smithi115 ceph-mon[104408]: pgmap v298: 129 pgs: 129 active+clean; 646 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2023-10-08T13:51:32.227 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:31 smithi115 ceph-mon[104408]: scrub queued for path: / 2023-10-08T13:51:32.228 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:31 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:51:32.228 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:31 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:51:32.228 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:31 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:51:32.228 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:31 smithi115 ceph-mon[104408]: from='client.15801 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:32.229 INFO:teuthology.run_tasks:Running task workunit... 2023-10-08T13:51:32.263 INFO:tasks.workunit:Pulling workunits from ref 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:51:32.264 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-08T13:51:32.264 INFO:tasks.workunit:timeout=3h 2023-10-08T13:51:32.264 INFO:tasks.workunit:cleanup=True 2023-10-08T13:51:32.264 DEBUG:teuthology.orchestra.run.smithi057:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-08T13:51:32.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:31 smithi099 ceph-mon[103849]: pgmap v298: 129 pgs: 129 active+clean; 646 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2023-10-08T13:51:32.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:31 smithi099 ceph-mon[103849]: scrub queued for path: / 2023-10-08T13:51:32.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:31 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:51:32.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:31 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:51:32.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:31 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:51:32.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:31 smithi099 ceph-mon[103849]: from='client.15801 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:32.282 INFO:teuthology.orchestra.run.smithi057.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-10-08T13:51:32.282 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-08T13:51:32.282 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 7ch/124d Inode: 1099511627783 Links: 2 2023-10-08T13:51:32.282 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T13:51:32.282 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-08T13:51:32.282 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-08 13:50:08.158910870 +0000 2023-10-08T13:51:32.282 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-08 13:50:08.158910870 +0000 2023-10-08T13:51:32.282 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-08 13:51:22.014991271 +0000 2023-10-08T13:51:32.283 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2023-10-08 13:50:08.158910870 +0000 2023-10-08T13:51:32.283 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-10-08T13:51:32.283 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-10-08T13:51:32.381 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T13:51:32.399 INFO:tasks.workunit.client.0.smithi057.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-08T13:51:33.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:32 smithi115 ceph-mon[104408]: from='client.15846 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:33.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:32 smithi115 ceph-mon[104408]: from='client.15852 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:33.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:32 smithi057 ceph-mon[99476]: from='client.15846 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:33.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:32 smithi057 ceph-mon[99476]: from='client.15852 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:33.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:32 smithi099 ceph-mon[103849]: from='client.15846 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:33.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:32 smithi099 ceph-mon[103849]: from='client.15852 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:34.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:33 smithi115 ceph-mon[104408]: from='client.15858 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:34.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:33 smithi115 ceph-mon[104408]: pgmap v299: 129 pgs: 129 active+clean; 646 KiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2023-10-08T13:51:34.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:33 smithi115 ceph-mon[104408]: mgrmap e23: x(active, since 7m), standbys: z, y 2023-10-08T13:51:34.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:33 smithi057 ceph-mon[99476]: from='client.15858 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:34.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:33 smithi057 ceph-mon[99476]: pgmap v299: 129 pgs: 129 active+clean; 646 KiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2023-10-08T13:51:34.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:33 smithi057 ceph-mon[99476]: mgrmap e23: x(active, since 7m), standbys: z, y 2023-10-08T13:51:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:33 smithi099 ceph-mon[103849]: from='client.15858 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T13:51:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:33 smithi099 ceph-mon[103849]: pgmap v299: 129 pgs: 129 active+clean; 646 KiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2023-10-08T13:51:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:33 smithi099 ceph-mon[103849]: mgrmap e23: x(active, since 7m), standbys: z, y 2023-10-08T13:51:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:35 smithi115 ceph-mon[104408]: pgmap v300: 129 pgs: 129 active+clean; 647 KiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2023-10-08T13:51:36.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:35 smithi057 ceph-mon[99476]: pgmap v300: 129 pgs: 129 active+clean; 647 KiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2023-10-08T13:51:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:35 smithi099 ceph-mon[103849]: pgmap v300: 129 pgs: 129 active+clean; 647 KiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2023-10-08T13:51:38.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:37 smithi115 ceph-mon[104408]: pgmap v301: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 44 op/s 2023-10-08T13:51:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:37 smithi057 ceph-mon[99476]: pgmap v301: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 44 op/s 2023-10-08T13:51:38.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:37 smithi099 ceph-mon[103849]: pgmap v301: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 44 op/s 2023-10-08T13:51:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:39 smithi115 ceph-mon[104408]: pgmap v302: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 115 KiB/s wr, 45 op/s 2023-10-08T13:51:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:39 smithi057 ceph-mon[99476]: pgmap v302: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 115 KiB/s wr, 45 op/s 2023-10-08T13:51:40.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:39 smithi099 ceph-mon[103849]: pgmap v302: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 115 KiB/s wr, 45 op/s 2023-10-08T13:51:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:41 smithi115 ceph-mon[104408]: pgmap v303: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 44 op/s 2023-10-08T13:51:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:41 smithi057 ceph-mon[99476]: pgmap v303: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 44 op/s 2023-10-08T13:51:42.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:41 smithi099 ceph-mon[103849]: pgmap v303: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 44 op/s 2023-10-08T13:51:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:43 smithi115 ceph-mon[104408]: pgmap v304: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 43 op/s 2023-10-08T13:51:44.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:43 smithi057 ceph-mon[99476]: pgmap v304: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 43 op/s 2023-10-08T13:51:44.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:43 smithi099 ceph-mon[103849]: pgmap v304: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 43 op/s 2023-10-08T13:51:45.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:44 smithi115 ceph-mon[104408]: scrub summary: idle 2023-10-08T13:51:45.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:44 smithi057 ceph-mon[99476]: scrub summary: idle 2023-10-08T13:51:45.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:44 smithi099 ceph-mon[103849]: scrub summary: idle 2023-10-08T13:51:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.397+0000 7f609a96d700 1 -- 172.21.15.57:0/236873379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60941045e0 msgr2=0x7f60941049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.397+0000 7f609a96d700 1 --2- 172.21.15.57:0/236873379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60941045e0 0x7f60941049c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f607c009990 tx=0x7f607c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.397+0000 7f609a96d700 1 -- 172.21.15.57:0/236873379 shutdown_connections 2023-10-08T13:51:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.397+0000 7f609a96d700 1 --2- 172.21.15.57:0/236873379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60940fff90 0x7f609410c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.397+0000 7f609a96d700 1 --2- 172.21.15.57:0/236873379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60941045e0 0x7f60941049c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.397+0000 7f609a96d700 1 --2- 172.21.15.57:0/236873379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60940ff5d0 0x7f60940ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.397+0000 7f609a96d700 1 -- 172.21.15.57:0/236873379 >> 172.21.15.57:0/236873379 conn(0x7f6094076790 msgr2=0x7f6094076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.397+0000 7f609a96d700 1 -- 172.21.15.57:0/236873379 shutdown_connections 2023-10-08T13:51:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.398+0000 7f609a96d700 1 -- 172.21.15.57:0/236873379 wait complete. 2023-10-08T13:51:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.398+0000 7f609a96d700 1 Processor -- start 2023-10-08T13:51:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.398+0000 7f609a96d700 1 -- start start 2023-10-08T13:51:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f609a96d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60940ff5d0 0x7f60941979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f609a96d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60940fff90 0x7f6094197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f609a96d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60941045e0 0x7f60941a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f609a96d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60941987c0 con 0x7f60940ff5d0 2023-10-08T13:51:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f609a96d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60941a2fa0 con 0x7f60940fff90 2023-10-08T13:51:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f609a96d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60941a30e0 con 0x7f60941045e0 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6093fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60940ff5d0 0x7f60941979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6093fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60940ff5d0 0x7f60941979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56382/0 (socket says 172.21.15.57:56382) 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6093fff700 1 -- 172.21.15.57:0/2893756923 learned_addr learned my addr 172.21.15.57:0/2893756923 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f60937fe700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60940fff90 0x7f6094197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6098f0a700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60941045e0 0x7f60941a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6093fff700 1 -- 172.21.15.57:0/2893756923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60941045e0 msgr2=0x7f60941a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6093fff700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60941045e0 0x7f60941a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6093fff700 1 -- 172.21.15.57:0/2893756923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60940fff90 msgr2=0x7f6094197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6093fff700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60940fff90 0x7f6094197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6093fff700 1 -- 172.21.15.57:0/2893756923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c0095f0 con 0x7f60940ff5d0 2023-10-08T13:51:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f60937fe700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60940fff90 0x7f6094197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6098f0a700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60941045e0 0x7f60941a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f6093fff700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60940ff5d0 0x7f60941979e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f607c004730 tx=0x7f607c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.399+0000 7f60917fa700 1 -- 172.21.15.57:0/2893756923 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c01d070 con 0x7f60940ff5d0 2023-10-08T13:51:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.400+0000 7f609a96d700 1 -- 172.21.15.57:0/2893756923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60941a3330 con 0x7f60940ff5d0 2023-10-08T13:51:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.400+0000 7f60917fa700 1 -- 172.21.15.57:0/2893756923 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f607c004810 con 0x7f60940ff5d0 2023-10-08T13:51:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.400+0000 7f60917fa700 1 -- 172.21.15.57:0/2893756923 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c005430 con 0x7f60940ff5d0 2023-10-08T13:51:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.400+0000 7f609a96d700 1 -- 172.21.15.57:0/2893756923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60941a36f0 con 0x7f60940ff5d0 2023-10-08T13:51:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.402+0000 7f60917fa700 1 -- 172.21.15.57:0/2893756923 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 23) v1 ==== 136077+0+0 (secure 0 0 0) 0x7f607c0055d0 con 0x7f60940ff5d0 2023-10-08T13:51:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.403+0000 7f60917fa700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f60800a1560 0x7f60800a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.403+0000 7f60937fe700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f60800a1560 0x7f60800a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.403+0000 7f60917fa700 1 -- 172.21.15.57:0/2893756923 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f607c0ce570 con 0x7f60940ff5d0 2023-10-08T13:51:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.403+0000 7f60937fe700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f60800a1560 0x7f60800a3a20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6084005950 tx=0x7f60840058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f60a6b56b80 1 Processor -- start 2023-10-08T13:51:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f60a6b56b80 1 -- start start 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561223114880 0x561223114c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612231151a0 0x56122310eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122310f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561223077400 con 0x56122310f0b0 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561223068bc0 con 0x5612231151a0 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612230664e0 con 0x561223114880 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f6098f0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122310f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f6098f0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122310f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56394/0 (socket says 172.21.15.57:56394) 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f6098f0a700 1 -- 172.21.15.57:0/21778910 learned_addr learned my addr 172.21.15.57:0/21778910 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f6093fff700 1 --2- 172.21.15.57:0/21778910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561223114880 0x561223114c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f60937fe700 1 --2- 172.21.15.57:0/21778910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612231151a0 0x56122310eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f6098f0a700 1 -- 172.21.15.57:0/21778910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561223114880 msgr2=0x561223114c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f6098f0a700 1 --2- 172.21.15.57:0/21778910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561223114880 0x561223114c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f6098f0a700 1 -- 172.21.15.57:0/21778910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612231151a0 msgr2=0x56122310eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.404+0000 7f6098f0a700 1 --2- 172.21.15.57:0/21778910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612231151a0 0x56122310eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.405+0000 7f6098f0a700 1 -- 172.21.15.57:0/21778910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561222e8cd80 con 0x56122310f0b0 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.405+0000 7f6098f0a700 1 --2- 172.21.15.57:0/21778910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122310f510 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f608c009400 tx=0x7f608c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38277f9671b5a1e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.405+0000 7f608a7fc700 1 -- 172.21.15.57:0/21778910 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c0153f0 con 0x56122310f0b0 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.405+0000 7f608a7fc700 1 -- 172.21.15.57:0/21778910 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f608c011040 con 0x56122310f0b0 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.405+0000 7f608a7fc700 1 -- 172.21.15.57:0/21778910 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c0148f0 con 0x56122310f0b0 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 -- 172.21.15.57:0/21778910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 msgr2=0x56122310f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/21778910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122310f510 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f608c009400 tx=0x7f608c009920 comp rx=0 tx=0).stop 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 -- 172.21.15.57:0/21778910 shutdown_connections 2023-10-08T13:51:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/21778910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122310f510 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/21778910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561223114880 0x561223114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/21778910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612231151a0 0x56122310eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 -- 172.21.15.57:0/21778910 >> 172.21.15.57:0/21778910 conn(0x561223083200 msgr2=0x561223109b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 -- 172.21.15.57:0/21778910 shutdown_connections 2023-10-08T13:51:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 -- 172.21.15.57:0/21778910 wait complete. 2023-10-08T13:51:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 -- start start 2023-10-08T13:51:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.406+0000 7f60a6b56b80 1 Processor -- start 2023-10-08T13:51:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561223114880 0x5612231184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612231151a0 0x561223118b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561223061dd0 con 0x56122310f0b0 2023-10-08T13:51:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561222ffe6b0 con 0x561223114880 2023-10-08T13:51:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561222ffe350 con 0x5612231151a0 2023-10-08T13:51:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60a6b56b80 10 client.0 _ll_register_callbacks cb 0x561222e5b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f6093fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f6093fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56396/0 (socket says 172.21.15.57:56396) 2023-10-08T13:51:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f6093fff700 1 -- 172.21.15.57:0/3236598732 learned_addr learned my addr 172.21.15.57:0/3236598732 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60937fe700 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561223114880 0x5612231184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f6098f0a700 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612231151a0 0x561223118b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f6093fff700 1 -- 172.21.15.57:0/3236598732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612231151a0 msgr2=0x561223118b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f6093fff700 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612231151a0 0x561223118b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f6093fff700 1 -- 172.21.15.57:0/3236598732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561223114880 msgr2=0x5612231184d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f6093fff700 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561223114880 0x5612231184d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f6093fff700 1 -- 172.21.15.57:0/3236598732 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c0090b0 con 0x56122310f0b0 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f6093fff700 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311b1e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f607c097840 tx=0x7f607c004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0b789c440b5f5f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60737fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60737fe700 1 -- 172.21.15.57:0/3236598732 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c09b340 con 0x56122310f0b0 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60a6b56b80 1 -- 172.21.15.57:0/3236598732 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561223029e50 con 0x56122310f0b0 2023-10-08T13:51:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.407+0000 7f60737fe700 1 -- 172.21.15.57:0/3236598732 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f607c09f490 con 0x56122310f0b0 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60737fe700 1 -- 172.21.15.57:0/3236598732 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f607c0a3640 con 0x56122310f0b0 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60a6b56b80 10 client.15870 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60a6b56b80 1 -- 172.21.15.57:0/3236598732 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561222e75c00 con 0x56122310f0b0 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60737fe700 1 -- 172.21.15.57:0/3236598732 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f607c066080 con 0x56122310f0b0 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60737fe700 1 -- 172.21.15.57:0/3236598732 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f607c09a830 con 0x56122310f0b0 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60a6b56b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60a6b56b80 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60a6b56b80 20 client.15870 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561223128970 0x56122312ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60a6b56b80 4 client.15870 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60a6b56b80 1 -- 172.21.15.57:0/3236598732 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56122305adb0 con 0x561223128970 2023-10-08T13:51:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.408+0000 7f60937fe700 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561223128970 0x56122312ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.409+0000 7f60737fe700 10 client.15870 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.409+0000 7f60937fe700 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561223128970 0x56122312ad60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552b98fe6de5d307 server_cookie=a24c626e00757177 in_seq=0 out_seq=0 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.409+0000 7f60737fe700 1 -- 172.21.15.57:0/3236598732 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56122305adb0 con 0x561223128970 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.409+0000 7f60737fe700 10 client.15870 handle_command_reply: tid=0 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 client.15870 shutdown 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 -- 172.21.15.57:0/3236598732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 msgr2=0x56122311b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311b1e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f607c097840 tx=0x7f607c004f40 comp rx=0 tx=0).stop 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 -- 172.21.15.57:0/3236598732 shutdown_connections 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561223128970 0x56122312ad60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612231151a0 0x561223118b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60737fe700 0 client.15870 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311b1e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/3236598732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561223114880 0x5612231184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 -- 172.21.15.57:0/3236598732 >> 172.21.15.57:0/3236598732 conn(0x561223083200 msgr2=0x561223109010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 -- 172.21.15.57:0/3236598732 shutdown_connections 2023-10-08T13:51:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 1 -- 172.21.15.57:0/3236598732 wait complete. 2023-10-08T13:51:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.410+0000 7f60a6b56b80 20 client.15870 trim_cache size 0 max 16384 2023-10-08T13:51:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f60a6b56b80 1 Processor -- start 2023-10-08T13:51:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f60a6b56b80 1 -- start start 2023-10-08T13:51:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56122310f0b0 0x56122310eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561223114880 0x561223109790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612231151a0 0x561223109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561222ee28c0 con 0x5612231151a0 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612231042b0 con 0x56122310f0b0 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561223115000 con 0x561223114880 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f6098f0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612231151a0 0x561223109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f6098f0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612231151a0 0x561223109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56408/0 (socket says 172.21.15.57:56408) 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f6098f0a700 1 -- 172.21.15.57:0/949359643 learned_addr learned my addr 172.21.15.57:0/949359643 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.425+0000 7f6093fff700 1 --2- 172.21.15.57:0/949359643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56122310f0b0 0x56122310eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.426+0000 7f60937fe700 1 --2- 172.21.15.57:0/949359643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561223114880 0x561223109790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.426+0000 7f6098f0a700 1 -- 172.21.15.57:0/949359643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561223114880 msgr2=0x561223109790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.426+0000 7f6098f0a700 1 --2- 172.21.15.57:0/949359643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561223114880 0x561223109790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.426+0000 7f6098f0a700 1 -- 172.21.15.57:0/949359643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56122310f0b0 msgr2=0x56122310eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.426+0000 7f6098f0a700 1 --2- 172.21.15.57:0/949359643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56122310f0b0 0x56122310eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.426+0000 7f6098f0a700 1 -- 172.21.15.57:0/949359643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561222eb31f0 con 0x5612231151a0 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.426+0000 7f6098f0a700 1 --2- 172.21.15.57:0/949359643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612231151a0 0x561223109cd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f608c009400 tx=0x7f608c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8045785520aa96b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.426+0000 7f608affd700 1 -- 172.21.15.57:0/949359643 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c004cf0 con 0x5612231151a0 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.426+0000 7f608affd700 1 -- 172.21.15.57:0/949359643 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f608c004e50 con 0x5612231151a0 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.426+0000 7f608affd700 1 -- 172.21.15.57:0/949359643 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c01f770 con 0x5612231151a0 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 -- 172.21.15.57:0/949359643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612231151a0 msgr2=0x561223109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/949359643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612231151a0 0x561223109cd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f608c009400 tx=0x7f608c009620 comp rx=0 tx=0).stop 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 -- 172.21.15.57:0/949359643 shutdown_connections 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/949359643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612231151a0 0x561223109cd0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/949359643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56122310f0b0 0x56122310eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/949359643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561223114880 0x561223109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 -- 172.21.15.57:0/949359643 >> 172.21.15.57:0/949359643 conn(0x561223083200 msgr2=0x56122312f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 -- 172.21.15.57:0/949359643 shutdown_connections 2023-10-08T13:51:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 -- 172.21.15.57:0/949359643 wait complete. 2023-10-08T13:51:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 -- start start 2023-10-08T13:51:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.427+0000 7f60a6b56b80 1 Processor -- start 2023-10-08T13:51:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561223114880 0x56122311fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f60a6b56b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612231151a0 0x56122311dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122311e350 con 0x56122310f0b0 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122311e4c0 con 0x561223114880 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f60a6b56b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122311e680 con 0x5612231151a0 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f60a6b56b80 10 client.0 _ll_register_callbacks cb 0x7f607c09ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f6093fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f6093fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56410/0 (socket says 172.21.15.57:56410) 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f6093fff700 1 -- 172.21.15.57:0/370164767 learned_addr learned my addr 172.21.15.57:0/370164767 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f60937fe700 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561223114880 0x56122311fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f6098f0a700 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612231151a0 0x56122311dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f6093fff700 1 -- 172.21.15.57:0/370164767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612231151a0 msgr2=0x56122311dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f6093fff700 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612231151a0 0x56122311dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f6093fff700 1 -- 172.21.15.57:0/370164767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561223114880 msgr2=0x56122311fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f6093fff700 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561223114880 0x56122311fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f6093fff700 1 -- 172.21.15.57:0/370164767 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561222e8cd80 con 0x56122310f0b0 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f6093fff700 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311f640 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f607c097010 tx=0x7f607c0a3540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb404aec86f49c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f60737fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.428+0000 7f60737fe700 1 -- 172.21.15.57:0/370164767 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c0a3640 con 0x56122310f0b0 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60737fe700 1 -- 172.21.15.57:0/370164767 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f607c09f490 con 0x56122310f0b0 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60a6b56b80 1 -- 172.21.15.57:0/370164767 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f608c011040 con 0x56122310f0b0 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60737fe700 1 -- 172.21.15.57:0/370164767 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f607c09b340 con 0x56122310f0b0 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60a6b56b80 10 client.15876 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60a6b56b80 1 -- 172.21.15.57:0/370164767 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561222e8b250 con 0x56122310f0b0 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60737fe700 1 -- 172.21.15.57:0/370164767 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f607c081080 con 0x56122310f0b0 2023-10-08T13:51:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60737fe700 1 -- 172.21.15.57:0/370164767 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f607c0a1240 con 0x56122310f0b0 2023-10-08T13:51:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60a6b56b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60a6b56b80 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:51:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60a6b56b80 20 client.15876 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561223128970 0x5612231839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60a6b56b80 4 client.15876 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:51:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60a6b56b80 1 -- 172.21.15.57:0/370164767 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56122305aad0 con 0x561223128970 2023-10-08T13:51:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.429+0000 7f60937fe700 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561223128970 0x5612231839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.430+0000 7f60737fe700 10 client.15876 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.430+0000 7f60937fe700 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561223128970 0x5612231839d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bd4bda4088a6daa server_cookie=7f0438d3604e722b in_seq=0 out_seq=0 2023-10-08T13:51:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.443+0000 7f60737fe700 1 -- 172.21.15.57:0/370164767 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56122305aad0 con 0x561223128970 2023-10-08T13:51:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60737fe700 10 client.15876 handle_command_reply: tid=0 2023-10-08T13:51:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 client.15876 shutdown 2023-10-08T13:51:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 -- 172.21.15.57:0/370164767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 msgr2=0x56122311f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311f640 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f607c097010 tx=0x7f607c0a3540 comp rx=0 tx=0).stop 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 -- 172.21.15.57:0/370164767 shutdown_connections 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561223128970 0x5612231839d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5612231151a0 0x56122311dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60737fe700 0 client.15876 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56122310f0b0 0x56122311f640 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 --2- 172.21.15.57:0/370164767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561223114880 0x56122311fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 -- 172.21.15.57:0/370164767 >> 172.21.15.57:0/370164767 conn(0x561223083200 msgr2=0x5612231126a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 -- 172.21.15.57:0/370164767 shutdown_connections 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 1 -- 172.21.15.57:0/370164767 wait complete. 2023-10-08T13:51:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f60a6b56b80 20 client.15876 trim_cache size 0 max 16384 2023-10-08T13:51:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:51:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:51:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:51:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:51:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f609a96d700 1 -- 172.21.15.57:0/2893756923 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f60800a1560 msgr2=0x7f60800a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.444+0000 7f609a96d700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f60800a1560 0x7f60800a3a20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6084005950 tx=0x7f60840058e0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.445+0000 7f609a96d700 1 -- 172.21.15.57:0/2893756923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60940ff5d0 msgr2=0x7f60941979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.445+0000 7f609a96d700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60940ff5d0 0x7f60941979e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f607c004730 tx=0x7f607c009580 comp rx=0 tx=0).stop 2023-10-08T13:51:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.445+0000 7f609a96d700 1 -- 172.21.15.57:0/2893756923 shutdown_connections 2023-10-08T13:51:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.445+0000 7f609a96d700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60941045e0 0x7f60941a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.445+0000 7f609a96d700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60940ff5d0 0x7f60941979e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.445+0000 7f609a96d700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f60800a1560 0x7f60800a3a20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.445+0000 7f609a96d700 1 --2- 172.21.15.57:0/2893756923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60940fff90 0x7f6094197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.445+0000 7f609a96d700 1 -- 172.21.15.57:0/2893756923 >> 172.21.15.57:0/2893756923 conn(0x7f6094076790 msgr2=0x7f60940fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.445+0000 7f609a96d700 1 -- 172.21.15.57:0/2893756923 shutdown_connections 2023-10-08T13:51:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:45.445+0000 7f609a96d700 1 -- 172.21.15.57:0/2893756923 wait complete. 2023-10-08T13:51:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:45 smithi115 ceph-mon[104408]: pgmap v305: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 45 op/s 2023-10-08T13:51:46.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:45 smithi057 ceph-mon[99476]: pgmap v305: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 45 op/s 2023-10-08T13:51:46.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:45 smithi099 ceph-mon[103849]: pgmap v305: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 45 op/s 2023-10-08T13:51:48.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:47 smithi115 ceph-mon[104408]: pgmap v306: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 44 op/s 2023-10-08T13:51:48.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:47 smithi057 ceph-mon[99476]: pgmap v306: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 44 op/s 2023-10-08T13:51:48.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:47 smithi099 ceph-mon[103849]: pgmap v306: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 115 KiB/s wr, 44 op/s 2023-10-08T13:51:50.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:49 smithi115 ceph-mon[104408]: pgmap v307: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-08T13:51:50.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:49 smithi057 ceph-mon[99476]: pgmap v307: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-08T13:51:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:49 smithi099 ceph-mon[103849]: pgmap v307: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-08T13:51:51.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:50 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:51:51.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:50 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:51:51.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:50 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:51:52.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:51 smithi057 ceph-mon[99476]: pgmap v308: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-10-08T13:51:52.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:51 smithi099 ceph-mon[103849]: pgmap v308: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-10-08T13:51:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:51 smithi115 ceph-mon[104408]: pgmap v308: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-10-08T13:51:54.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:53 smithi057 ceph-mon[99476]: pgmap v309: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-10-08T13:51:54.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:53 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:53 smithi099 ceph-mon[103849]: pgmap v309: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-10-08T13:51:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:53 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:53 smithi115 ceph-mon[104408]: pgmap v309: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-10-08T13:51:54.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:53 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:51:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:51:55.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:55.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:51:55.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:51:55.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:55.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:51:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:51:55.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:51:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:56 smithi099 ceph-mon[103849]: pgmap v310: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2023-10-08T13:51:56.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:56 smithi115 ceph-mon[104408]: pgmap v310: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2023-10-08T13:51:56.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:56 smithi057 ceph-mon[99476]: pgmap v310: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2023-10-08T13:51:58.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:51:58 smithi115 ceph-mon[104408]: pgmap v311: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-10-08T13:51:58.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:51:58 smithi057 ceph-mon[99476]: pgmap v311: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-10-08T13:51:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:51:58 smithi099 ceph-mon[103849]: pgmap v311: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-10-08T13:51:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.556+0000 7f9e1450b700 1 -- 172.21.15.57:0/2599770543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e0c105ae0 msgr2=0x7f9e0c10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.556+0000 7f9e1450b700 1 --2- 172.21.15.57:0/2599770543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e0c105ae0 0x7f9e0c10d4f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400b0d0 tx=0x7f9e0400b3e0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.557+0000 7f9e1450b700 1 -- 172.21.15.57:0/2599770543 shutdown_connections 2023-10-08T13:51:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.557+0000 7f9e1450b700 1 --2- 172.21.15.57:0/2599770543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e0c105ae0 0x7f9e0c10d4f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.557+0000 7f9e1450b700 1 --2- 172.21.15.57:0/2599770543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e0c067fd0 0x7f9e0c0683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.557+0000 7f9e1450b700 1 --2- 172.21.15.57:0/2599770543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e0c105120 0x7f9e0c1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.557+0000 7f9e1450b700 1 -- 172.21.15.57:0/2599770543 >> 172.21.15.57:0/2599770543 conn(0x7f9e0c076790 msgr2=0x7f9e0c076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.557+0000 7f9e1450b700 1 -- 172.21.15.57:0/2599770543 shutdown_connections 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.557+0000 7f9e1450b700 1 -- 172.21.15.57:0/2599770543 wait complete. 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.557+0000 7f9e1450b700 1 Processor -- start 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.557+0000 7f9e1450b700 1 -- start start 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e1450b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e0c067fd0 0x7f9e0c0ff660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e1450b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e0c105120 0x7f9e0c0ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e1450b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e0c105ae0 0x7f9e0c1000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e1450b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c1a4ff0 con 0x7f9e0c105120 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e1450b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c1a5160 con 0x7f9e0c067fd0 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e1450b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c1a5320 con 0x7f9e0c105ae0 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e11aa6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e0c105120 0x7f9e0c0ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e11aa6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e0c105120 0x7f9e0c0ffba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56426/0 (socket says 172.21.15.57:56426) 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e11aa6700 1 -- 172.21.15.57:0/3194240699 learned_addr learned my addr 172.21.15.57:0/3194240699 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e122a7700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e0c067fd0 0x7f9e0c0ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e12aa8700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e0c105ae0 0x7f9e0c1000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e11aa6700 1 -- 172.21.15.57:0/3194240699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e0c105ae0 msgr2=0x7f9e0c1000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e0c105ae0 0x7f9e0c1000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e11aa6700 1 -- 172.21.15.57:0/3194240699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e0c067fd0 msgr2=0x7f9e0c0ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e0c067fd0 0x7f9e0c0ff660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e11aa6700 1 -- 172.21.15.57:0/3194240699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e04009d00 con 0x7f9e0c105120 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e0c105120 0x7f9e0c0ffba0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00e9a0 tx=0x7f9dfc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e037fe700 1 -- 172.21.15.57:0/3194240699 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc009790 con 0x7f9e0c105120 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.558+0000 7f9e1450b700 1 -- 172.21.15.57:0/3194240699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0c1a5600 con 0x7f9e0c105120 2023-10-08T13:51:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.559+0000 7f9e037fe700 1 -- 172.21.15.57:0/3194240699 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dfc010430 con 0x7f9e0c105120 2023-10-08T13:51:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.559+0000 7f9e037fe700 1 -- 172.21.15.57:0/3194240699 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc018680 con 0x7f9e0c105120 2023-10-08T13:51:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.559+0000 7f9e1450b700 1 -- 172.21.15.57:0/3194240699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c1a5b60 con 0x7f9e0c105120 2023-10-08T13:51:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.562+0000 7f9e037fe700 1 -- 172.21.15.57:0/3194240699 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 23) v1 ==== 136077+0+0 (secure 0 0 0) 0x7f9dfc0105a0 con 0x7f9e0c105120 2023-10-08T13:51:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.562+0000 7f9e037fe700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9df80a1560 0x7f9df80a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.562+0000 7f9e037fe700 1 -- 172.21.15.57:0/3194240699 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9dfc014070 con 0x7f9e0c105120 2023-10-08T13:51:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.562+0000 7f9e122a7700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9df80a1560 0x7f9df80a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e206f4b80 1 Processor -- start 2023-10-08T13:51:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e122a7700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9df80a1560 0x7f9df80a3a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009c60 tx=0x7f9e08009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e206f4b80 1 -- start start 2023-10-08T13:51:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c1826880 0x5617c1826c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 0x5617c1820b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c1821510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c1789400 con 0x5617c18210b0 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c177abc0 con 0x5617c1826880 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c17784e0 con 0x5617c18271a0 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e12aa8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c1821510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e12aa8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c1821510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56436/0 (socket says 172.21.15.57:56436) 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.563+0000 7f9e12aa8700 1 -- 172.21.15.57:0/2115637807 learned_addr learned my addr 172.21.15.57:0/2115637807 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e122a7700 1 --2- 172.21.15.57:0/2115637807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c1826880 0x5617c1826c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/2115637807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 0x5617c1820b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e12aa8700 1 -- 172.21.15.57:0/2115637807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 msgr2=0x5617c1820b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e12aa8700 1 --2- 172.21.15.57:0/2115637807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 0x5617c1820b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e12aa8700 1 -- 172.21.15.57:0/2115637807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c1826880 msgr2=0x5617c1826c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e12aa8700 1 --2- 172.21.15.57:0/2115637807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c1826880 0x5617c1826c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e12aa8700 1 -- 172.21.15.57:0/2115637807 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617c159ed80 con 0x5617c18210b0 2023-10-08T13:51:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/2115637807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 0x5617c1820b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:51:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e12aa8700 1 --2- 172.21.15.57:0/2115637807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c1821510 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009cb0 tx=0x7f9e04005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9f1bece54c2069 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e00ff9700 1 -- 172.21.15.57:0/2115637807 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0400e040 con 0x5617c18210b0 2023-10-08T13:51:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e00ff9700 1 -- 172.21.15.57:0/2115637807 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e040060c0 con 0x5617c18210b0 2023-10-08T13:51:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.564+0000 7f9e00ff9700 1 -- 172.21.15.57:0/2115637807 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04003a00 con 0x5617c18210b0 2023-10-08T13:51:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 -- 172.21.15.57:0/2115637807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 msgr2=0x5617c1821510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/2115637807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c1821510 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009cb0 tx=0x7f9e04005a20 comp rx=0 tx=0).stop 2023-10-08T13:51:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 -- 172.21.15.57:0/2115637807 shutdown_connections 2023-10-08T13:51:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/2115637807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c1821510 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/2115637807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c1826880 0x5617c1826c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/2115637807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 0x5617c1820b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 -- 172.21.15.57:0/2115637807 >> 172.21.15.57:0/2115637807 conn(0x5617c1795200 msgr2=0x5617c181bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 -- 172.21.15.57:0/2115637807 shutdown_connections 2023-10-08T13:51:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 -- 172.21.15.57:0/2115637807 wait complete. 2023-10-08T13:51:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 -- start start 2023-10-08T13:51:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.565+0000 7f9e206f4b80 1 Processor -- start 2023-10-08T13:51:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18210b0 0x5617c182d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c182a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 0x5617c182abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c1773dd0 con 0x5617c1826880 2023-10-08T13:51:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c17106b0 con 0x5617c18271a0 2023-10-08T13:51:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c1710350 con 0x5617c18210b0 2023-10-08T13:51:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e206f4b80 10 client.0 _ll_register_callbacks cb 0x5617c156d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e11aa6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c182a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e11aa6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c182a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56440/0 (socket says 172.21.15.57:56440) 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e11aa6700 1 -- 172.21.15.57:0/1013931594 learned_addr learned my addr 172.21.15.57:0/1013931594 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e12aa8700 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 0x5617c182abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e122a7700 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18210b0 0x5617c182d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e11aa6700 1 -- 172.21.15.57:0/1013931594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18210b0 msgr2=0x5617c182d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18210b0 0x5617c182d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e11aa6700 1 -- 172.21.15.57:0/1013931594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 msgr2=0x5617c182abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 0x5617c182abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e11aa6700 1 -- 172.21.15.57:0/1013931594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c10de20 con 0x5617c1826880 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c182a490 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc096700 tx=0x7f9dfc09a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304f568990381519 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.566+0000 7f9df5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9df5ffb700 1 -- 172.21.15.57:0/1013931594 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc09e120 con 0x5617c1826880 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9df5ffb700 1 -- 172.21.15.57:0/1013931594 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9dfc0a2590 con 0x5617c1826880 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9e206f4b80 1 -- 172.21.15.57:0/1013931594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617c173be50 con 0x5617c1826880 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9df5ffb700 1 -- 172.21.15.57:0/1013931594 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dfc010880 con 0x5617c1826880 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9e206f4b80 10 client.15888 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9e206f4b80 1 -- 172.21.15.57:0/1013931594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5617c1587c00 con 0x5617c1826880 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9df5ffb700 1 -- 172.21.15.57:0/1013931594 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617c173be50 con 0x5617c1826880 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9df5ffb700 1 -- 172.21.15.57:0/1013931594 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9dfc099280 con 0x5617c1826880 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9e206f4b80 10 client.15888 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9e206f4b80 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9e206f4b80 20 client.15888 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5617c183aba0 0x5617c183cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9e206f4b80 4 client.15888 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.567+0000 7f9e206f4b80 1 -- 172.21.15.57:0/1013931594 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617c176cdb0 con 0x5617c183aba0 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.568+0000 7f9e122a7700 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5617c183aba0 0x5617c183cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.568+0000 7f9df5ffb700 10 client.15888 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.568+0000 7f9e122a7700 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5617c183aba0 0x5617c183cfc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0d531e2d6676ff server_cookie=c1676a8ed36f0f90 in_seq=0 out_seq=0 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9df5ffb700 1 -- 172.21.15.57:0/1013931594 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5617c176cdb0 con 0x5617c183aba0 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9df5ffb700 10 client.15888 handle_command_reply: tid=0 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 client.15888 shutdown 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 -- 172.21.15.57:0/1013931594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 msgr2=0x5617c182a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c182a490 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc096700 tx=0x7f9dfc09a740 comp rx=0 tx=0).stop 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 -- 172.21.15.57:0/1013931594 shutdown_connections 2023-10-08T13:51:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5617c183aba0 0x5617c183cfc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 0x5617c182abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9df5ffb700 0 client.15888 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18210b0 0x5617c182d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/1013931594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c182a490 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 -- 172.21.15.57:0/1013931594 >> 172.21.15.57:0/1013931594 conn(0x5617c1795200 msgr2=0x5617c181b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 -- 172.21.15.57:0/1013931594 shutdown_connections 2023-10-08T13:51:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 1 -- 172.21.15.57:0/1013931594 wait complete. 2023-10-08T13:51:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.569+0000 7f9e206f4b80 20 client.15888 trim_cache size 0 max 16384 2023-10-08T13:51:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.584+0000 7f9e206f4b80 1 Processor -- start 2023-10-08T13:51:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.584+0000 7f9e206f4b80 1 -- start start 2023-10-08T13:51:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.584+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18210b0 0x5617c1820b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.584+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c181b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.584+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 0x5617c181bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.584+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c15f48c0 con 0x5617c1826880 2023-10-08T13:51:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.584+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c181c7c0 con 0x5617c18210b0 2023-10-08T13:51:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.584+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c183a990 con 0x5617c18271a0 2023-10-08T13:51:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e11aa6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c181b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e11aa6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c181b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56448/0 (socket says 172.21.15.57:56448) 2023-10-08T13:51:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e11aa6700 1 -- 172.21.15.57:0/3223969220 learned_addr learned my addr 172.21.15.57:0/3223969220 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e12aa8700 1 --2- 172.21.15.57:0/3223969220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 0x5617c181bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e122a7700 1 --2- 172.21.15.57:0/3223969220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18210b0 0x5617c1820b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e11aa6700 1 -- 172.21.15.57:0/3223969220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 msgr2=0x5617c181bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/3223969220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 0x5617c181bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e11aa6700 1 -- 172.21.15.57:0/3223969220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18210b0 msgr2=0x5617c1820b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/3223969220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18210b0 0x5617c1820b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e11aa6700 1 -- 172.21.15.57:0/3223969220 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617c15c51f0 con 0x5617c1826880 2023-10-08T13:51:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/3223969220 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c181b790 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc096c70 tx=0x7f9dfc09e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498db638c739ba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e017fa700 1 -- 172.21.15.57:0/3223969220 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0b84f0 con 0x5617c1826880 2023-10-08T13:51:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e017fa700 1 -- 172.21.15.57:0/3223969220 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dfc099b60 con 0x5617c1826880 2023-10-08T13:51:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.585+0000 7f9e017fa700 1 -- 172.21.15.57:0/3223969220 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0993c0 con 0x5617c1826880 2023-10-08T13:51:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.586+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3223969220 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 msgr2=0x5617c181b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.586+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/3223969220 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c181b790 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc096c70 tx=0x7f9dfc09e4d0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.586+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3223969220 shutdown_connections 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.586+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/3223969220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c18271a0 0x5617c181bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.586+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/3223969220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18210b0 0x5617c1820b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.586+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/3223969220 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c1826880 0x5617c181b790 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.586+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3223969220 >> 172.21.15.57:0/3223969220 conn(0x5617c1795200 msgr2=0x5617c1824600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.586+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3223969220 shutdown_connections 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.586+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3223969220 wait complete. 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.586+0000 7f9e206f4b80 1 -- start start 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e206f4b80 1 Processor -- start 2023-10-08T13:51:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c18236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c1826880 0x5617c1823be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e206f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 0x5617c1822400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c1822940 con 0x5617c18210b0 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c1822ab0 con 0x5617c18271a0 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e206f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c1822c70 con 0x5617c1826880 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e206f4b80 10 client.0 _ll_register_callbacks cb 0x7f9dfc09a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e122a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c18236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e122a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c18236a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56452/0 (socket says 172.21.15.57:56452) 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e122a7700 1 -- 172.21.15.57:0/3109683405 learned_addr learned my addr 172.21.15.57:0/3109683405 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c1826880 0x5617c1823be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e122a7700 1 -- 172.21.15.57:0/3109683405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c1826880 msgr2=0x5617c1823be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e122a7700 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c1826880 0x5617c1823be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e12aa8700 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 0x5617c1822400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e122a7700 1 -- 172.21.15.57:0/3109683405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 msgr2=0x5617c1822400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e122a7700 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 0x5617c1822400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e122a7700 1 -- 172.21.15.57:0/3109683405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617c1587c00 con 0x5617c18210b0 2023-10-08T13:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.587+0000 7f9e12aa8700 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 0x5617c1822400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9e122a7700 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c18236a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9e08005c50 tx=0x7f9e080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d824309f7f5b1acd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9df5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9df5ffb700 1 -- 172.21.15.57:0/3109683405 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08023040 con 0x5617c18210b0 2023-10-08T13:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9df5ffb700 1 -- 172.21.15.57:0/3109683405 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9e08003680 con 0x5617c18210b0 2023-10-08T13:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3109683405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e040060c0 con 0x5617c18210b0 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9df5ffb700 1 -- 172.21.15.57:0/3109683405 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e08027730 con 0x5617c18210b0 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9e206f4b80 10 client.15894 fetch_fsmap learned FSMap version 27 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3109683405 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5617c159d250 con 0x5617c18210b0 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9df5ffb700 1 -- 172.21.15.57:0/3109683405 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e040060c0 con 0x5617c18210b0 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9df5ffb700 1 -- 172.21.15.57:0/3109683405 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9e0801d4a0 con 0x5617c18210b0 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9e206f4b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9e206f4b80 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9e206f4b80 20 client.15894 populate_metadata read hostname 'smithi057' 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5617c188fa30 0x5617c1891e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9e206f4b80 4 client.15894 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:51:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.588+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3109683405 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617c176cad0 con 0x5617c188fa30 2023-10-08T13:51:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.589+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5617c188fa30 0x5617c1891e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:51:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.589+0000 7f9df5ffb700 10 client.15894 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.589+0000 7f9e11aa6700 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5617c188fa30 0x5617c1891e50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0ee6508a29f50a2 server_cookie=fdd807f4ec888256 in_seq=0 out_seq=0 2023-10-08T13:51:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9df5ffb700 1 -- 172.21.15.57:0/3109683405 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617c176cad0 con 0x5617c188fa30 2023-10-08T13:51:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9df5ffb700 10 client.15894 handle_command_reply: tid=0 2023-10-08T13:51:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 client.15894 shutdown 2023-10-08T13:51:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3109683405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 msgr2=0x5617c18236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c18236a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9e08005c50 tx=0x7f9e080047c0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3109683405 shutdown_connections 2023-10-08T13:51:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5617c188fa30 0x5617c1891e50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617c18271a0 0x5617c1822400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5617c18210b0 0x5617c18236a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 --2- 172.21.15.57:0/3109683405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617c1826880 0x5617c1823be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3109683405 >> 172.21.15.57:0/3109683405 conn(0x5617c1795200 msgr2=0x5617c18299f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9df5ffb700 0 client.15894 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:51:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3109683405 shutdown_connections 2023-10-08T13:51:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 1 -- 172.21.15.57:0/3109683405 wait complete. 2023-10-08T13:51:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.593+0000 7f9e206f4b80 20 client.15894 trim_cache size 0 max 16384 2023-10-08T13:51:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:51:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:51:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:51:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:51:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 -- 172.21.15.57:0/3194240699 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9df80a1560 msgr2=0x7f9df80a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9df80a1560 0x7f9df80a3a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009c60 tx=0x7f9e08009260 comp rx=0 tx=0).stop 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 -- 172.21.15.57:0/3194240699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e0c105120 msgr2=0x7f9e0c0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e0c105120 0x7f9e0c0ffba0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00e9a0 tx=0x7f9dfc00ecb0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 -- 172.21.15.57:0/3194240699 shutdown_connections 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e0c105ae0 0x7f9e0c1000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9df80a1560 0x7f9df80a3a20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e0c067fd0 0x7f9e0c0ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 --2- 172.21.15.57:0/3194240699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e0c105120 0x7f9e0c0ffba0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 -- 172.21.15.57:0/3194240699 >> 172.21.15.57:0/3194240699 conn(0x7f9e0c076790 msgr2=0x7f9e0c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 -- 172.21.15.57:0/3194240699 shutdown_connections 2023-10-08T13:51:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:51:58.594+0000 7f9e1450b700 1 -- 172.21.15.57:0/3194240699 wait complete. 2023-10-08T13:52:00.293 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:00 smithi057 ceph-mon[99476]: pgmap v312: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2023-10-08T13:52:00.293 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:52:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:52:00.289+0000 7fe485c96700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T13:52:00.293 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:52:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:52:00.289+0000 7fe485c96700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T13:52:00.293 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:52:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:52:00.289+0000 7fe485c96700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T13:52:00.293 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:52:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:52:00.289+0000 7fe485c96700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T13:52:00.294 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 13:52:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T13:52:00.289+0000 7fe485c96700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T13:52:00.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:00 smithi115 ceph-mon[104408]: pgmap v312: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2023-10-08T13:52:00.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:00 smithi099 ceph-mon[103849]: pgmap v312: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2023-10-08T13:52:00.660 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:52:00.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.811+0000 7f876b3f4700 1 -- 172.21.15.57:0/240443123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87640ff640 msgr2=0x7f87640ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.811+0000 7f876b3f4700 1 --2- 172.21.15.57:0/240443123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87640ff640 0x7f87640ffac0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8754009990 tx=0x7f8754009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.811+0000 7f876b3f4700 1 -- 172.21.15.57:0/240443123 shutdown_connections 2023-10-08T13:52:00.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.811+0000 7f876b3f4700 1 --2- 172.21.15.57:0/240443123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8764100000 0x7f876410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.811+0000 7f876b3f4700 1 --2- 172.21.15.57:0/240443123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764105650 0x7f8764105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.811+0000 7f876b3f4700 1 --2- 172.21.15.57:0/240443123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87640ff640 0x7f87640ffac0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.811+0000 7f876b3f4700 1 -- 172.21.15.57:0/240443123 >> 172.21.15.57:0/240443123 conn(0x7f87640767c0 msgr2=0x7f8764076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:00.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.811+0000 7f876b3f4700 1 -- 172.21.15.57:0/240443123 shutdown_connections 2023-10-08T13:52:00.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.811+0000 7f876b3f4700 1 -- 172.21.15.57:0/240443123 wait complete. 2023-10-08T13:52:00.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f876b3f4700 1 Processor -- start 2023-10-08T13:52:00.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f876b3f4700 1 -- start start 2023-10-08T13:52:00.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f876b3f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87640ff640 0x7f876419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f876b3f4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764100000 0x7f876419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f876b3f4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8764105650 0x7f8764195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f876b3f4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8764195f70 con 0x7f8764105650 2023-10-08T13:52:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f876b3f4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87641960e0 con 0x7f87640ff640 2023-10-08T13:52:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f876b3f4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87641962a0 con 0x7f8764100000 2023-10-08T13:52:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f8769991700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8764105650 0x7f8764195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f8769991700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8764105650 0x7f8764195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56486/0 (socket says 172.21.15.57:56486) 2023-10-08T13:52:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f8769991700 1 -- 172.21.15.57:0/1960189496 learned_addr learned my addr 172.21.15.57:0/1960189496 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.812+0000 7f8769190700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87640ff640 0x7f876419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f876898f700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764100000 0x7f876419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f8769991700 1 -- 172.21.15.57:0/1960189496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764100000 msgr2=0x7f876419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f8769991700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764100000 0x7f876419bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f8769991700 1 -- 172.21.15.57:0/1960189496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87640ff640 msgr2=0x7f876419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f8769991700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87640ff640 0x7f876419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f8769991700 1 -- 172.21.15.57:0/1960189496 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540095f0 con 0x7f8764105650 2023-10-08T13:52:00.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f8769991700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8764105650 0x7f8764195910 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f875c007ee0 tx=0x7f875c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:00.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f875a7fc700 1 -- 172.21.15.57:0/1960189496 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c017070 con 0x7f8764105650 2023-10-08T13:52:00.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f875a7fc700 1 -- 172.21.15.57:0/1960189496 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f875c00f040 con 0x7f8764105650 2023-10-08T13:52:00.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f876b3f4700 1 -- 172.21.15.57:0/1960189496 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8764196580 con 0x7f8764105650 2023-10-08T13:52:00.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f875a7fc700 1 -- 172.21.15.57:0/1960189496 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c005580 con 0x7f8764105650 2023-10-08T13:52:00.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.813+0000 7f876b3f4700 1 -- 172.21.15.57:0/1960189496 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87641a27d0 con 0x7f8764105650 2023-10-08T13:52:00.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.816+0000 7f875a7fc700 1 -- 172.21.15.57:0/1960189496 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 23) v1 ==== 136077+0+0 (secure 0 0 0) 0x7f875c00db20 con 0x7f8764105650 2023-10-08T13:52:00.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.816+0000 7f875a7fc700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87500a1560 0x7f87500a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.817+0000 7f8769190700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87500a1560 0x7f87500a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.817+0000 7f875a7fc700 1 -- 172.21.15.57:0/1960189496 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f875c0cca70 con 0x7f8764105650 2023-10-08T13:52:00.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.817+0000 7f87775ddb80 1 Processor -- start 2023-10-08T13:52:00.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f87775ddb80 1 -- start start 2023-10-08T13:52:00.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f8769190700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87500a1560 0x7f87500a3a20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8760005950 tx=0x7f876000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:00.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 0x55da236e1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e17a0 0x55da236ee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236ee860 0x55da236f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f87775ddb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da2364a460 con 0x55da236e17a0 2023-10-08T13:52:00.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f87775ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da2363bc20 con 0x55da236ee860 2023-10-08T13:52:00.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f87775ddb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da23639540 con 0x55da236e0e80 2023-10-08T13:52:00.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f876898f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e17a0 0x55da236ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f876898f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e17a0 0x55da236ee320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56492/0 (socket says 172.21.15.57:56492) 2023-10-08T13:52:00.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f876898f700 1 -- 172.21.15.57:0/2132482994 learned_addr learned my addr 172.21.15.57:0/2132482994 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:00.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f8769991700 1 --2- 172.21.15.57:0/2132482994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236ee860 0x55da236f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f8769190700 1 --2- 172.21.15.57:0/2132482994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 0x55da236e1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f876898f700 1 -- 172.21.15.57:0/2132482994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 msgr2=0x55da236e1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:00.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f876898f700 1 --2- 172.21.15.57:0/2132482994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 0x55da236e1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f876898f700 1 -- 172.21.15.57:0/2132482994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236ee860 msgr2=0x55da236f0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:00.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f876898f700 1 --2- 172.21.15.57:0/2132482994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236ee860 0x55da236f0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.818+0000 7f876898f700 1 -- 172.21.15.57:0/2132482994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da2345fde0 con 0x55da236e17a0 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.819+0000 7f876898f700 1 --2- 172.21.15.57:0/2132482994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e17a0 0x55da236ee320 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8754015000 tx=0x7f8754009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b217aa91975f4e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.819+0000 7f874f7fe700 1 -- 172.21.15.57:0/2132482994 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400b7d0 con 0x55da236e17a0 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.819+0000 7f874f7fe700 1 -- 172.21.15.57:0/2132482994 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f875401e040 con 0x55da236e17a0 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.819+0000 7f874f7fe700 1 -- 172.21.15.57:0/2132482994 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754003b10 con 0x55da236e17a0 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.819+0000 7f87775ddb80 1 -- 172.21.15.57:0/2132482994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e17a0 msgr2=0x55da236ee320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.819+0000 7f87775ddb80 1 --2- 172.21.15.57:0/2132482994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e17a0 0x55da236ee320 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8754015000 tx=0x7f8754009550 comp rx=0 tx=0).stop 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.819+0000 7f87775ddb80 1 -- 172.21.15.57:0/2132482994 shutdown_connections 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.819+0000 7f87775ddb80 1 --2- 172.21.15.57:0/2132482994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236ee860 0x55da236f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.820+0000 7f87775ddb80 1 --2- 172.21.15.57:0/2132482994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 0x55da236e1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.820+0000 7f87775ddb80 1 --2- 172.21.15.57:0/2132482994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e17a0 0x55da236ee320 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.820+0000 7f87775ddb80 1 -- 172.21.15.57:0/2132482994 >> 172.21.15.57:0/2132482994 conn(0x55da23656260 msgr2=0x55da236dd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:00.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.820+0000 7f87775ddb80 1 -- 172.21.15.57:0/2132482994 shutdown_connections 2023-10-08T13:52:00.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.820+0000 7f87775ddb80 1 -- 172.21.15.57:0/2132482994 wait complete. 2023-10-08T13:52:00.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.820+0000 7f87775ddb80 1 -- start start 2023-10-08T13:52:00.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.820+0000 7f87775ddb80 1 Processor -- start 2023-10-08T13:52:00.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.820+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.820+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.820+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 0x55da236ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f87775ddb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da23634e30 con 0x55da236e0e80 2023-10-08T13:52:00.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f87775ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da235d1710 con 0x55da236e17a0 2023-10-08T13:52:00.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f87775ddb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da235d13b0 con 0x55da236ee860 2023-10-08T13:52:00.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f87775ddb80 10 client.0 _ll_register_callbacks cb 0x55da2342e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:00.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769190700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769190700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ee2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56496/0 (socket says 172.21.15.57:56496) 2023-10-08T13:52:00.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769190700 1 -- 172.21.15.57:0/741328553 learned_addr learned my addr 172.21.15.57:0/741328553 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:00.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f876898f700 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769991700 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 0x55da236ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769190700 1 -- 172.21.15.57:0/741328553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 msgr2=0x55da236ebc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:00.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769190700 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 0x55da236ebc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769190700 1 -- 172.21.15.57:0/741328553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 msgr2=0x55da236eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769190700 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236eb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769190700 1 -- 172.21.15.57:0/741328553 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876410ce10 con 0x55da236e0e80 2023-10-08T13:52:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769991700 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 0x55da236ebc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f876898f700 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f8769190700 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ee2a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f87600114e0 tx=0x7f87600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a969c31c1b0cc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f874cff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f874cff9700 1 -- 172.21.15.57:0/741328553 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760004a30 con 0x55da236e0e80 2023-10-08T13:52:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f874cff9700 1 -- 172.21.15.57:0/741328553 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8760017070 con 0x55da236e0e80 2023-10-08T13:52:00.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.821+0000 7f87775ddb80 1 -- 172.21.15.57:0/741328553 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da235fceb0 con 0x55da236e0e80 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f874cff9700 1 -- 172.21.15.57:0/741328553 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8760004b90 con 0x55da236e0e80 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f87775ddb80 10 client.15912 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f87775ddb80 1 -- 172.21.15.57:0/741328553 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55da23448c60 con 0x55da236e0e80 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f874cff9700 1 -- 172.21.15.57:0/741328553 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da235fceb0 con 0x55da236e0e80 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f874cff9700 1 -- 172.21.15.57:0/741328553 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f876001f5d0 con 0x55da236e0e80 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f87775ddb80 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f87775ddb80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f87775ddb80 20 client.15912 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f87775ddb80 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da236fe3b0 0x55da237007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f87775ddb80 4 client.15912 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:00.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f87775ddb80 1 -- 172.21.15.57:0/741328553 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da2362de10 con 0x55da236fe3b0 2023-10-08T13:52:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.822+0000 7f876898f700 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da236fe3b0 0x55da237007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.823+0000 7f874cff9700 10 client.15912 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.823+0000 7f876898f700 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da236fe3b0 0x55da237007d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c36c89350b3971 server_cookie=5feefc40062b5526 in_seq=0 out_seq=0 2023-10-08T13:52:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.823+0000 7f874cff9700 1 -- 172.21.15.57:0/741328553 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55da2362de10 con 0x55da236fe3b0 2023-10-08T13:52:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f874cff9700 10 client.15912 handle_command_reply: tid=0 2023-10-08T13:52:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 client.15912 shutdown 2023-10-08T13:52:00.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 -- 172.21.15.57:0/741328553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 msgr2=0x55da236ee2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ee2a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f87600114e0 tx=0x7f87600047c0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 -- 172.21.15.57:0/741328553 shutdown_connections 2023-10-08T13:52:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da236fe3b0 0x55da237007d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 0x55da236ebc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f874cff9700 0 client.15912 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ee2a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 --2- 172.21.15.57:0/741328553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 -- 172.21.15.57:0/741328553 >> 172.21.15.57:0/741328553 conn(0x55da23656260 msgr2=0x55da236dcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 -- 172.21.15.57:0/741328553 shutdown_connections 2023-10-08T13:52:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 1 -- 172.21.15.57:0/741328553 wait complete. 2023-10-08T13:52:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.824+0000 7f87775ddb80 20 client.15912 trim_cache size 0 max 16384 2023-10-08T13:52:00.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.841+0000 7f87775ddb80 1 Processor -- start 2023-10-08T13:52:00.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.841+0000 7f87775ddb80 1 -- start start 2023-10-08T13:52:00.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.841+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.841+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.841+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 0x55da236ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.841+0000 7f87775ddb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da234b5920 con 0x55da236e0e80 2023-10-08T13:52:00.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.841+0000 7f87775ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da236d9b40 con 0x55da236e17a0 2023-10-08T13:52:00.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.841+0000 7f87775ddb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da236ec4b0 con 0x55da236ee860 2023-10-08T13:52:00.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f8769190700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f8769190700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ebc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56504/0 (socket says 172.21.15.57:56504) 2023-10-08T13:52:00.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f8769190700 1 -- 172.21.15.57:0/1284773246 learned_addr learned my addr 172.21.15.57:0/1284773246 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:00.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f876898f700 1 --2- 172.21.15.57:0/1284773246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f8769991700 1 --2- 172.21.15.57:0/1284773246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 0x55da236ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f8769190700 1 -- 172.21.15.57:0/1284773246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 msgr2=0x55da236ee310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f8769190700 1 --2- 172.21.15.57:0/1284773246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 0x55da236ee310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f8769190700 1 -- 172.21.15.57:0/1284773246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 msgr2=0x55da236eddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f8769190700 1 --2- 172.21.15.57:0/1284773246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236eddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f8769190700 1 -- 172.21.15.57:0/1284773246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da23486250 con 0x55da236e0e80 2023-10-08T13:52:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f8769190700 1 --2- 172.21.15.57:0/1284773246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ebc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8760000c00 tx=0x7f876000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cb18beb860564d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:00.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f874ffff700 1 -- 172.21.15.57:0/1284773246 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876001fd70 con 0x55da236e0e80 2023-10-08T13:52:00.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f874ffff700 1 -- 172.21.15.57:0/1284773246 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8760015070 con 0x55da236e0e80 2023-10-08T13:52:00.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.842+0000 7f874ffff700 1 -- 172.21.15.57:0/1284773246 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760034da0 con 0x55da236e0e80 2023-10-08T13:52:00.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 -- 172.21.15.57:0/1284773246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 msgr2=0x55da236ebc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 --2- 172.21.15.57:0/1284773246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ebc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8760000c00 tx=0x7f876000b8f0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 -- 172.21.15.57:0/1284773246 shutdown_connections 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 --2- 172.21.15.57:0/1284773246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236ee860 0x55da236ee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 --2- 172.21.15.57:0/1284773246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236e0e80 0x55da236ebc60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 --2- 172.21.15.57:0/1284773246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 -- 172.21.15.57:0/1284773246 >> 172.21.15.57:0/1284773246 conn(0x55da23656260 msgr2=0x55da236e6880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 -- 172.21.15.57:0/1284773246 shutdown_connections 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 -- 172.21.15.57:0/1284773246 wait complete. 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 -- start start 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.843+0000 7f87775ddb80 1 Processor -- start 2023-10-08T13:52:00.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 0x55da236f1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f87775ddb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236ee860 0x55da2374b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f87775ddb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da2374b700 con 0x55da236ee860 2023-10-08T13:52:00.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f87775ddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da2374b840 con 0x55da236e17a0 2023-10-08T13:52:00.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f87775ddb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da2374b9d0 con 0x55da236e0e80 2023-10-08T13:52:00.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f87775ddb80 10 client.0 _ll_register_callbacks cb 0x7f8760010c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:00.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769991700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236ee860 0x55da2374b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769991700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236ee860 0x55da2374b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56514/0 (socket says 172.21.15.57:56514) 2023-10-08T13:52:00.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769991700 1 -- 172.21.15.57:0/2581600538 learned_addr learned my addr 172.21.15.57:0/2581600538 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:00.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f876898f700 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769190700 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 0x55da236f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769991700 1 -- 172.21.15.57:0/2581600538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 msgr2=0x55da236f1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:00.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769991700 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 0x55da236f1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769991700 1 -- 172.21.15.57:0/2581600538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 msgr2=0x55da236f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:00.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769991700 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236f1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769991700 1 -- 172.21.15.57:0/2581600538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da23448c60 con 0x55da236ee860 2023-10-08T13:52:00.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769190700 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 0x55da236f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:00.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f8769991700 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236ee860 0x55da2374b1c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f875c099510 tx=0x7f875c095c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e963a8aff6249469 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:00.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.844+0000 7f874cff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:00.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f874cff9700 1 -- 172.21.15.57:0/2581600538 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0a1ba0 con 0x55da236ee860 2023-10-08T13:52:00.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f874cff9700 1 -- 172.21.15.57:0/2581600538 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f875c004c40 con 0x55da236ee860 2023-10-08T13:52:00.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f87775ddb80 1 -- 172.21.15.57:0/2581600538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f875401e040 con 0x55da236ee860 2023-10-08T13:52:00.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f874cff9700 1 -- 172.21.15.57:0/2581600538 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f875c0ca250 con 0x55da236ee860 2023-10-08T13:52:00.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f87775ddb80 10 client.15918 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:00.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f87775ddb80 1 -- 172.21.15.57:0/2581600538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55da2345e2b0 con 0x55da236ee860 2023-10-08T13:52:00.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f874cff9700 1 -- 172.21.15.57:0/2581600538 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f875401e040 con 0x55da236ee860 2023-10-08T13:52:00.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f874cff9700 1 -- 172.21.15.57:0/2581600538 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f875c0222e0 con 0x55da236ee860 2023-10-08T13:52:00.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f87775ddb80 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:00.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f87775ddb80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f87775ddb80 20 client.15918 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f87775ddb80 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da237532d0 0x55da237556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f87775ddb80 4 client.15918 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:52:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.845+0000 7f87775ddb80 1 -- 172.21.15.57:0/2581600538 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55da2362db30 con 0x55da237532d0 2023-10-08T13:52:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.846+0000 7f8769190700 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da237532d0 0x55da237556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.846+0000 7f8769190700 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da237532d0 0x55da237556f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9a22203806ed20 server_cookie=2f16e8c7d5be556a in_seq=0 out_seq=0 2023-10-08T13:52:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.846+0000 7f874cff9700 10 client.15918 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.846+0000 7f874cff9700 1 -- 172.21.15.57:0/2581600538 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55da2362db30 con 0x55da237532d0 2023-10-08T13:52:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.846+0000 7f874cff9700 10 client.15918 handle_command_reply: tid=0 2023-10-08T13:52:00.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.846+0000 7f87775ddb80 1 client.15918 shutdown 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 1 -- 172.21.15.57:0/2581600538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236ee860 msgr2=0x55da2374b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236ee860 0x55da2374b1c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f875c099510 tx=0x7f875c095c80 comp rx=0 tx=0).stop 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 1 -- 172.21.15.57:0/2581600538 shutdown_connections 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da237532d0 0x55da237556f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da236ee860 0x55da2374b1c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f874cff9700 0 client.15918 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da236e0e80 0x55da236f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 1 --2- 172.21.15.57:0/2581600538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da236e17a0 0x55da236f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 1 -- 172.21.15.57:0/2581600538 >> 172.21.15.57:0/2581600538 conn(0x55da23656260 msgr2=0x55da236e2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 1 -- 172.21.15.57:0/2581600538 shutdown_connections 2023-10-08T13:52:00.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 1 -- 172.21.15.57:0/2581600538 wait complete. 2023-10-08T13:52:00.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f87775ddb80 20 client.15918 trim_cache size 0 max 16384 2023-10-08T13:52:00.860 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:52:00.860 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-10-08T13:52:00.860 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-10-08T13:52:00.860 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:52:00.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f876b3f4700 1 -- 172.21.15.57:0/1960189496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87500a1560 msgr2=0x7f87500a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f876b3f4700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87500a1560 0x7f87500a3a20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8760005950 tx=0x7f876000b410 comp rx=0 tx=0).stop 2023-10-08T13:52:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f876b3f4700 1 -- 172.21.15.57:0/1960189496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8764105650 msgr2=0x7f8764195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f876b3f4700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8764105650 0x7f8764195910 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f875c007ee0 tx=0x7f875c00d3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f876b3f4700 1 -- 172.21.15.57:0/1960189496 shutdown_connections 2023-10-08T13:52:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f876b3f4700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8764105650 0x7f8764195910 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f876b3f4700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f87500a1560 0x7f87500a3a20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f876b3f4700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87640ff640 0x7f876419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f876b3f4700 1 --2- 172.21.15.57:0/1960189496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764100000 0x7f876419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.847+0000 7f876b3f4700 1 -- 172.21.15.57:0/1960189496 >> 172.21.15.57:0/1960189496 conn(0x7f87640767c0 msgr2=0x7f87640fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:00.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.848+0000 7f876b3f4700 1 -- 172.21.15.57:0/1960189496 shutdown_connections 2023-10-08T13:52:00.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.848+0000 7f876b3f4700 1 -- 172.21.15.57:0/1960189496 wait complete. 2023-10-08T13:52:00.863 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-08T13:52:00.863 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:52:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.991+0000 7f7e5f737700 1 -- 172.21.15.57:0/1741076654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e580ff510 msgr2=0x7f7e5810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.991+0000 7f7e5f737700 1 --2- 172.21.15.57:0/1741076654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e580ff510 0x7f7e5810d4f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000b0a0 tx=0x7f7e5000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.991+0000 7f7e5f737700 1 -- 172.21.15.57:0/1741076654 shutdown_connections 2023-10-08T13:52:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.991+0000 7f7e5f737700 1 --2- 172.21.15.57:0/1741076654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e580ff510 0x7f7e5810d4f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.991+0000 7f7e5f737700 1 --2- 172.21.15.57:0/1741076654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e58106630 0x7f7e58106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.991+0000 7f7e5f737700 1 --2- 172.21.15.57:0/1741076654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e580feb50 0x7f7e580fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.991+0000 7f7e5f737700 1 -- 172.21.15.57:0/1741076654 >> 172.21.15.57:0/1741076654 conn(0x7f7e58075a90 msgr2=0x7f7e580fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.991+0000 7f7e5f737700 1 -- 172.21.15.57:0/1741076654 shutdown_connections 2023-10-08T13:52:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.991+0000 7f7e5f737700 1 -- 172.21.15.57:0/1741076654 wait complete. 2023-10-08T13:52:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5f737700 1 Processor -- start 2023-10-08T13:52:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5f737700 1 -- start start 2023-10-08T13:52:00.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5f737700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e580feb50 0x7f7e58196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5f737700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e580ff510 0x7f7e58196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5f737700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e58106630 0x7f7e5819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:00.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5f737700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5819b900 con 0x7f7e58106630 2023-10-08T13:52:00.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5f737700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5819ba70 con 0x7f7e580ff510 2023-10-08T13:52:00.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5f737700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5819bc30 con 0x7f7e580feb50 2023-10-08T13:52:00.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5dcd4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e58106630 0x7f7e5819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5dcd4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e58106630 0x7f7e5819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56528/0 (socket says 172.21.15.57:56528) 2023-10-08T13:52:00.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5dcd4700 1 -- 172.21.15.57:0/3358729076 learned_addr learned my addr 172.21.15.57:0/3358729076 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5ccd2700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e580ff510 0x7f7e58196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5d4d3700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e580feb50 0x7f7e58196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5dcd4700 1 -- 172.21.15.57:0/3358729076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e580feb50 msgr2=0x7f7e58196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5dcd4700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e580feb50 0x7f7e58196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5dcd4700 1 -- 172.21.15.57:0/3358729076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e580ff510 msgr2=0x7f7e58196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5dcd4700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e580ff510 0x7f7e58196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5dcd4700 1 -- 172.21.15.57:0/3358729076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e50009d00 con 0x7f7e58106630 2023-10-08T13:52:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.992+0000 7f7e5ccd2700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e580ff510 0x7f7e58196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.993+0000 7f7e5dcd4700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e58106630 0x7f7e5819b330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7e50007b60 tx=0x7f7e50005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.993+0000 7f7e4e7fc700 1 -- 172.21.15.57:0/3358729076 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000e040 con 0x7f7e58106630 2023-10-08T13:52:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.993+0000 7f7e5f737700 1 -- 172.21.15.57:0/3358729076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5819beb0 con 0x7f7e58106630 2023-10-08T13:52:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.993+0000 7f7e4e7fc700 1 -- 172.21.15.57:0/3358729076 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e5001d480 con 0x7f7e58106630 2023-10-08T13:52:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.993+0000 7f7e4e7fc700 1 -- 172.21.15.57:0/3358729076 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5001b6b0 con 0x7f7e58106630 2023-10-08T13:52:00.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.993+0000 7f7e5f737700 1 -- 172.21.15.57:0/3358729076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e581a37c0 con 0x7f7e58106630 2023-10-08T13:52:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.994+0000 7f7e5f737700 1 -- 172.21.15.57:0/3358729076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e58108c40 con 0x7f7e58106630 2023-10-08T13:52:01.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.996+0000 7f7e4e7fc700 1 -- 172.21.15.57:0/3358729076 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 23) v1 ==== 136077+0+0 (secure 0 0 0) 0x7f7e5001d5f0 con 0x7f7e58106630 2023-10-08T13:52:01.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.996+0000 7f7e4e7fc700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7e440a1560 0x7f7e440a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.997+0000 7f7e5d4d3700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7e440a1560 0x7f7e440a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.997+0000 7f7e4e7fc700 1 -- 172.21.15.57:0/3358729076 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7e500cf980 con 0x7f7e58106630 2023-10-08T13:52:01.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.997+0000 7f7e5d4d3700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7e440a1560 0x7f7e440a3a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7e54005fd0 tx=0x7f7e54005ee0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:00.999+0000 7f7e4e7fc700 1 -- 172.21.15.57:0/3358729076 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7e50098af0 con 0x7f7e58106630 2023-10-08T13:52:01.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.048+0000 7f7e4e7fc700 1 -- 172.21.15.57:0/3358729076 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7e50098ec0 con 0x7f7e58106630 2023-10-08T13:52:01.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.292+0000 7f7e5f737700 1 -- 172.21.15.57:0/3358729076 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e58197d20 con 0x7f7e58106630 2023-10-08T13:52:01.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.293+0000 7f7e4e7fc700 1 -- 172.21.15.57:0/3358729076 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f7e5005e710 con 0x7f7e58106630 2023-10-08T13:52:01.294 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:52:01.294 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:50:07.719573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:52:01.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 -- 172.21.15.57:0/3358729076 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7e440a1560 msgr2=0x7f7e440a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7e440a1560 0x7f7e440a3a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7e54005fd0 tx=0x7f7e54005ee0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 -- 172.21.15.57:0/3358729076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e58106630 msgr2=0x7f7e5819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e58106630 0x7f7e5819b330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7e50007b60 tx=0x7f7e50005b10 comp rx=0 tx=0).stop 2023-10-08T13:52:01.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 -- 172.21.15.57:0/3358729076 shutdown_connections 2023-10-08T13:52:01.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e58106630 0x7f7e5819b330 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e580feb50 0x7f7e58196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7e440a1560 0x7f7e440a3a20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 --2- 172.21.15.57:0/3358729076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e580ff510 0x7f7e58196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 -- 172.21.15.57:0/3358729076 >> 172.21.15.57:0/3358729076 conn(0x7f7e58075a90 msgr2=0x7f7e580fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 -- 172.21.15.57:0/3358729076 shutdown_connections 2023-10-08T13:52:01.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.295+0000 7f7e5f737700 1 -- 172.21.15.57:0/3358729076 wait complete. 2023-10-08T13:52:01.298 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:52:01.310 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-08T13:52:01.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.439+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1998216385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106740 msgr2=0x7f93f4106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.439+0000 7f93f8d9f700 1 --2- 172.21.15.57:0/1998216385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106740 0x7f93f4106b20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009990 tx=0x7f93e4009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.439+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1998216385 shutdown_connections 2023-10-08T13:52:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.439+0000 7f93f8d9f700 1 --2- 172.21.15.57:0/1998216385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93f41010f0 0x7f93f410d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.439+0000 7f93f8d9f700 1 --2- 172.21.15.57:0/1998216385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106740 0x7f93f4106b20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.439+0000 7f93f8d9f700 1 --2- 172.21.15.57:0/1998216385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f4100730 0x7f93f4100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.439+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1998216385 >> 172.21.15.57:0/1998216385 conn(0x7f93f40fc200 msgr2=0x7f93f40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.439+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1998216385 shutdown_connections 2023-10-08T13:52:01.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.439+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1998216385 wait complete. 2023-10-08T13:52:01.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f8d9f700 1 Processor -- start 2023-10-08T13:52:01.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f8d9f700 1 -- start start 2023-10-08T13:52:01.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f8d9f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f4100730 0x7f93f4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f8d9f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93f41010f0 0x7f93f4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f8d9f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106740 0x7f93f4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f8d9f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41a4f70 con 0x7f93f4106740 2023-10-08T13:52:01.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f8d9f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41a50e0 con 0x7f93f41010f0 2023-10-08T13:52:01.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f8d9f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41a52a0 con 0x7f93f4100730 2023-10-08T13:52:01.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106740 0x7f93f4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106740 0x7f93f4073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56540/0 (socket says 172.21.15.57:56540) 2023-10-08T13:52:01.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f2d9d700 1 -- 172.21.15.57:0/1342806397 learned_addr learned my addr 172.21.15.57:0/1342806397 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f1d9b700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93f41010f0 0x7f93f4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f259c700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f4100730 0x7f93f4072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f2d9d700 1 -- 172.21.15.57:0/1342806397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f4100730 msgr2=0x7f93f4072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f2d9d700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f4100730 0x7f93f4072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f2d9d700 1 -- 172.21.15.57:0/1342806397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93f41010f0 msgr2=0x7f93f4073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f2d9d700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93f41010f0 0x7f93f4073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f2d9d700 1 -- 172.21.15.57:0/1342806397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40095f0 con 0x7f93f4106740 2023-10-08T13:52:01.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.440+0000 7f93f259c700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f4100730 0x7f93f4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:01.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.441+0000 7f93f2d9d700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106740 0x7f93f4073570 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f93ec007ee0 tx=0x7f93ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.441+0000 7f93e37fe700 1 -- 172.21.15.57:0/1342806397 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00ddd0 con 0x7f93f4106740 2023-10-08T13:52:01.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.441+0000 7f93e37fe700 1 -- 172.21.15.57:0/1342806397 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93ec00f040 con 0x7f93f4106740 2023-10-08T13:52:01.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.441+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1342806397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f41a5580 con 0x7f93f4106740 2023-10-08T13:52:01.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.441+0000 7f93e37fe700 1 -- 172.21.15.57:0/1342806397 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec0048e0 con 0x7f93f4106740 2023-10-08T13:52:01.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.441+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1342806397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f41a5a40 con 0x7f93f4106740 2023-10-08T13:52:01.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.444+0000 7f93e37fe700 1 -- 172.21.15.57:0/1342806397 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f93ec003a00 con 0x7f93f4106740 2023-10-08T13:52:01.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f93e37fe700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93dc0a15f0 0x7f93dc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f93f259c700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93dc0a15f0 0x7f93dc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f93e37fe700 1 -- 172.21.15.57:0/1342806397 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f93ec0ccd60 con 0x7f93f4106740 2023-10-08T13:52:01.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f9404f88b80 1 Processor -- start 2023-10-08T13:52:01.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f9404f88b80 1 -- start start 2023-10-08T13:52:01.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f93f259c700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93dc0a15f0 0x7f93dc0a3ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b5c0 tx=0x7f93e4005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fae7a0 0x555e95fb9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fb9650 0x555e95fbba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f9404f88b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95f17460 con 0x555e95fade80 2023-10-08T13:52:01.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f9404f88b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95f08c20 con 0x555e95fae7a0 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.445+0000 7f9404f88b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95f06540 con 0x555e95fb9650 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56544/0 (socket says 172.21.15.57:56544) 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f259c700 1 -- 172.21.15.57:0/3639704933 learned_addr learned my addr 172.21.15.57:0/3639704933 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f2d9d700 1 --2- 172.21.15.57:0/3639704933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fb9650 0x555e95fbba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f1d9b700 1 --2- 172.21.15.57:0/3639704933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fae7a0 0x555e95fb9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f259c700 1 -- 172.21.15.57:0/3639704933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fb9650 msgr2=0x555e95fbba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f259c700 1 --2- 172.21.15.57:0/3639704933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fb9650 0x555e95fbba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f259c700 1 -- 172.21.15.57:0/3639704933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fae7a0 msgr2=0x555e95fb9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f259c700 1 --2- 172.21.15.57:0/3639704933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fae7a0 0x555e95fb9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f259c700 1 -- 172.21.15.57:0/3639704933 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e95d2cde0 con 0x555e95fade80 2023-10-08T13:52:01.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93f259c700 1 --2- 172.21.15.57:0/3639704933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fae260 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b850 tx=0x7f93e401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9354ff419532b01a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93e0ff9700 1 -- 172.21.15.57:0/3639704933 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e401d070 con 0x555e95fade80 2023-10-08T13:52:01.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93e0ff9700 1 -- 172.21.15.57:0/3639704933 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e40054e0 con 0x555e95fade80 2023-10-08T13:52:01.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.446+0000 7f93e0ff9700 1 -- 172.21.15.57:0/3639704933 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e400f900 con 0x555e95fade80 2023-10-08T13:52:01.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.447+0000 7f9404f88b80 1 -- 172.21.15.57:0/3639704933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 msgr2=0x555e95fae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.447+0000 7f9404f88b80 1 --2- 172.21.15.57:0/3639704933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fae260 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b850 tx=0x7f93e401d000 comp rx=0 tx=0).stop 2023-10-08T13:52:01.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.447+0000 7f9404f88b80 1 -- 172.21.15.57:0/3639704933 shutdown_connections 2023-10-08T13:52:01.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.447+0000 7f9404f88b80 1 --2- 172.21.15.57:0/3639704933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fb9650 0x555e95fbba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.447+0000 7f9404f88b80 1 --2- 172.21.15.57:0/3639704933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fae260 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.447+0000 7f9404f88b80 1 --2- 172.21.15.57:0/3639704933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fae7a0 0x555e95fb9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.447+0000 7f9404f88b80 1 -- 172.21.15.57:0/3639704933 >> 172.21.15.57:0/3639704933 conn(0x555e95f23260 msgr2=0x555e95faa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.447+0000 7f9404f88b80 1 -- 172.21.15.57:0/3639704933 shutdown_connections 2023-10-08T13:52:01.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.447+0000 7f9404f88b80 1 -- 172.21.15.57:0/3639704933 wait complete. 2023-10-08T13:52:01.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f9404f88b80 1 -- start start 2023-10-08T13:52:01.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f9404f88b80 1 Processor -- start 2023-10-08T13:52:01.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fb9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fae7a0 0x555e95fb6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fb9650 0x555e95fb6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f9404f88b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95f01e30 con 0x555e95fade80 2023-10-08T13:52:01.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f9404f88b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95e9e710 con 0x555e95fb9650 2023-10-08T13:52:01.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f9404f88b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95e9e3b0 con 0x555e95fae7a0 2023-10-08T13:52:01.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f9404f88b80 10 client.0 _ll_register_callbacks cb 0x555e95cfb800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:01.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f93f259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fb9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f93f259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fb9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56550/0 (socket says 172.21.15.57:56550) 2023-10-08T13:52:01.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f93f259c700 1 -- 172.21.15.57:0/2450559007 learned_addr learned my addr 172.21.15.57:0/2450559007 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f93f1d9b700 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fae7a0 0x555e95fb6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.448+0000 7f93f2d9d700 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fb9650 0x555e95fb6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93f259c700 1 -- 172.21.15.57:0/2450559007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fae7a0 msgr2=0x555e95fb6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93f259c700 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fae7a0 0x555e95fb6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93f259c700 1 -- 172.21.15.57:0/2450559007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fb9650 msgr2=0x555e95fb6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93f259c700 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fb9650 0x555e95fb6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93f259c700 1 -- 172.21.15.57:0/2450559007 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f410df00 con 0x555e95fade80 2023-10-08T13:52:01.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93f259c700 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fb9050 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f93e40059d0 tx=0x7f93e400f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560c70873d2f3c88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:01.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93d9ffb700 1 -- 172.21.15.57:0/2450559007 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e400fb00 con 0x555e95fade80 2023-10-08T13:52:01.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93d9ffb700 1 -- 172.21.15.57:0/2450559007 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f93e400fc60 con 0x555e95fade80 2023-10-08T13:52:01.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f9404f88b80 1 -- 172.21.15.57:0/2450559007 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e95ec9eb0 con 0x555e95fade80 2023-10-08T13:52:01.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93d9ffb700 1 -- 172.21.15.57:0/2450559007 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e400fe20 con 0x555e95fade80 2023-10-08T13:52:01.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f9404f88b80 10 client.15936 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:01.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f9404f88b80 1 -- 172.21.15.57:0/2450559007 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555e95d15c60 con 0x555e95fade80 2023-10-08T13:52:01.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.449+0000 7f93d9ffb700 1 -- 172.21.15.57:0/2450559007 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e95ec9eb0 con 0x555e95fade80 2023-10-08T13:52:01.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.450+0000 7f93d9ffb700 1 -- 172.21.15.57:0/2450559007 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93e402fba0 con 0x555e95fade80 2023-10-08T13:52:01.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.450+0000 7f9404f88b80 10 client.15936 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:01.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.450+0000 7f9404f88b80 10 client.15936 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-08T13:52:01.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.450+0000 7f9404f88b80 20 client.15936 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:01.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.450+0000 7f9404f88b80 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x555e95fc8e00 0x555e95fcb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.450+0000 7f9404f88b80 4 client.15936 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:01.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.450+0000 7f9404f88b80 1 -- 172.21.15.57:0/2450559007 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e95efae10 con 0x555e95fc8e00 2023-10-08T13:52:01.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.450+0000 7f93f1d9b700 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x555e95fc8e00 0x555e95fcb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.450+0000 7f93f1d9b700 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x555e95fc8e00 0x555e95fcb220 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1af150c4c616853c server_cookie=89af64128ceabd9f in_seq=0 out_seq=0 2023-10-08T13:52:01.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.450+0000 7f93d9ffb700 10 client.15936 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:52:01.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f93d9ffb700 1 -- 172.21.15.57:0/2450559007 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555e95efae10 con 0x555e95fc8e00 2023-10-08T13:52:01.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f93d9ffb700 10 client.15936 handle_command_reply: tid=0 2023-10-08T13:52:01.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 client.15936 shutdown 2023-10-08T13:52:01.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 -- 172.21.15.57:0/2450559007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 msgr2=0x555e95fb9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fb9050 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f93e40059d0 tx=0x7f93e400f9c0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 -- 172.21.15.57:0/2450559007 shutdown_connections 2023-10-08T13:52:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x555e95fc8e00 0x555e95fcb220 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fb9650 0x555e95fb6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fb9050 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f93d9ffb700 0 client.15936 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:52:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 --2- 172.21.15.57:0/2450559007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fae7a0 0x555e95fb6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 -- 172.21.15.57:0/2450559007 >> 172.21.15.57:0/2450559007 conn(0x555e95f23260 msgr2=0x555e95fa9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 -- 172.21.15.57:0/2450559007 shutdown_connections 2023-10-08T13:52:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.451+0000 7f9404f88b80 1 -- 172.21.15.57:0/2450559007 wait complete. 2023-10-08T13:52:01.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.452+0000 7f9404f88b80 20 client.15936 trim_cache size 0 max 16384 2023-10-08T13:52:01.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.477+0000 7f9404f88b80 1 Processor -- start 2023-10-08T13:52:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.477+0000 7f9404f88b80 1 -- start start 2023-10-08T13:52:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fc01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fae7a0 0x555e95fa9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fb9650 0x555e95faa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f9404f88b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95d82920 con 0x555e95fade80 2023-10-08T13:52:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f9404f88b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95fa9c10 con 0x555e95fae7a0 2023-10-08T13:52:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f9404f88b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e95faeb80 con 0x555e95fb9650 2023-10-08T13:52:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fc01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fc01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56558/0 (socket says 172.21.15.57:56558) 2023-10-08T13:52:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f259c700 1 -- 172.21.15.57:0/3143274344 learned_addr learned my addr 172.21.15.57:0/3143274344 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f1d9b700 1 --2- 172.21.15.57:0/3143274344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fae7a0 0x555e95fa9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f259c700 1 -- 172.21.15.57:0/3143274344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fb9650 msgr2=0x555e95faa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f2d9d700 1 --2- 172.21.15.57:0/3143274344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fb9650 0x555e95faa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f259c700 1 --2- 172.21.15.57:0/3143274344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fb9650 0x555e95faa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f259c700 1 -- 172.21.15.57:0/3143274344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fae7a0 msgr2=0x555e95fa9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f259c700 1 --2- 172.21.15.57:0/3143274344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fae7a0 0x555e95fa9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f259c700 1 -- 172.21.15.57:0/3143274344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e95d53250 con 0x555e95fade80 2023-10-08T13:52:01.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93f259c700 1 --2- 172.21.15.57:0/3143274344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fc01a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f570 tx=0x7f93e4038510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519d56b09ad8bad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93e17fa700 1 -- 172.21.15.57:0/3143274344 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e401d070 con 0x555e95fade80 2023-10-08T13:52:01.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93e17fa700 1 -- 172.21.15.57:0/3143274344 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e4033070 con 0x555e95fade80 2023-10-08T13:52:01.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.478+0000 7f93e17fa700 1 -- 172.21.15.57:0/3143274344 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4038920 con 0x555e95fade80 2023-10-08T13:52:01.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.479+0000 7f9404f88b80 1 -- 172.21.15.57:0/3143274344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 msgr2=0x555e95fc01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.479+0000 7f9404f88b80 1 --2- 172.21.15.57:0/3143274344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fc01a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f570 tx=0x7f93e4038510 comp rx=0 tx=0).stop 2023-10-08T13:52:01.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.479+0000 7f9404f88b80 1 -- 172.21.15.57:0/3143274344 shutdown_connections 2023-10-08T13:52:01.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.479+0000 7f9404f88b80 1 --2- 172.21.15.57:0/3143274344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fb9650 0x555e95faa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.479+0000 7f9404f88b80 1 --2- 172.21.15.57:0/3143274344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fade80 0x555e95fc01a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.479+0000 7f9404f88b80 1 --2- 172.21.15.57:0/3143274344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fae7a0 0x555e95fa9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.479+0000 7f9404f88b80 1 -- 172.21.15.57:0/3143274344 >> 172.21.15.57:0/3143274344 conn(0x555e95f23260 msgr2=0x555e95fa94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.479+0000 7f9404f88b80 1 -- 172.21.15.57:0/3143274344 shutdown_connections 2023-10-08T13:52:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.479+0000 7f9404f88b80 1 -- 172.21.15.57:0/3143274344 wait complete. 2023-10-08T13:52:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f9404f88b80 1 -- start start 2023-10-08T13:52:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f9404f88b80 1 Processor -- start 2023-10-08T13:52:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fade80 0x555e95fbe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fae7a0 0x555e95fbe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f9404f88b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fb9650 0x555e95fbec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f9404f88b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9601bbf0 con 0x555e95fb9650 2023-10-08T13:52:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f9404f88b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9601bd30 con 0x555e95fade80 2023-10-08T13:52:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f9404f88b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9601be70 con 0x555e95fae7a0 2023-10-08T13:52:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f9404f88b80 10 client.0 _ll_register_callbacks cb 0x7f93ec007f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f93f2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fb9650 0x555e95fbec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f93f2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fb9650 0x555e95fbec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56568/0 (socket says 172.21.15.57:56568) 2023-10-08T13:52:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f93f2d9d700 1 -- 172.21.15.57:0/335575356 learned_addr learned my addr 172.21.15.57:0/335575356 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f93f1d9b700 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fae7a0 0x555e95fbe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f93f259c700 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fade80 0x555e95fbe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f93f2d9d700 1 -- 172.21.15.57:0/335575356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fae7a0 msgr2=0x555e95fbe6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.480+0000 7f93f2d9d700 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fae7a0 0x555e95fbe6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f93f2d9d700 1 -- 172.21.15.57:0/335575356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fade80 msgr2=0x555e95fbe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f93f2d9d700 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fade80 0x555e95fbe180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f93f2d9d700 1 -- 172.21.15.57:0/335575356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e95d15c60 con 0x555e95fb9650 2023-10-08T13:52:01.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f93f2d9d700 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fb9650 0x555e95fbec00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0ca1b0 tx=0x7f93ec0993b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189b482892cff019 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f93d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:01.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f93d9ffb700 1 -- 172.21.15.57:0/335575356 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec09db30 con 0x555e95fb9650 2023-10-08T13:52:01.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f93d9ffb700 1 -- 172.21.15.57:0/335575356 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f93ec0a1d70 con 0x555e95fb9650 2023-10-08T13:52:01.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f9404f88b80 1 -- 172.21.15.57:0/335575356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e40054e0 con 0x555e95fb9650 2023-10-08T13:52:01.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f93d9ffb700 1 -- 172.21.15.57:0/335575356 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93ec003cb0 con 0x555e95fb9650 2023-10-08T13:52:01.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f9404f88b80 10 client.15942 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:01.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f9404f88b80 1 -- 172.21.15.57:0/335575356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555e95d2b2b0 con 0x555e95fb9650 2023-10-08T13:52:01.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f93d9ffb700 1 -- 172.21.15.57:0/335575356 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e40054e0 con 0x555e95fb9650 2023-10-08T13:52:01.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f93d9ffb700 1 -- 172.21.15.57:0/335575356 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93ec098fd0 con 0x555e95fb9650 2023-10-08T13:52:01.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f9404f88b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:01.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f9404f88b80 10 client.15942 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-08T13:52:01.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f9404f88b80 20 client.15942 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:01.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f9404f88b80 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x555e95fc8e00 0x555e9601f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f9404f88b80 4 client.15942 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.481+0000 7f9404f88b80 1 -- 172.21.15.57:0/335575356 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e95efab30 con 0x555e95fc8e00 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.482+0000 7f93f259c700 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x555e95fc8e00 0x555e9601f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.482+0000 7f93d9ffb700 10 client.15942 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.482+0000 7f93f259c700 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x555e95fc8e00 0x555e9601f5c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ecd8647932110a8 server_cookie=552d34de412081c9 in_seq=0 out_seq=0 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.482+0000 7f93d9ffb700 1 -- 172.21.15.57:0/335575356 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555e95efab30 con 0x555e95fc8e00 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.482+0000 7f93d9ffb700 10 client.15942 handle_command_reply: tid=0 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.482+0000 7f9404f88b80 1 client.15942 shutdown 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.482+0000 7f9404f88b80 1 -- 172.21.15.57:0/335575356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fb9650 msgr2=0x555e95fbec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.482+0000 7f9404f88b80 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fb9650 0x555e95fbec00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0ca1b0 tx=0x7f93ec0993b0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f9404f88b80 1 -- 172.21.15.57:0/335575356 shutdown_connections 2023-10-08T13:52:01.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f9404f88b80 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x555e95fc8e00 0x555e9601f5c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f9404f88b80 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555e95fb9650 0x555e95fbec00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f93d9ffb700 0 client.15942 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:52:01.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f9404f88b80 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e95fade80 0x555e95fbe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f9404f88b80 1 --2- 172.21.15.57:0/335575356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555e95fae7a0 0x555e95fbe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f9404f88b80 1 -- 172.21.15.57:0/335575356 >> 172.21.15.57:0/335575356 conn(0x555e95f23260 msgr2=0x555e95fb16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f9404f88b80 1 -- 172.21.15.57:0/335575356 shutdown_connections 2023-10-08T13:52:01.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f9404f88b80 1 -- 172.21.15.57:0/335575356 wait complete. 2023-10-08T13:52:01.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f9404f88b80 20 client.15942 trim_cache size 0 max 16384 2023-10-08T13:52:01.495 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:52:01.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1342806397 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93dc0a15f0 msgr2=0x7f93dc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f93f8d9f700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93dc0a15f0 0x7f93dc0a3ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b5c0 tx=0x7f93e4005fb0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1342806397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106740 msgr2=0x7f93f4073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.483+0000 7f93f8d9f700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106740 0x7f93f4073570 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f93ec007ee0 tx=0x7f93ec00d3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.484+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1342806397 shutdown_connections 2023-10-08T13:52:01.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.484+0000 7f93f8d9f700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f93f4106740 0x7f93f4073570 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.484+0000 7f93f8d9f700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93f4100730 0x7f93f4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.484+0000 7f93f8d9f700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93dc0a15f0 0x7f93dc0a3ab0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.484+0000 7f93f8d9f700 1 --2- 172.21.15.57:0/1342806397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93f41010f0 0x7f93f4073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.484+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1342806397 >> 172.21.15.57:0/1342806397 conn(0x7f93f40fc200 msgr2=0x7f93f40fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.484+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1342806397 shutdown_connections 2023-10-08T13:52:01.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.484+0000 7f93f8d9f700 1 -- 172.21.15.57:0/1342806397 wait complete. 2023-10-08T13:52:01.498 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-08T13:52:01.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.627+0000 7fca2d8be700 1 -- 172.21.15.57:0/662474634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca280fffe0 msgr2=0x7fca2810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.627+0000 7fca2d8be700 1 --2- 172.21.15.57:0/662474634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca280fffe0 0x7fca2810c4e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fca2000b0a0 tx=0x7fca2000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.627+0000 7fca2d8be700 1 -- 172.21.15.57:0/662474634 shutdown_connections 2023-10-08T13:52:01.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.627+0000 7fca2d8be700 1 --2- 172.21.15.57:0/662474634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca280fffe0 0x7fca2810c4e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.627+0000 7fca2d8be700 1 --2- 172.21.15.57:0/662474634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca281056b0 0x7fca28105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.627+0000 7fca2d8be700 1 --2- 172.21.15.57:0/662474634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280ff620 0x7fca280ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.627+0000 7fca2d8be700 1 -- 172.21.15.57:0/662474634 >> 172.21.15.57:0/662474634 conn(0x7fca28076780 msgr2=0x7fca28076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.627+0000 7fca2d8be700 1 -- 172.21.15.57:0/662474634 shutdown_connections 2023-10-08T13:52:01.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.627+0000 7fca2d8be700 1 -- 172.21.15.57:0/662474634 wait complete. 2023-10-08T13:52:01.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.628+0000 7fca2d8be700 1 Processor -- start 2023-10-08T13:52:01.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.628+0000 7fca2d8be700 1 -- start start 2023-10-08T13:52:01.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca2d8be700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280ff620 0x7fca2819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca2d8be700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca280fffe0 0x7fca2819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca2d8be700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca281056b0 0x7fca28198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca2d8be700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca28199170 con 0x7fca280fffe0 2023-10-08T13:52:01.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca2d8be700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca281992e0 con 0x7fca280ff620 2023-10-08T13:52:01.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca2d8be700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca281994a0 con 0x7fca281056b0 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca267fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca280fffe0 0x7fca2819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca267fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca280fffe0 0x7fca2819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56578/0 (socket says 172.21.15.57:56578) 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca267fc700 1 -- 172.21.15.57:0/1628643160 learned_addr learned my addr 172.21.15.57:0/1628643160 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca26ffd700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280ff620 0x7fca2819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca277fe700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca281056b0 0x7fca28198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca267fc700 1 -- 172.21.15.57:0/1628643160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca281056b0 msgr2=0x7fca28198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca267fc700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca281056b0 0x7fca28198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca267fc700 1 -- 172.21.15.57:0/1628643160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280ff620 msgr2=0x7fca2819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca267fc700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280ff620 0x7fca2819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca267fc700 1 -- 172.21.15.57:0/1628643160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca20009d00 con 0x7fca280fffe0 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca267fc700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca280fffe0 0x7fca2819eed0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00d980 tx=0x7fca1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca2c8bc700 1 -- 172.21.15.57:0/1628643160 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c004220 con 0x7fca280fffe0 2023-10-08T13:52:01.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca2c8bc700 1 -- 172.21.15.57:0/1628643160 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca1c007640 con 0x7fca280fffe0 2023-10-08T13:52:01.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca2d8be700 1 -- 172.21.15.57:0/1628643160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca28199780 con 0x7fca280fffe0 2023-10-08T13:52:01.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.629+0000 7fca2c8bc700 1 -- 172.21.15.57:0/1628643160 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c005690 con 0x7fca280fffe0 2023-10-08T13:52:01.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.630+0000 7fca2d8be700 1 -- 172.21.15.57:0/1628643160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca281a5980 con 0x7fca280fffe0 2023-10-08T13:52:01.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.633+0000 7fca2c8bc700 1 -- 172.21.15.57:0/1628643160 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fca1c004750 con 0x7fca280fffe0 2023-10-08T13:52:01.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.633+0000 7fca2c8bc700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca100a1670 0x7fca100a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca39aa7b80 1 Processor -- start 2023-10-08T13:52:01.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca26ffd700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca100a1670 0x7fca100a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca39aa7b80 1 -- start start 2023-10-08T13:52:01.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca2c8bc700 1 -- 172.21.15.57:0/1628643160 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fca1c0cd1a0 con 0x7fca280fffe0 2023-10-08T13:52:01.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1e5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1e57a0 0x56495d1f0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1f0650 0x56495d1f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d14e460 con 0x56495d1e4e80 2023-10-08T13:52:01.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d13fc20 con 0x56495d1f0650 2023-10-08T13:52:01.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d13d540 con 0x56495d1e57a0 2023-10-08T13:52:01.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca26ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca26ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1e5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56586/0 (socket says 172.21.15.57:56586) 2023-10-08T13:52:01.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca26ffd700 1 -- 172.21.15.57:0/2068036120 learned_addr learned my addr 172.21.15.57:0/2068036120 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca277fe700 1 --2- 172.21.15.57:0/2068036120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1f0650 0x56495d1f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca26ffd700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca100a1670 0x7fca100a3b30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fca28077ab0 tx=0x7fca1800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca26ffd700 1 -- 172.21.15.57:0/2068036120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1e57a0 msgr2=0x56495d1f0110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:52:01.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca26ffd700 1 --2- 172.21.15.57:0/2068036120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1e57a0 0x56495d1f0110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca26ffd700 1 -- 172.21.15.57:0/2068036120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1f0650 msgr2=0x56495d1f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca26ffd700 1 --2- 172.21.15.57:0/2068036120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1f0650 0x56495d1f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.634+0000 7fca26ffd700 1 -- 172.21.15.57:0/2068036120 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56495cf63de0 con 0x56495d1e4e80 2023-10-08T13:52:01.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.635+0000 7fca26ffd700 1 --2- 172.21.15.57:0/2068036120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1e5260 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fca18010e90 tx=0x7fca1801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a8d675ca8dbb7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.635+0000 7fca0dffb700 1 -- 172.21.15.57:0/2068036120 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18011db0 con 0x56495d1e4e80 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.635+0000 7fca0dffb700 1 -- 172.21.15.57:0/2068036120 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca18010ec0 con 0x56495d1e4e80 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.635+0000 7fca0dffb700 1 -- 172.21.15.57:0/2068036120 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1801d430 con 0x56495d1e4e80 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 -- 172.21.15.57:0/2068036120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 msgr2=0x56495d1e5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/2068036120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1e5260 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fca18010e90 tx=0x7fca1801b000 comp rx=0 tx=0).stop 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 -- 172.21.15.57:0/2068036120 shutdown_connections 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/2068036120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1f0650 0x56495d1f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/2068036120 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1e5260 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/2068036120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1e57a0 0x56495d1f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 -- 172.21.15.57:0/2068036120 >> 172.21.15.57:0/2068036120 conn(0x56495d15a260 msgr2=0x56495d1e12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 -- 172.21.15.57:0/2068036120 shutdown_connections 2023-10-08T13:52:01.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 -- 172.21.15.57:0/2068036120 wait complete. 2023-10-08T13:52:01.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 -- start start 2023-10-08T13:52:01.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.636+0000 7fca39aa7b80 1 Processor -- start 2023-10-08T13:52:01.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e57a0 0x56495d1ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1edb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d138e30 con 0x56495d1e4e80 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d0d5710 con 0x56495d1e57a0 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d0d53b0 con 0x56495d1f0650 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca39aa7b80 10 client.0 _ll_register_callbacks cb 0x56495cf32800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca26ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca26ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56590/0 (socket says 172.21.15.57:56590) 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca26ffd700 1 -- 172.21.15.57:0/3214247092 learned_addr learned my addr 172.21.15.57:0/3214247092 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca277fe700 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1edb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca267fc700 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e57a0 0x56495d1ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca26ffd700 1 -- 172.21.15.57:0/3214247092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 msgr2=0x56495d1edb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca26ffd700 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1edb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca26ffd700 1 -- 172.21.15.57:0/3214247092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e57a0 msgr2=0x56495d1ed410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca26ffd700 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e57a0 0x56495d1ed410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca26ffd700 1 -- 172.21.15.57:0/3214247092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca18010b70 con 0x56495d1e4e80 2023-10-08T13:52:01.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca26ffd700 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f00d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fca18011a00 tx=0x7fca180038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137319dc8581a4d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:01.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca06ffd700 1 -- 172.21.15.57:0/3214247092 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1802a070 con 0x56495d1e4e80 2023-10-08T13:52:01.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.637+0000 7fca06ffd700 1 -- 172.21.15.57:0/3214247092 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fca18017070 con 0x56495d1e4e80 2023-10-08T13:52:01.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3214247092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56495d131e10 con 0x56495d1e4e80 2023-10-08T13:52:01.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca06ffd700 1 -- 172.21.15.57:0/3214247092 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca18003c30 con 0x56495d1e4e80 2023-10-08T13:52:01.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca39aa7b80 10 client.15954 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:01.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3214247092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56495cf4cc60 con 0x56495d1e4e80 2023-10-08T13:52:01.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca06ffd700 1 -- 172.21.15.57:0/3214247092 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56495d131e10 con 0x56495d1e4e80 2023-10-08T13:52:01.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca06ffd700 1 -- 172.21.15.57:0/3214247092 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fca1801d840 con 0x56495d1e4e80 2023-10-08T13:52:01.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca39aa7b80 10 client.15954 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:01.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca39aa7b80 10 client.15954 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-08T13:52:01.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca39aa7b80 20 client.15954 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:01.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56495d2000b0 0x56495d2024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca39aa7b80 4 client.15954 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3214247092 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56495d131b30 con 0x56495d2000b0 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.638+0000 7fca267fc700 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56495d2000b0 0x56495d2024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.639+0000 7fca267fc700 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56495d2000b0 0x56495d2024d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5679d85cd8f1812f server_cookie=77a979d7505841c8 in_seq=0 out_seq=0 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.639+0000 7fca06ffd700 10 client.15954 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.639+0000 7fca06ffd700 1 -- 172.21.15.57:0/3214247092 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56495d131b30 con 0x56495d2000b0 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.639+0000 7fca06ffd700 10 client.15954 handle_command_reply: tid=0 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.639+0000 7fca39aa7b80 1 client.15954 shutdown 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3214247092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 msgr2=0x56495d1f00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f00d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fca18011a00 tx=0x7fca180038b0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3214247092 shutdown_connections 2023-10-08T13:52:01.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56495d2000b0 0x56495d2024d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1edb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca06ffd700 0 client.15954 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:52:01.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f00d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/3214247092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e57a0 0x56495d1ed410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3214247092 >> 172.21.15.57:0/3214247092 conn(0x56495d15a260 msgr2=0x56495d1e0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3214247092 shutdown_connections 2023-10-08T13:52:01.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3214247092 wait complete. 2023-10-08T13:52:01.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.640+0000 7fca39aa7b80 20 client.15954 trim_cache size 0 max 16384 2023-10-08T13:52:01.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.653+0000 7fca39aa7b80 1 Processor -- start 2023-10-08T13:52:01.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.653+0000 7fca39aa7b80 1 -- start start 2023-10-08T13:52:01.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.653+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e4e80 0x56495d1e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.653+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e57a0 0x56495d1f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.653+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1ea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.653+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495cfb9920 con 0x56495d1e57a0 2023-10-08T13:52:01.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.653+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d1f76e0 con 0x56495d1e4e80 2023-10-08T13:52:01.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.653+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d1eb980 con 0x56495d1f0650 2023-10-08T13:52:01.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca267fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e57a0 0x56495d1f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca267fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e57a0 0x56495d1f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56600/0 (socket says 172.21.15.57:56600) 2023-10-08T13:52:01.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca267fc700 1 -- 172.21.15.57:0/3168447746 learned_addr learned my addr 172.21.15.57:0/3168447746 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca277fe700 1 --2- 172.21.15.57:0/3168447746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1ea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca26ffd700 1 --2- 172.21.15.57:0/3168447746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e4e80 0x56495d1e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca267fc700 1 -- 172.21.15.57:0/3168447746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 msgr2=0x56495d1ea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca267fc700 1 --2- 172.21.15.57:0/3168447746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1ea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca267fc700 1 -- 172.21.15.57:0/3168447746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e4e80 msgr2=0x56495d1e7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca267fc700 1 --2- 172.21.15.57:0/3168447746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e4e80 0x56495d1e7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca267fc700 1 -- 172.21.15.57:0/3168447746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56495cf8a250 con 0x56495d1e57a0 2023-10-08T13:52:01.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca267fc700 1 --2- 172.21.15.57:0/3168447746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e57a0 0x56495d1f71a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fca1c096380 tx=0x7fca1c0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9b390e9f128ee0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca0e7fc700 1 -- 172.21.15.57:0/3168447746 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0b6070 con 0x56495d1e57a0 2023-10-08T13:52:01.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca0e7fc700 1 -- 172.21.15.57:0/3168447746 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca1c09dfd0 con 0x56495d1e57a0 2023-10-08T13:52:01.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.654+0000 7fca0e7fc700 1 -- 172.21.15.57:0/3168447746 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c09a0e0 con 0x56495d1e57a0 2023-10-08T13:52:01.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.655+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3168447746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e57a0 msgr2=0x56495d1f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.655+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/3168447746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e57a0 0x56495d1f71a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fca1c096380 tx=0x7fca1c0cc040 comp rx=0 tx=0).stop 2023-10-08T13:52:01.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.655+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3168447746 shutdown_connections 2023-10-08T13:52:01.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.655+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/3168447746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.655+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/3168447746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e4e80 0x56495d1e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.655+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/3168447746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e57a0 0x56495d1f71a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.655+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3168447746 >> 172.21.15.57:0/3168447746 conn(0x56495d15a260 msgr2=0x56495d1ebd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.655+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3168447746 shutdown_connections 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.655+0000 7fca39aa7b80 1 -- 172.21.15.57:0/3168447746 wait complete. 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.655+0000 7fca39aa7b80 1 -- start start 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca39aa7b80 1 Processor -- start 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e57a0 0x56495d1f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca39aa7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d1f6250 con 0x56495d1e4e80 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d1f63c0 con 0x56495d1e57a0 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca39aa7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56495d253ef0 con 0x56495d1f0650 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca39aa7b80 10 client.0 _ll_register_callbacks cb 0x7fca18003950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:01.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca26ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca26ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56604/0 (socket says 172.21.15.57:56604) 2023-10-08T13:52:01.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca26ffd700 1 -- 172.21.15.57:0/898514855 learned_addr learned my addr 172.21.15.57:0/898514855 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca277fe700 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca267fc700 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e57a0 0x56495d1f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca26ffd700 1 -- 172.21.15.57:0/898514855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 msgr2=0x56495d1f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca26ffd700 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca26ffd700 1 -- 172.21.15.57:0/898514855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e57a0 msgr2=0x56495d1f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca26ffd700 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e57a0 0x56495d1f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.656+0000 7fca26ffd700 1 -- 172.21.15.57:0/898514855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca281a5980 con 0x56495d1e4e80 2023-10-08T13:52:01.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca26ffd700 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f7030 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fca1800b4f0 tx=0x7fca18005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81629eefddd8cffd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca06ffd700 1 -- 172.21.15.57:0/898514855 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1802a020 con 0x56495d1e4e80 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca06ffd700 1 -- 172.21.15.57:0/898514855 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fca18017070 con 0x56495d1e4e80 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca39aa7b80 1 -- 172.21.15.57:0/898514855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56495d100eb0 con 0x56495d1e4e80 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca06ffd700 1 -- 172.21.15.57:0/898514855 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca18011ab0 con 0x56495d1e4e80 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca39aa7b80 10 client.15960 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca39aa7b80 1 -- 172.21.15.57:0/898514855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56495cf622b0 con 0x56495d1e4e80 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca06ffd700 1 -- 172.21.15.57:0/898514855 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56495d100eb0 con 0x56495d1e4e80 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca06ffd700 1 -- 172.21.15.57:0/898514855 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fca1802fcd0 con 0x56495d1e4e80 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca39aa7b80 10 client.15960 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca39aa7b80 10 client.15960 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-08T13:52:01.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca39aa7b80 20 client.15960 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56495d255360 0x56495d257750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.657+0000 7fca39aa7b80 4 client.15960 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:52:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca39aa7b80 1 -- 172.21.15.57:0/898514855 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56495cf84f70 con 0x56495d255360 2023-10-08T13:52:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca267fc700 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56495d255360 0x56495d257750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca267fc700 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56495d255360 0x56495d257750 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e19dde4b0081cca server_cookie=41c7930ea3f237c3 in_seq=0 out_seq=0 2023-10-08T13:52:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca06ffd700 10 client.15960 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:52:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca06ffd700 1 -- 172.21.15.57:0/898514855 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56495cf84f70 con 0x56495d255360 2023-10-08T13:52:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca06ffd700 10 client.15960 handle_command_reply: tid=0 2023-10-08T13:52:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca39aa7b80 1 client.15960 shutdown 2023-10-08T13:52:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca39aa7b80 1 -- 172.21.15.57:0/898514855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 msgr2=0x56495d1f7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f7030 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fca1800b4f0 tx=0x7fca18005a00 comp rx=0 tx=0).stop 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca39aa7b80 1 -- 172.21.15.57:0/898514855 shutdown_connections 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56495d255360 0x56495d257750 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56495d1f0650 0x56495d1f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.658+0000 7fca06ffd700 0 client.15960 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56495d1e4e80 0x56495d1f7030 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca39aa7b80 1 --2- 172.21.15.57:0/898514855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56495d1e57a0 0x56495d1f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca39aa7b80 1 -- 172.21.15.57:0/898514855 >> 172.21.15.57:0/898514855 conn(0x56495d15a260 msgr2=0x56495d1ec970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca39aa7b80 1 -- 172.21.15.57:0/898514855 shutdown_connections 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca39aa7b80 1 -- 172.21.15.57:0/898514855 wait complete. 2023-10-08T13:52:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca39aa7b80 20 client.15960 trim_cache size 0 max 16384 2023-10-08T13:52:01.668 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:52:01.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 -- 172.21.15.57:0/1628643160 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca100a1670 msgr2=0x7fca100a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca100a1670 0x7fca100a3b30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fca28077ab0 tx=0x7fca1800b410 comp rx=0 tx=0).stop 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 -- 172.21.15.57:0/1628643160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca280fffe0 msgr2=0x7fca2819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca280fffe0 0x7fca2819eed0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00d980 tx=0x7fca1c00dc90 comp rx=0 tx=0).stop 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 -- 172.21.15.57:0/1628643160 shutdown_connections 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca281056b0 0x7fca28198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fca100a1670 0x7fca100a3b30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280ff620 0x7fca2819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 --2- 172.21.15.57:0/1628643160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fca280fffe0 0x7fca2819eed0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 -- 172.21.15.57:0/1628643160 >> 172.21.15.57:0/1628643160 conn(0x7fca28076780 msgr2=0x7fca28067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 -- 172.21.15.57:0/1628643160 shutdown_connections 2023-10-08T13:52:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.659+0000 7fca2d8be700 1 -- 172.21.15.57:0/1628643160 wait complete. 2023-10-08T13:52:01.673 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-08T13:52:01.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.802+0000 7febd5646700 1 -- 172.21.15.57:0/3859154576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febd0067ce0 msgr2=0x7febd00680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.802+0000 7febd5646700 1 --2- 172.21.15.57:0/3859154576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febd0067ce0 0x7febd00680c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7febc00099c0 tx=0x7febc0009cd0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.802+0000 7febd5646700 1 -- 172.21.15.57:0/3859154576 shutdown_connections 2023-10-08T13:52:01.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.802+0000 7febd5646700 1 --2- 172.21.15.57:0/3859154576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd0105b80 0x7febd010d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.802+0000 7febd5646700 1 --2- 172.21.15.57:0/3859154576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febd0067ce0 0x7febd00680c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.802+0000 7febd5646700 1 --2- 172.21.15.57:0/3859154576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd01051c0 0x7febd0105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.802+0000 7febd5646700 1 -- 172.21.15.57:0/3859154576 >> 172.21.15.57:0/3859154576 conn(0x7febd0076780 msgr2=0x7febd0076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.802+0000 7febd5646700 1 -- 172.21.15.57:0/3859154576 shutdown_connections 2023-10-08T13:52:01.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.802+0000 7febd5646700 1 -- 172.21.15.57:0/3859154576 wait complete. 2023-10-08T13:52:01.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febd5646700 1 Processor -- start 2023-10-08T13:52:01.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febd5646700 1 -- start start 2023-10-08T13:52:01.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febd5646700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febd0067ce0 0x7febd0198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febd5646700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd01051c0 0x7febd0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febd5646700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0105b80 0x7febd019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febd5646700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd019dd30 con 0x7febd0067ce0 2023-10-08T13:52:01.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febd5646700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd019dea0 con 0x7febd0105b80 2023-10-08T13:52:01.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febd5646700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd019e060 con 0x7febd01051c0 2023-10-08T13:52:01.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febd0067ce0 0x7febd0198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febd0067ce0 0x7febd0198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56618/0 (socket says 172.21.15.57:56618) 2023-10-08T13:52:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febceffd700 1 -- 172.21.15.57:0/1667511520 learned_addr learned my addr 172.21.15.57:0/1667511520 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febcf7fe700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0105b80 0x7febd019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.803+0000 7febce7fc700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd01051c0 0x7febd0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febceffd700 1 -- 172.21.15.57:0/1667511520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd01051c0 msgr2=0x7febd0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febceffd700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd01051c0 0x7febd0199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febceffd700 1 -- 172.21.15.57:0/1667511520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0105b80 msgr2=0x7febd019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febceffd700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0105b80 0x7febd019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febceffd700 1 -- 172.21.15.57:0/1667511520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc00095f0 con 0x7febd0067ce0 2023-10-08T13:52:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febceffd700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febd0067ce0 0x7febd0198e10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7febc000b580 tx=0x7febc000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febb7fff700 1 -- 172.21.15.57:0/1667511520 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc001c070 con 0x7febd0067ce0 2023-10-08T13:52:01.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febb7fff700 1 -- 172.21.15.57:0/1667511520 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc000fd80 con 0x7febd0067ce0 2023-10-08T13:52:01.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febb7fff700 1 -- 172.21.15.57:0/1667511520 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0022c40 con 0x7febd0067ce0 2023-10-08T13:52:01.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febd5646700 1 -- 172.21.15.57:0/1667511520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd019e2e0 con 0x7febd0067ce0 2023-10-08T13:52:01.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.804+0000 7febd5646700 1 -- 172.21.15.57:0/1667511520 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd019e560 con 0x7febd0067ce0 2023-10-08T13:52:01.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.807+0000 7febb7fff700 1 -- 172.21.15.57:0/1667511520 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7febc0020070 con 0x7febd0067ce0 2023-10-08T13:52:01.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.807+0000 7febb7fff700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7febb80a15f0 0x7febb80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febb7fff700 1 -- 172.21.15.57:0/1667511520 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7febc00ce140 con 0x7febd0067ce0 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febce7fc700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7febb80a15f0 0x7febb80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febe182fb80 1 Processor -- start 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febce7fc700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7febb80a15f0 0x7febb80a3ab0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7febd019a360 tx=0x7febc400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febe182fb80 1 -- start start 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febe182fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febe182fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369bb7a0 0x55a7369c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febe182fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febe182fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a736924460 con 0x55a7369c6650 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febe182fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a736915c20 con 0x55a7369bb7a0 2023-10-08T13:52:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.808+0000 7febe182fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a736913540 con 0x55a7369bae80 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56630/0 (socket says 172.21.15.57:56630) 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febcf7fe700 1 -- 172.21.15.57:0/3539956670 learned_addr learned my addr 172.21.15.57:0/3539956670 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febce7fc700 1 --2- 172.21.15.57:0/3539956670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369bb7a0 0x55a7369c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febceffd700 1 --2- 172.21.15.57:0/3539956670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febcf7fe700 1 -- 172.21.15.57:0/3539956670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 msgr2=0x55a7369bb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febcf7fe700 1 --2- 172.21.15.57:0/3539956670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369bb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febcf7fe700 1 -- 172.21.15.57:0/3539956670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369bb7a0 msgr2=0x55a7369c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febcf7fe700 1 --2- 172.21.15.57:0/3539956670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369bb7a0 0x55a7369c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febcf7fe700 1 -- 172.21.15.57:0/3539956670 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a736739de0 con 0x55a7369c6650 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febcf7fe700 1 --2- 172.21.15.57:0/3539956670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c8a40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7febc8011680 tx=0x7febc8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29af813ddbdec3fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febb57fa700 1 -- 172.21.15.57:0/3539956670 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc800e040 con 0x55a7369c6650 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febb57fa700 1 -- 172.21.15.57:0/3539956670 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc800f040 con 0x55a7369c6650 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.809+0000 7febb57fa700 1 -- 172.21.15.57:0/3539956670 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8004850 con 0x55a7369c6650 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.810+0000 7febe182fb80 1 -- 172.21.15.57:0/3539956670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 msgr2=0x55a7369c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.810+0000 7febe182fb80 1 --2- 172.21.15.57:0/3539956670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c8a40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7febc8011680 tx=0x7febc8009920 comp rx=0 tx=0).stop 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.810+0000 7febe182fb80 1 -- 172.21.15.57:0/3539956670 shutdown_connections 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.810+0000 7febe182fb80 1 --2- 172.21.15.57:0/3539956670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c8a40 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.810+0000 7febe182fb80 1 --2- 172.21.15.57:0/3539956670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369bb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.810+0000 7febe182fb80 1 --2- 172.21.15.57:0/3539956670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369bb7a0 0x55a7369c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.810+0000 7febe182fb80 1 -- 172.21.15.57:0/3539956670 >> 172.21.15.57:0/3539956670 conn(0x55a736930260 msgr2=0x55a7369b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.810+0000 7febe182fb80 1 -- 172.21.15.57:0/3539956670 shutdown_connections 2023-10-08T13:52:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.810+0000 7febe182fb80 1 -- 172.21.15.57:0/3539956670 wait complete. 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febe182fb80 1 -- start start 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febe182fb80 1 Processor -- start 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febe182fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bae80 0x55a7369c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febe182fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bb7a0 0x55a7369c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febe182fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 0x55a7369c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febe182fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73690ee30 con 0x55a7369bae80 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febe182fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7368ab710 con 0x55a7369c6650 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febe182fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7368ab3b0 con 0x55a7369bb7a0 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febe182fb80 10 client.0 _ll_register_callbacks cb 0x55a736708800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bae80 0x55a7369c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bae80 0x55a7369c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56632/0 (socket says 172.21.15.57:56632) 2023-10-08T13:52:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febceffd700 1 -- 172.21.15.57:0/4085737239 learned_addr learned my addr 172.21.15.57:0/4085737239 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febce7fc700 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bb7a0 0x55a7369c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febcf7fe700 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 0x55a7369c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febceffd700 1 -- 172.21.15.57:0/4085737239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bb7a0 msgr2=0x55a7369c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febceffd700 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bb7a0 0x55a7369c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febceffd700 1 -- 172.21.15.57:0/4085737239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 msgr2=0x55a7369c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febceffd700 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 0x55a7369c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febceffd700 1 -- 172.21.15.57:0/4085737239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc8011330 con 0x55a7369bae80 2023-10-08T13:52:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.811+0000 7febcf7fe700 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 0x55a7369c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febceffd700 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bae80 0x55a7369c5fc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7febc0097920 tx=0x7febc009b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785459c57253c743 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febb2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febb2ff5700 1 -- 172.21.15.57:0/4085737239 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc009f000 con 0x55a7369bae80 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febb2ff5700 1 -- 172.21.15.57:0/4085737239 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7febc00a31b0 con 0x55a7369bae80 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febe182fb80 1 -- 172.21.15.57:0/4085737239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7368d6eb0 con 0x55a7369bae80 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febb2ff5700 1 -- 172.21.15.57:0/4085737239 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7febc0022420 con 0x55a7369bae80 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febe182fb80 10 client.15972 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febe182fb80 1 -- 172.21.15.57:0/4085737239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a736722c60 con 0x55a7369bae80 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febb2ff5700 1 -- 172.21.15.57:0/4085737239 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc0064080 con 0x55a7369bae80 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febb2ff5700 1 -- 172.21.15.57:0/4085737239 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7febc00a2d20 con 0x55a7369bae80 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febe182fb80 10 client.15972 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febe182fb80 10 client.15972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febe182fb80 20 client.15972 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febe182fb80 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55a7369d5cb0 0x55a7369d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febe182fb80 4 client.15972 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.812+0000 7febe182fb80 1 -- 172.21.15.57:0/4085737239 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a736907e10 con 0x55a7369d5cb0 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.813+0000 7febce7fc700 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55a7369d5cb0 0x55a7369d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.813+0000 7febb2ff5700 10 client.15972 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.813+0000 7febce7fc700 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55a7369d5cb0 0x55a7369d80d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23849a351f3e5a0d server_cookie=755b79453b815f37 in_seq=0 out_seq=0 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febb2ff5700 1 -- 172.21.15.57:0/4085737239 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a736907e10 con 0x55a7369d5cb0 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febb2ff5700 10 client.15972 handle_command_reply: tid=0 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 client.15972 shutdown 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 -- 172.21.15.57:0/4085737239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bae80 msgr2=0x55a7369c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bae80 0x55a7369c5fc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7febc0097920 tx=0x7febc009b5b0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 -- 172.21.15.57:0/4085737239 shutdown_connections 2023-10-08T13:52:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55a7369d5cb0 0x55a7369d80d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 0x55a7369c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bae80 0x55a7369c5fc0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febb2ff5700 0 client.15972 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:52:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 --2- 172.21.15.57:0/4085737239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bb7a0 0x55a7369c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 -- 172.21.15.57:0/4085737239 >> 172.21.15.57:0/4085737239 conn(0x55a736930260 msgr2=0x55a7369b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 -- 172.21.15.57:0/4085737239 shutdown_connections 2023-10-08T13:52:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 1 -- 172.21.15.57:0/4085737239 wait complete. 2023-10-08T13:52:01.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.814+0000 7febe182fb80 20 client.15972 trim_cache size 0 max 16384 2023-10-08T13:52:01.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.827+0000 7febe182fb80 1 Processor -- start 2023-10-08T13:52:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.827+0000 7febe182fb80 1 -- start start 2023-10-08T13:52:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febe182fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febe182fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369bb7a0 0x55a7369d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febe182fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febe182fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73678f920 con 0x55a7369c6650 2023-10-08T13:52:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febe182fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7369d8760 con 0x55a7369bb7a0 2023-10-08T13:52:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febe182fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7369c1800 con 0x55a7369bae80 2023-10-08T13:52:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56644/0 (socket says 172.21.15.57:56644) 2023-10-08T13:52:01.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febcf7fe700 1 -- 172.21.15.57:0/1218013932 learned_addr learned my addr 172.21.15.57:0/1218013932 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febce7fc700 1 --2- 172.21.15.57:0/1218013932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369bb7a0 0x55a7369d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febceffd700 1 --2- 172.21.15.57:0/1218013932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febcf7fe700 1 -- 172.21.15.57:0/1218013932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 msgr2=0x55a7369d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febcf7fe700 1 --2- 172.21.15.57:0/1218013932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369d18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febcf7fe700 1 -- 172.21.15.57:0/1218013932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369bb7a0 msgr2=0x55a7369d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febcf7fe700 1 --2- 172.21.15.57:0/1218013932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369bb7a0 0x55a7369d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febcf7fe700 1 -- 172.21.15.57:0/1218013932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a736760250 con 0x55a7369c6650 2023-10-08T13:52:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febcf7fe700 1 --2- 172.21.15.57:0/1218013932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c0510 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7febc8011620 tx=0x7febc8009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c73ef028d1155e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febb5ffb700 1 -- 172.21.15.57:0/1218013932 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc800e050 con 0x55a7369c6650 2023-10-08T13:52:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febb5ffb700 1 -- 172.21.15.57:0/1218013932 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc8005110 con 0x55a7369c6650 2023-10-08T13:52:01.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.828+0000 7febb5ffb700 1 -- 172.21.15.57:0/1218013932 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8009420 con 0x55a7369c6650 2023-10-08T13:52:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.829+0000 7febe182fb80 1 -- 172.21.15.57:0/1218013932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 msgr2=0x55a7369c0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.829+0000 7febe182fb80 1 --2- 172.21.15.57:0/1218013932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c0510 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7febc8011620 tx=0x7febc8009dd0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.829+0000 7febe182fb80 1 -- 172.21.15.57:0/1218013932 shutdown_connections 2023-10-08T13:52:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.829+0000 7febe182fb80 1 --2- 172.21.15.57:0/1218013932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369c6650 0x55a7369c0510 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.829+0000 7febe182fb80 1 --2- 172.21.15.57:0/1218013932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.829+0000 7febe182fb80 1 --2- 172.21.15.57:0/1218013932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369bb7a0 0x55a7369d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.829+0000 7febe182fb80 1 -- 172.21.15.57:0/1218013932 >> 172.21.15.57:0/1218013932 conn(0x55a736930260 msgr2=0x55a7369c1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.829+0000 7febe182fb80 1 -- 172.21.15.57:0/1218013932 shutdown_connections 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.829+0000 7febe182fb80 1 -- 172.21.15.57:0/1218013932 wait complete. 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febe182fb80 1 -- start start 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febe182fb80 1 Processor -- start 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febe182fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febe182fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bb7a0 0x55a7369cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febe182fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 0x55a7369cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febe182fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7369cbef0 con 0x55a7369bb7a0 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febe182fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7369cc030 con 0x55a7369c6650 2023-10-08T13:52:01.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febe182fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7369cc1f0 con 0x55a7369bae80 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febe182fb80 10 client.0 _ll_register_callbacks cb 0x7febc009ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febce7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bb7a0 0x55a7369cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febce7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bb7a0 0x55a7369cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56648/0 (socket says 172.21.15.57:56648) 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febce7fc700 1 -- 172.21.15.57:0/3188090471 learned_addr learned my addr 172.21.15.57:0/3188090471 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febcf7fe700 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 0x55a7369cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.830+0000 7febceffd700 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febce7fc700 1 -- 172.21.15.57:0/3188090471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 msgr2=0x55a7369ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febce7fc700 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febce7fc700 1 -- 172.21.15.57:0/3188090471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 msgr2=0x55a7369cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febce7fc700 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 0x55a7369cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febce7fc700 1 -- 172.21.15.57:0/3188090471 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a736739de0 con 0x55a7369bb7a0 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febcf7fe700 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 0x55a7369cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febceffd700 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febce7fc700 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bb7a0 0x55a7369cb470 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55a7369c47f0 tx=0x7febc40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f55ed6bc762bd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febb2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febb2ff5700 1 -- 172.21.15.57:0/3188090471 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4026250 con 0x55a7369bb7a0 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febb2ff5700 1 -- 172.21.15.57:0/3188090471 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7febc4017070 con 0x55a7369bb7a0 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febe182fb80 1 -- 172.21.15.57:0/3188090471 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febc800f040 con 0x55a7369bb7a0 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febb2ff5700 1 -- 172.21.15.57:0/3188090471 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7febc40263b0 con 0x55a7369bb7a0 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febe182fb80 10 client.15978 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febe182fb80 1 -- 172.21.15.57:0/3188090471 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a7367382b0 con 0x55a7369bb7a0 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.831+0000 7febb2ff5700 1 -- 172.21.15.57:0/3188090471 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc800f040 con 0x55a7369bb7a0 2023-10-08T13:52:01.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febb2ff5700 1 -- 172.21.15.57:0/3188090471 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7febc4026b70 con 0x55a7369bb7a0 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febe182fb80 10 client.15978 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febe182fb80 10 client.15978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febe182fb80 20 client.15978 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febe182fb80 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55a736a2b120 0x55a736a2d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febe182fb80 4 client.15978 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febe182fb80 1 -- 172.21.15.57:0/3188090471 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a736907b30 con 0x55a736a2b120 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febceffd700 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55a736a2b120 0x55a736a2d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febceffd700 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55a736a2b120 0x55a736a2d540 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34b668f3146f689c server_cookie=bb2b303a342cad97 in_seq=0 out_seq=0 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febb2ff5700 10 client.15978 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febb2ff5700 1 -- 172.21.15.57:0/3188090471 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a736907b30 con 0x55a736a2b120 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febb2ff5700 10 client.15978 handle_command_reply: tid=0 2023-10-08T13:52:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.832+0000 7febe182fb80 1 client.15978 shutdown 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 1 -- 172.21.15.57:0/3188090471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bb7a0 msgr2=0x55a7369cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bb7a0 0x55a7369cb470 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55a7369c47f0 tx=0x7febc40047c0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 1 -- 172.21.15.57:0/3188090471 shutdown_connections 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55a736a2b120 0x55a736a2d540 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7369c6650 0x55a7369cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7369bae80 0x55a7369ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 1 --2- 172.21.15.57:0/3188090471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7369bb7a0 0x55a7369cb470 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febb2ff5700 0 client.15978 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 1 -- 172.21.15.57:0/3188090471 >> 172.21.15.57:0/3188090471 conn(0x55a736930260 msgr2=0x55a7369d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 1 -- 172.21.15.57:0/3188090471 shutdown_connections 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 1 -- 172.21.15.57:0/3188090471 wait complete. 2023-10-08T13:52:01.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febe182fb80 20 client.15978 trim_cache size 0 max 16384 2023-10-08T13:52:01.842 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:52:01.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febd5646700 1 -- 172.21.15.57:0/1667511520 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7febb80a15f0 msgr2=0x7febb80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febd5646700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7febb80a15f0 0x7febb80a3ab0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7febd019a360 tx=0x7febc400b410 comp rx=0 tx=0).stop 2023-10-08T13:52:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febd5646700 1 -- 172.21.15.57:0/1667511520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febd0067ce0 msgr2=0x7febd0198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.833+0000 7febd5646700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febd0067ce0 0x7febd0198e10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7febc000b580 tx=0x7febc000f690 comp rx=0 tx=0).stop 2023-10-08T13:52:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.834+0000 7febd5646700 1 -- 172.21.15.57:0/1667511520 shutdown_connections 2023-10-08T13:52:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.834+0000 7febd5646700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7febb80a15f0 0x7febb80a3ab0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.834+0000 7febd5646700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0105b80 0x7febd019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.834+0000 7febd5646700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7febd0067ce0 0x7febd0198e10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.834+0000 7febd5646700 1 --2- 172.21.15.57:0/1667511520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd01051c0 0x7febd0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.834+0000 7febd5646700 1 -- 172.21.15.57:0/1667511520 >> 172.21.15.57:0/1667511520 conn(0x7febd0076780 msgr2=0x7febd00fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.834+0000 7febd5646700 1 -- 172.21.15.57:0/1667511520 shutdown_connections 2023-10-08T13:52:01.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.834+0000 7febd5646700 1 -- 172.21.15.57:0/1667511520 wait complete. 2023-10-08T13:52:01.848 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-08T13:52:01.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 -- 172.21.15.57:0/848397293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf4c0ff620 msgr2=0x7fdf4c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 --2- 172.21.15.57:0/848397293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf4c0ff620 0x7fdf4c0ffaa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009990 tx=0x7fdf3c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 -- 172.21.15.57:0/848397293 shutdown_connections 2023-10-08T13:52:01.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 --2- 172.21.15.57:0/848397293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf4c0fffe0 0x7fdf4c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 --2- 172.21.15.57:0/848397293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4c1056b0 0x7fdf4c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 --2- 172.21.15.57:0/848397293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf4c0ff620 0x7fdf4c0ffaa0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 -- 172.21.15.57:0/848397293 >> 172.21.15.57:0/848397293 conn(0x7fdf4c076780 msgr2=0x7fdf4c076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 -- 172.21.15.57:0/848397293 shutdown_connections 2023-10-08T13:52:01.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 -- 172.21.15.57:0/848397293 wait complete. 2023-10-08T13:52:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 Processor -- start 2023-10-08T13:52:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.979+0000 7fdf53617700 1 -- start start 2023-10-08T13:52:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf53617700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf4c0ff620 0x7fdf4c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf53617700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4c0fffe0 0x7fdf4c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf53617700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf4c1056b0 0x7fdf4c196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf53617700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c196fc0 con 0x7fdf4c0ff620 2023-10-08T13:52:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf53617700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c197130 con 0x7fdf4c0fffe0 2023-10-08T13:52:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf53617700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c1972f0 con 0x7fdf4c1056b0 2023-10-08T13:52:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf513b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf4c0ff620 0x7fdf4c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf513b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf4c0ff620 0x7fdf4c19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56660/0 (socket says 172.21.15.57:56660) 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf513b3700 1 -- 172.21.15.57:0/2418519953 learned_addr learned my addr 172.21.15.57:0/2418519953 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf50bb2700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4c0fffe0 0x7fdf4c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf4c1056b0 0x7fdf4c196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf513b3700 1 -- 172.21.15.57:0/2418519953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf4c1056b0 msgr2=0x7fdf4c196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf513b3700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf4c1056b0 0x7fdf4c196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf513b3700 1 -- 172.21.15.57:0/2418519953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4c0fffe0 msgr2=0x7fdf4c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf513b3700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4c0fffe0 0x7fdf4c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf513b3700 1 -- 172.21.15.57:0/2418519953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0095f0 con 0x7fdf4c0ff620 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf4c1056b0 0x7fdf4c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf513b3700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf4c0ff620 0x7fdf4c19c830 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800d980 tx=0x7fdf4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.980+0000 7fdf427fc700 1 -- 172.21.15.57:0/2418519953 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800b950 con 0x7fdf4c0ff620 2023-10-08T13:52:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.981+0000 7fdf427fc700 1 -- 172.21.15.57:0/2418519953 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf48007620 con 0x7fdf4c0ff620 2023-10-08T13:52:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.981+0000 7fdf53617700 1 -- 172.21.15.57:0/2418519953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c1975d0 con 0x7fdf4c0ff620 2023-10-08T13:52:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.981+0000 7fdf427fc700 1 -- 172.21.15.57:0/2418519953 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800f3c0 con 0x7fdf4c0ff620 2023-10-08T13:52:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.981+0000 7fdf53617700 1 -- 172.21.15.57:0/2418519953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c1a3820 con 0x7fdf4c0ff620 2023-10-08T13:52:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.983+0000 7fdf427fc700 1 -- 172.21.15.57:0/2418519953 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdf48010640 con 0x7fdf4c0ff620 2023-10-08T13:52:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.984+0000 7fdf427fc700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdf380a1670 0x7fdf380a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.984+0000 7fdf427fc700 1 -- 172.21.15.57:0/2418519953 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdf480cd190 con 0x7fdf4c0ff620 2023-10-08T13:52:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.984+0000 7fdf50bb2700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdf380a1670 0x7fdf380a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.984+0000 7fdf50bb2700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdf380a1670 0x7fdf380a3b30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00b5c0 tx=0x7fdf3c005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.985+0000 7fdf5f800b80 1 Processor -- start 2023-10-08T13:52:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.985+0000 7fdf5f800b80 1 -- start start 2023-10-08T13:52:01.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.985+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353db260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.985+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353e6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.985+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 0x5592353e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.985+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559235344460 con 0x5592353dae80 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.985+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559235335c20 con 0x5592353e6650 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.985+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559235333540 con 0x5592353db7a0 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.985+0000 7fdf513b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353db260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.985+0000 7fdf513b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353db260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56668/0 (socket says 172.21.15.57:56668) 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf513b3700 1 -- 172.21.15.57:0/3313219909 learned_addr learned my addr 172.21.15.57:0/3313219909 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf50bb2700 1 --2- 172.21.15.57:0/3313219909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/3313219909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 0x5592353e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf513b3700 1 -- 172.21.15.57:0/3313219909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 msgr2=0x5592353e6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf513b3700 1 --2- 172.21.15.57:0/3313219909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353e6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf513b3700 1 -- 172.21.15.57:0/3313219909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 msgr2=0x5592353e8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf513b3700 1 --2- 172.21.15.57:0/3313219909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 0x5592353e8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf513b3700 1 -- 172.21.15.57:0/3313219909 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559235159de0 con 0x5592353dae80 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/3313219909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 0x5592353e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf513b3700 1 --2- 172.21.15.57:0/3313219909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353db260 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdf48096370 tx=0x7fdf48010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94120c59898bc1cc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf377fe700 1 -- 172.21.15.57:0/3313219909 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48099e70 con 0x5592353dae80 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf377fe700 1 -- 172.21.15.57:0/3313219909 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf4809dfc0 con 0x5592353dae80 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.986+0000 7fdf377fe700 1 -- 172.21.15.57:0/3313219909 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48099360 con 0x5592353dae80 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.987+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3313219909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 msgr2=0x5592353db260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.987+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/3313219909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353db260 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdf48096370 tx=0x7fdf48010530 comp rx=0 tx=0).stop 2023-10-08T13:52:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.987+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3313219909 shutdown_connections 2023-10-08T13:52:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.987+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/3313219909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 0x5592353e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.987+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/3313219909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353db260 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.987+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/3313219909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.987+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3313219909 >> 172.21.15.57:0/3313219909 conn(0x559235350260 msgr2=0x5592353d72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.987+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3313219909 shutdown_connections 2023-10-08T13:52:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.987+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3313219909 wait complete. 2023-10-08T13:52:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.987+0000 7fdf5f800b80 1 -- start start 2023-10-08T13:52:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf5f800b80 1 Processor -- start 2023-10-08T13:52:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353dae80 0x5592353e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353db7a0 0x5592353e32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x5592353e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923532ee30 con 0x5592353e6650 2023-10-08T13:52:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592352cb710 con 0x5592353db7a0 2023-10-08T13:52:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592352cb3b0 con 0x5592353dae80 2023-10-08T13:52:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf5f800b80 10 client.0 _ll_register_callbacks cb 0x559235128800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf51bb4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x5592353e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf51bb4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x5592353e3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56678/0 (socket says 172.21.15.57:56678) 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf51bb4700 1 -- 172.21.15.57:0/3706255603 learned_addr learned my addr 172.21.15.57:0/3706255603 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf513b3700 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353dae80 0x5592353e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.988+0000 7fdf50bb2700 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353db7a0 0x5592353e32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf51bb4700 1 -- 172.21.15.57:0/3706255603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353dae80 msgr2=0x5592353e5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353dae80 0x5592353e5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf51bb4700 1 -- 172.21.15.57:0/3706255603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353db7a0 msgr2=0x5592353e32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353db7a0 0x5592353e32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf51bb4700 1 -- 172.21.15.57:0/3706255603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c1a3820 con 0x5592353e6650 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x5592353e3890 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdf44007af0 tx=0x7fdf44007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8745222b7c1b72 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf34ff9700 1 -- 172.21.15.57:0/3706255603 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400fa50 con 0x5592353e6650 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3706255603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592352f6eb0 con 0x5592353e6650 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf34ff9700 1 -- 172.21.15.57:0/3706255603 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdf4400fbb0 con 0x5592353e6650 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf34ff9700 1 -- 172.21.15.57:0/3706255603 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf44009e60 con 0x5592353e6650 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf5f800b80 10 client.15990 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3706255603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559235142c60 con 0x5592353e6650 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.989+0000 7fdf34ff9700 1 -- 172.21.15.57:0/3706255603 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592352f6eb0 con 0x5592353e6650 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.990+0000 7fdf34ff9700 1 -- 172.21.15.57:0/3706255603 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf4401c3a0 con 0x5592353e6650 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.990+0000 7fdf5f800b80 10 client.15990 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.990+0000 7fdf5f800b80 10 client.15990 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.990+0000 7fdf5f800b80 20 client.15990 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.990+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5592353f60f0 0x5592353f8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.990+0000 7fdf5f800b80 4 client.15990 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.990+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3706255603 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559235327e10 con 0x5592353f60f0 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.990+0000 7fdf513b3700 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5592353f60f0 0x5592353f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.990+0000 7fdf34ff9700 10 client.15990 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.990+0000 7fdf513b3700 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5592353f60f0 0x5592353f8510 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c299af8be54478ec server_cookie=cb5f12ef702f68d4 in_seq=0 out_seq=0 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf34ff9700 1 -- 172.21.15.57:0/3706255603 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559235327e10 con 0x5592353f60f0 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf34ff9700 10 client.15990 handle_command_reply: tid=0 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf5f800b80 1 client.15990 shutdown 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3706255603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 msgr2=0x5592353e3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x5592353e3890 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdf44007af0 tx=0x7fdf44007e00 comp rx=0 tx=0).stop 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3706255603 shutdown_connections 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x5592353f60f0 0x5592353f8510 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x5592353e3890 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353dae80 0x5592353e5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf34ff9700 0 client.15990 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:52:01.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/3706255603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353db7a0 0x5592353e32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:01.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.991+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3706255603 >> 172.21.15.57:0/3706255603 conn(0x559235350260 msgr2=0x5592353d6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:01.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.992+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3706255603 shutdown_connections 2023-10-08T13:52:01.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.992+0000 7fdf5f800b80 1 -- 172.21.15.57:0/3706255603 wait complete. 2023-10-08T13:52:01.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:01.992+0000 7fdf5f800b80 20 client.15990 trim_cache size 0 max 16384 2023-10-08T13:52:02.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf5f800b80 1 Processor -- start 2023-10-08T13:52:02.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf5f800b80 1 -- start start 2023-10-08T13:52:02.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353e3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353ed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 0x5592353e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592351af920 con 0x5592353dae80 2023-10-08T13:52:02.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592353d3b40 con 0x5592353e6650 2023-10-08T13:52:02.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592353d80e0 con 0x5592353db7a0 2023-10-08T13:52:02.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf513b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf513b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353e3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56682/0 (socket says 172.21.15.57:56682) 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf513b3700 1 -- 172.21.15.57:0/4065710071 learned_addr learned my addr 172.21.15.57:0/4065710071 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf50bb2700 1 --2- 172.21.15.57:0/4065710071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353ed0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/4065710071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 0x5592353e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf513b3700 1 -- 172.21.15.57:0/4065710071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 msgr2=0x5592353ed0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf513b3700 1 --2- 172.21.15.57:0/4065710071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353ed0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf513b3700 1 -- 172.21.15.57:0/4065710071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 msgr2=0x5592353e1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf513b3700 1 --2- 172.21.15.57:0/4065710071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 0x5592353e1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.005+0000 7fdf513b3700 1 -- 172.21.15.57:0/4065710071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592353d67a0 con 0x5592353dae80 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.006+0000 7fdf513b3700 1 --2- 172.21.15.57:0/4065710071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353e3840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800d950 tx=0x7fdf4809e130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d91401a1fcbe035 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.006+0000 7fdf37fff700 1 -- 172.21.15.57:0/4065710071 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4807c070 con 0x5592353dae80 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.006+0000 7fdf37fff700 1 -- 172.21.15.57:0/4065710071 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf48099ef0 con 0x5592353dae80 2023-10-08T13:52:02.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.006+0000 7fdf37fff700 1 -- 172.21.15.57:0/4065710071 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480994e0 con 0x5592353dae80 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.006+0000 7fdf5f800b80 1 -- 172.21.15.57:0/4065710071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 msgr2=0x5592353e3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.006+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/4065710071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353e3840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800d950 tx=0x7fdf4809e130 comp rx=0 tx=0).stop 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.006+0000 7fdf5f800b80 1 -- 172.21.15.57:0/4065710071 shutdown_connections 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.006+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/4065710071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353e6650 0x5592353e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.006+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/4065710071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353dae80 0x5592353e3840 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.006+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/4065710071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353ed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 -- 172.21.15.57:0/4065710071 >> 172.21.15.57:0/4065710071 conn(0x559235350260 msgr2=0x5592353d6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 -- 172.21.15.57:0/4065710071 shutdown_connections 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 -- 172.21.15.57:0/4065710071 wait complete. 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 -- start start 2023-10-08T13:52:02.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 Processor -- start 2023-10-08T13:52:02.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353dae80 0x5592353eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x559235442ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559235443430 con 0x5592353e6650 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559235443570 con 0x5592353dae80 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559235443730 con 0x5592353db7a0 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.007+0000 7fdf5f800b80 10 client.0 _ll_register_callbacks cb 0x7fdf44009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf51bb4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x559235442ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf51bb4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x559235442ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56692/0 (socket says 172.21.15.57:56692) 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf51bb4700 1 -- 172.21.15.57:0/224849791 learned_addr learned my addr 172.21.15.57:0/224849791 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf513b3700 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353dae80 0x5592353eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf50bb2700 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf51bb4700 1 -- 172.21.15.57:0/224849791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 msgr2=0x5592353eb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353eb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf51bb4700 1 -- 172.21.15.57:0/224849791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353dae80 msgr2=0x5592353eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353dae80 0x5592353eaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf51bb4700 1 -- 172.21.15.57:0/224849791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf480b1040 con 0x5592353e6650 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf51bb4700 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x559235442ef0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdf4401c800 tx=0x7fdf44009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13540bce18562906 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf34ff9700 1 -- 172.21.15.57:0/224849791 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4401cd80 con 0x5592353e6650 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf34ff9700 1 -- 172.21.15.57:0/224849791 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdf4402b070 con 0x5592353e6650 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf5f800b80 1 -- 172.21.15.57:0/224849791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf4809dfc0 con 0x5592353e6650 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf34ff9700 1 -- 172.21.15.57:0/224849791 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf4401c520 con 0x5592353e6650 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf5f800b80 10 client.15996 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.008+0000 7fdf5f800b80 1 -- 172.21.15.57:0/224849791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5592353f2aa0 con 0x5592353e6650 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.009+0000 7fdf34ff9700 1 -- 172.21.15.57:0/224849791 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf4809dfc0 con 0x5592353e6650 2023-10-08T13:52:02.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.009+0000 7fdf34ff9700 1 -- 172.21.15.57:0/224849791 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf44014950 con 0x5592353e6650 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.009+0000 7fdf5f800b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.009+0000 7fdf5f800b80 10 client.15996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.009+0000 7fdf5f800b80 20 client.15996 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.009+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55923544af90 0x55923544d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.009+0000 7fdf5f800b80 4 client.15996 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.009+0000 7fdf5f800b80 1 -- 172.21.15.57:0/224849791 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559235327b30 con 0x55923544af90 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.009+0000 7fdf513b3700 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55923544af90 0x55923544d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf34ff9700 10 client.15996 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf513b3700 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55923544af90 0x55923544d3b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5acef3e226a0cab8 server_cookie=19989bcf7dd89463 in_seq=0 out_seq=0 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf34ff9700 1 -- 172.21.15.57:0/224849791 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559235327b30 con 0x55923544af90 2023-10-08T13:52:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf34ff9700 10 client.15996 handle_command_reply: tid=0 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 client.15996 shutdown 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 -- 172.21.15.57:0/224849791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 msgr2=0x559235442ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x559235442ef0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdf4401c800 tx=0x7fdf44009240 comp rx=0 tx=0).stop 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 -- 172.21.15.57:0/224849791 shutdown_connections 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x55923544af90 0x55923544d3b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5592353e6650 0x559235442ef0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592353dae80 0x5592353eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf34ff9700 0 client.15996 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 --2- 172.21.15.57:0/224849791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592353db7a0 0x5592353eb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 -- 172.21.15.57:0/224849791 >> 172.21.15.57:0/224849791 conn(0x559235350260 msgr2=0x5592353f1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:02.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 -- 172.21.15.57:0/224849791 shutdown_connections 2023-10-08T13:52:02.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 1 -- 172.21.15.57:0/224849791 wait complete. 2023-10-08T13:52:02.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.010+0000 7fdf5f800b80 20 client.15996 trim_cache size 0 max 16384 2023-10-08T13:52:02.019 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:52:02.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 -- 172.21.15.57:0/2418519953 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdf380a1670 msgr2=0x7fdf380a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdf380a1670 0x7fdf380a3b30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00b5c0 tx=0x7fdf3c005960 comp rx=0 tx=0).stop 2023-10-08T13:52:02.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 -- 172.21.15.57:0/2418519953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf4c0ff620 msgr2=0x7fdf4c19c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf4c0ff620 0x7fdf4c19c830 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800d980 tx=0x7fdf4800dc90 comp rx=0 tx=0).stop 2023-10-08T13:52:02.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 -- 172.21.15.57:0/2418519953 shutdown_connections 2023-10-08T13:52:02.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf4c1056b0 0x7fdf4c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf4c0ff620 0x7fdf4c19c830 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdf380a1670 0x7fdf380a3b30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 --2- 172.21.15.57:0/2418519953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf4c0fffe0 0x7fdf4c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 -- 172.21.15.57:0/2418519953 >> 172.21.15.57:0/2418519953 conn(0x7fdf4c076780 msgr2=0x7fdf4c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:02.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 -- 172.21.15.57:0/2418519953 shutdown_connections 2023-10-08T13:52:02.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.011+0000 7fdf53617700 1 -- 172.21.15.57:0/2418519953 wait complete. 2023-10-08T13:52:02.024 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-08T13:52:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.155+0000 7f0c4505f700 1 -- 172.21.15.57:0/336892942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c40102940 msgr2=0x7f0c40102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.155+0000 7f0c4505f700 1 --2- 172.21.15.57:0/336892942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c40102940 0x7f0c40102dc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009990 tx=0x7f0c34009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.155+0000 7f0c4505f700 1 -- 172.21.15.57:0/336892942 shutdown_connections 2023-10-08T13:52:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.155+0000 7f0c4505f700 1 --2- 172.21.15.57:0/336892942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40103300 0x7f0c4010f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.155+0000 7f0c4505f700 1 --2- 172.21.15.57:0/336892942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c40108950 0x7f0c40108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.155+0000 7f0c4505f700 1 --2- 172.21.15.57:0/336892942 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c40102940 0x7f0c40102dc0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.155+0000 7f0c4505f700 1 -- 172.21.15.57:0/336892942 >> 172.21.15.57:0/336892942 conn(0x7f0c400fe3d0 msgr2=0x7f0c401007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:02.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.155+0000 7f0c4505f700 1 -- 172.21.15.57:0/336892942 shutdown_connections 2023-10-08T13:52:02.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.155+0000 7f0c4505f700 1 -- 172.21.15.57:0/336892942 wait complete. 2023-10-08T13:52:02.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c4505f700 1 Processor -- start 2023-10-08T13:52:02.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c4505f700 1 -- start start 2023-10-08T13:52:02.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c4505f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c40102940 0x7f0c40198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c4505f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c40103300 0x7f0c401992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c4505f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40108950 0x7f0c4019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c4505f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4019dca0 con 0x7f0c40102940 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c4505f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4019de10 con 0x7f0c40103300 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c4505f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4019dfd0 con 0x7f0c40108950 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c3ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c40102940 0x7f0c40198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c3ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c40102940 0x7f0c40198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56702/0 (socket says 172.21.15.57:56702) 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/1017761216 learned_addr learned my addr 172.21.15.57:0/1017761216 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c40103300 0x7f0c401992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.156+0000 7f0c3f59e700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40108950 0x7f0c4019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/1017761216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40108950 msgr2=0x7f0c4019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40108950 0x7f0c4019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/1017761216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c40103300 msgr2=0x7f0c401992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c40103300 0x7f0c401992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/1017761216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c340095f0 con 0x7f0c40102940 2023-10-08T13:52:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c3f59e700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40108950 0x7f0c4019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c40103300 0x7f0c401992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c40102940 0x7f0c40198d80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000d950 tx=0x7f0c3000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c27fff700 1 -- 172.21.15.57:0/1017761216 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30009940 con 0x7f0c40102940 2023-10-08T13:52:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c27fff700 1 -- 172.21.15.57:0/1017761216 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c30010460 con 0x7f0c40102940 2023-10-08T13:52:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c4505f700 1 -- 172.21.15.57:0/1017761216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4019e2b0 con 0x7f0c40102940 2023-10-08T13:52:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c27fff700 1 -- 172.21.15.57:0/1017761216 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000f620 con 0x7f0c40102940 2023-10-08T13:52:02.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.157+0000 7f0c4505f700 1 -- 172.21.15.57:0/1017761216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c401a5af0 con 0x7f0c40102940 2023-10-08T13:52:02.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.160+0000 7f0c27fff700 1 -- 172.21.15.57:0/1017761216 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0c3000f780 con 0x7f0c40102940 2023-10-08T13:52:02.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.161+0000 7f0c27fff700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c280a15f0 0x7f0c280a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.161+0000 7f0c27fff700 1 -- 172.21.15.57:0/1017761216 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0c300cd210 con 0x7f0c40102940 2023-10-08T13:52:02.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.161+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c280a15f0 0x7f0c280a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.161+0000 7f0c51248b80 1 Processor -- start 2023-10-08T13:52:02.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.161+0000 7f0c51248b80 1 -- start start 2023-10-08T13:52:02.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.161+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c280a15f0 0x7f0c280a3ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c340098e0 tx=0x7f0c34005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:02.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb28f57a0 0x559eb2900110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb2900650 0x559eb2902a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c51248b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb285e460 con 0x559eb28f4e80 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c51248b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb284fc20 con 0x559eb28f57a0 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c51248b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb284d540 con 0x559eb2900650 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c3ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c3ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28f5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56710/0 (socket says 172.21.15.57:56710) 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/3204685829 learned_addr learned my addr 172.21.15.57:0/3204685829 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/3204685829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb28f57a0 0x559eb2900110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/3204685829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb2900650 msgr2=0x559eb2902a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/3204685829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb2900650 0x559eb2902a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/3204685829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb28f57a0 msgr2=0x559eb2900110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/3204685829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb28f57a0 0x559eb2900110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/3204685829 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eb2673de0 con 0x559eb28f4e80 2023-10-08T13:52:02.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/3204685829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28f5260 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0c30096960 tx=0x7f0c30099920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55eeb5e9fe4bfc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:02.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c257fa700 1 -- 172.21.15.57:0/3204685829 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3009e040 con 0x559eb28f4e80 2023-10-08T13:52:02.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c257fa700 1 -- 172.21.15.57:0/3204685829 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c300a2280 con 0x559eb28f4e80 2023-10-08T13:52:02.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.162+0000 7f0c257fa700 1 -- 172.21.15.57:0/3204685829 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3009a3b0 con 0x559eb28f4e80 2023-10-08T13:52:02.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.163+0000 7f0c51248b80 1 -- 172.21.15.57:0/3204685829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 msgr2=0x559eb28f5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.163+0000 7f0c51248b80 1 --2- 172.21.15.57:0/3204685829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28f5260 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0c30096960 tx=0x7f0c30099920 comp rx=0 tx=0).stop 2023-10-08T13:52:02.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.163+0000 7f0c51248b80 1 -- 172.21.15.57:0/3204685829 shutdown_connections 2023-10-08T13:52:02.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.163+0000 7f0c51248b80 1 --2- 172.21.15.57:0/3204685829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb2900650 0x559eb2902a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.163+0000 7f0c51248b80 1 --2- 172.21.15.57:0/3204685829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28f5260 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.163+0000 7f0c51248b80 1 --2- 172.21.15.57:0/3204685829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb28f57a0 0x559eb2900110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.163+0000 7f0c51248b80 1 -- 172.21.15.57:0/3204685829 >> 172.21.15.57:0/3204685829 conn(0x559eb286a260 msgr2=0x559eb28f12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:02.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.163+0000 7f0c51248b80 1 -- 172.21.15.57:0/3204685829 shutdown_connections 2023-10-08T13:52:02.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.163+0000 7f0c51248b80 1 -- 172.21.15.57:0/3204685829 wait complete. 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c51248b80 1 -- start start 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c51248b80 1 Processor -- start 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb28f57a0 0x559eb28fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2900650 0x559eb28fde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c51248b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb2848e30 con 0x559eb28f4e80 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c51248b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb27e5710 con 0x559eb2900650 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c51248b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb27e53b0 con 0x559eb28f57a0 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c51248b80 10 client.0 _ll_register_callbacks cb 0x559eb2642800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c3ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c3ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56716/0 (socket says 172.21.15.57:56716) 2023-10-08T13:52:02.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/2289571703 learned_addr learned my addr 172.21.15.57:0/2289571703 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb28f57a0 0x559eb28fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/2289571703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb28f57a0 msgr2=0x559eb28fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb28f57a0 0x559eb28fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/2289571703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2900650 msgr2=0x559eb28fde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c3f59e700 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2900650 0x559eb28fde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.164+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2900650 0x559eb28fde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c3ed9d700 1 -- 172.21.15.57:0/2289571703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c401a5af0 con 0x559eb28f4e80 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28fd3b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0c300ca710 tx=0x7f0c30005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad727d95395e4b2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c22ff5700 1 -- 172.21.15.57:0/2289571703 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c300b6070 con 0x559eb28f4e80 2023-10-08T13:52:02.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c22ff5700 1 -- 172.21.15.57:0/2289571703 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0c30098290 con 0x559eb28f4e80 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c51248b80 1 -- 172.21.15.57:0/2289571703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559eb2810eb0 con 0x559eb28f4e80 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c22ff5700 1 -- 172.21.15.57:0/2289571703 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c3009e0c0 con 0x559eb28f4e80 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c51248b80 10 client.16008 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c51248b80 1 -- 172.21.15.57:0/2289571703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559eb265cc60 con 0x559eb28f4e80 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c22ff5700 1 -- 172.21.15.57:0/2289571703 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559eb2810eb0 con 0x559eb28f4e80 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c22ff5700 1 -- 172.21.15.57:0/2289571703 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0c300a1df0 con 0x559eb28f4e80 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c51248b80 10 client.16008 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.165+0000 7f0c51248b80 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.166+0000 7f0c51248b80 20 client.16008 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.166+0000 7f0c51248b80 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559eb290fc70 0x559eb2912060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.166+0000 7f0c51248b80 4 client.16008 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:02.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.166+0000 7f0c51248b80 1 -- 172.21.15.57:0/2289571703 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559eb2841e10 con 0x559eb290fc70 2023-10-08T13:52:02.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.166+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559eb290fc70 0x559eb2912060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.166+0000 7f0c22ff5700 10 client.16008 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:02.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.166+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559eb290fc70 0x559eb2912060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f333edbd5fae7ce1 server_cookie=15995b03b7e7b2f7 in_seq=0 out_seq=0 2023-10-08T13:52:02.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c22ff5700 1 -- 172.21.15.57:0/2289571703 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559eb2841e10 con 0x559eb290fc70 2023-10-08T13:52:02.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c22ff5700 10 client.16008 handle_command_reply: tid=0 2023-10-08T13:52:02.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 client.16008 shutdown 2023-10-08T13:52:02.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 -- 172.21.15.57:0/2289571703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 msgr2=0x559eb28fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28fd3b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0c300ca710 tx=0x7f0c30005910 comp rx=0 tx=0).stop 2023-10-08T13:52:02.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 -- 172.21.15.57:0/2289571703 shutdown_connections 2023-10-08T13:52:02.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559eb290fc70 0x559eb2912060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2900650 0x559eb28fde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f4e80 0x559eb28fd3b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 --2- 172.21.15.57:0/2289571703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb28f57a0 0x559eb28fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 -- 172.21.15.57:0/2289571703 >> 172.21.15.57:0/2289571703 conn(0x559eb286a260 msgr2=0x559eb28f0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:02.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c22ff5700 0 client.16008 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:02.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 -- 172.21.15.57:0/2289571703 shutdown_connections 2023-10-08T13:52:02.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 1 -- 172.21.15.57:0/2289571703 wait complete. 2023-10-08T13:52:02.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.167+0000 7f0c51248b80 20 client.16008 trim_cache size 0 max 16384 2023-10-08T13:52:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.180+0000 7f0c51248b80 1 Processor -- start 2023-10-08T13:52:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.180+0000 7f0c51248b80 1 -- start start 2023-10-08T13:52:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.180+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb28f4e80 0x559eb28fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.180+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb28fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.180+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb2900650 0x559eb28fab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.180+0000 7f0c51248b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb26c9920 con 0x559eb28f57a0 2023-10-08T13:52:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.180+0000 7f0c51248b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb28f60d0 con 0x559eb28f4e80 2023-10-08T13:52:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.180+0000 7f0c51248b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb28fea20 con 0x559eb2900650 2023-10-08T13:52:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb28fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb28fa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56726/0 (socket says 172.21.15.57:56726) 2023-10-08T13:52:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3e59c700 1 -- 172.21.15.57:0/2195345446 learned_addr learned my addr 172.21.15.57:0/2195345446 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/2195345446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb28f4e80 0x559eb28fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3f59e700 1 --2- 172.21.15.57:0/2195345446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb2900650 0x559eb28fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3e59c700 1 -- 172.21.15.57:0/2195345446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb2900650 msgr2=0x559eb28fab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/2195345446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb2900650 0x559eb28fab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3e59c700 1 -- 172.21.15.57:0/2195345446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb28f4e80 msgr2=0x559eb28fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/2195345446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb28f4e80 0x559eb28fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3e59c700 1 -- 172.21.15.57:0/2195345446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eb269a250 con 0x559eb28f57a0 2023-10-08T13:52:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/2195345446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb28fa650 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009990 tx=0x7f0c3401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4dc3073b474c85d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c25ffb700 1 -- 172.21.15.57:0/2195345446 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34030990 con 0x559eb28f57a0 2023-10-08T13:52:02.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c25ffb700 1 -- 172.21.15.57:0/2195345446 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c34029070 con 0x559eb28f57a0 2023-10-08T13:52:02.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.181+0000 7f0c25ffb700 1 -- 172.21.15.57:0/2195345446 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400f460 con 0x559eb28f57a0 2023-10-08T13:52:02.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.182+0000 7f0c51248b80 1 -- 172.21.15.57:0/2195345446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 msgr2=0x559eb28fa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.182+0000 7f0c51248b80 1 --2- 172.21.15.57:0/2195345446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb28fa650 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009990 tx=0x7f0c3401f000 comp rx=0 tx=0).stop 2023-10-08T13:52:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.182+0000 7f0c51248b80 1 -- 172.21.15.57:0/2195345446 shutdown_connections 2023-10-08T13:52:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.182+0000 7f0c51248b80 1 --2- 172.21.15.57:0/2195345446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb2900650 0x559eb28fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.182+0000 7f0c51248b80 1 --2- 172.21.15.57:0/2195345446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb28f4e80 0x559eb28fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.182+0000 7f0c51248b80 1 --2- 172.21.15.57:0/2195345446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb28fa650 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.182+0000 7f0c51248b80 1 -- 172.21.15.57:0/2195345446 >> 172.21.15.57:0/2195345446 conn(0x559eb286a260 msgr2=0x559eb28f86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.182+0000 7f0c51248b80 1 -- 172.21.15.57:0/2195345446 shutdown_connections 2023-10-08T13:52:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.182+0000 7f0c51248b80 1 -- 172.21.15.57:0/2195345446 wait complete. 2023-10-08T13:52:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.182+0000 7f0c51248b80 1 -- start start 2023-10-08T13:52:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c51248b80 1 Processor -- start 2023-10-08T13:52:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb28f4e80 0x559eb2906cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb2907230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c51248b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2900650 0x559eb2905110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c51248b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb2905820 con 0x559eb28f57a0 2023-10-08T13:52:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c51248b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb2905990 con 0x559eb2900650 2023-10-08T13:52:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c51248b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb2905b50 con 0x559eb28f4e80 2023-10-08T13:52:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c51248b80 10 client.0 _ll_register_callbacks cb 0x7f0c34005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb2907230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb2907230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56732/0 (socket says 172.21.15.57:56732) 2023-10-08T13:52:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3e59c700 1 -- 172.21.15.57:0/1546508396 learned_addr learned my addr 172.21.15.57:0/1546508396 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3f59e700 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2900650 0x559eb2905110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3e59c700 1 -- 172.21.15.57:0/1546508396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb28f4e80 msgr2=0x559eb2906cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb28f4e80 0x559eb2906cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb28f4e80 0x559eb2906cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3e59c700 1 -- 172.21.15.57:0/1546508396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2900650 msgr2=0x559eb2905110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2900650 0x559eb2905110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3e59c700 1 -- 172.21.15.57:0/1546508396 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c40110110 con 0x559eb28f57a0 2023-10-08T13:52:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c3e59c700 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb2907230 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0c34034040 tx=0x7f0c34030430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ea2c2aac4bcf05 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.183+0000 7f0c22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c22ff5700 1 -- 172.21.15.57:0/1546508396 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34003990 con 0x559eb28f57a0 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c22ff5700 1 -- 172.21.15.57:0/1546508396 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0c34030990 con 0x559eb28f57a0 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c51248b80 1 -- 172.21.15.57:0/1546508396 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c300a2280 con 0x559eb28f57a0 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c22ff5700 1 -- 172.21.15.57:0/1546508396 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c34030b50 con 0x559eb28f57a0 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c51248b80 10 client.16014 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c51248b80 1 -- 172.21.15.57:0/1546508396 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559eb26722b0 con 0x559eb28f57a0 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c22ff5700 1 -- 172.21.15.57:0/1546508396 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c300a2280 con 0x559eb28f57a0 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c22ff5700 1 -- 172.21.15.57:0/1546508396 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0c3400f460 con 0x559eb28f57a0 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c51248b80 10 client.16014 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c51248b80 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c51248b80 20 client.16014 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c51248b80 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559eb2964840 0x559eb2966c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c51248b80 4 client.16014 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.184+0000 7f0c51248b80 1 -- 172.21.15.57:0/1546508396 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559eb2841b30 con 0x559eb2964840 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559eb2964840 0x559eb2966c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c22ff5700 10 client.16014 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c3ed9d700 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559eb2964840 0x559eb2966c60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39f478a2da9b573 server_cookie=814ecbcb331219dc in_seq=0 out_seq=0 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c22ff5700 1 -- 172.21.15.57:0/1546508396 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559eb2841b30 con 0x559eb2964840 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c22ff5700 10 client.16014 handle_command_reply: tid=0 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 client.16014 shutdown 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 -- 172.21.15.57:0/1546508396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 msgr2=0x559eb2907230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb2907230 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0c34034040 tx=0x7f0c34030430 comp rx=0 tx=0).stop 2023-10-08T13:52:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 -- 172.21.15.57:0/1546508396 shutdown_connections 2023-10-08T13:52:02.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559eb2964840 0x559eb2966c60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2900650 0x559eb2905110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c22ff5700 0 client.16014 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:02.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559eb28f4e80 0x559eb2906cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 --2- 172.21.15.57:0/1546508396 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559eb28f57a0 0x559eb2907230 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 -- 172.21.15.57:0/1546508396 >> 172.21.15.57:0/1546508396 conn(0x559eb286a260 msgr2=0x559eb28fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:02.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 -- 172.21.15.57:0/1546508396 shutdown_connections 2023-10-08T13:52:02.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.185+0000 7f0c51248b80 1 -- 172.21.15.57:0/1546508396 wait complete. 2023-10-08T13:52:02.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c51248b80 20 client.16014 trim_cache size 0 max 16384 2023-10-08T13:52:02.195 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:52:02.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 -- 172.21.15.57:0/1017761216 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c280a15f0 msgr2=0x7f0c280a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c280a15f0 0x7f0c280a3ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c340098e0 tx=0x7f0c34005960 comp rx=0 tx=0).stop 2023-10-08T13:52:02.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 -- 172.21.15.57:0/1017761216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c40102940 msgr2=0x7f0c40198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:02.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c40102940 0x7f0c40198d80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000d950 tx=0x7f0c3000dc60 comp rx=0 tx=0).stop 2023-10-08T13:52:02.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 -- 172.21.15.57:0/1017761216 shutdown_connections 2023-10-08T13:52:02.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40108950 0x7f0c4019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c40102940 0x7f0c40198d80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c280a15f0 0x7f0c280a3ab0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 --2- 172.21.15.57:0/1017761216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c40103300 0x7f0c401992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:02.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 -- 172.21.15.57:0/1017761216 >> 172.21.15.57:0/1017761216 conn(0x7f0c400fe3d0 msgr2=0x7f0c400fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:02.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 -- 172.21.15.57:0/1017761216 shutdown_connections 2023-10-08T13:52:02.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:02.186+0000 7f0c4505f700 1 -- 172.21.15.57:0/1017761216 wait complete. 2023-10-08T13:52:02.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:02 smithi115 ceph-mon[104408]: pgmap v313: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 13 op/s 2023-10-08T13:52:02.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:02 smithi115 ceph-mon[104408]: mgrmap e24: x(active, since 7m), standbys: z, y 2023-10-08T13:52:02.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:02 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3358729076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:52:02.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:02 smithi057 ceph-mon[99476]: pgmap v313: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 13 op/s 2023-10-08T13:52:02.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:02 smithi057 ceph-mon[99476]: mgrmap e24: x(active, since 7m), standbys: z, y 2023-10-08T13:52:02.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:02 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3358729076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:52:02.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:02 smithi099 ceph-mon[103849]: pgmap v313: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 13 op/s 2023-10-08T13:52:02.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:02 smithi099 ceph-mon[103849]: mgrmap e24: x(active, since 7m), standbys: z, y 2023-10-08T13:52:02.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:02 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3358729076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:52:03.201 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-08T13:52:03.201 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-08T13:52:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.332+0000 7f62a9d83700 1 -- 172.21.15.57:0/3543965060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a40ff640 msgr2=0x7f62a40ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.332+0000 7f62a9d83700 1 --2- 172.21.15.57:0/3543965060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a40ff640 0x7f62a40ffac0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6298009990 tx=0x7f6298009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.332+0000 7f62a9d83700 1 -- 172.21.15.57:0/3543965060 shutdown_connections 2023-10-08T13:52:03.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.332+0000 7f62a9d83700 1 --2- 172.21.15.57:0/3543965060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a4100000 0x7f62a410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.332+0000 7f62a9d83700 1 --2- 172.21.15.57:0/3543965060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a4105650 0x7f62a4105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.332+0000 7f62a9d83700 1 --2- 172.21.15.57:0/3543965060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a40ff640 0x7f62a40ffac0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.332+0000 7f62a9d83700 1 -- 172.21.15.57:0/3543965060 >> 172.21.15.57:0/3543965060 conn(0x7f62a40767c0 msgr2=0x7f62a4076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.332+0000 7f62a9d83700 1 -- 172.21.15.57:0/3543965060 shutdown_connections 2023-10-08T13:52:03.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.332+0000 7f62a9d83700 1 -- 172.21.15.57:0/3543965060 wait complete. 2023-10-08T13:52:03.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a9d83700 1 Processor -- start 2023-10-08T13:52:03.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a9d83700 1 -- start start 2023-10-08T13:52:03.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a9d83700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a40ff640 0x7f62a419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a9d83700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a4100000 0x7f62a419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a9d83700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a4105650 0x7f62a41a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a9d83700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41a2160 con 0x7f62a40ff640 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a9d83700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41a22d0 con 0x7f62a4100000 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a9d83700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41a2490 con 0x7f62a4105650 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a40ff640 0x7f62a419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a40ff640 0x7f62a419d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56744/0 (socket says 172.21.15.57:56744) 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.333+0000 7f62a37fe700 1 -- 172.21.15.57:0/4170362566 learned_addr learned my addr 172.21.15.57:0/4170362566 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a3fff700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a4105650 0x7f62a41a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a4100000 0x7f62a419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a37fe700 1 -- 172.21.15.57:0/4170362566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a4105650 msgr2=0x7f62a41a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a37fe700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a4105650 0x7f62a41a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a37fe700 1 -- 172.21.15.57:0/4170362566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a4100000 msgr2=0x7f62a419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a37fe700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a4100000 0x7f62a419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a37fe700 1 -- 172.21.15.57:0/4170362566 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62980095f0 con 0x7f62a40ff640 2023-10-08T13:52:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a3fff700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a4105650 0x7f62a41a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a4100000 0x7f62a419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a37fe700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a40ff640 0x7f62a419d560 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f629400e9a0 tx=0x7f629400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a0ff9700 1 -- 172.21.15.57:0/4170362566 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294009790 con 0x7f62a40ff640 2023-10-08T13:52:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a0ff9700 1 -- 172.21.15.57:0/4170362566 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f629400cbd0 con 0x7f62a40ff640 2023-10-08T13:52:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a0ff9700 1 -- 172.21.15.57:0/4170362566 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294018680 con 0x7f62a40ff640 2023-10-08T13:52:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a9d83700 1 -- 172.21.15.57:0/4170362566 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a41a2770 con 0x7f62a40ff640 2023-10-08T13:52:03.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.334+0000 7f62a9d83700 1 -- 172.21.15.57:0/4170362566 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a41a9f60 con 0x7f62a40ff640 2023-10-08T13:52:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.337+0000 7f62a0ff9700 1 -- 172.21.15.57:0/4170362566 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f629400cd40 con 0x7f62a40ff640 2023-10-08T13:52:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.338+0000 7f62a0ff9700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f628c0a1670 0x7f628c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.338+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f628c0a1670 0x7f628c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.338+0000 7f62a0ff9700 1 -- 172.21.15.57:0/4170362566 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6294014070 con 0x7f62a40ff640 2023-10-08T13:52:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.338+0000 7f62b5f6cb80 1 Processor -- start 2023-10-08T13:52:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.338+0000 7f62b5f6cb80 1 -- start start 2023-10-08T13:52:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.338+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f628c0a1670 0x7f628c0a3b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f629800b2e0 tx=0x7f6298005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 0x55e9b606e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b6079190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b607bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5fd74e0 con 0x55e9b60796d0 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5fc8ca0 con 0x55e9b606e820 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5fc65c0 con 0x55e9b606df00 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b607bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b607bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56756/0 (socket says 172.21.15.57:56756) 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a3fff700 1 -- 172.21.15.57:0/2533132812 learned_addr learned my addr 172.21.15.57:0/2533132812 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a37fe700 1 --2- 172.21.15.57:0/2533132812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 0x55e9b606e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/2533132812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b6079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a3fff700 1 -- 172.21.15.57:0/2533132812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 msgr2=0x55e9b606e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a3fff700 1 --2- 172.21.15.57:0/2533132812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 0x55e9b606e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a3fff700 1 -- 172.21.15.57:0/2533132812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 msgr2=0x55e9b6079190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a3fff700 1 --2- 172.21.15.57:0/2533132812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b6079190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a3fff700 1 -- 172.21.15.57:0/2533132812 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9b5dece60 con 0x55e9b60796d0 2023-10-08T13:52:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.339+0000 7f62a3fff700 1 --2- 172.21.15.57:0/2533132812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b607bac0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f629c009400 tx=0x7f629c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a2e09d62073aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.340+0000 7f6289ffb700 1 -- 172.21.15.57:0/2533132812 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c014610 con 0x55e9b60796d0 2023-10-08T13:52:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.340+0000 7f6289ffb700 1 -- 172.21.15.57:0/2533132812 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f629c00f050 con 0x55e9b60796d0 2023-10-08T13:52:03.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.340+0000 7f6289ffb700 1 -- 172.21.15.57:0/2533132812 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c013960 con 0x55e9b60796d0 2023-10-08T13:52:03.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.340+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/2533132812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 msgr2=0x55e9b607bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.340+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/2533132812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b607bac0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f629c009400 tx=0x7f629c009920 comp rx=0 tx=0).stop 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.340+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/2533132812 shutdown_connections 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.340+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/2533132812 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b607bac0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.340+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/2533132812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 0x55e9b606e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.340+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/2533132812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b6079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.340+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/2533132812 >> 172.21.15.57:0/2533132812 conn(0x55e9b5fe32e0 msgr2=0x55e9b606a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.341+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/2533132812 shutdown_connections 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.341+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/2533132812 wait complete. 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.341+0000 7f62b5f6cb80 1 -- start start 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.341+0000 7f62b5f6cb80 1 Processor -- start 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.341+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 0x55e9b6079040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.341+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b6076330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.341+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b6076960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.341+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5fc1eb0 con 0x55e9b60796d0 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.341+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5f5e790 con 0x55e9b606e820 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.341+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5f5e430 con 0x55e9b606df00 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62b5f6cb80 10 client.0 _ll_register_callbacks cb 0x55e9b5dbb880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b6076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b6076960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56762/0 (socket says 172.21.15.57:56762) 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a3fff700 1 -- 172.21.15.57:0/86841230 learned_addr learned my addr 172.21.15.57:0/86841230 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a37fe700 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 0x55e9b6079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b6076330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a3fff700 1 -- 172.21.15.57:0/86841230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 msgr2=0x55e9b6079040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a3fff700 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 0x55e9b6079040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a3fff700 1 -- 172.21.15.57:0/86841230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 msgr2=0x55e9b6076330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a3fff700 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b6076330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a3fff700 1 -- 172.21.15.57:0/86841230 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c0090b0 con 0x55e9b60796d0 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a37fe700 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 0x55e9b6079040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b6076330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62a3fff700 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b6076960 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f629c007ac0 tx=0x7f629c0146f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156906536e98e640 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f6282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f6282ffd700 1 -- 172.21.15.57:0/86841230 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c0095e0 con 0x55e9b60796d0 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f6282ffd700 1 -- 172.21.15.57:0/86841230 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f629c0135e0 con 0x55e9b60796d0 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.342+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/86841230 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9b5f89f30 con 0x55e9b60796d0 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f6282ffd700 1 -- 172.21.15.57:0/86841230 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f629c0137c0 con 0x55e9b60796d0 2023-10-08T13:52:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f62b5f6cb80 10 client.16026 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/86841230 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e9b5dd5ce0 con 0x55e9b60796d0 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f6282ffd700 1 -- 172.21.15.57:0/86841230 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9b5f89f30 con 0x55e9b60796d0 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f6282ffd700 1 -- 172.21.15.57:0/86841230 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f629c027830 con 0x55e9b60796d0 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f62b5f6cb80 10 client.16026 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f62b5f6cb80 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f62b5f6cb80 20 client.16026 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e9b6089100 0x55e9b608b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f62b5f6cb80 4 client.16026 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/86841230 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9b5fbae90 con 0x55e9b6089100 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.343+0000 7f62a37fe700 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e9b6089100 0x55e9b608b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.344+0000 7f6282ffd700 10 client.16026 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.344+0000 7f62a37fe700 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e9b6089100 0x55e9b608b4f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f3a84ea38ef2f71 server_cookie=cf0ba623cf65d746 in_seq=0 out_seq=0 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f6282ffd700 1 -- 172.21.15.57:0/86841230 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9b5fbae90 con 0x55e9b6089100 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f6282ffd700 10 client.16026 handle_command_reply: tid=0 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 client.16026 shutdown 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/86841230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 msgr2=0x55e9b6076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b6076960 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f629c007ac0 tx=0x7f629c0146f0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/86841230 shutdown_connections 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e9b6089100 0x55e9b608b4f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b60796d0 0x55e9b6076960 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b606df00 0x55e9b6079040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/86841230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b6076330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/86841230 >> 172.21.15.57:0/86841230 conn(0x55e9b5fe32e0 msgr2=0x55e9b6069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f6282ffd700 0 client.16026 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:03.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/86841230 shutdown_connections 2023-10-08T13:52:03.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/86841230 wait complete. 2023-10-08T13:52:03.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.345+0000 7f62b5f6cb80 20 client.16026 trim_cache size 0 max 16384 2023-10-08T13:52:03.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.362+0000 7f62b5f6cb80 1 Processor -- start 2023-10-08T13:52:03.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.362+0000 7f62b5f6cb80 1 -- start start 2023-10-08T13:52:03.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b6085240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b60752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 0x55e9b60768c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b5e429a0 con 0x55e9b606df00 2023-10-08T13:52:03.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b6088ee0 con 0x55e9b606e820 2023-10-08T13:52:03.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b60645f0 con 0x55e9b60796d0 2023-10-08T13:52:03.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b6085240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b6085240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56766/0 (socket says 172.21.15.57:56766) 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a37fe700 1 -- 172.21.15.57:0/2437480531 learned_addr learned my addr 172.21.15.57:0/2437480531 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/2437480531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b60752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a3fff700 1 --2- 172.21.15.57:0/2437480531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 0x55e9b60768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a37fe700 1 -- 172.21.15.57:0/2437480531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 msgr2=0x55e9b60768c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a37fe700 1 --2- 172.21.15.57:0/2437480531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 0x55e9b60768c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a37fe700 1 -- 172.21.15.57:0/2437480531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 msgr2=0x55e9b60752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a37fe700 1 --2- 172.21.15.57:0/2437480531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b60752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a37fe700 1 -- 172.21.15.57:0/2437480531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9b6090120 con 0x55e9b606df00 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a3fff700 1 --2- 172.21.15.57:0/2437480531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 0x55e9b60768c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.363+0000 7f62a37fe700 1 --2- 172.21.15.57:0/2437480531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b6085240 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6294096da0 tx=0x7f629400efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684592dd3e453e15 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f628a7fc700 1 -- 172.21.15.57:0/2437480531 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62940a26c0 con 0x55e9b606df00 2023-10-08T13:52:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f628a7fc700 1 -- 172.21.15.57:0/2437480531 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62940986d0 con 0x55e9b606df00 2023-10-08T13:52:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f628a7fc700 1 -- 172.21.15.57:0/2437480531 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294099a80 con 0x55e9b606df00 2023-10-08T13:52:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/2437480531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 msgr2=0x55e9b6085240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/2437480531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b6085240 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6294096da0 tx=0x7f629400efa0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/2437480531 shutdown_connections 2023-10-08T13:52:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/2437480531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 0x55e9b60768c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/2437480531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b6085240 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/2437480531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b60752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/2437480531 >> 172.21.15.57:0/2437480531 conn(0x55e9b5fe32e0 msgr2=0x55e9b6077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/2437480531 shutdown_connections 2023-10-08T13:52:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.364+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/2437480531 wait complete. 2023-10-08T13:52:03.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62b5f6cb80 1 -- start start 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62b5f6cb80 1 Processor -- start 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b60d7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b60d7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62b5f6cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 0x55e9b607f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b607fb60 con 0x55e9b606df00 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b607fcd0 con 0x55e9b606e820 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62b5f6cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9b607fe90 con 0x55e9b60796d0 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62b5f6cb80 10 client.0 _ll_register_callbacks cb 0x7f6298009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b60d7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62a37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b60d7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56772/0 (socket says 172.21.15.57:56772) 2023-10-08T13:52:03.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.365+0000 7f62a37fe700 1 -- 172.21.15.57:0/1903481588 learned_addr learned my addr 172.21.15.57:0/1903481588 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b60d7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62a3fff700 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 0x55e9b607f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62a37fe700 1 -- 172.21.15.57:0/1903481588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 msgr2=0x55e9b607f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62a37fe700 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 0x55e9b607f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62a37fe700 1 -- 172.21.15.57:0/1903481588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 msgr2=0x55e9b60d7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62a37fe700 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b60d7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62a37fe700 1 -- 172.21.15.57:0/1903481588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a41a9f60 con 0x55e9b606df00 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62a3fff700 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 0x55e9b607f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62a37fe700 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b60d7520 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f62940a2190 tx=0x7f629409a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e8688518ae0262 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f6282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:03.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f6282ffd700 1 -- 172.21.15.57:0/1903481588 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294099a20 con 0x55e9b606df00 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f6282ffd700 1 -- 172.21.15.57:0/1903481588 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f629409a590 con 0x55e9b606df00 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/1903481588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f629c00f050 con 0x55e9b606df00 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f6282ffd700 1 -- 172.21.15.57:0/1903481588 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f629409a770 con 0x55e9b606df00 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62b5f6cb80 10 client.16032 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/1903481588 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e9b5e132d0 con 0x55e9b606df00 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.366+0000 7f6282ffd700 1 -- 172.21.15.57:0/1903481588 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f629c00f050 con 0x55e9b606df00 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.367+0000 7f6282ffd700 1 -- 172.21.15.57:0/1903481588 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6294099320 con 0x55e9b606df00 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.367+0000 7f62b5f6cb80 10 client.16032 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.367+0000 7f62b5f6cb80 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.367+0000 7f62b5f6cb80 20 client.16032 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.367+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e9b60dee90 0x55e9b60e1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.367+0000 7f62b5f6cb80 4 client.16032 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.367+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/1903481588 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e9b5fbabb0 con 0x55e9b60dee90 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.367+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e9b60dee90 0x55e9b60e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.368+0000 7f6282ffd700 10 client.16032 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.368+0000 7f62a2ffd700 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e9b60dee90 0x55e9b60e1280 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d289b5b5d3ce110f server_cookie=98443b9a6e3bb1e4 in_seq=0 out_seq=0 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f6282ffd700 1 -- 172.21.15.57:0/1903481588 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e9b5fbabb0 con 0x55e9b60dee90 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f6282ffd700 10 client.16032 handle_command_reply: tid=0 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 client.16032 shutdown 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/1903481588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 msgr2=0x55e9b60d7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b60d7520 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f62940a2190 tx=0x7f629409a3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/1903481588 shutdown_connections 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e9b60dee90 0x55e9b60e1280 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e9b60796d0 0x55e9b607f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e9b606df00 0x55e9b60d7520 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 --2- 172.21.15.57:0/1903481588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9b606e820 0x55e9b60d7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/1903481588 >> 172.21.15.57:0/1903481588 conn(0x55e9b5fe32e0 msgr2=0x55e9b6078460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f6282ffd700 0 client.16032 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/1903481588 shutdown_connections 2023-10-08T13:52:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 1 -- 172.21.15.57:0/1903481588 wait complete. 2023-10-08T13:52:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.369+0000 7f62b5f6cb80 20 client.16032 trim_cache size 0 max 16384 2023-10-08T13:52:03.376 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:52:03.376 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-10-08T13:52:03.377 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "93fdd18c-2542-496b-ab74-9ea93ca55e19", 2023-10-08T13:52:03.377 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-10-08T13:52:03.377 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:52:03.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 -- 172.21.15.57:0/4170362566 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f628c0a1670 msgr2=0x7f628c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f628c0a1670 0x7f628c0a3b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f629800b2e0 tx=0x7f6298005e80 comp rx=0 tx=0).stop 2023-10-08T13:52:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 -- 172.21.15.57:0/4170362566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a40ff640 msgr2=0x7f62a419d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a40ff640 0x7f62a419d560 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f629400e9a0 tx=0x7f629400ed60 comp rx=0 tx=0).stop 2023-10-08T13:52:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 -- 172.21.15.57:0/4170362566 shutdown_connections 2023-10-08T13:52:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a4105650 0x7f62a41a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62a40ff640 0x7f62a419d560 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f628c0a1670 0x7f628c0a3b30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 --2- 172.21.15.57:0/4170362566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a4100000 0x7f62a419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 -- 172.21.15.57:0/4170362566 >> 172.21.15.57:0/4170362566 conn(0x7f62a40767c0 msgr2=0x7f62a40689e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 -- 172.21.15.57:0/4170362566 shutdown_connections 2023-10-08T13:52:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.370+0000 7f62a9d83700 1 -- 172.21.15.57:0/4170362566 wait complete. 2023-10-08T13:52:03.384 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:52:03.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.519+0000 7f7b209c2700 1 -- 172.21.15.57:0/682806655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b1c105b80 msgr2=0x7f7b1c10d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.519+0000 7f7b209c2700 1 --2- 172.21.15.57:0/682806655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b1c105b80 0x7f7b1c10d590 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400b0d0 tx=0x7f7b1400b3e0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.519+0000 7f7b209c2700 1 -- 172.21.15.57:0/682806655 shutdown_connections 2023-10-08T13:52:03.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.519+0000 7f7b209c2700 1 --2- 172.21.15.57:0/682806655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b1c105b80 0x7f7b1c10d590 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.519+0000 7f7b209c2700 1 --2- 172.21.15.57:0/682806655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b1c067ce0 0x7f7b1c0680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.519+0000 7f7b209c2700 1 --2- 172.21.15.57:0/682806655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c1051c0 0x7f7b1c105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.519+0000 7f7b209c2700 1 -- 172.21.15.57:0/682806655 >> 172.21.15.57:0/682806655 conn(0x7f7b1c076780 msgr2=0x7f7b1c076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.519+0000 7f7b209c2700 1 -- 172.21.15.57:0/682806655 shutdown_connections 2023-10-08T13:52:03.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.519+0000 7f7b209c2700 1 -- 172.21.15.57:0/682806655 wait complete. 2023-10-08T13:52:03.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b209c2700 1 Processor -- start 2023-10-08T13:52:03.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b209c2700 1 -- start start 2023-10-08T13:52:03.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b209c2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b1c067ce0 0x7f7b1c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b209c2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b1c1051c0 0x7f7b1c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b209c2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c105b80 0x7f7b1c196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b209c2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c197180 con 0x7f7b1c067ce0 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b209c2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c1972f0 con 0x7f7b1c105b80 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b209c2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c1974b0 con 0x7f7b1c1051c0 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b1a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b1c067ce0 0x7f7b1c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b1a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b1c067ce0 0x7f7b1c19c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56786/0 (socket says 172.21.15.57:56786) 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.520+0000 7f7b1a59c700 1 -- 172.21.15.57:0/1702058129 learned_addr learned my addr 172.21.15.57:0/1702058129 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b1c1051c0 0x7f7b1c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b1ad9d700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c105b80 0x7f7b1c196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b1a59c700 1 -- 172.21.15.57:0/1702058129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b1c1051c0 msgr2=0x7f7b1c19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b1c1051c0 0x7f7b1c19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b1a59c700 1 -- 172.21.15.57:0/1702058129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c105b80 msgr2=0x7f7b1c196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c105b80 0x7f7b1c196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b1a59c700 1 -- 172.21.15.57:0/1702058129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b14009d00 con 0x7f7b1c067ce0 2023-10-08T13:52:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b1c067ce0 0x7f7b1c19c9f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00d950 tx=0x7f7b0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b0b7fe700 1 -- 172.21.15.57:0/1702058129 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c009940 con 0x7f7b1c067ce0 2023-10-08T13:52:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b0b7fe700 1 -- 172.21.15.57:0/1702058129 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b0c010460 con 0x7f7b1c067ce0 2023-10-08T13:52:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b209c2700 1 -- 172.21.15.57:0/1702058129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1c197790 con 0x7f7b1c067ce0 2023-10-08T13:52:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b0b7fe700 1 -- 172.21.15.57:0/1702058129 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c00f620 con 0x7f7b1c067ce0 2023-10-08T13:52:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.521+0000 7f7b209c2700 1 -- 172.21.15.57:0/1702058129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c1a39e0 con 0x7f7b1c067ce0 2023-10-08T13:52:03.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.523+0000 7f7b0b7fe700 1 -- 172.21.15.57:0/1702058129 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7b0c0108d0 con 0x7f7b1c067ce0 2023-10-08T13:52:03.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.524+0000 7f7b0b7fe700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b040a15f0 0x7f7b040a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.524+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b040a15f0 0x7f7b040a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.524+0000 7f7b0b7fe700 1 -- 172.21.15.57:0/1702058129 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7b0c0cd2e0 con 0x7f7b1c067ce0 2023-10-08T13:52:03.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.525+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b040a15f0 0x7f7b040a3ab0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c197ce0 tx=0x7f7b10009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.526+0000 7f7b2cbabb80 1 Processor -- start 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b2cbabb80 1 -- start start 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138be80 0x5569b138c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b139b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b139ba60 0x5569b139de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b12f5460 con 0x5569b138c7a0 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b12e6c20 con 0x5569b138be80 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b12e4540 con 0x5569b139ba60 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b19d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b139b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b19d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b139b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56796/0 (socket says 172.21.15.57:56796) 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b19d9b700 1 -- 172.21.15.57:0/2149112900 learned_addr learned my addr 172.21.15.57:0/2149112900 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/2149112900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138be80 0x5569b138c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b1ad9d700 1 --2- 172.21.15.57:0/2149112900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b139ba60 0x5569b139de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b19d9b700 1 -- 172.21.15.57:0/2149112900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b139ba60 msgr2=0x5569b139de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/2149112900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b139ba60 0x5569b139de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b19d9b700 1 -- 172.21.15.57:0/2149112900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138be80 msgr2=0x5569b138c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/2149112900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138be80 0x5569b138c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b19d9b700 1 -- 172.21.15.57:0/2149112900 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569b110ade0 con 0x5569b138c7a0 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.527+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/2149112900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138be80 0x5569b138c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.528+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/2149112900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b139b520 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7b10011d40 tx=0x7f7b10014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6655f376106f7353 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.528+0000 7f7b08ff9700 1 -- 172.21.15.57:0/2149112900 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10011d70 con 0x5569b138c7a0 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.528+0000 7f7b08ff9700 1 -- 172.21.15.57:0/2149112900 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b100044e0 con 0x5569b138c7a0 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.528+0000 7f7b08ff9700 1 -- 172.21.15.57:0/2149112900 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10003d50 con 0x5569b138c7a0 2023-10-08T13:52:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.528+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/2149112900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 msgr2=0x5569b139b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.528+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/2149112900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b139b520 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7b10011d40 tx=0x7f7b10014000 comp rx=0 tx=0).stop 2023-10-08T13:52:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.529+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/2149112900 shutdown_connections 2023-10-08T13:52:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.529+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/2149112900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b139ba60 0x5569b139de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.529+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/2149112900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138be80 0x5569b138c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.529+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/2149112900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b139b520 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.529+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/2149112900 >> 172.21.15.57:0/2149112900 conn(0x5569b1301260 msgr2=0x5569b13882d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.529+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/2149112900 shutdown_connections 2023-10-08T13:52:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.529+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/2149112900 wait complete. 2023-10-08T13:52:03.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.529+0000 7f7b2cbabb80 1 -- start start 2023-10-08T13:52:03.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.529+0000 7f7b2cbabb80 1 Processor -- start 2023-10-08T13:52:03.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b139b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b1398740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b139ba60 0x5569b1398d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b12dfe30 con 0x5569b138c7a0 2023-10-08T13:52:03.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b127c710 con 0x5569b139ba60 2023-10-08T13:52:03.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b127c3b0 con 0x5569b138be80 2023-10-08T13:52:03.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b2cbabb80 10 client.0 _ll_register_callbacks cb 0x5569b10d9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b19d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b1398740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b19d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b1398740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56802/0 (socket says 172.21.15.57:56802) 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b19d9b700 1 -- 172.21.15.57:0/4088642209 learned_addr learned my addr 172.21.15.57:0/4088642209 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b1ad9d700 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b139ba60 0x5569b1398d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b139b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b19d9b700 1 -- 172.21.15.57:0/4088642209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 msgr2=0x5569b139b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b139b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b19d9b700 1 -- 172.21.15.57:0/4088642209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b139ba60 msgr2=0x5569b1398d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b139ba60 0x5569b1398d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b19d9b700 1 -- 172.21.15.57:0/4088642209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b10017070 con 0x5569b138c7a0 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b139b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:03.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b1398740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000efa0 tx=0x7f7b10003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1a2b626fc65b96 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.530+0000 7f7b01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b01ffb700 1 -- 172.21.15.57:0/4088642209 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b100040e0 con 0x5569b138c7a0 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b01ffb700 1 -- 172.21.15.57:0/4088642209 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7b10004240 con 0x5569b138c7a0 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/4088642209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569b12a7eb0 con 0x5569b138c7a0 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b01ffb700 1 -- 172.21.15.57:0/4088642209 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b1001e420 con 0x5569b138c7a0 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b2cbabb80 10 client.16044 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/4088642209 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5569b10f3c60 con 0x5569b138c7a0 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b01ffb700 1 -- 172.21.15.57:0/4088642209 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5569b12a7eb0 con 0x5569b138c7a0 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b01ffb700 1 -- 172.21.15.57:0/4088642209 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7b10024030 con 0x5569b138c7a0 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b2cbabb80 10 client.16044 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:03.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b2cbabb80 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b2cbabb80 20 client.16044 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5569b13ab3d0 0x5569b13ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b2cbabb80 4 client.16044 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.531+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/4088642209 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569b12d8e10 con 0x5569b13ab3d0 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.532+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5569b13ab3d0 0x5569b13ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.532+0000 7f7b01ffb700 10 client.16044 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.532+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5569b13ab3d0 0x5569b13ad7f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e480b9cbec52382 server_cookie=7ae45ccce181808f in_seq=0 out_seq=0 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b01ffb700 1 -- 172.21.15.57:0/4088642209 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5569b12d8e10 con 0x5569b13ab3d0 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b01ffb700 10 client.16044 handle_command_reply: tid=0 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 client.16044 shutdown 2023-10-08T13:52:03.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/4088642209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 msgr2=0x5569b1398740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b1398740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000efa0 tx=0x7f7b10003c00 comp rx=0 tx=0).stop 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/4088642209 shutdown_connections 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5569b13ab3d0 0x5569b13ad7f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b139ba60 0x5569b1398d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b139b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b01ffb700 0 client.16044 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/4088642209 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b1398740 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/4088642209 >> 172.21.15.57:0/4088642209 conn(0x5569b1301260 msgr2=0x5569b1387ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/4088642209 shutdown_connections 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/4088642209 wait complete. 2023-10-08T13:52:03.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.533+0000 7f7b2cbabb80 20 client.16044 trim_cache size 0 max 16384 2023-10-08T13:52:03.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b2cbabb80 1 Processor -- start 2023-10-08T13:52:03.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b2cbabb80 1 -- start start 2023-10-08T13:52:03.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b1388480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b13a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b139ba60 0x5569b1397fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b1160920 con 0x5569b138c7a0 2023-10-08T13:52:03.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b13a2af0 con 0x5569b139ba60 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b1399630 con 0x5569b138be80 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b19d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b13a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b19d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b13a25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56810/0 (socket says 172.21.15.57:56810) 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b19d9b700 1 -- 172.21.15.57:0/3493558195 learned_addr learned my addr 172.21.15.57:0/3493558195 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b1ad9d700 1 --2- 172.21.15.57:0/3493558195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b139ba60 0x5569b1397fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.551+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/3493558195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b1388480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.552+0000 7f7b19d9b700 1 -- 172.21.15.57:0/3493558195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 msgr2=0x5569b1388480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.552+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/3493558195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b1388480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.552+0000 7f7b19d9b700 1 -- 172.21.15.57:0/3493558195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b139ba60 msgr2=0x5569b1397fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.552+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/3493558195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b139ba60 0x5569b1397fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.552+0000 7f7b19d9b700 1 -- 172.21.15.57:0/3493558195 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569b1131250 con 0x5569b138c7a0 2023-10-08T13:52:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.552+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/3493558195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b1388480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:03.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.552+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/3493558195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b13a25b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7b10005950 tx=0x7f7b10026400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5058f933c13efe0c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.552+0000 7f7b097fa700 1 -- 172.21.15.57:0/3493558195 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10003d20 con 0x5569b138c7a0 2023-10-08T13:52:03.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.552+0000 7f7b097fa700 1 -- 172.21.15.57:0/3493558195 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b10003740 con 0x5569b138c7a0 2023-10-08T13:52:03.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.552+0000 7f7b097fa700 1 -- 172.21.15.57:0/3493558195 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10004380 con 0x5569b138c7a0 2023-10-08T13:52:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3493558195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 msgr2=0x5569b13a25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/3493558195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b13a25b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7b10005950 tx=0x7f7b10026400 comp rx=0 tx=0).stop 2023-10-08T13:52:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3493558195 shutdown_connections 2023-10-08T13:52:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/3493558195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b139ba60 0x5569b1397fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/3493558195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b1388480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/3493558195 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b138c7a0 0x5569b13a25b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3493558195 >> 172.21.15.57:0/3493558195 conn(0x5569b1301260 msgr2=0x5569b1387d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3493558195 shutdown_connections 2023-10-08T13:52:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3493558195 wait complete. 2023-10-08T13:52:03.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 -- start start 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.553+0000 7f7b2cbabb80 1 Processor -- start 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b13a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138c7a0 0x5569b13a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b2cbabb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b139ba60 0x5569b13a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b13a1370 con 0x5569b139ba60 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b13a14e0 con 0x5569b138c7a0 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b2cbabb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b13ff210 con 0x5569b138be80 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b2cbabb80 10 client.0 _ll_register_callbacks cb 0x7f7b10011c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b139ba60 0x5569b13a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b139ba60 0x5569b13a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56818/0 (socket says 172.21.15.57:56818) 2023-10-08T13:52:03.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1ad9d700 1 -- 172.21.15.57:0/3259133791 learned_addr learned my addr 172.21.15.57:0/3259133791 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b19d9b700 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138c7a0 0x5569b13a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b13a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1ad9d700 1 -- 172.21.15.57:0/3259133791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 msgr2=0x5569b13a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1ad9d700 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b13a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1ad9d700 1 -- 172.21.15.57:0/3259133791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138c7a0 msgr2=0x5569b13a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1ad9d700 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138c7a0 0x5569b13a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1ad9d700 1 -- 172.21.15.57:0/3259133791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569b10f3c60 con 0x5569b139ba60 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b13a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b1ad9d700 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b139ba60 0x5569b13a0ca0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7b14015000 tx=0x7f7b14007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5125b57dc0c74a42 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.554+0000 7f7b01ffb700 1 -- 172.21.15.57:0/3259133791 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400e040 con 0x5569b139ba60 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b01ffb700 1 -- 172.21.15.57:0/3259133791 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7b14004a50 con 0x5569b139ba60 2023-10-08T13:52:03.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3259133791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b100044e0 con 0x5569b139ba60 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b01ffb700 1 -- 172.21.15.57:0/3259133791 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b1400be50 con 0x5569b139ba60 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b2cbabb80 10 client.16050 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3259133791 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5569b11092b0 con 0x5569b139ba60 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b01ffb700 1 -- 172.21.15.57:0/3259133791 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b100044e0 con 0x5569b139ba60 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b01ffb700 1 -- 172.21.15.57:0/3259133791 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7b14019070 con 0x5569b139ba60 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b2cbabb80 10 client.16050 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b2cbabb80 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b2cbabb80 20 client.16050 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5569b1400930 0x5569b1402d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b2cbabb80 4 client.16050 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:52:03.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.555+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3259133791 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5569b12d8b30 con 0x5569b1400930 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5569b1400930 0x5569b1402d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b01ffb700 10 client.16050 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b1a59c700 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5569b1400930 0x5569b1402d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b13c8df53bebde server_cookie=1f1e33eef34eb1a8 in_seq=0 out_seq=0 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b01ffb700 1 -- 172.21.15.57:0/3259133791 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5569b12d8b30 con 0x5569b1400930 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b01ffb700 10 client.16050 handle_command_reply: tid=0 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b2cbabb80 1 client.16050 shutdown 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3259133791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b139ba60 msgr2=0x5569b13a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b139ba60 0x5569b13a0ca0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7b14015000 tx=0x7f7b14007f00 comp rx=0 tx=0).stop 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3259133791 shutdown_connections 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5569b1400930 0x5569b1402d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5569b139ba60 0x5569b13a0ca0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569b138be80 0x5569b13a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b01ffb700 0 client.16050 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b2cbabb80 1 --2- 172.21.15.57:0/3259133791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569b138c7a0 0x5569b13a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.556+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3259133791 >> 172.21.15.57:0/3259133791 conn(0x5569b1301260 msgr2=0x5569b1387b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3259133791 shutdown_connections 2023-10-08T13:52:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b2cbabb80 1 -- 172.21.15.57:0/3259133791 wait complete. 2023-10-08T13:52:03.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b2cbabb80 20 client.16050 trim_cache size 0 max 16384 2023-10-08T13:52:03.565 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:52:03.565 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T13:52:03.565 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:52:03.565 INFO:teuthology.orchestra.run.smithi057.stdout: "93fdd18c-2542-496b-ab74-9ea93ca55e19": { 2023-10-08T13:52:03.565 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:52:03.565 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "93fdd18c-2542-496b-ab74-9ea93ca55e19", 2023-10-08T13:52:03.565 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:52:03.566 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:52:03.566 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:52:03.566 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:52:03.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 -- 172.21.15.57:0/1702058129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b040a15f0 msgr2=0x7f7b040a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b040a15f0 0x7f7b040a3ab0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c197ce0 tx=0x7f7b10009260 comp rx=0 tx=0).stop 2023-10-08T13:52:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 -- 172.21.15.57:0/1702058129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b1c067ce0 msgr2=0x7f7b1c19c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b1c067ce0 0x7f7b1c19c9f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00d950 tx=0x7f7b0c00dc60 comp rx=0 tx=0).stop 2023-10-08T13:52:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 -- 172.21.15.57:0/1702058129 shutdown_connections 2023-10-08T13:52:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b040a15f0 0x7f7b040a3ab0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b1c105b80 0x7f7b1c196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b1c067ce0 0x7f7b1c19c9f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 --2- 172.21.15.57:0/1702058129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b1c1051c0 0x7f7b1c19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 -- 172.21.15.57:0/1702058129 >> 172.21.15.57:0/1702058129 conn(0x7f7b1c076780 msgr2=0x7f7b1c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 -- 172.21.15.57:0/1702058129 shutdown_connections 2023-10-08T13:52:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:03.557+0000 7f7b209c2700 1 -- 172.21.15.57:0/1702058129 wait complete. 2023-10-08T13:52:03.570 INFO:tasks.cephfs.filesystem:scrub status for tag:93fdd18c-2542-496b-ab74-9ea93ca55e19 - {'path': '/', 'tag': '93fdd18c-2542-496b-ab74-9ea93ca55e19', 'options': 'recursive,force'} 2023-10-08T13:52:03.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:03 smithi057 ceph-mon[99476]: pgmap v314: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-08T13:52:04.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:03 smithi099 ceph-mon[103849]: pgmap v314: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-08T13:52:04.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:03 smithi115 ceph-mon[104408]: pgmap v314: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-08T13:52:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:04 smithi099 ceph-mon[103849]: scrub queued for path: / 2023-10-08T13:52:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:04 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:04 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:52:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:04 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:05.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:04 smithi099 ceph-mon[103849]: scrub summary: idle 2023-10-08T13:52:05.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:04 smithi115 ceph-mon[104408]: scrub queued for path: / 2023-10-08T13:52:05.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:04 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:05.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:04 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:52:05.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:04 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:05.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:04 smithi115 ceph-mon[104408]: scrub summary: idle 2023-10-08T13:52:05.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:04 smithi057 ceph-mon[99476]: scrub queued for path: / 2023-10-08T13:52:05.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:04 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:05.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:04 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:52:05.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:04 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:05.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:04 smithi057 ceph-mon[99476]: scrub summary: idle 2023-10-08T13:52:06.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:05 smithi099 ceph-mon[103849]: pgmap v315: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2023-10-08T13:52:06.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:05 smithi115 ceph-mon[104408]: pgmap v315: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2023-10-08T13:52:06.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:05 smithi057 ceph-mon[99476]: pgmap v315: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2023-10-08T13:52:08.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:07 smithi099 ceph-mon[103849]: pgmap v316: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2023-10-08T13:52:08.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:07 smithi115 ceph-mon[104408]: pgmap v316: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2023-10-08T13:52:08.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:07 smithi057 ceph-mon[99476]: pgmap v316: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2023-10-08T13:52:10.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:09 smithi115 ceph-mon[104408]: pgmap v317: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 25 op/s 2023-10-08T13:52:10.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:09 smithi057 ceph-mon[99476]: pgmap v317: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 25 op/s 2023-10-08T13:52:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:09 smithi099 ceph-mon[103849]: pgmap v317: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 25 op/s 2023-10-08T13:52:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.706+0000 7fb7dadf5700 1 -- 172.21.15.57:0/3864955605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d4105690 msgr2=0x7fb7d4105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.706+0000 7fb7dadf5700 1 --2- 172.21.15.57:0/3864955605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d4105690 0x7fb7d4105a70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009990 tx=0x7fb7c8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.707+0000 7fb7dadf5700 1 -- 172.21.15.57:0/3864955605 shutdown_connections 2023-10-08T13:52:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.707+0000 7fb7dadf5700 1 --2- 172.21.15.57:0/3864955605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d40fffb0 0x7fb7d410c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.707+0000 7fb7dadf5700 1 --2- 172.21.15.57:0/3864955605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d4105690 0x7fb7d4105a70 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.707+0000 7fb7dadf5700 1 --2- 172.21.15.57:0/3864955605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d40ff5f0 0x7fb7d40ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.707+0000 7fb7dadf5700 1 -- 172.21.15.57:0/3864955605 >> 172.21.15.57:0/3864955605 conn(0x7fb7d4076790 msgr2=0x7fb7d4076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.707+0000 7fb7dadf5700 1 -- 172.21.15.57:0/3864955605 shutdown_connections 2023-10-08T13:52:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.707+0000 7fb7dadf5700 1 -- 172.21.15.57:0/3864955605 wait complete. 2023-10-08T13:52:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.707+0000 7fb7dadf5700 1 Processor -- start 2023-10-08T13:52:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7dadf5700 1 -- start start 2023-10-08T13:52:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7dadf5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d40ff5f0 0x7fb7d419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7dadf5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d40fffb0 0x7fb7d419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7dadf5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d4105690 0x7fb7d41957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7dadf5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4195e00 con 0x7fb7d40fffb0 2023-10-08T13:52:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7dadf5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4195f70 con 0x7fb7d40ff5f0 2023-10-08T13:52:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7dadf5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4196130 con 0x7fb7d4105690 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d40fffb0 0x7fb7d419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d40fffb0 0x7fb7d419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56830/0 (socket says 172.21.15.57:56830) 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7d3fff700 1 -- 172.21.15.57:0/2154618190 learned_addr learned my addr 172.21.15.57:0/2154618190 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7d8b91700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d40ff5f0 0x7fb7d419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7d9392700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d4105690 0x7fb7d41957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7d3fff700 1 -- 172.21.15.57:0/2154618190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d4105690 msgr2=0x7fb7d41957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d4105690 0x7fb7d41957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7d3fff700 1 -- 172.21.15.57:0/2154618190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d40ff5f0 msgr2=0x7fb7d419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d40ff5f0 0x7fb7d419b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.708+0000 7fb7d3fff700 1 -- 172.21.15.57:0/2154618190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c80095f0 con 0x7fb7d40fffb0 2023-10-08T13:52:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.709+0000 7fb7d9392700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d4105690 0x7fb7d41957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.709+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d40fffb0 0x7fb7d419bb60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c9b0 tx=0x7fb7c000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.709+0000 7fb7d1ffb700 1 -- 172.21.15.57:0/2154618190 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0007b00 con 0x7fb7d40fffb0 2023-10-08T13:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.709+0000 7fb7dadf5700 1 -- 172.21.15.57:0/2154618190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d4196410 con 0x7fb7d40fffb0 2023-10-08T13:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.709+0000 7fb7d1ffb700 1 -- 172.21.15.57:0/2154618190 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c0007c60 con 0x7fb7d40fffb0 2023-10-08T13:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.709+0000 7fb7dadf5700 1 -- 172.21.15.57:0/2154618190 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d4068af0 con 0x7fb7d40fffb0 2023-10-08T13:52:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.709+0000 7fb7d1ffb700 1 -- 172.21.15.57:0/2154618190 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0018660 con 0x7fb7d40fffb0 2023-10-08T13:52:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.712+0000 7fb7d1ffb700 1 -- 172.21.15.57:0/2154618190 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb7c0004830 con 0x7fb7d40fffb0 2023-10-08T13:52:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.712+0000 7fb7d1ffb700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7c40a15f0 0x7fb7c40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.713+0000 7fb7d8b91700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7c40a15f0 0x7fb7c40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.713+0000 7fb7d1ffb700 1 -- 172.21.15.57:0/2154618190 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb7c00cd240 con 0x7fb7d40fffb0 2023-10-08T13:52:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.713+0000 7fb7e6fdeb80 1 Processor -- start 2023-10-08T13:52:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.713+0000 7fb7e6fdeb80 1 -- start start 2023-10-08T13:52:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52765b400 con 0x55b5276f30b0 2023-10-08T13:52:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52764cbc0 con 0x55b5276f8880 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52764a4e0 con 0x55b5276f91a0 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d9392700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d9392700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56838/0 (socket says 172.21.15.57:56838) 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d9392700 1 -- 172.21.15.57:0/2795553332 learned_addr learned my addr 172.21.15.57:0/2795553332 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/2795553332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d9392700 1 -- 172.21.15.57:0/2795553332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 msgr2=0x55b5276f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d9392700 1 --2- 172.21.15.57:0/2795553332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d9392700 1 -- 172.21.15.57:0/2795553332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 msgr2=0x55b5276f8c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d9392700 1 --2- 172.21.15.57:0/2795553332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276f8c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d9392700 1 -- 172.21.15.57:0/2795553332 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b527470d80 con 0x55b5276f30b0 2023-10-08T13:52:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/2795553332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d8b91700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7c40a15f0 0x7fb7c40a3ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8006010 tx=0x7fb7c8005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.714+0000 7fb7d9392700 1 --2- 172.21.15.57:0/2795553332 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f3510 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009e50 tx=0x7fb7cc0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81126ee93e179a4a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.715+0000 7fb7beffd700 1 -- 172.21.15.57:0/2795553332 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc012660 con 0x55b5276f30b0 2023-10-08T13:52:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.715+0000 7fb7beffd700 1 -- 172.21.15.57:0/2795553332 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7cc012c80 con 0x55b5276f30b0 2023-10-08T13:52:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.715+0000 7fb7beffd700 1 -- 172.21.15.57:0/2795553332 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc011c80 con 0x55b5276f30b0 2023-10-08T13:52:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/2795553332 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 msgr2=0x55b5276f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/2795553332 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f3510 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009e50 tx=0x7fb7cc0077e0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/2795553332 shutdown_connections 2023-10-08T13:52:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/2795553332 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f3510 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/2795553332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/2795553332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/2795553332 >> 172.21.15.57:0/2795553332 conn(0x55b527667200 msgr2=0x55b5276edb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/2795553332 shutdown_connections 2023-10-08T13:52:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/2795553332 wait complete. 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 -- start start 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.716+0000 7fb7e6fdeb80 1 Processor -- start 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276ff120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b527645dd0 con 0x55b5276f30b0 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5275e26b0 con 0x55b5276f8880 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5275e2350 con 0x55b5276f91a0 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7e6fdeb80 10 client.0 _ll_register_callbacks cb 0x55b52743f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7d8b91700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276ff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7d8b91700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276ff120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56840/0 (socket says 172.21.15.57:56840) 2023-10-08T13:52:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7d8b91700 1 -- 172.21.15.57:0/4010608500 learned_addr learned my addr 172.21.15.57:0/4010608500 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7d9392700 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7d8b91700 1 -- 172.21.15.57:0/4010608500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 msgr2=0x55b5276fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.717+0000 7fb7d8b91700 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7d8b91700 1 -- 172.21.15.57:0/4010608500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 msgr2=0x55b5276fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7d8b91700 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7d8b91700 1 -- 172.21.15.57:0/4010608500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc009a80 con 0x55b5276f30b0 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7d8b91700 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276ff120 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800b5c0 tx=0x7fb7c8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37163fa64f01556f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7b3fff700 1 -- 172.21.15.57:0/4010608500 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8029070 con 0x55b5276f30b0 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7b3fff700 1 -- 172.21.15.57:0/4010608500 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb7c8003d80 con 0x55b5276f30b0 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/4010608500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b52760de50 con 0x55b5276f30b0 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7b3fff700 1 -- 172.21.15.57:0/4010608500 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7c800f460 con 0x55b5276f30b0 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7e6fdeb80 10 client.16062 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/4010608500 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b527459c00 con 0x55b5276f30b0 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.718+0000 7fb7b3fff700 1 -- 172.21.15.57:0/4010608500 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b52760de50 con 0x55b5276f30b0 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.719+0000 7fb7b3fff700 1 -- 172.21.15.57:0/4010608500 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb7c8035070 con 0x55b5276f30b0 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.719+0000 7fb7e6fdeb80 10 client.16062 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.719+0000 7fb7e6fdeb80 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.719+0000 7fb7e6fdeb80 20 client.16062 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.719+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b52770cb90 0x55b52770efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.719+0000 7fb7e6fdeb80 4 client.16062 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.719+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/4010608500 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b52763edb0 con 0x55b52770cb90 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.719+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b52770cb90 0x55b52770efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.719+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b52770cb90 0x55b52770efb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b69a65becab4eeb5 server_cookie=3336be295122ec54 in_seq=0 out_seq=0 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.719+0000 7fb7b3fff700 10 client.16062 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7b3fff700 1 -- 172.21.15.57:0/4010608500 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b52763edb0 con 0x55b52770cb90 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7b3fff700 10 client.16062 handle_command_reply: tid=0 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 client.16062 shutdown 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/4010608500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 msgr2=0x55b5276ff120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276ff120 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800b5c0 tx=0x7fb7c8009310 comp rx=0 tx=0).stop 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/4010608500 shutdown_connections 2023-10-08T13:52:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b52770cb90 0x55b52770efb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276ff120 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/4010608500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/4010608500 >> 172.21.15.57:0/4010608500 conn(0x55b527667200 msgr2=0x55b5276ed010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7b3fff700 0 client.16062 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/4010608500 shutdown_connections 2023-10-08T13:52:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.720+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/4010608500 wait complete. 2023-10-08T13:52:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.721+0000 7fb7e6fdeb80 20 client.16062 trim_cache size 0 max 16384 2023-10-08T13:52:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.738+0000 7fb7e6fdeb80 1 Processor -- start 2023-10-08T13:52:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.738+0000 7fb7e6fdeb80 1 -- start start 2023-10-08T13:52:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.738+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f30b0 0x55b5276f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.738+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b527703c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.738+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f91a0 0x55b5276ed790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.738+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5274c68c0 con 0x55b5276f91a0 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.738+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5276fda10 con 0x55b5276f8880 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.738+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5276ee940 con 0x55b5276f30b0 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b527703c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b527703c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:58276/0 (socket says 172.21.15.57:58276) 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d3fff700 1 -- 172.21.15.57:0/1612660941 learned_addr learned my addr 172.21.15.57:0/1612660941 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d8b91700 1 --2- 172.21.15.57:0/1612660941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f30b0 0x55b5276f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d9392700 1 --2- 172.21.15.57:0/1612660941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f91a0 0x55b5276ed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d3fff700 1 -- 172.21.15.57:0/1612660941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f30b0 msgr2=0x55b5276f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/1612660941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f30b0 0x55b5276f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d3fff700 1 -- 172.21.15.57:0/1612660941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f91a0 msgr2=0x55b5276ed790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/1612660941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f91a0 0x55b5276ed790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d3fff700 1 -- 172.21.15.57:0/1612660941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b527713bc0 con 0x55b5276f8880 2023-10-08T13:52:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d9392700 1 --2- 172.21.15.57:0/1612660941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f91a0 0x55b5276ed790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/1612660941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b527703c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00074a0 tx=0x7fb7c0096040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=602a4d5271ccfa96 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.739+0000 7fb7bf7fe700 1 -- 172.21.15.57:0/1612660941 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0007dd0 con 0x55b5276f8880 2023-10-08T13:52:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7bf7fe700 1 -- 172.21.15.57:0/1612660941 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c0099f20 con 0x55b5276f8880 2023-10-08T13:52:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7bf7fe700 1 -- 172.21.15.57:0/1612660941 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00980f0 con 0x55b5276f8880 2023-10-08T13:52:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/1612660941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 msgr2=0x55b527703c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/1612660941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b527703c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00074a0 tx=0x7fb7c0096040 comp rx=0 tx=0).stop 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/1612660941 shutdown_connections 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/1612660941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f91a0 0x55b5276ed790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/1612660941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f30b0 0x55b5276f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/1612660941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b527703c40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/1612660941 >> 172.21.15.57:0/1612660941 conn(0x55b527667200 msgr2=0x55b5276f6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/1612660941 shutdown_connections 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.740+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/1612660941 wait complete. 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7e6fdeb80 1 -- start start 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7e6fdeb80 1 Processor -- start 2023-10-08T13:52:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7e6fdeb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5276f54f0 con 0x55b5276f30b0 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5276f5660 con 0x55b5276f8880 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7e6fdeb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5276f5820 con 0x55b5276f91a0 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7e6fdeb80 10 client.0 _ll_register_callbacks cb 0x7fb7c80050d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7d8b91700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7d8b91700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56854/0 (socket says 172.21.15.57:56854) 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.741+0000 7fb7d8b91700 1 -- 172.21.15.57:0/724079860 learned_addr learned my addr 172.21.15.57:0/724079860 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7d9392700 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7d8b91700 1 -- 172.21.15.57:0/724079860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 msgr2=0x55b5276f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7d8b91700 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7d8b91700 1 -- 172.21.15.57:0/724079860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 msgr2=0x55b5276f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7d8b91700 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7d8b91700 1 -- 172.21.15.57:0/724079860 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d4068af0 con 0x55b5276f30b0 2023-10-08T13:52:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7d8b91700 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f4390 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80227e0 tx=0x7fb7c8003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ec84ae2843ff1c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7b3fff700 1 -- 172.21.15.57:0/724079860 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8029070 con 0x55b5276f30b0 2023-10-08T13:52:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/724079860 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7cc012c80 con 0x55b5276f30b0 2023-10-08T13:52:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7b3fff700 1 -- 172.21.15.57:0/724079860 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb7c800f460 con 0x55b5276f30b0 2023-10-08T13:52:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7b3fff700 1 -- 172.21.15.57:0/724079860 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7c800f620 con 0x55b5276f30b0 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7e6fdeb80 10 client.16068 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.742+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/724079860 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b5274971f0 con 0x55b5276f30b0 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.743+0000 7fb7b3fff700 1 -- 172.21.15.57:0/724079860 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7cc012c80 con 0x55b5276f30b0 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.743+0000 7fb7b3fff700 1 -- 172.21.15.57:0/724079860 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb7c8035070 con 0x55b5276f30b0 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.743+0000 7fb7e6fdeb80 10 client.16068 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.743+0000 7fb7e6fdeb80 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.743+0000 7fb7e6fdeb80 20 client.16068 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.743+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b5277612f0 0x55b527763710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.743+0000 7fb7e6fdeb80 4 client.16068 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.743+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/724079860 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b52763ead0 con 0x55b5277612f0 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.743+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b5277612f0 0x55b527763710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.744+0000 7fb7d3fff700 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b5277612f0 0x55b527763710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fe18af3414e7fe5 server_cookie=383fcc1167deae8e in_seq=0 out_seq=0 2023-10-08T13:52:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.744+0000 7fb7b3fff700 10 client.16068 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7b3fff700 1 -- 172.21.15.57:0/724079860 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b52763ead0 con 0x55b5277612f0 2023-10-08T13:52:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7b3fff700 10 client.16068 handle_command_reply: tid=0 2023-10-08T13:52:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 client.16068 shutdown 2023-10-08T13:52:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/724079860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 msgr2=0x55b5276f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f4390 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80227e0 tx=0x7fb7c8003ae0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/724079860 shutdown_connections 2023-10-08T13:52:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b5277612f0 0x55b527763710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5276f91a0 0x55b5276f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7b3fff700 0 client.16068 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b5276f30b0 0x55b5276f4390 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 --2- 172.21.15.57:0/724079860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5276f8880 0x55b5276f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/724079860 >> 172.21.15.57:0/724079860 conn(0x55b527667200 msgr2=0x55b5276fab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/724079860 shutdown_connections 2023-10-08T13:52:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 1 -- 172.21.15.57:0/724079860 wait complete. 2023-10-08T13:52:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.752+0000 7fb7e6fdeb80 20 client.16068 trim_cache size 0 max 16384 2023-10-08T13:52:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:52:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:52:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:52:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:52:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 -- 172.21.15.57:0/2154618190 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7c40a15f0 msgr2=0x7fb7c40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7c40a15f0 0x7fb7c40a3ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8006010 tx=0x7fb7c8005960 comp rx=0 tx=0).stop 2023-10-08T13:52:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 -- 172.21.15.57:0/2154618190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d40fffb0 msgr2=0x7fb7d419bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d40fffb0 0x7fb7d419bb60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c9b0 tx=0x7fb7c000ccc0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 -- 172.21.15.57:0/2154618190 shutdown_connections 2023-10-08T13:52:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d4105690 0x7fb7d41957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb7c40a15f0 0x7fb7c40a3ab0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d40ff5f0 0x7fb7d419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 --2- 172.21.15.57:0/2154618190 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7d40fffb0 0x7fb7d419bb60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 -- 172.21.15.57:0/2154618190 >> 172.21.15.57:0/2154618190 conn(0x7fb7d4076790 msgr2=0x7fb7d40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 -- 172.21.15.57:0/2154618190 shutdown_connections 2023-10-08T13:52:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:11.753+0000 7fb7dadf5700 1 -- 172.21.15.57:0/2154618190 wait complete. 2023-10-08T13:52:12.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:11 smithi115 ceph-mon[104408]: pgmap v318: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2023-10-08T13:52:12.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:11 smithi057 ceph-mon[99476]: pgmap v318: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2023-10-08T13:52:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:11 smithi099 ceph-mon[103849]: pgmap v318: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2023-10-08T13:52:14.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:13 smithi115 ceph-mon[104408]: pgmap v319: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-10-08T13:52:14.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:13 smithi057 ceph-mon[99476]: pgmap v319: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-10-08T13:52:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:13 smithi099 ceph-mon[103849]: pgmap v319: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-10-08T13:52:16.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:15 smithi115 ceph-mon[104408]: pgmap v320: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 27 op/s 2023-10-08T13:52:16.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:15 smithi057 ceph-mon[99476]: pgmap v320: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 27 op/s 2023-10-08T13:52:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:15 smithi099 ceph-mon[103849]: pgmap v320: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 27 op/s 2023-10-08T13:52:18.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:17 smithi115 ceph-mon[104408]: pgmap v321: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-10-08T13:52:18.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:17 smithi057 ceph-mon[99476]: pgmap v321: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-10-08T13:52:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:17 smithi099 ceph-mon[103849]: pgmap v321: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-10-08T13:52:20.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:19 smithi115 ceph-mon[104408]: pgmap v322: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-08T13:52:20.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:19 smithi057 ceph-mon[99476]: pgmap v322: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-08T13:52:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:19 smithi099 ceph-mon[103849]: pgmap v322: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-08T13:52:22.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:21 smithi115 ceph-mon[104408]: pgmap v323: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2023-10-08T13:52:22.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:21 smithi057 ceph-mon[99476]: pgmap v323: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2023-10-08T13:52:22.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:21 smithi099 ceph-mon[103849]: pgmap v323: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2023-10-08T13:52:24.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:23 smithi115 ceph-mon[104408]: pgmap v324: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-10-08T13:52:24.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:23 smithi057 ceph-mon[99476]: pgmap v324: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-10-08T13:52:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:23 smithi099 ceph-mon[103849]: pgmap v324: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-10-08T13:52:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.924+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/2023890821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eb81010a0 msgr2=0x7f1eb810d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.924+0000 7f1ebf5bf700 1 --2- 172.21.15.57:0/2023890821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eb81010a0 0x7f1eb810d5b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000b0a0 tx=0x7f1eb000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.925+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/2023890821 shutdown_connections 2023-10-08T13:52:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.925+0000 7f1ebf5bf700 1 --2- 172.21.15.57:0/2023890821 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eb81010a0 0x7f1eb810d5b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.925+0000 7f1ebf5bf700 1 --2- 172.21.15.57:0/2023890821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1eb81066f0 0x7f1eb8106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.925+0000 7f1ebf5bf700 1 --2- 172.21.15.57:0/2023890821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eb81006e0 0x7f1eb8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.925+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/2023890821 >> 172.21.15.57:0/2023890821 conn(0x7f1eb80fc1f0 msgr2=0x7f1eb80fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.925+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/2023890821 shutdown_connections 2023-10-08T13:52:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.925+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/2023890821 wait complete. 2023-10-08T13:52:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebf5bf700 1 Processor -- start 2023-10-08T13:52:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebf5bf700 1 -- start start 2023-10-08T13:52:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebf5bf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1eb81006e0 0x7f1eb819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebf5bf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eb81010a0 0x7f1eb819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebf5bf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eb81066f0 0x7f1eb81968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebf5bf700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8196f40 con 0x7f1eb81066f0 2023-10-08T13:52:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebf5bf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb81970b0 con 0x7f1eb81010a0 2023-10-08T13:52:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebf5bf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8197270 con 0x7f1eb81006e0 2023-10-08T13:52:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebd35b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1eb81006e0 0x7f1eb819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebd35b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1eb81006e0 0x7f1eb819c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:57680/0 (socket says 172.21.15.57:57680) 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebd35b700 1 -- 172.21.15.57:0/3339756885 learned_addr learned my addr 172.21.15.57:0/3339756885 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebcb5a700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eb81010a0 0x7f1eb819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.926+0000 7f1ebdb5c700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eb81066f0 0x7f1eb81968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1ebd35b700 1 -- 172.21.15.57:0/3339756885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eb81010a0 msgr2=0x7f1eb819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eb81010a0 0x7f1eb819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1ebd35b700 1 -- 172.21.15.57:0/3339756885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eb81066f0 msgr2=0x7f1eb81968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eb81066f0 0x7f1eb81968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1ebd35b700 1 -- 172.21.15.57:0/3339756885 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0009d00 con 0x7f1eb81006e0 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1ebcb5a700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eb81010a0 0x7f1eb819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1eb81006e0 0x7f1eb819c7b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d950 tx=0x7f1eb400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1eae7fc700 1 -- 172.21.15.57:0/3339756885 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4009940 con 0x7f1eb81006e0 2023-10-08T13:52:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/3339756885 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb8197550 con 0x7f1eb81006e0 2023-10-08T13:52:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1eae7fc700 1 -- 172.21.15.57:0/3339756885 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb4010460 con 0x7f1eb81006e0 2023-10-08T13:52:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1eae7fc700 1 -- 172.21.15.57:0/3339756885 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400f620 con 0x7f1eb81006e0 2023-10-08T13:52:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.927+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/3339756885 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb81a37a0 con 0x7f1eb81006e0 2023-10-08T13:52:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.930+0000 7f1eae7fc700 1 -- 172.21.15.57:0/3339756885 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1eb4009aa0 con 0x7f1eb81006e0 2023-10-08T13:52:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.931+0000 7f1eae7fc700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1ea40a15f0 0x7f1ea40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.931+0000 7f1eae7fc700 1 -- 172.21.15.57:0/3339756885 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1eb40cd1a0 con 0x7f1eb81006e0 2023-10-08T13:52:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.931+0000 7f1ebcb5a700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1ea40a15f0 0x7f1ea40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.932+0000 7f1ecb7a8b80 1 Processor -- start 2023-10-08T13:52:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.932+0000 7f1ecb7a8b80 1 -- start start 2023-10-08T13:52:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.932+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d797b880 0x5578d797bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d797c1a0 0x5578d7975b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d79760b0 0x5578d7976510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d78de400 con 0x5578d79760b0 2023-10-08T13:52:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d78cfbc0 con 0x5578d797b880 2023-10-08T13:52:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d78cd4e0 con 0x5578d797c1a0 2023-10-08T13:52:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ebdb5c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d79760b0 0x5578d7976510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ebdb5c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d79760b0 0x5578d7976510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56878/0 (socket says 172.21.15.57:56878) 2023-10-08T13:52:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ebdb5c700 1 -- 172.21.15.57:0/3423662473 learned_addr learned my addr 172.21.15.57:0/3423662473 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ebdb5c700 1 -- 172.21.15.57:0/3423662473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d797c1a0 msgr2=0x5578d7975b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:52:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ebdb5c700 1 --2- 172.21.15.57:0/3423662473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d797c1a0 0x5578d7975b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ebdb5c700 1 -- 172.21.15.57:0/3423662473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d797b880 msgr2=0x5578d797bc60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:52:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ebdb5c700 1 --2- 172.21.15.57:0/3423662473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d797b880 0x5578d797bc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ebdb5c700 1 -- 172.21.15.57:0/3423662473 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578d76f3d80 con 0x5578d79760b0 2023-10-08T13:52:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ebdb5c700 1 --2- 172.21.15.57:0/3423662473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d79760b0 0x5578d7976510 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009500 tx=0x7f1eb0007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ee51f28cd47552 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.933+0000 7f1ea37fe700 1 -- 172.21.15.57:0/3423662473 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000e040 con 0x5578d79760b0 2023-10-08T13:52:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.934+0000 7f1ea37fe700 1 -- 172.21.15.57:0/3423662473 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb0007e50 con 0x5578d79760b0 2023-10-08T13:52:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.934+0000 7f1ea37fe700 1 -- 172.21.15.57:0/3423662473 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb001c440 con 0x5578d79760b0 2023-10-08T13:52:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.934+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/3423662473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d79760b0 msgr2=0x5578d7976510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.934+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/3423662473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d79760b0 0x5578d7976510 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009500 tx=0x7f1eb0007b80 comp rx=0 tx=0).stop 2023-10-08T13:52:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.935+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/3423662473 shutdown_connections 2023-10-08T13:52:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.935+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/3423662473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d79760b0 0x5578d7976510 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.935+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/3423662473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d797b880 0x5578d797bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.935+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/3423662473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d797c1a0 0x5578d7975b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.935+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/3423662473 >> 172.21.15.57:0/3423662473 conn(0x5578d78ea200 msgr2=0x5578d7970b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.936+0000 7f1ebcb5a700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1ea40a15f0 0x7f1ea40a3ab0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8197aa0 tx=0x7f1ea8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.936+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/3423662473 shutdown_connections 2023-10-08T13:52:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.936+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/3423662473 wait complete. 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.936+0000 7f1ecb7a8b80 1 -- start start 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.936+0000 7f1ecb7a8b80 1 Processor -- start 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d797f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d797b880 0x5578d797fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d7980080 0x5578d798bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d78c8dd0 con 0x5578d797b880 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d78656b0 con 0x5578d79760b0 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d7865350 con 0x5578d7980080 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ecb7a8b80 10 client.0 _ll_register_callbacks cb 0x5578d76c27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ebd35b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d797f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ebd35b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d797f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:58308/0 (socket says 172.21.15.57:58308) 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ebd35b700 1 -- 172.21.15.57:0/1474356410 learned_addr learned my addr 172.21.15.57:0/1474356410 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ebcb5a700 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d797b880 0x5578d797fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ebd35b700 1 -- 172.21.15.57:0/1474356410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d7980080 msgr2=0x5578d798bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d7980080 0x5578d798bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ebd35b700 1 -- 172.21.15.57:0/1474356410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d797b880 msgr2=0x5578d797fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d797b880 0x5578d797fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.937+0000 7f1ebd35b700 1 -- 172.21.15.57:0/1474356410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb810deb0 con 0x5578d79760b0 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d797f600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40a1ce0 tx=0x7f1eb409a600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b69d28d53570c3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ea0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1474356410 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb409a2e0 con 0x5578d79760b0 2023-10-08T13:52:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1474356410 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1eb40a2210 con 0x5578d79760b0 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1474356410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578d7890e50 con 0x5578d79760b0 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1474356410 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1eb4099370 con 0x5578d79760b0 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ecb7a8b80 10 client.25706 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1474356410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5578d76dcc00 con 0x5578d79760b0 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1474356410 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578d7890e50 con 0x5578d79760b0 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1474356410 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1eb4098de0 con 0x5578d79760b0 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ecb7a8b80 10 client.25706 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ecb7a8b80 10 client.25706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.938+0000 7f1ecb7a8b80 20 client.25706 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.939+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578d7990070 0x5578d7992490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.939+0000 7f1ecb7a8b80 4 client.25706 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.939+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1474356410 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578d78c1db0 con 0x5578d7990070 2023-10-08T13:52:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.939+0000 7f1ebcb5a700 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578d7990070 0x5578d7992490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.939+0000 7f1ea0ff9700 10 client.25706 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.939+0000 7f1ebcb5a700 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578d7990070 0x5578d7992490 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d1b78fc7912f8b server_cookie=c6ea79e0fc28ba91 in_seq=0 out_seq=0 2023-10-08T13:52:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1474356410 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5578d78c1db0 con 0x5578d7990070 2023-10-08T13:52:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ea0ff9700 10 client.25706 handle_command_reply: tid=0 2023-10-08T13:52:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ecb7a8b80 1 client.25706 shutdown 2023-10-08T13:52:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1474356410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 msgr2=0x5578d797f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d797f600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40a1ce0 tx=0x7f1eb409a600 comp rx=0 tx=0).stop 2023-10-08T13:52:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1474356410 shutdown_connections 2023-10-08T13:52:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578d7990070 0x5578d7992490 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d7980080 0x5578d798bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d797f600 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1474356410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d797b880 0x5578d797fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1474356410 >> 172.21.15.57:0/1474356410 conn(0x5578d78ea200 msgr2=0x5578d7970010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.940+0000 7f1ea0ff9700 0 client.25706 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.941+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1474356410 shutdown_connections 2023-10-08T13:52:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.941+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1474356410 wait complete. 2023-10-08T13:52:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.941+0000 7f1ecb7a8b80 20 client.25706 trim_cache size 0 max 16384 2023-10-08T13:52:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.969+0000 7f1ecb7a8b80 1 Processor -- start 2023-10-08T13:52:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.969+0000 7f1ecb7a8b80 1 -- start start 2023-10-08T13:52:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d7975b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d797b880 0x5578d797af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d7980080 0x5578d797eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d77498c0 con 0x5578d797b880 2023-10-08T13:52:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d797f450 con 0x5578d79760b0 2023-10-08T13:52:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d797c000 con 0x5578d7980080 2023-10-08T13:52:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ebd35b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d7975b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ebdb5c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d7980080 0x5578d797eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ebd35b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d7975b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:58316/0 (socket says 172.21.15.57:58316) 2023-10-08T13:52:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ebd35b700 1 -- 172.21.15.57:0/2021955804 learned_addr learned my addr 172.21.15.57:0/2021955804 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ebd35b700 1 -- 172.21.15.57:0/2021955804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d7980080 msgr2=0x5578d797eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/2021955804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d7980080 0x5578d797eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ebd35b700 1 -- 172.21.15.57:0/2021955804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d797b880 msgr2=0x5578d797af50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:52:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/2021955804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d797b880 0x5578d797af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ebd35b700 1 -- 172.21.15.57:0/2021955804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578d771a1f0 con 0x5578d79760b0 2023-10-08T13:52:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.970+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/2021955804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d7975b70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40ca6a0 tx=0x7f1eb40962f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79fde94ab1c8960f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.971+0000 7f1ea3fff700 1 -- 172.21.15.57:0/2021955804 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4014070 con 0x5578d79760b0 2023-10-08T13:52:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.971+0000 7f1ea3fff700 1 -- 172.21.15.57:0/2021955804 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb40a2210 con 0x5578d79760b0 2023-10-08T13:52:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.971+0000 7f1ea3fff700 1 -- 172.21.15.57:0/2021955804 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40a1fe0 con 0x5578d79760b0 2023-10-08T13:52:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.972+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/2021955804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 msgr2=0x5578d7975b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.972+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/2021955804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d7975b70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40ca6a0 tx=0x7f1eb40962f0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.976+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/2021955804 shutdown_connections 2023-10-08T13:52:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.976+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/2021955804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d7980080 0x5578d797eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.976+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/2021955804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d79760b0 0x5578d7975b70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.976+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/2021955804 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d797b880 0x5578d797af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.976+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/2021955804 >> 172.21.15.57:0/2021955804 conn(0x5578d78ea200 msgr2=0x5578d7980ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.976+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/2021955804 shutdown_connections 2023-10-08T13:52:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.976+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/2021955804 wait complete. 2023-10-08T13:52:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.976+0000 7f1ecb7a8b80 1 -- start start 2023-10-08T13:52:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.976+0000 7f1ecb7a8b80 1 Processor -- start 2023-10-08T13:52:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d79760b0 0x5578d79866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d797b880 0x5578d7986be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ecb7a8b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d7980080 0x5578d7984c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d7985340 con 0x5578d7980080 2023-10-08T13:52:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d79854b0 con 0x5578d797b880 2023-10-08T13:52:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ecb7a8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d7985670 con 0x5578d79760b0 2023-10-08T13:52:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ecb7a8b80 10 client.0 _ll_register_callbacks cb 0x7f1eb40999c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ebd35b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d79760b0 0x5578d79866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ebd35b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d79760b0 0x5578d79866a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:57706/0 (socket says 172.21.15.57:57706) 2023-10-08T13:52:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ebd35b700 1 -- 172.21.15.57:0/1720235392 learned_addr learned my addr 172.21.15.57:0/1720235392 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ebdb5c700 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d7980080 0x5578d7984c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ebcb5a700 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d797b880 0x5578d7986be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ebdb5c700 1 -- 172.21.15.57:0/1720235392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d79760b0 msgr2=0x5578d79866a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ebdb5c700 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d79760b0 0x5578d79866a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ebdb5c700 1 -- 172.21.15.57:0/1720235392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d797b880 msgr2=0x5578d7986be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.977+0000 7f1ebdb5c700 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d797b880 0x5578d7986be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.978+0000 7f1ebdb5c700 1 -- 172.21.15.57:0/1720235392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578d76dcc00 con 0x5578d7980080 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.978+0000 7f1ebdb5c700 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d7980080 0x5578d7984c00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0007ca0 tx=0x7f1eb000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22e8220ee0b9690 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.978+0000 7f1ea0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.978+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1720235392 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000e040 con 0x5578d7980080 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.978+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1720235392 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1eb00078b0 con 0x5578d7980080 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.978+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1720235392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1eb0007e50 con 0x5578d7980080 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.978+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1720235392 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1eb0005050 con 0x5578d7980080 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.978+0000 7f1ecb7a8b80 10 client.16083 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.978+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1720235392 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5578d76f2250 con 0x5578d7980080 2023-10-08T13:52:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.978+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1720235392 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb0007e50 con 0x5578d7980080 2023-10-08T13:52:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.979+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1720235392 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1eb0019030 con 0x5578d7980080 2023-10-08T13:52:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.979+0000 7f1ecb7a8b80 10 client.16083 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.979+0000 7f1ecb7a8b80 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.979+0000 7f1ecb7a8b80 20 client.16083 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.979+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578d79d9ff0 0x5578d79dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.979+0000 7f1ecb7a8b80 4 client.16083 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:52:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.979+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1720235392 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5578d78c1ad0 con 0x5578d79d9ff0 2023-10-08T13:52:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.979+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578d79d9ff0 0x5578d79dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.979+0000 7f1ea0ff9700 10 client.16083 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.979+0000 7f1ebd35b700 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578d79d9ff0 0x5578d79dc3e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738ed1687c72c07c server_cookie=205f27390c9425fb in_seq=0 out_seq=0 2023-10-08T13:52:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.983+0000 7f1ea0ff9700 1 -- 172.21.15.57:0/1720235392 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5578d78c1ad0 con 0x5578d79d9ff0 2023-10-08T13:52:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.983+0000 7f1ea0ff9700 10 client.16083 handle_command_reply: tid=0 2023-10-08T13:52:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.983+0000 7f1ecb7a8b80 1 client.16083 shutdown 2023-10-08T13:52:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.983+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1720235392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d7980080 msgr2=0x5578d7984c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.983+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d7980080 0x5578d7984c00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0007ca0 tx=0x7f1eb000cda0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.984+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1720235392 shutdown_connections 2023-10-08T13:52:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.984+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578d79d9ff0 0x5578d79dc3e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.984+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578d7980080 0x5578d7984c00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.984+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d79760b0 0x5578d79866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.984+0000 7f1ecb7a8b80 1 --2- 172.21.15.57:0/1720235392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d797b880 0x5578d7986be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.984+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1720235392 >> 172.21.15.57:0/1720235392 conn(0x5578d78ea200 msgr2=0x5578d797cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.984+0000 7f1ea0ff9700 0 client.16083 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.984+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1720235392 shutdown_connections 2023-10-08T13:52:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.984+0000 7f1ecb7a8b80 1 -- 172.21.15.57:0/1720235392 wait complete. 2023-10-08T13:52:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ecb7a8b80 20 client.16083 trim_cache size 0 max 16384 2023-10-08T13:52:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:52:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:52:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:52:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:52:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/3339756885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1ea40a15f0 msgr2=0x7f1ea40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1ea40a15f0 0x7f1ea40a3ab0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8197aa0 tx=0x7f1ea8009260 comp rx=0 tx=0).stop 2023-10-08T13:52:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/3339756885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1eb81006e0 msgr2=0x7f1eb819c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1eb81006e0 0x7f1eb819c7b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d950 tx=0x7f1eb400dc60 comp rx=0 tx=0).stop 2023-10-08T13:52:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/3339756885 shutdown_connections 2023-10-08T13:52:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eb81066f0 0x7f1eb81968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1eb81006e0 0x7f1eb819c7b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1ea40a15f0 0x7f1ea40a3ab0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 --2- 172.21.15.57:0/3339756885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1eb81010a0 0x7f1eb819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/3339756885 >> 172.21.15.57:0/3339756885 conn(0x7f1eb80fc1f0 msgr2=0x7f1eb80fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.985+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/3339756885 shutdown_connections 2023-10-08T13:52:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:24.986+0000 7f1ebf5bf700 1 -- 172.21.15.57:0/3339756885 wait complete. 2023-10-08T13:52:26.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:25 smithi115 ceph-mon[104408]: pgmap v325: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-10-08T13:52:26.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:25 smithi057 ceph-mon[99476]: pgmap v325: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-10-08T13:52:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:25 smithi099 ceph-mon[103849]: pgmap v325: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-10-08T13:52:28.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:27 smithi115 ceph-mon[104408]: pgmap v326: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T13:52:28.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:27 smithi057 ceph-mon[99476]: pgmap v326: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T13:52:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:27 smithi099 ceph-mon[103849]: pgmap v326: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T13:52:30.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:29 smithi115 ceph-mon[104408]: pgmap v327: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-10-08T13:52:30.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:29 smithi057 ceph-mon[99476]: pgmap v327: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-10-08T13:52:30.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:29 smithi099 ceph-mon[103849]: pgmap v327: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-10-08T13:52:32.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:31 smithi115 ceph-mon[104408]: pgmap v328: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-10-08T13:52:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:31 smithi057 ceph-mon[99476]: pgmap v328: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-10-08T13:52:32.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:31 smithi099 ceph-mon[103849]: pgmap v328: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-10-08T13:52:33.571 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:52:33.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.704+0000 7f0111c5b700 1 -- 172.21.15.57:0/3884345430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f010c101100 msgr2=0x7f010c10d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.704+0000 7f0111c5b700 1 --2- 172.21.15.57:0/3884345430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f010c101100 0x7f010c10d610 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f010400b0a0 tx=0x7f010400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.704+0000 7f0111c5b700 1 -- 172.21.15.57:0/3884345430 shutdown_connections 2023-10-08T13:52:33.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.704+0000 7f0111c5b700 1 --2- 172.21.15.57:0/3884345430 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f010c101100 0x7f010c10d610 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.704+0000 7f0111c5b700 1 --2- 172.21.15.57:0/3884345430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f010c106750 0x7f010c106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.704+0000 7f0111c5b700 1 --2- 172.21.15.57:0/3884345430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f010c100740 0x7f010c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.704+0000 7f0111c5b700 1 -- 172.21.15.57:0/3884345430 >> 172.21.15.57:0/3884345430 conn(0x7f010c0fc230 msgr2=0x7f010c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:33.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.704+0000 7f0111c5b700 1 -- 172.21.15.57:0/3884345430 shutdown_connections 2023-10-08T13:52:33.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.704+0000 7f0111c5b700 1 -- 172.21.15.57:0/3884345430 wait complete. 2023-10-08T13:52:33.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f0111c5b700 1 Processor -- start 2023-10-08T13:52:33.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f0111c5b700 1 -- start start 2023-10-08T13:52:33.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f0111c5b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f010c100740 0x7f010c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f0111c5b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f010c101100 0x7f010c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f0111c5b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f010c106750 0x7f010c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f0111c5b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c1971e0 con 0x7f010c101100 2023-10-08T13:52:33.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f0111c5b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c197350 con 0x7f010c100740 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f0111c5b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c197510 con 0x7f010c106750 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f010affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f010c101100 0x7f010c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f010affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f010c101100 0x7f010c19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56908/0 (socket says 172.21.15.57:56908) 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f010affd700 1 -- 172.21.15.57:0/1375987780 learned_addr learned my addr 172.21.15.57:0/1375987780 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f010b7fe700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f010c100740 0x7f010c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.705+0000 7f010bfff700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f010c106750 0x7f010c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f010affd700 1 -- 172.21.15.57:0/1375987780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f010c106750 msgr2=0x7f010c196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f010affd700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f010c106750 0x7f010c196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f010affd700 1 -- 172.21.15.57:0/1375987780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f010c100740 msgr2=0x7f010c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f010affd700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f010c100740 0x7f010c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f010affd700 1 -- 172.21.15.57:0/1375987780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0104009d00 con 0x7f010c101100 2023-10-08T13:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f010bfff700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f010c106750 0x7f010c196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f010b7fe700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f010c100740 0x7f010c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f010affd700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f010c101100 0x7f010c19ceb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f010000d950 tx=0x7f010000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f0108ff9700 1 -- 172.21.15.57:0/1375987780 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100009b40 con 0x7f010c101100 2023-10-08T13:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f0111c5b700 1 -- 172.21.15.57:0/1375987780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c1977f0 con 0x7f010c101100 2023-10-08T13:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f0108ff9700 1 -- 172.21.15.57:0/1375987780 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0100009ca0 con 0x7f010c101100 2023-10-08T13:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f0108ff9700 1 -- 172.21.15.57:0/1375987780 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000f620 con 0x7f010c101100 2023-10-08T13:52:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.706+0000 7f0111c5b700 1 -- 172.21.15.57:0/1375987780 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c1a38f0 con 0x7f010c101100 2023-10-08T13:52:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.708+0000 7f0108ff9700 1 -- 172.21.15.57:0/1375987780 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0100010460 con 0x7f010c101100 2023-10-08T13:52:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.709+0000 7f0108ff9700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00f40a15f0 0x7f00f40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.709+0000 7f010b7fe700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00f40a15f0 0x7f00f40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.709+0000 7f0108ff9700 1 -- 172.21.15.57:0/1375987780 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f01000cd7f0 con 0x7f010c101100 2023-10-08T13:52:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.709+0000 7f010b7fe700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00f40a15f0 0x7f00f40a3ab0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005e10 tx=0x7f00fc005d60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.711+0000 7f011de44b80 1 Processor -- start 2023-10-08T13:52:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.711+0000 7f011de44b80 1 -- start start 2023-10-08T13:52:33.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.711+0000 7f011de44b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bdc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.711+0000 7f011de44b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2be7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.711+0000 7f011de44b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 0x55d6e2be9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.711+0000 7f011de44b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2b45460 con 0x55d6e2bdbe80 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.711+0000 7f011de44b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2b36c20 con 0x55d6e2be7650 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.711+0000 7f011de44b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2b34540 con 0x55d6e2bdc7a0 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bdc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bdc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56914/0 (socket says 172.21.15.57:56914) 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010b7fe700 1 -- 172.21.15.57:0/3878261736 learned_addr learned my addr 172.21.15.57:0/3878261736 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010affd700 1 --2- 172.21.15.57:0/3878261736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2be7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010bfff700 1 --2- 172.21.15.57:0/3878261736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 0x55d6e2be9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010b7fe700 1 -- 172.21.15.57:0/3878261736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdc7a0 msgr2=0x55d6e2be7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010b7fe700 1 --2- 172.21.15.57:0/3878261736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2be7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010b7fe700 1 -- 172.21.15.57:0/3878261736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 msgr2=0x55d6e2be9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:33.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010b7fe700 1 --2- 172.21.15.57:0/3878261736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 0x55d6e2be9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010b7fe700 1 -- 172.21.15.57:0/3878261736 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6e295ade0 con 0x55d6e2bdbe80 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010bfff700 1 --2- 172.21.15.57:0/3878261736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 0x55d6e2be9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f010b7fe700 1 --2- 172.21.15.57:0/3878261736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bdc260 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0044e0 tx=0x7f00fc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5048ce7376b220d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f00f1ffb700 1 -- 172.21.15.57:0/3878261736 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc017040 con 0x55d6e2bdbe80 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f00f1ffb700 1 -- 172.21.15.57:0/3878261736 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00fc004510 con 0x55d6e2bdbe80 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.712+0000 7f00f1ffb700 1 -- 172.21.15.57:0/3878261736 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc01fc10 con 0x55d6e2bdbe80 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.713+0000 7f011de44b80 1 -- 172.21.15.57:0/3878261736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 msgr2=0x55d6e2bdc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.713+0000 7f011de44b80 1 --2- 172.21.15.57:0/3878261736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bdc260 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0044e0 tx=0x7f00fc012000 comp rx=0 tx=0).stop 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.713+0000 7f011de44b80 1 -- 172.21.15.57:0/3878261736 shutdown_connections 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.713+0000 7f011de44b80 1 --2- 172.21.15.57:0/3878261736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 0x55d6e2be9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.713+0000 7f011de44b80 1 --2- 172.21.15.57:0/3878261736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bdc260 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.713+0000 7f011de44b80 1 --2- 172.21.15.57:0/3878261736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2be7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.713+0000 7f011de44b80 1 -- 172.21.15.57:0/3878261736 >> 172.21.15.57:0/3878261736 conn(0x55d6e2b51260 msgr2=0x55d6e2bd82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:33.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.713+0000 7f011de44b80 1 -- 172.21.15.57:0/3878261736 shutdown_connections 2023-10-08T13:52:33.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.713+0000 7f011de44b80 1 -- 172.21.15.57:0/3878261736 wait complete. 2023-10-08T13:52:33.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.713+0000 7f011de44b80 1 -- start start 2023-10-08T13:52:33.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f011de44b80 1 Processor -- start 2023-10-08T13:52:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f011de44b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdbe80 0x55d6e2be70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f011de44b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2be4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f011de44b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2be7650 0x55d6e2be4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f011de44b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2b2fe30 con 0x55d6e2be7650 2023-10-08T13:52:33.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f011de44b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2acc710 con 0x55d6e2bdbe80 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f011de44b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2acc3b0 con 0x55d6e2bdc7a0 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f011de44b80 10 client.0 _ll_register_callbacks cb 0x55d6e2929800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f010bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2be7650 0x55d6e2be4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f010bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2be7650 0x55d6e2be4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56924/0 (socket says 172.21.15.57:56924) 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f010bfff700 1 -- 172.21.15.57:0/18198605 learned_addr learned my addr 172.21.15.57:0/18198605 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f010b7fe700 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdbe80 0x55d6e2be70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f010affd700 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2be4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f010bfff700 1 -- 172.21.15.57:0/18198605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdc7a0 msgr2=0x55d6e2be4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f010bfff700 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2be4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f010bfff700 1 -- 172.21.15.57:0/18198605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdbe80 msgr2=0x55d6e2be70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f010bfff700 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdbe80 0x55d6e2be70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.714+0000 7f010bfff700 1 -- 172.21.15.57:0/18198605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc0041c0 con 0x55d6e2be7650 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.715+0000 7f010bfff700 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2be7650 0x55d6e2be4b50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f010400b070 tx=0x7f01040095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1143d9eeb65a453 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.715+0000 7f00eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.715+0000 7f00eaffd700 1 -- 172.21.15.57:0/18198605 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010400e050 con 0x55d6e2be7650 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.715+0000 7f011de44b80 1 -- 172.21.15.57:0/18198605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6e2af7eb0 con 0x55d6e2be7650 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.715+0000 7f00eaffd700 1 -- 172.21.15.57:0/18198605 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f01040076e0 con 0x55d6e2be7650 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.715+0000 7f00eaffd700 1 -- 172.21.15.57:0/18198605 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01040078a0 con 0x55d6e2be7650 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.715+0000 7f011de44b80 10 client.16095 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.715+0000 7f011de44b80 1 -- 172.21.15.57:0/18198605 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d6e2943c60 con 0x55d6e2be7650 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.715+0000 7f00eaffd700 1 -- 172.21.15.57:0/18198605 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d6e2af7eb0 con 0x55d6e2be7650 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.715+0000 7f00eaffd700 1 -- 172.21.15.57:0/18198605 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f010400b0d0 con 0x55d6e2be7650 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.716+0000 7f011de44b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.716+0000 7f011de44b80 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.716+0000 7f011de44b80 20 client.16095 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.716+0000 7f011de44b80 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55d6e2bf7100 0x55d6e2bf9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.716+0000 7f011de44b80 4 client.16095 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.716+0000 7f011de44b80 1 -- 172.21.15.57:0/18198605 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6e2b28e10 con 0x55d6e2bf7100 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.716+0000 7f010b7fe700 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55d6e2bf7100 0x55d6e2bf9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.716+0000 7f00eaffd700 10 client.16095 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.716+0000 7f010b7fe700 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55d6e2bf7100 0x55d6e2bf9520 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d61b0d4b3ba2e66f server_cookie=b9b32298d87f16e6 in_seq=0 out_seq=0 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f00eaffd700 1 -- 172.21.15.57:0/18198605 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d6e2b28e10 con 0x55d6e2bf7100 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f00eaffd700 10 client.16095 handle_command_reply: tid=0 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 client.16095 shutdown 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 -- 172.21.15.57:0/18198605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2be7650 msgr2=0x55d6e2be4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2be7650 0x55d6e2be4b50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f010400b070 tx=0x7f01040095d0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 -- 172.21.15.57:0/18198605 shutdown_connections 2023-10-08T13:52:33.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55d6e2bf7100 0x55d6e2bf9520 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2be7650 0x55d6e2be4b50 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdbe80 0x55d6e2be70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 --2- 172.21.15.57:0/18198605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2be4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f00eaffd700 0 client.16095 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:33.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 -- 172.21.15.57:0/18198605 >> 172.21.15.57:0/18198605 conn(0x55d6e2b51260 msgr2=0x55d6e2bd7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:33.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 -- 172.21.15.57:0/18198605 shutdown_connections 2023-10-08T13:52:33.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 1 -- 172.21.15.57:0/18198605 wait complete. 2023-10-08T13:52:33.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.717+0000 7f011de44b80 20 client.16095 trim_cache size 0 max 16384 2023-10-08T13:52:33.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.734+0000 7f011de44b80 1 Processor -- start 2023-10-08T13:52:33.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.734+0000 7f011de44b80 1 -- start start 2023-10-08T13:52:33.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.734+0000 7f011de44b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdbe80 0x55d6e2be3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.734+0000 7f011de44b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2bded50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.734+0000 7f011de44b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 0x55d6e2bf28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.734+0000 7f011de44b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e29b0920 con 0x55d6e2bdc7a0 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.734+0000 7f011de44b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2bd4b10 con 0x55d6e2be7650 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.734+0000 7f011de44b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2be42d0 con 0x55d6e2bdbe80 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.734+0000 7f010affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2bded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2bded50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56930/0 (socket says 172.21.15.57:56930) 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010affd700 1 -- 172.21.15.57:0/2182913977 learned_addr learned my addr 172.21.15.57:0/2182913977 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010b7fe700 1 --2- 172.21.15.57:0/2182913977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdbe80 0x55d6e2be3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010bfff700 1 --2- 172.21.15.57:0/2182913977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 0x55d6e2bf28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010affd700 1 -- 172.21.15.57:0/2182913977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdbe80 msgr2=0x55d6e2be3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:33.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010affd700 1 --2- 172.21.15.57:0/2182913977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdbe80 0x55d6e2be3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010affd700 1 -- 172.21.15.57:0/2182913977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 msgr2=0x55d6e2bf28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:33.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010affd700 1 --2- 172.21.15.57:0/2182913977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 0x55d6e2bf28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010affd700 1 -- 172.21.15.57:0/2182913977 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6e2981250 con 0x55d6e2bdc7a0 2023-10-08T13:52:33.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010bfff700 1 --2- 172.21.15.57:0/2182913977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 0x55d6e2bf28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:33.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f010affd700 1 --2- 172.21.15.57:0/2182913977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2bded50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f01000961f0 tx=0x7f010000df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa01b35f834a73ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:33.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f00f27fc700 1 -- 172.21.15.57:0/2182913977 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010009a470 con 0x55d6e2bdc7a0 2023-10-08T13:52:33.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f00f27fc700 1 -- 172.21.15.57:0/2182913977 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f010009e5c0 con 0x55d6e2bdc7a0 2023-10-08T13:52:33.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.735+0000 7f00f27fc700 1 -- 172.21.15.57:0/2182913977 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010009e170 con 0x55d6e2bdc7a0 2023-10-08T13:52:33.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.736+0000 7f011de44b80 1 -- 172.21.15.57:0/2182913977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdc7a0 msgr2=0x55d6e2bded50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.736+0000 7f011de44b80 1 --2- 172.21.15.57:0/2182913977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2bded50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f01000961f0 tx=0x7f010000df70 comp rx=0 tx=0).stop 2023-10-08T13:52:33.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.736+0000 7f011de44b80 1 -- 172.21.15.57:0/2182913977 shutdown_connections 2023-10-08T13:52:33.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.736+0000 7f011de44b80 1 --2- 172.21.15.57:0/2182913977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2be7650 0x55d6e2bf28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.736+0000 7f011de44b80 1 --2- 172.21.15.57:0/2182913977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2bdbe80 0x55d6e2be3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.736+0000 7f011de44b80 1 --2- 172.21.15.57:0/2182913977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2bded50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.736+0000 7f011de44b80 1 -- 172.21.15.57:0/2182913977 >> 172.21.15.57:0/2182913977 conn(0x55d6e2b51260 msgr2=0x55d6e2bd3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:33.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.736+0000 7f011de44b80 1 -- 172.21.15.57:0/2182913977 shutdown_connections 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.736+0000 7f011de44b80 1 -- 172.21.15.57:0/2182913977 wait complete. 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.736+0000 7f011de44b80 1 -- start start 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f011de44b80 1 Processor -- start 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f011de44b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bec9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f011de44b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2becf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f011de44b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2be7650 0x55d6e2bed470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f011de44b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2c4f3e0 con 0x55d6e2bdbe80 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f011de44b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2c4f520 con 0x55d6e2bdc7a0 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f011de44b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6e2c4f6e0 con 0x55d6e2be7650 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f011de44b80 10 client.0 _ll_register_callbacks cb 0x7f0104009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bec9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56934/0 (socket says 172.21.15.57:56934) 2023-10-08T13:52:33.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010b7fe700 1 -- 172.21.15.57:0/937161417 learned_addr learned my addr 172.21.15.57:0/937161417 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:33.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010affd700 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2becf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010bfff700 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2be7650 0x55d6e2bed470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010b7fe700 1 -- 172.21.15.57:0/937161417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2be7650 msgr2=0x55d6e2bed470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:33.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010b7fe700 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2be7650 0x55d6e2bed470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010b7fe700 1 -- 172.21.15.57:0/937161417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdc7a0 msgr2=0x55d6e2becf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:33.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010b7fe700 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2becf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010b7fe700 1 -- 172.21.15.57:0/937161417 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c1a38f0 con 0x55d6e2bdbe80 2023-10-08T13:52:33.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.737+0000 7f010affd700 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2becf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:33.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f010b7fe700 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bec9f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f00fc01f800 tx=0x7f00fc01f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7edac764a0ac4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:33.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f00eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f00eaffd700 1 -- 172.21.15.57:0/937161417 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc030040 con 0x55d6e2bdbe80 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f00eaffd700 1 -- 172.21.15.57:0/937161417 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f00fc01fc20 con 0x55d6e2bdbe80 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f011de44b80 1 -- 172.21.15.57:0/937161417 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00fc004510 con 0x55d6e2bdbe80 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f00eaffd700 1 -- 172.21.15.57:0/937161417 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00fc034730 con 0x55d6e2bdbe80 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f011de44b80 10 client.16101 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f011de44b80 1 -- 172.21.15.57:0/937161417 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d6e29592b0 con 0x55d6e2bdbe80 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f00eaffd700 1 -- 172.21.15.57:0/937161417 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00fc004510 con 0x55d6e2bdbe80 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f00eaffd700 1 -- 172.21.15.57:0/937161417 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f00fc03a020 con 0x55d6e2bdbe80 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f011de44b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f011de44b80 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f011de44b80 20 client.16101 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:33.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f011de44b80 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55d6e2c50c00 0x55d6e2c52ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f011de44b80 4 client.16101 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.738+0000 7f011de44b80 1 -- 172.21.15.57:0/937161417 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d6e2b28b30 con 0x55d6e2c50c00 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.739+0000 7f010affd700 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55d6e2c50c00 0x55d6e2c52ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.739+0000 7f010affd700 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55d6e2c50c00 0x55d6e2c52ff0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8da3dd9bcf18d6 server_cookie=3025c051bc5400be in_seq=0 out_seq=0 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.739+0000 7f00eaffd700 10 client.16101 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.739+0000 7f00eaffd700 1 -- 172.21.15.57:0/937161417 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d6e2b28b30 con 0x55d6e2c50c00 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.739+0000 7f00eaffd700 10 client.16101 handle_command_reply: tid=0 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.739+0000 7f011de44b80 1 client.16101 shutdown 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 1 -- 172.21.15.57:0/937161417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 msgr2=0x55d6e2bec9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bec9f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f00fc01f800 tx=0x7f00fc01f460 comp rx=0 tx=0).stop 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 1 -- 172.21.15.57:0/937161417 shutdown_connections 2023-10-08T13:52:33.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55d6e2c50c00 0x55d6e2c52ff0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6e2be7650 0x55d6e2bed470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d6e2bdbe80 0x55d6e2bec9f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 1 --2- 172.21.15.57:0/937161417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6e2bdc7a0 0x55d6e2becf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 1 -- 172.21.15.57:0/937161417 >> 172.21.15.57:0/937161417 conn(0x55d6e2b51260 msgr2=0x55d6e2bf28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f00eaffd700 0 client.16101 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 1 -- 172.21.15.57:0/937161417 shutdown_connections 2023-10-08T13:52:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 1 -- 172.21.15.57:0/937161417 wait complete. 2023-10-08T13:52:33.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f011de44b80 20 client.16101 trim_cache size 0 max 16384 2023-10-08T13:52:33.749 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:52:33.749 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-10-08T13:52:33.749 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-10-08T13:52:33.749 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:52:33.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f0111c5b700 1 -- 172.21.15.57:0/1375987780 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00f40a15f0 msgr2=0x7f00f40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f0111c5b700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00f40a15f0 0x7f00f40a3ab0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005e10 tx=0x7f00fc005d60 comp rx=0 tx=0).stop 2023-10-08T13:52:33.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f0111c5b700 1 -- 172.21.15.57:0/1375987780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f010c101100 msgr2=0x7f010c19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f0111c5b700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f010c101100 0x7f010c19ceb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f010000d950 tx=0x7f010000dd10 comp rx=0 tx=0).stop 2023-10-08T13:52:33.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f0111c5b700 1 -- 172.21.15.57:0/1375987780 shutdown_connections 2023-10-08T13:52:33.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f0111c5b700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f010c106750 0x7f010c196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f0111c5b700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f00f40a15f0 0x7f00f40a3ab0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f0111c5b700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f010c100740 0x7f010c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f0111c5b700 1 --2- 172.21.15.57:0/1375987780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f010c101100 0x7f010c19ceb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.740+0000 7f0111c5b700 1 -- 172.21.15.57:0/1375987780 >> 172.21.15.57:0/1375987780 conn(0x7f010c0fc230 msgr2=0x7f010c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:33.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.741+0000 7f0111c5b700 1 -- 172.21.15.57:0/1375987780 shutdown_connections 2023-10-08T13:52:33.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.741+0000 7f0111c5b700 1 -- 172.21.15.57:0/1375987780 wait complete. 2023-10-08T13:52:33.753 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-08T13:52:33.754 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:52:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.883+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1011358709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2cb8100ff0 msgr2=0x7f2cb810d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.883+0000 7f2cbde0b700 1 --2- 172.21.15.57:0/1011358709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2cb8100ff0 0x7f2cb810d500 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000b0a0 tx=0x7f2cb000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.884+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1011358709 shutdown_connections 2023-10-08T13:52:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.884+0000 7f2cbde0b700 1 --2- 172.21.15.57:0/1011358709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2cb8100ff0 0x7f2cb810d500 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.884+0000 7f2cbde0b700 1 --2- 172.21.15.57:0/1011358709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cb8106640 0x7f2cb8106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.884+0000 7f2cbde0b700 1 --2- 172.21.15.57:0/1011358709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cb8100630 0x7f2cb8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.884+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1011358709 >> 172.21.15.57:0/1011358709 conn(0x7f2cb80fc140 msgr2=0x7f2cb80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:33.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.884+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1011358709 shutdown_connections 2023-10-08T13:52:33.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.884+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1011358709 wait complete. 2023-10-08T13:52:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.884+0000 7f2cbde0b700 1 Processor -- start 2023-10-08T13:52:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cbde0b700 1 -- start start 2023-10-08T13:52:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cbde0b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2cb8100630 0x7f2cb8196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cbde0b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cb8100ff0 0x7f2cb8196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cbde0b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cb8106640 0x7f2cb819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cbde0b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb819b910 con 0x7f2cb8100630 2023-10-08T13:52:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cbde0b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb819ba80 con 0x7f2cb8100ff0 2023-10-08T13:52:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cbde0b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb819bc40 con 0x7f2cb8106640 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cb77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2cb8100630 0x7f2cb8196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cb77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2cb8100630 0x7f2cb8196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56948/0 (socket says 172.21.15.57:56948) 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cb77fe700 1 -- 172.21.15.57:0/1363793679 learned_addr learned my addr 172.21.15.57:0/1363793679 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cb6ffd700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cb8100ff0 0x7f2cb8196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cb77fe700 1 -- 172.21.15.57:0/1363793679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cb8106640 msgr2=0x7f2cb819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cb7fff700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cb8106640 0x7f2cb819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cb77fe700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cb8106640 0x7f2cb819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cb77fe700 1 -- 172.21.15.57:0/1363793679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cb8100ff0 msgr2=0x7f2cb8196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cb77fe700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cb8100ff0 0x7f2cb8196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.885+0000 7f2cb77fe700 1 -- 172.21.15.57:0/1363793679 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb0009d00 con 0x7f2cb8100630 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.886+0000 7f2cb7fff700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cb8106640 0x7f2cb819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.886+0000 7f2cb6ffd700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cb8100ff0 0x7f2cb8196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.886+0000 7f2cb77fe700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2cb8100630 0x7f2cb8196a80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800d950 tx=0x7f2ca800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.886+0000 7f2cb4ff9700 1 -- 172.21.15.57:0/1363793679 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8009b40 con 0x7f2cb8100630 2023-10-08T13:52:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.886+0000 7f2cb4ff9700 1 -- 172.21.15.57:0/1363793679 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca8009ca0 con 0x7f2cb8100630 2023-10-08T13:52:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.886+0000 7f2cb4ff9700 1 -- 172.21.15.57:0/1363793679 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800f620 con 0x7f2cb8100630 2023-10-08T13:52:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.886+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1363793679 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb819bf20 con 0x7f2cb8100630 2023-10-08T13:52:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.886+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1363793679 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb81a37d0 con 0x7f2cb8100630 2023-10-08T13:52:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.887+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1363793679 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb8108c50 con 0x7f2cb8100630 2023-10-08T13:52:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.888+0000 7f2cb4ff9700 1 -- 172.21.15.57:0/1363793679 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2ca8010460 con 0x7f2cb8100630 2023-10-08T13:52:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.889+0000 7f2cb4ff9700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ca00a1670 0x7f2ca00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.889+0000 7f2cb4ff9700 1 -- 172.21.15.57:0/1363793679 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f2ca80cd6e0 con 0x7f2cb8100630 2023-10-08T13:52:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.889+0000 7f2cb6ffd700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ca00a1670 0x7f2ca00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.889+0000 7f2cb6ffd700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ca00a1670 0x7f2ca00a3b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8197f80 tx=0x7f2cac00b5f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:33.893+0000 7f2cb4ff9700 1 -- 172.21.15.57:0/1363793679 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2ca80966a0 con 0x7f2cb8100630 2023-10-08T13:52:33.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:33 smithi057 ceph-mon[99476]: pgmap v329: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-08T13:52:34.148 INFO:tasks.workunit.client.0.smithi057.stderr:Updating files: 96% (11594/12065) Updating files: 97% (11704/12065) Updating files: 98% (11824/12065) Updating files: 99% (11945/12065) Updating files: 100% (12065/12065) Updating files: 100% (12065/12065), done. 2023-10-08T13:52:34.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.159+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1363793679 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cb8101d70 con 0x7f2cb8100630 2023-10-08T13:52:34.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.160+0000 7f2cb4ff9700 1 -- 172.21.15.57:0/1363793679 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f2ca8099cc0 con 0x7f2cb8100630 2023-10-08T13:52:34.161 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:52:34.161 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:50:07.719573+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:52:34.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1363793679 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ca00a1670 msgr2=0x7f2ca00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ca00a1670 0x7f2ca00a3b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8197f80 tx=0x7f2cac00b5f0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1363793679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2cb8100630 msgr2=0x7f2cb8196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2cb8100630 0x7f2cb8196a80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800d950 tx=0x7f2ca800dd10 comp rx=0 tx=0).stop 2023-10-08T13:52:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1363793679 shutdown_connections 2023-10-08T13:52:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cb8106640 0x7f2cb819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2cb8100630 0x7f2cb8196a80 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2ca00a1670 0x7f2ca00a3b30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 --2- 172.21.15.57:0/1363793679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cb8100ff0 0x7f2cb8196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1363793679 >> 172.21.15.57:0/1363793679 conn(0x7f2cb80fc140 msgr2=0x7f2cb80fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1363793679 shutdown_connections 2023-10-08T13:52:34.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.162+0000 7f2cbde0b700 1 -- 172.21.15.57:0/1363793679 wait complete. 2023-10-08T13:52:34.165 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 27 2023-10-08T13:52:34.176 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-08T13:52:34.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:33 smithi115 ceph-mon[104408]: pgmap v329: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-08T13:52:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:33 smithi099 ceph-mon[103849]: pgmap v329: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-10-08T13:52:34.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.308+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2715864418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c141051c0 msgr2=0x7f4c14105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.308+0000 7f4c1bd16700 1 --2- 172.21.15.57:0/2715864418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c141051c0 0x7f4c14105640 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4c040099c0 tx=0x7f4c04009cd0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.308+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2715864418 shutdown_connections 2023-10-08T13:52:34.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.308+0000 7f4c1bd16700 1 --2- 172.21.15.57:0/2715864418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c14105b80 0x7f4c1410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.308+0000 7f4c1bd16700 1 --2- 172.21.15.57:0/2715864418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c14067ce0 0x7f4c140680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.308+0000 7f4c1bd16700 1 --2- 172.21.15.57:0/2715864418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c141051c0 0x7f4c14105640 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.308+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2715864418 >> 172.21.15.57:0/2715864418 conn(0x7f4c14076780 msgr2=0x7f4c14076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.309+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2715864418 shutdown_connections 2023-10-08T13:52:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.309+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2715864418 wait complete. 2023-10-08T13:52:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.309+0000 7f4c1bd16700 1 Processor -- start 2023-10-08T13:52:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.309+0000 7f4c1bd16700 1 -- start start 2023-10-08T13:52:34.312 INFO:tasks.workunit.client.0.smithi057.stderr:Note: switching to '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de'. 2023-10-08T13:52:34.312 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-08T13:52:34.312 INFO:tasks.workunit.client.0.smithi057.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-08T13:52:34.312 INFO:tasks.workunit.client.0.smithi057.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-08T13:52:34.312 INFO:tasks.workunit.client.0.smithi057.stderr:state without impacting any branches by switching back to a branch. 2023-10-08T13:52:34.312 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-08T13:52:34.312 INFO:tasks.workunit.client.0.smithi057.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr: git switch -c 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr:Or undo this operation with: 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr: git switch - 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-08T13:52:34.313 INFO:tasks.workunit.client.0.smithi057.stderr:HEAD is now at 7b3143bae3b Merge PR #48130 into wip-batrick-testing-20231007.154158 2023-10-08T13:52:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1bd16700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c14067ce0 0x7f4c1419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1bd16700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c141051c0 0x7f4c1419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1bd16700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c14105b80 0x7f4c14196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1bd16700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14197060 con 0x7f4c14105b80 2023-10-08T13:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1bd16700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c141971d0 con 0x7f4c141051c0 2023-10-08T13:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1bd16700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14197390 con 0x7f4c14067ce0 2023-10-08T13:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1a2b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c14105b80 0x7f4c14196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1a2b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c14105b80 0x7f4c14196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56966/0 (socket says 172.21.15.57:56966) 2023-10-08T13:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/2517624423 learned_addr learned my addr 172.21.15.57:0/2517624423 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c14067ce0 0x7f4c1419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c192b1700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c141051c0 0x7f4c1419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/2517624423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c14067ce0 msgr2=0x7f4c1419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c14067ce0 0x7f4c1419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/2517624423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c141051c0 msgr2=0x7f4c1419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c141051c0 0x7f4c1419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/2517624423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c040095f0 con 0x7f4c14105b80 2023-10-08T13:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.310+0000 7f4c192b1700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c141051c0 0x7f4c1419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.311+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c14105b80 0x7f4c14196a00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c007ee0 tx=0x7f4c0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.311+0000 7f4c0affd700 1 -- 172.21.15.57:0/2517624423 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c017070 con 0x7f4c14105b80 2023-10-08T13:52:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.311+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2517624423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c14197670 con 0x7f4c14105b80 2023-10-08T13:52:34.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.311+0000 7f4c0affd700 1 -- 172.21.15.57:0/2517624423 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c0c00f040 con 0x7f4c14105b80 2023-10-08T13:52:34.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.311+0000 7f4c0affd700 1 -- 172.21.15.57:0/2517624423 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c005740 con 0x7f4c14105b80 2023-10-08T13:52:34.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.311+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2517624423 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c141a38c0 con 0x7f4c14105b80 2023-10-08T13:52:34.319 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-08T13:52:34.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.313+0000 7f4c0affd700 1 -- 172.21.15.57:0/2517624423 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4c0c00dc60 con 0x7f4c14105b80 2023-10-08T13:52:34.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.314+0000 7f4c0affd700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4c000a15f0 0x7f4c000a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.314+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4c000a15f0 0x7f4c000a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.314+0000 7f4c0affd700 1 -- 172.21.15.57:0/2517624423 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4c0c0ccb80 con 0x7f4c14105b80 2023-10-08T13:52:34.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.314+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4c000a15f0 0x7f4c000a3ab0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4c10007be0 tx=0x7f4c100073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.314+0000 7f4c27effb80 1 Processor -- start 2023-10-08T13:52:34.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c27effb80 1 -- start start 2023-10-08T13:52:34.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 0x55fed000f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000f7a0 0x55fed001a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c27effb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecff78460 con 0x55fed001a650 2023-10-08T13:52:34.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c27effb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecff69c20 con 0x55fed000f7a0 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c27effb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecff67540 con 0x55fed000ee80 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c1a2b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c1a2b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56974/0 (socket says 172.21.15.57:56974) 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/378477750 learned_addr learned my addr 172.21.15.57:0/378477750 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/378477750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 0x55fed000f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/378477750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 msgr2=0x55fed000f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/378477750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 0x55fed000f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c192b1700 1 --2- 172.21.15.57:0/378477750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000f7a0 0x55fed001a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/378477750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000f7a0 msgr2=0x55fed001a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/378477750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000f7a0 0x55fed001a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.315+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/378477750 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fecfd8dde0 con 0x55fed001a650 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.316+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/378477750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001ca40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0a1660 tx=0x7f4c0c00df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9cac081b567e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.316+0000 7f4bfffff700 1 -- 172.21.15.57:0/378477750 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c09d950 con 0x55fed001a650 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.316+0000 7f4bfffff700 1 -- 172.21.15.57:0/378477750 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c0c0a1b90 con 0x55fed001a650 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.316+0000 7f4bfffff700 1 -- 172.21.15.57:0/378477750 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0842e0 con 0x55fed001a650 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.316+0000 7f4c27effb80 1 -- 172.21.15.57:0/378477750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 msgr2=0x55fed001ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.316+0000 7f4c27effb80 1 --2- 172.21.15.57:0/378477750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001ca40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0a1660 tx=0x7f4c0c00df60 comp rx=0 tx=0).stop 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.317+0000 7f4c27effb80 1 -- 172.21.15.57:0/378477750 shutdown_connections 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.317+0000 7f4c27effb80 1 --2- 172.21.15.57:0/378477750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001ca40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.317+0000 7f4c27effb80 1 --2- 172.21.15.57:0/378477750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 0x55fed000f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.317+0000 7f4c27effb80 1 --2- 172.21.15.57:0/378477750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000f7a0 0x55fed001a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.317+0000 7f4c27effb80 1 -- 172.21.15.57:0/378477750 >> 172.21.15.57:0/378477750 conn(0x55fecff84260 msgr2=0x55fed000b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.317+0000 7f4c27effb80 1 -- 172.21.15.57:0/378477750 shutdown_connections 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.317+0000 7f4c27effb80 1 -- 172.21.15.57:0/378477750 wait complete. 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.317+0000 7f4c27effb80 1 -- start start 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.317+0000 7f4c27effb80 1 Processor -- start 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000ee80 0x55fed001a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000f7a0 0x55fed00173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed001a650 0x55fed0017b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c27effb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecff62e30 con 0x55fed000ee80 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c27effb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecfeff710 con 0x55fed000f7a0 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c27effb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecfeff3b0 con 0x55fed001a650 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c27effb80 10 client.0 _ll_register_callbacks cb 0x55fecfd5c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:34.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c19ab2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000ee80 0x55fed001a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c19ab2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000ee80 0x55fed001a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56976/0 (socket says 172.21.15.57:56976) 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c19ab2700 1 -- 172.21.15.57:0/4124763525 learned_addr learned my addr 172.21.15.57:0/4124763525 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c192b1700 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000f7a0 0x55fed00173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed001a650 0x55fed0017b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c19ab2700 1 -- 172.21.15.57:0/4124763525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed001a650 msgr2=0x55fed0017b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed001a650 0x55fed0017b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c19ab2700 1 -- 172.21.15.57:0/4124763525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000f7a0 msgr2=0x55fed00173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000f7a0 0x55fed00173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c19ab2700 1 -- 172.21.15.57:0/4124763525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c141a38c0 con 0x55fed000ee80 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000ee80 0x55fed001a100 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4c100042e0 tx=0x7f4c100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7bef1d7b32079b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4bfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/4124763525 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10003a90 con 0x55fed000ee80 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.318+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/4124763525 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4c10017070 con 0x55fed000ee80 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4c27effb80 1 -- 172.21.15.57:0/4124763525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fecff2aeb0 con 0x55fed000ee80 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/4124763525 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c10003bf0 con 0x55fed000ee80 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4c27effb80 10 client.16119 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4c27effb80 1 -- 172.21.15.57:0/4124763525 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fecfd76c60 con 0x55fed000ee80 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/4124763525 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fecff2aeb0 con 0x55fed000ee80 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/4124763525 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4c10020410 con 0x55fed000ee80 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4c27effb80 10 client.16119 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4c27effb80 10 client.16119 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4c27effb80 20 client.16119 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:34.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4c27effb80 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55fed002a210 0x55fed002c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4c27effb80 4 client.16119 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4c27effb80 1 -- 172.21.15.57:0/4124763525 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fecff5be10 con 0x55fed002a210 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.319+0000 7f4c192b1700 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55fed002a210 0x55fed002c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.320+0000 7f4bfd7fa700 10 client.16119 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.320+0000 7f4c192b1700 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55fed002a210 0x55fed002c630 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d36f953d37bd6f7 server_cookie=c0dd75b2134171c6 in_seq=0 out_seq=0 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.320+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/4124763525 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fecff5be10 con 0x55fed002a210 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.320+0000 7f4bfd7fa700 10 client.16119 handle_command_reply: tid=0 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 client.16119 shutdown 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 -- 172.21.15.57:0/4124763525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000ee80 msgr2=0x55fed001a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000ee80 0x55fed001a100 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4c100042e0 tx=0x7f4c100047c0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 -- 172.21.15.57:0/4124763525 shutdown_connections 2023-10-08T13:52:34.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55fed002a210 0x55fed002c630 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed001a650 0x55fed0017b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000ee80 0x55fed001a100 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 --2- 172.21.15.57:0/4124763525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000f7a0 0x55fed00173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4bfd7fa700 0 client.16119 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:52:34.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 -- 172.21.15.57:0/4124763525 >> 172.21.15.57:0/4124763525 conn(0x55fecff84260 msgr2=0x55fed000ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 -- 172.21.15.57:0/4124763525 shutdown_connections 2023-10-08T13:52:34.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 1 -- 172.21.15.57:0/4124763525 wait complete. 2023-10-08T13:52:34.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.321+0000 7f4c27effb80 20 client.16119 trim_cache size 0 max 16384 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.334+0000 7f4c27effb80 1 Processor -- start 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.334+0000 7f4c27effb80 1 -- start start 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 0x55fed0017b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000f7a0 0x55fed00211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed001a650 0x55fed0019c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c27effb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecfde3920 con 0x55fed000f7a0 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c27effb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed0027920 con 0x55fed001a650 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c27effb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed0011db0 con 0x55fed000ee80 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c192b1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000f7a0 0x55fed00211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c192b1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000f7a0 0x55fed00211a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56986/0 (socket says 172.21.15.57:56986) 2023-10-08T13:52:34.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c192b1700 1 -- 172.21.15.57:0/1663731502 learned_addr learned my addr 172.21.15.57:0/1663731502 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/1663731502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed001a650 0x55fed0019c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/1663731502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 0x55fed0017b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c192b1700 1 -- 172.21.15.57:0/1663731502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 msgr2=0x55fed0017b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c192b1700 1 --2- 172.21.15.57:0/1663731502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 0x55fed0017b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c192b1700 1 -- 172.21.15.57:0/1663731502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed001a650 msgr2=0x55fed0019c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c192b1700 1 --2- 172.21.15.57:0/1663731502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed001a650 0x55fed0019c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c192b1700 1 -- 172.21.15.57:0/1663731502 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fecfdb4250 con 0x55fed000f7a0 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/1663731502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 0x55fed0017b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.335+0000 7f4c192b1700 1 --2- 172.21.15.57:0/1663731502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000f7a0 0x55fed00211a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4c040098e0 tx=0x7f4c04005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb61e77123bf7d3e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c08ff9700 1 -- 172.21.15.57:0/1663731502 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04029af0 con 0x55fed000f7a0 2023-10-08T13:52:34.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c08ff9700 1 -- 172.21.15.57:0/1663731502 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c04029c50 con 0x55fed000f7a0 2023-10-08T13:52:34.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c08ff9700 1 -- 172.21.15.57:0/1663731502 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0400b990 con 0x55fed000f7a0 2023-10-08T13:52:34.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c27effb80 1 -- 172.21.15.57:0/1663731502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000f7a0 msgr2=0x55fed00211a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c27effb80 1 --2- 172.21.15.57:0/1663731502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000f7a0 0x55fed00211a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4c040098e0 tx=0x7f4c04005e80 comp rx=0 tx=0).stop 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c27effb80 1 -- 172.21.15.57:0/1663731502 shutdown_connections 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c27effb80 1 --2- 172.21.15.57:0/1663731502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed001a650 0x55fed0019c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c27effb80 1 --2- 172.21.15.57:0/1663731502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000ee80 0x55fed0017b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c27effb80 1 --2- 172.21.15.57:0/1663731502 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed000f7a0 0x55fed00211a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c27effb80 1 -- 172.21.15.57:0/1663731502 >> 172.21.15.57:0/1663731502 conn(0x55fecff84260 msgr2=0x55fed0012750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c27effb80 1 -- 172.21.15.57:0/1663731502 shutdown_connections 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.336+0000 7f4c27effb80 1 -- 172.21.15.57:0/1663731502 wait complete. 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c27effb80 1 -- start start 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c27effb80 1 Processor -- start 2023-10-08T13:52:34.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000ee80 0x55fed00208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000f7a0 0x55fed0020e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c27effb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c27effb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed001f550 con 0x55fed001a650 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c27effb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed001f6c0 con 0x55fed000ee80 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c27effb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed001f880 con 0x55fed000f7a0 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c27effb80 10 client.0 _ll_register_callbacks cb 0x7f4c10010cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c1a2b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c1a2b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56994/0 (socket says 172.21.15.57:56994) 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.337+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/1700522258 learned_addr learned my addr 172.21.15.57:0/1700522258 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c192b1700 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000f7a0 0x55fed0020e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000ee80 0x55fed00208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/1700522258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000f7a0 msgr2=0x55fed0020e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000f7a0 0x55fed0020e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/1700522258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000ee80 msgr2=0x55fed00208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000ee80 0x55fed00208e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c1a2b3700 1 -- 172.21.15.57:0/1700522258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1410dec0 con 0x55fed001a650 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c1a2b3700 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001eec0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0a1660 tx=0x7f4c0c007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fee9126af6a1cc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4bfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/1700522258 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c005aa0 con 0x55fed001a650 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/1700522258 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4c0c022250 con 0x55fed001a650 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c27effb80 1 -- 172.21.15.57:0/1700522258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c0c0a1b90 con 0x55fed001a650 2023-10-08T13:52:34.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/1700522258 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c0c022410 con 0x55fed001a650 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c27effb80 10 client.16125 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.338+0000 7f4c27effb80 1 -- 172.21.15.57:0/1700522258 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fecfd8c2b0 con 0x55fed001a650 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/1700522258 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c0c0a1b90 con 0x55fed001a650 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/1700522258 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4c0c099a60 con 0x55fed001a650 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4c27effb80 10 client.16125 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4c27effb80 10 client.16125 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4c27effb80 20 client.16125 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4c27effb80 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55fed007ed90 0x55fed0081180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4c27effb80 4 client.16125 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4c27effb80 1 -- 172.21.15.57:0/1700522258 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fecff5bb30 con 0x55fed007ed90 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55fed007ed90 0x55fed0081180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4bfd7fa700 10 client.16125 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.339+0000 7f4c19ab2700 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55fed007ed90 0x55fed0081180 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b104c09199ece153 server_cookie=de5b957123f5bbe2 in_seq=0 out_seq=0 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4bfd7fa700 1 -- 172.21.15.57:0/1700522258 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fecff5bb30 con 0x55fed007ed90 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4bfd7fa700 10 client.16125 handle_command_reply: tid=0 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 client.16125 shutdown 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 -- 172.21.15.57:0/1700522258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 msgr2=0x55fed001eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001eec0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0a1660 tx=0x7f4c0c007f10 comp rx=0 tx=0).stop 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 -- 172.21.15.57:0/1700522258 shutdown_connections 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55fed007ed90 0x55fed0081180 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fed001a650 0x55fed001eec0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fed000ee80 0x55fed00208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 --2- 172.21.15.57:0/1700522258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fed000f7a0 0x55fed0020e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4bfd7fa700 0 client.16125 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:52:34.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 -- 172.21.15.57:0/1700522258 >> 172.21.15.57:0/1700522258 conn(0x55fecff84260 msgr2=0x55fed0010d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 -- 172.21.15.57:0/1700522258 shutdown_connections 2023-10-08T13:52:34.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 1 -- 172.21.15.57:0/1700522258 wait complete. 2023-10-08T13:52:34.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.340+0000 7f4c27effb80 20 client.16125 trim_cache size 0 max 16384 2023-10-08T13:52:34.348 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:52:34.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2517624423 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4c000a15f0 msgr2=0x7f4c000a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4c000a15f0 0x7f4c000a3ab0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4c10007be0 tx=0x7f4c100073d0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2517624423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c14105b80 msgr2=0x7f4c14196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c14105b80 0x7f4c14196a00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c007ee0 tx=0x7f4c0c00d3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2517624423 shutdown_connections 2023-10-08T13:52:34.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4c14105b80 0x7f4c14196a00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c14067ce0 0x7f4c1419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4c000a15f0 0x7f4c000a3ab0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 --2- 172.21.15.57:0/2517624423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c141051c0 0x7f4c1419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2517624423 >> 172.21.15.57:0/2517624423 conn(0x7f4c14076780 msgr2=0x7f4c140fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2517624423 shutdown_connections 2023-10-08T13:52:34.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.341+0000 7f4c1bd16700 1 -- 172.21.15.57:0/2517624423 wait complete. 2023-10-08T13:52:34.350 INFO:tasks.workunit.client.0.smithi057.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-08T13:52:34.350 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-08T13:52:34.350 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-08T13:52:34.354 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-08T13:52:34.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.522+0000 7f6eb98f2700 1 -- 172.21.15.57:0/2877460923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6eb4068db0 msgr2=0x7f6eb4069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.522+0000 7f6eb98f2700 1 --2- 172.21.15.57:0/2877460923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6eb4068db0 0x7f6eb4069230 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009990 tx=0x7f6ea8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.523+0000 7f6eb98f2700 1 -- 172.21.15.57:0/2877460923 shutdown_connections 2023-10-08T13:52:34.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.523+0000 7f6eb98f2700 1 --2- 172.21.15.57:0/2877460923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6eb4069770 0x7f6eb410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.523+0000 7f6eb98f2700 1 --2- 172.21.15.57:0/2877460923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6eb4106720 0x7f6eb4106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.523+0000 7f6eb98f2700 1 --2- 172.21.15.57:0/2877460923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6eb4068db0 0x7f6eb4069230 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.523+0000 7f6eb98f2700 1 -- 172.21.15.57:0/2877460923 >> 172.21.15.57:0/2877460923 conn(0x7f6eb4076820 msgr2=0x7f6eb4076c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.523+0000 7f6eb98f2700 1 -- 172.21.15.57:0/2877460923 shutdown_connections 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.523+0000 7f6eb98f2700 1 -- 172.21.15.57:0/2877460923 wait complete. 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.523+0000 7f6eb98f2700 1 Processor -- start 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.523+0000 7f6eb98f2700 1 -- start start 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb98f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6eb4068db0 0x7f6eb419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb98f2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6eb4069770 0x7f6eb419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb98f2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6eb4106720 0x7f6eb41969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb98f2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb4197030 con 0x7f6eb4068db0 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb98f2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb41971a0 con 0x7f6eb4069770 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb98f2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb4197360 con 0x7f6eb4106720 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6eb4068db0 0x7f6eb419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6eb4068db0 0x7f6eb419c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57004/0 (socket says 172.21.15.57:57004) 2023-10-08T13:52:34.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb2ffd700 1 -- 172.21.15.57:0/581924000 learned_addr learned my addr 172.21.15.57:0/581924000 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6eb4069770 0x7f6eb419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb37fe700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6eb4106720 0x7f6eb41969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb2ffd700 1 -- 172.21.15.57:0/581924000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6eb4106720 msgr2=0x7f6eb41969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6eb4106720 0x7f6eb41969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb2ffd700 1 -- 172.21.15.57:0/581924000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6eb4069770 msgr2=0x7f6eb419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6eb4069770 0x7f6eb419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb2ffd700 1 -- 172.21.15.57:0/581924000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac009c40 con 0x7f6eb4068db0 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb37fe700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6eb4106720 0x7f6eb41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6eb4069770 0x7f6eb419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6eb4068db0 0x7f6eb419c8a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400e9a0 tx=0x7f6ea400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb88f0700 1 -- 172.21.15.57:0/581924000 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4009790 con 0x7f6eb4068db0 2023-10-08T13:52:34.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb88f0700 1 -- 172.21.15.57:0/581924000 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ea400cbd0 con 0x7f6eb4068db0 2023-10-08T13:52:34.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb98f2700 1 -- 172.21.15.57:0/581924000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea80095f0 con 0x7f6eb4068db0 2023-10-08T13:52:34.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb88f0700 1 -- 172.21.15.57:0/581924000 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4018680 con 0x7f6eb4068db0 2023-10-08T13:52:34.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.524+0000 7f6eb98f2700 1 -- 172.21.15.57:0/581924000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb4197830 con 0x7f6eb4068db0 2023-10-08T13:52:34.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.527+0000 7f6eb88f0700 1 -- 172.21.15.57:0/581924000 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6ea400cd40 con 0x7f6eb4068db0 2023-10-08T13:52:34.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.527+0000 7f6eb88f0700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6e9c0a1670 0x7f6e9c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.527+0000 7f6eb88f0700 1 -- 172.21.15.57:0/581924000 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6ea4014070 con 0x7f6eb4068db0 2023-10-08T13:52:34.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.527+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6e9c0a1670 0x7f6e9c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6e9c0a1670 0x7f6e9c0a3b30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8000c00 tx=0x7f6ea8005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6ec5adbb80 1 Processor -- start 2023-10-08T13:52:34.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6ec5adbb80 1 -- start start 2023-10-08T13:52:34.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef35e80 0x56369ef36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef367a0 0x56369ef45520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 0x56369ef47e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ee9f460 con 0x56369ef35e80 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ee90c20 con 0x56369ef367a0 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ee8e540 con 0x56369ef45a60 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef35e80 0x56369ef36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef35e80 0x56369ef36260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57012/0 (socket says 172.21.15.57:57012) 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.528+0000 7f6eb2ffd700 1 -- 172.21.15.57:0/1573233133 learned_addr learned my addr 172.21.15.57:0/1573233133 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6eb2ffd700 1 -- 172.21.15.57:0/1573233133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 msgr2=0x56369ef47e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6eb37fe700 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 0x56369ef47e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef367a0 0x56369ef45520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 0x56369ef47e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6eb2ffd700 1 -- 172.21.15.57:0/1573233133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef367a0 msgr2=0x56369ef45520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef367a0 0x56369ef45520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6eb2ffd700 1 -- 172.21.15.57:0/1573233133 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56369ecb4de0 con 0x56369ef35e80 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6eb37fe700 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 0x56369ef47e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef367a0 0x56369ef45520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef35e80 0x56369ef36260 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40a21d0 tx=0x7f6ea4099fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eb2fe5d77ee6b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6e99ffb700 1 -- 172.21.15.57:0/1573233133 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40a2700 con 0x56369ef35e80 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6e99ffb700 1 -- 172.21.15.57:0/1573233133 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ea40cd370 con 0x56369ef35e80 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.529+0000 7f6e99ffb700 1 -- 172.21.15.57:0/1573233133 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4099860 con 0x56369ef35e80 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.530+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/1573233133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef35e80 msgr2=0x56369ef36260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.530+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef35e80 0x56369ef36260 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40a21d0 tx=0x7f6ea4099fe0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.530+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/1573233133 shutdown_connections 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.530+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 0x56369ef47e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.530+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef35e80 0x56369ef36260 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.530+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/1573233133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef367a0 0x56369ef45520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.530+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/1573233133 >> 172.21.15.57:0/1573233133 conn(0x56369eeab260 msgr2=0x56369ef322d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.530+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/1573233133 shutdown_connections 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.530+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/1573233133 wait complete. 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.530+0000 7f6ec5adbb80 1 -- start start 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6ec5adbb80 1 Processor -- start 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef35e80 0x56369ef45420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef42760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 0x56369ef42de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.538 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ee89e30 con 0x56369ef367a0 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ee26710 con 0x56369ef45a60 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ee263b0 con 0x56369ef35e80 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6ec5adbb80 10 client.0 _ll_register_callbacks cb 0x56369ec83800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6eb27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef42760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6eb27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef42760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57020/0 (socket says 172.21.15.57:57020) 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6eb27fc700 1 -- 172.21.15.57:0/3945262291 learned_addr learned my addr 172.21.15.57:0/3945262291 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef35e80 0x56369ef45420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6eb37fe700 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 0x56369ef42de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6eb27fc700 1 -- 172.21.15.57:0/3945262291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef35e80 msgr2=0x56369ef45420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef35e80 0x56369ef45420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6eb27fc700 1 -- 172.21.15.57:0/3945262291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 msgr2=0x56369ef42de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 0x56369ef42de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.531+0000 7f6eb27fc700 1 -- 172.21.15.57:0/3945262291 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb4197830 con 0x56369ef367a0 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef42760 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800bc70 tx=0x7f6ea8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c6c35bff6f055 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6e92ffd700 1 -- 172.21.15.57:0/3945262291 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea802b070 con 0x56369ef367a0 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/3945262291 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56369ee51eb0 con 0x56369ef367a0 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6e92ffd700 1 -- 172.21.15.57:0/3945262291 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6ea8004910 con 0x56369ef367a0 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6e92ffd700 1 -- 172.21.15.57:0/3945262291 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ea801e070 con 0x56369ef367a0 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6ec5adbb80 10 client.16137 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/3945262291 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56369ec9dc60 con 0x56369ef367a0 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6e92ffd700 1 -- 172.21.15.57:0/3945262291 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56369ee51eb0 con 0x56369ef367a0 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6e92ffd700 1 -- 172.21.15.57:0/3945262291 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ea800fbc0 con 0x56369ef367a0 2023-10-08T13:52:34.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6ec5adbb80 10 client.16137 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6ec5adbb80 10 client.16137 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6ec5adbb80 20 client.16137 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56369ef555f0 0x56369ef57a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6ec5adbb80 4 client.16137 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.532+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/3945262291 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56369ee82e10 con 0x56369ef555f0 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.533+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56369ef555f0 0x56369ef57a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.533+0000 7f6e92ffd700 10 client.16137 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.533+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56369ef555f0 0x56369ef57a10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d5d1862506392182 server_cookie=9b7a516c890879e3 in_seq=0 out_seq=0 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6e92ffd700 1 -- 172.21.15.57:0/3945262291 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56369ee82e10 con 0x56369ef555f0 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6e92ffd700 10 client.16137 handle_command_reply: tid=0 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 client.16137 shutdown 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/3945262291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 msgr2=0x56369ef42760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef42760 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800bc70 tx=0x7f6ea8009310 comp rx=0 tx=0).stop 2023-10-08T13:52:34.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/3945262291 shutdown_connections 2023-10-08T13:52:34.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56369ef555f0 0x56369ef57a10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 0x56369ef42de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef35e80 0x56369ef45420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/3945262291 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef42760 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6e92ffd700 0 client.16137 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:52:34.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/3945262291 >> 172.21.15.57:0/3945262291 conn(0x56369eeab260 msgr2=0x56369ef31ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/3945262291 shutdown_connections 2023-10-08T13:52:34.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/3945262291 wait complete. 2023-10-08T13:52:34.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.534+0000 7f6ec5adbb80 20 client.16137 trim_cache size 0 max 16384 2023-10-08T13:52:34.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6ec5adbb80 1 Processor -- start 2023-10-08T13:52:34.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6ec5adbb80 1 -- start start 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef35e80 0x56369ef579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 0x56369ef450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ed0a920 con 0x56369ef367a0 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ef436e0 con 0x56369ef45a60 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ef37120 con 0x56369ef35e80 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6eb27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6eb27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57028/0 (socket says 172.21.15.57:57028) 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6eb27fc700 1 -- 172.21.15.57:0/575856281 learned_addr learned my addr 172.21.15.57:0/575856281 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6eb27fc700 1 -- 172.21.15.57:0/575856281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef35e80 msgr2=0x56369ef579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/575856281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef35e80 0x56369ef579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6eb37fe700 1 --2- 172.21.15.57:0/575856281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 0x56369ef450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.547+0000 7f6eb27fc700 1 -- 172.21.15.57:0/575856281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 msgr2=0x56369ef450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/575856281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 0x56369ef450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6eb27fc700 1 -- 172.21.15.57:0/575856281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56369ecdb250 con 0x56369ef367a0 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6eb37fe700 1 --2- 172.21.15.57:0/575856281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 0x56369ef450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/575856281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4c5b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80304a0 tx=0x7f6ea800fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248d97a10585e055 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6e9a7fc700 1 -- 172.21.15.57:0/575856281 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea802b040 con 0x56369ef367a0 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6e9a7fc700 1 -- 172.21.15.57:0/575856281 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ea800f920 con 0x56369ef367a0 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6e9a7fc700 1 -- 172.21.15.57:0/575856281 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8030870 con 0x56369ef367a0 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/575856281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 msgr2=0x56369ef4c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/575856281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4c5b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80304a0 tx=0x7f6ea800fbc0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/575856281 shutdown_connections 2023-10-08T13:52:34.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/575856281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef45a60 0x56369ef450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/575856281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef35e80 0x56369ef579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/575856281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4c5b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/575856281 >> 172.21.15.57:0/575856281 conn(0x56369eeab260 msgr2=0x56369ef2df30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.548+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/575856281 shutdown_connections 2023-10-08T13:52:34.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/575856281 wait complete. 2023-10-08T13:52:34.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6ec5adbb80 1 -- start start 2023-10-08T13:52:34.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6ec5adbb80 1 Processor -- start 2023-10-08T13:52:34.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef35e80 0x56369ef4a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6ec5adbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 0x56369efa83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ef4af70 con 0x56369ef367a0 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369efa8920 con 0x56369ef35e80 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6ec5adbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369efa8a60 con 0x56369ef45a60 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6ec5adbb80 10 client.0 _ll_register_callbacks cb 0x7f6ea800ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6eb27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6eb27fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57034/0 (socket says 172.21.15.57:57034) 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.549+0000 7f6eb27fc700 1 -- 172.21.15.57:0/926381602 learned_addr learned my addr 172.21.15.57:0/926381602 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6eb37fe700 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 0x56369efa83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef35e80 0x56369ef4a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6eb27fc700 1 -- 172.21.15.57:0/926381602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 msgr2=0x56369efa83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 0x56369efa83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6eb27fc700 1 -- 172.21.15.57:0/926381602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef35e80 msgr2=0x56369ef4a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef35e80 0x56369ef4a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6eb27fc700 1 -- 172.21.15.57:0/926381602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56369ec9dc60 con 0x56369ef367a0 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6eb27fc700 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4a690 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8020040 tx=0x7f6ea8004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8b58d18cbea8e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6e92ffd700 1 -- 172.21.15.57:0/926381602 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea801e070 con 0x56369ef367a0 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6e92ffd700 1 -- 172.21.15.57:0/926381602 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6ea80305c0 con 0x56369ef367a0 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/926381602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ea40cd370 con 0x56369ef367a0 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6e92ffd700 1 -- 172.21.15.57:0/926381602 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ea802b040 con 0x56369ef367a0 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6ec5adbb80 10 client.16143 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/926381602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56369ecb32b0 con 0x56369ef367a0 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.550+0000 7f6e92ffd700 1 -- 172.21.15.57:0/926381602 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ea40cd370 con 0x56369ef367a0 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.551+0000 7f6e92ffd700 1 -- 172.21.15.57:0/926381602 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ea8024640 con 0x56369ef367a0 2023-10-08T13:52:34.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.551+0000 7f6ec5adbb80 10 client.16143 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.551+0000 7f6ec5adbb80 10 client.16143 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.551+0000 7f6ec5adbb80 20 client.16143 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.551+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56369efa9cd0 0x56369efac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.551+0000 7f6ec5adbb80 4 client.16143 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.551+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/926381602 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56369ee82b30 con 0x56369efa9cd0 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.551+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56369efa9cd0 0x56369efac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.551+0000 7f6e92ffd700 10 client.16143 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.551+0000 7f6eb2ffd700 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56369efa9cd0 0x56369efac0c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3bddd6d7d33bf4d server_cookie=c6e152944a66f745 in_seq=0 out_seq=0 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6e92ffd700 1 -- 172.21.15.57:0/926381602 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56369ee82b30 con 0x56369efa9cd0 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6e92ffd700 10 client.16143 handle_command_reply: tid=0 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 client.16143 shutdown 2023-10-08T13:52:34.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/926381602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 msgr2=0x56369ef4a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4a690 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8020040 tx=0x7f6ea8004a90 comp rx=0 tx=0).stop 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/926381602 shutdown_connections 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x56369efa9cd0 0x56369efac0c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56369ef45a60 0x56369efa83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56369ef35e80 0x56369ef4a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6e92ffd700 0 client.16143 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 --2- 172.21.15.57:0/926381602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56369ef367a0 0x56369ef4a690 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/926381602 >> 172.21.15.57:0/926381602 conn(0x56369eeab260 msgr2=0x56369ef450c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/926381602 shutdown_connections 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 1 -- 172.21.15.57:0/926381602 wait complete. 2023-10-08T13:52:34.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6ec5adbb80 20 client.16143 trim_cache size 0 max 16384 2023-10-08T13:52:34.560 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:52:34.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6eb98f2700 1 -- 172.21.15.57:0/581924000 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6e9c0a1670 msgr2=0x7f6e9c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6eb98f2700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6e9c0a1670 0x7f6e9c0a3b30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8000c00 tx=0x7f6ea8005fb0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6eb98f2700 1 -- 172.21.15.57:0/581924000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6eb4068db0 msgr2=0x7f6eb419c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.552+0000 7f6eb98f2700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6eb4068db0 0x7f6eb419c8a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400e9a0 tx=0x7f6ea400ed60 comp rx=0 tx=0).stop 2023-10-08T13:52:34.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.553+0000 7f6eb98f2700 1 -- 172.21.15.57:0/581924000 shutdown_connections 2023-10-08T13:52:34.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.553+0000 7f6eb98f2700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6eb4106720 0x7f6eb41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.553+0000 7f6eb98f2700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6eb4068db0 0x7f6eb419c8a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.553+0000 7f6eb98f2700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6e9c0a1670 0x7f6e9c0a3b30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.553+0000 7f6eb98f2700 1 --2- 172.21.15.57:0/581924000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6eb4069770 0x7f6eb419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.553+0000 7f6eb98f2700 1 -- 172.21.15.57:0/581924000 >> 172.21.15.57:0/581924000 conn(0x7f6eb4076820 msgr2=0x7f6eb40fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.553+0000 7f6eb98f2700 1 -- 172.21.15.57:0/581924000 shutdown_connections 2023-10-08T13:52:34.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.553+0000 7f6eb98f2700 1 -- 172.21.15.57:0/581924000 wait complete. 2023-10-08T13:52:34.566 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-08T13:52:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.695+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1476813424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72b8106770 msgr2=0x7f72b8106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.695+0000 7f72bd7a7700 1 --2- 172.21.15.57:0/1476813424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72b8106770 0x7f72b8106b50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009990 tx=0x7f72a8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.695+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1476813424 shutdown_connections 2023-10-08T13:52:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.695+0000 7f72bd7a7700 1 --2- 172.21.15.57:0/1476813424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b8069470 0x7f72b810d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.695+0000 7f72bd7a7700 1 --2- 172.21.15.57:0/1476813424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72b8106770 0x7f72b8106b50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.695+0000 7f72bd7a7700 1 --2- 172.21.15.57:0/1476813424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72b8068ab0 0x7f72b8068f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.695+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1476813424 >> 172.21.15.57:0/1476813424 conn(0x7f72b8076790 msgr2=0x7f72b8076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.695+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1476813424 shutdown_connections 2023-10-08T13:52:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.695+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1476813424 wait complete. 2023-10-08T13:52:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.695+0000 7f72bd7a7700 1 Processor -- start 2023-10-08T13:52:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72bd7a7700 1 -- start start 2023-10-08T13:52:34.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72bd7a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72b8068ab0 0x7f72b819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72bd7a7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b8069470 0x7f72b819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72bd7a7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72b8106770 0x7f72b8196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72bd7a7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b8197060 con 0x7f72b8068ab0 2023-10-08T13:52:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72bd7a7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b81971d0 con 0x7f72b8106770 2023-10-08T13:52:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72bd7a7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b8197390 con 0x7f72b8069470 2023-10-08T13:52:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72b8068ab0 0x7f72b819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72b8068ab0 0x7f72b819c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57046/0 (socket says 172.21.15.57:57046) 2023-10-08T13:52:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b6ffd700 1 -- 172.21.15.57:0/1089669703 learned_addr learned my addr 172.21.15.57:0/1089669703 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b77fe700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72b8106770 0x7f72b8196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b67fc700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b8069470 0x7f72b819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b6ffd700 1 -- 172.21.15.57:0/1089669703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b8069470 msgr2=0x7f72b819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b8069470 0x7f72b819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b6ffd700 1 -- 172.21.15.57:0/1089669703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72b8106770 msgr2=0x7f72b8196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72b8106770 0x7f72b8196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b6ffd700 1 -- 172.21.15.57:0/1089669703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a80095f0 con 0x7f72b8068ab0 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.696+0000 7f72b77fe700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72b8106770 0x7f72b8196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.697+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72b8068ab0 0x7f72b819c8d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f72a80095c0 tx=0x7f72a800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.697+0000 7f72affff700 1 -- 172.21.15.57:0/1089669703 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a801d070 con 0x7f72b8068ab0 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.697+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1089669703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b8197610 con 0x7f72b8068ab0 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.697+0000 7f72affff700 1 -- 172.21.15.57:0/1089669703 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72a8005050 con 0x7f72b8068ab0 2023-10-08T13:52:34.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.697+0000 7f72affff700 1 -- 172.21.15.57:0/1089669703 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8004bb0 con 0x7f72b8068ab0 2023-10-08T13:52:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.697+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1089669703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b81a38c0 con 0x7f72b8068ab0 2023-10-08T13:52:34.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.699+0000 7f72affff700 1 -- 172.21.15.57:0/1089669703 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f72a8004d30 con 0x7f72b8068ab0 2023-10-08T13:52:34.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.699+0000 7f72affff700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f72a40a15f0 0x7f72a40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.700+0000 7f72b67fc700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f72a40a15f0 0x7f72a40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.700+0000 7f72affff700 1 -- 172.21.15.57:0/1089669703 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f72a80ce310 con 0x7f72b8068ab0 2023-10-08T13:52:34.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.700+0000 7f72b67fc700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f72a40a15f0 0x7f72a40a3ab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f72b8197bc0 tx=0x7f72a0005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72c9990b80 1 Processor -- start 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72c9990b80 1 -- start start 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb760e80 0x55c9eb761260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb7617a0 0x55c9eb76c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb76c650 0x55c9eb76ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72c9990b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb6ca460 con 0x55c9eb760e80 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72c9990b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb6bbc20 con 0x55c9eb76c650 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72c9990b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb6b9540 con 0x55c9eb7617a0 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72b6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb760e80 0x55c9eb761260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72b6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb760e80 0x55c9eb761260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57054/0 (socket says 172.21.15.57:57054) 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.701+0000 7f72b6ffd700 1 -- 172.21.15.57:0/2672107095 learned_addr learned my addr 172.21.15.57:0/2672107095 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72b67fc700 1 --2- 172.21.15.57:0/2672107095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb7617a0 0x55c9eb76c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72b77fe700 1 --2- 172.21.15.57:0/2672107095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb76c650 0x55c9eb76ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72b6ffd700 1 -- 172.21.15.57:0/2672107095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb7617a0 msgr2=0x55c9eb76c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/2672107095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb7617a0 0x55c9eb76c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72b6ffd700 1 -- 172.21.15.57:0/2672107095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb76c650 msgr2=0x55c9eb76ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/2672107095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb76c650 0x55c9eb76ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72b6ffd700 1 -- 172.21.15.57:0/2672107095 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9eb4dfde0 con 0x55c9eb760e80 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/2672107095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb760e80 0x55c9eb761260 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f72a8097520 tx=0x7f72a809b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bbdcccb44e2703 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72ad7fa700 1 -- 172.21.15.57:0/2672107095 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a80a3320 con 0x55c9eb760e80 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72ad7fa700 1 -- 172.21.15.57:0/2672107095 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72a80ce030 con 0x55c9eb760e80 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.702+0000 7f72ad7fa700 1 -- 172.21.15.57:0/2672107095 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a809b280 con 0x55c9eb760e80 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 -- 172.21.15.57:0/2672107095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb760e80 msgr2=0x55c9eb761260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 --2- 172.21.15.57:0/2672107095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb760e80 0x55c9eb761260 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f72a8097520 tx=0x7f72a809b720 comp rx=0 tx=0).stop 2023-10-08T13:52:34.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 -- 172.21.15.57:0/2672107095 shutdown_connections 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 --2- 172.21.15.57:0/2672107095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb76c650 0x55c9eb76ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 --2- 172.21.15.57:0/2672107095 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb760e80 0x55c9eb761260 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 --2- 172.21.15.57:0/2672107095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb7617a0 0x55c9eb76c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 -- 172.21.15.57:0/2672107095 >> 172.21.15.57:0/2672107095 conn(0x55c9eb6d6260 msgr2=0x55c9eb75d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 -- 172.21.15.57:0/2672107095 shutdown_connections 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 -- 172.21.15.57:0/2672107095 wait complete. 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 -- start start 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.703+0000 7f72c9990b80 1 Processor -- start 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb76ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb76cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb76d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72c9990b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb6b4e30 con 0x55c9eb76c650 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72c9990b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb651710 con 0x55c9eb7617a0 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72c9990b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb6513b0 con 0x55c9eb760e80 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72c9990b80 10 client.0 _ll_register_callbacks cb 0x55c9eb4ae800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb76d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb76d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57064/0 (socket says 172.21.15.57:57064) 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b77fe700 1 -- 172.21.15.57:0/15187734 learned_addr learned my addr 172.21.15.57:0/15187734 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b67fc700 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb76cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b77fe700 1 -- 172.21.15.57:0/15187734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 msgr2=0x55c9eb76ca30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b77fe700 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb76ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b77fe700 1 -- 172.21.15.57:0/15187734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 msgr2=0x55c9eb76cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b77fe700 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb76cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b77fe700 1 -- 172.21.15.57:0/15187734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b81a38c0 con 0x55c9eb76c650 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b67fc700 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb76cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f72b77fe700 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb76d4d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f72b00078f0 tx=0x7f72b0007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d3bb66c01a972f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f729e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.704+0000 7f729e7fc700 1 -- 172.21.15.57:0/15187734 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0010860 con 0x55c9eb76c650 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f729e7fc700 1 -- 172.21.15.57:0/15187734 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f72b00109c0 con 0x55c9eb76c650 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f72c9990b80 1 -- 172.21.15.57:0/15187734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9eb67ceb0 con 0x55c9eb76c650 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f729e7fc700 1 -- 172.21.15.57:0/15187734 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72b0010b80 con 0x55c9eb76c650 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f72c9990b80 10 client.16155 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f72c9990b80 1 -- 172.21.15.57:0/15187734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c9eb4c8c60 con 0x55c9eb76c650 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f729e7fc700 1 -- 172.21.15.57:0/15187734 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9eb67ceb0 con 0x55c9eb76c650 2023-10-08T13:52:34.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f729e7fc700 1 -- 172.21.15.57:0/15187734 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f72b00153e0 con 0x55c9eb76c650 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f72c9990b80 10 client.16155 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f72c9990b80 10 client.16155 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f72c9990b80 20 client.16155 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f72c9990b80 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55c9eb77bef0 0x55c9eb77e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f72c9990b80 4 client.16155 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.705+0000 7f72c9990b80 1 -- 172.21.15.57:0/15187734 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9eb6ade10 con 0x55c9eb77bef0 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.706+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55c9eb77bef0 0x55c9eb77e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.706+0000 7f729e7fc700 10 client.16155 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.706+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55c9eb77bef0 0x55c9eb77e310 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39fa4e2d585ca06c server_cookie=66205ba59441df57 in_seq=0 out_seq=0 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.706+0000 7f729e7fc700 1 -- 172.21.15.57:0/15187734 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c9eb6ade10 con 0x55c9eb77bef0 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.706+0000 7f729e7fc700 10 client.16155 handle_command_reply: tid=0 2023-10-08T13:52:34.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 client.16155 shutdown 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 -- 172.21.15.57:0/15187734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 msgr2=0x55c9eb76d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb76d4d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f72b00078f0 tx=0x7f72b0007cb0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 -- 172.21.15.57:0/15187734 shutdown_connections 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55c9eb77bef0 0x55c9eb77e310 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb76d4d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb76ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 --2- 172.21.15.57:0/15187734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb76cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 -- 172.21.15.57:0/15187734 >> 172.21.15.57:0/15187734 conn(0x55c9eb6d6260 msgr2=0x55c9eb75cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f729e7fc700 0 client.16155 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 -- 172.21.15.57:0/15187734 shutdown_connections 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 1 -- 172.21.15.57:0/15187734 wait complete. 2023-10-08T13:52:34.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.707+0000 7f72c9990b80 20 client.16155 trim_cache size 0 max 16384 2023-10-08T13:52:34.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72c9990b80 1 Processor -- start 2023-10-08T13:52:34.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72c9990b80 1 -- start start 2023-10-08T13:52:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb771a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb75c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb766ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72c9990b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb535920 con 0x55c9eb76c650 2023-10-08T13:52:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72c9990b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb757570 con 0x55c9eb7617a0 2023-10-08T13:52:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72c9990b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb771fb0 con 0x55c9eb760e80 2023-10-08T13:52:34.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb766ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb766ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57072/0 (socket says 172.21.15.57:57072) 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.720+0000 7f72b77fe700 1 -- 172.21.15.57:0/2429387733 learned_addr learned my addr 172.21.15.57:0/2429387733 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/2429387733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb771a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72b67fc700 1 --2- 172.21.15.57:0/2429387733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb75c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72b77fe700 1 -- 172.21.15.57:0/2429387733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 msgr2=0x55c9eb771a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72b77fe700 1 --2- 172.21.15.57:0/2429387733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb771a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72b77fe700 1 -- 172.21.15.57:0/2429387733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 msgr2=0x55c9eb75c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72b77fe700 1 --2- 172.21.15.57:0/2429387733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb75c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72b77fe700 1 -- 172.21.15.57:0/2429387733 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9eb506250 con 0x55c9eb76c650 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72b77fe700 1 --2- 172.21.15.57:0/2429387733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb766ad0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f72b0000f80 tx=0x7f72b0010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59f555868bf77c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72adffb700 1 -- 172.21.15.57:0/2429387733 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b000d040 con 0x55c9eb76c650 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72adffb700 1 -- 172.21.15.57:0/2429387733 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72b00153e0 con 0x55c9eb76c650 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.721+0000 7f72adffb700 1 -- 172.21.15.57:0/2429387733 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0010860 con 0x55c9eb76c650 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 -- 172.21.15.57:0/2429387733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 msgr2=0x55c9eb766ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 --2- 172.21.15.57:0/2429387733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb766ad0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f72b0000f80 tx=0x7f72b0010440 comp rx=0 tx=0).stop 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 -- 172.21.15.57:0/2429387733 shutdown_connections 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 --2- 172.21.15.57:0/2429387733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb766ad0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 --2- 172.21.15.57:0/2429387733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb771a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 --2- 172.21.15.57:0/2429387733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb75c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 -- 172.21.15.57:0/2429387733 >> 172.21.15.57:0/2429387733 conn(0x55c9eb6d6260 msgr2=0x55c9eb76b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 -- 172.21.15.57:0/2429387733 shutdown_connections 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 -- 172.21.15.57:0/2429387733 wait complete. 2023-10-08T13:52:34.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 -- start start 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 Processor -- start 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb771890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb76fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb7702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb7709e0 con 0x55c9eb76c650 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb770b50 con 0x55c9eb7617a0 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9eb7cfd70 con 0x55c9eb760e80 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72c9990b80 10 client.0 _ll_register_callbacks cb 0x7f72b0009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.722+0000 7f72b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb7702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb7702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57078/0 (socket says 172.21.15.57:57078) 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72b77fe700 1 -- 172.21.15.57:0/946802139 learned_addr learned my addr 172.21.15.57:0/946802139 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72b67fc700 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb76fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb771890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72b77fe700 1 -- 172.21.15.57:0/946802139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 msgr2=0x55c9eb771890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72b77fe700 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb771890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72b77fe700 1 -- 172.21.15.57:0/946802139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 msgr2=0x55c9eb76fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72b77fe700 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb76fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72b77fe700 1 -- 172.21.15.57:0/946802139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9eb4c8c60 con 0x55c9eb76c650 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72b77fe700 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb7702d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f72b0007fd0 tx=0x7f72b00155c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7b22bef462333d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f729e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f729e7fc700 1 -- 172.21.15.57:0/946802139 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0003a00 con 0x55c9eb76c650 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f729e7fc700 1 -- 172.21.15.57:0/946802139 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f72b0003b60 con 0x55c9eb76c650 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72c9990b80 1 -- 172.21.15.57:0/946802139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72a80ce030 con 0x55c9eb76c650 2023-10-08T13:52:34.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f729e7fc700 1 -- 172.21.15.57:0/946802139 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72b0003d20 con 0x55c9eb76c650 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72c9990b80 10 client.16161 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.723+0000 7f72c9990b80 1 -- 172.21.15.57:0/946802139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c9eb4de2b0 con 0x55c9eb76c650 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.724+0000 7f729e7fc700 1 -- 172.21.15.57:0/946802139 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72a80ce030 con 0x55c9eb76c650 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.724+0000 7f729e7fc700 1 -- 172.21.15.57:0/946802139 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f72b00167e0 con 0x55c9eb76c650 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.724+0000 7f72c9990b80 10 client.16161 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.724+0000 7f72c9990b80 10 client.16161 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.724+0000 7f72c9990b80 20 client.16161 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.724+0000 7f72c9990b80 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55c9eb7d1130 0x55c9eb7d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.724+0000 7f72c9990b80 4 client.16161 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.724+0000 7f72c9990b80 1 -- 172.21.15.57:0/946802139 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c9eb6adb30 con 0x55c9eb7d1130 2023-10-08T13:52:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.724+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55c9eb7d1130 0x55c9eb7d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72b6ffd700 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55c9eb7d1130 0x55c9eb7d3520 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69e9893f9ebedfbd server_cookie=a145624d1f5e83df in_seq=0 out_seq=0 2023-10-08T13:52:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f729e7fc700 10 client.16161 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:52:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f729e7fc700 1 -- 172.21.15.57:0/946802139 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c9eb6adb30 con 0x55c9eb7d1130 2023-10-08T13:52:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f729e7fc700 10 client.16161 handle_command_reply: tid=0 2023-10-08T13:52:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 client.16161 shutdown 2023-10-08T13:52:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 -- 172.21.15.57:0/946802139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 msgr2=0x55c9eb7702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb7702d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f72b0007fd0 tx=0x7f72b00155c0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 -- 172.21.15.57:0/946802139 shutdown_connections 2023-10-08T13:52:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55c9eb7d1130 0x55c9eb7d3520 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c9eb76c650 0x55c9eb7702d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f729e7fc700 0 client.16161 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:52:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9eb760e80 0x55c9eb771890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 --2- 172.21.15.57:0/946802139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9eb7617a0 0x55c9eb76fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 -- 172.21.15.57:0/946802139 >> 172.21.15.57:0/946802139 conn(0x55c9eb6d6260 msgr2=0x55c9eb766ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 -- 172.21.15.57:0/946802139 shutdown_connections 2023-10-08T13:52:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 1 -- 172.21.15.57:0/946802139 wait complete. 2023-10-08T13:52:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72c9990b80 20 client.16161 trim_cache size 0 max 16384 2023-10-08T13:52:34.733 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:52:34.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1089669703 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f72a40a15f0 msgr2=0x7f72a40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72bd7a7700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f72a40a15f0 0x7f72a40a3ab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f72b8197bc0 tx=0x7f72a0005d50 comp rx=0 tx=0).stop 2023-10-08T13:52:34.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.725+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1089669703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72b8068ab0 msgr2=0x7f72b819c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.726+0000 7f72bd7a7700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72b8068ab0 0x7f72b819c8d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f72a80095c0 tx=0x7f72a800f690 comp rx=0 tx=0).stop 2023-10-08T13:52:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.726+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1089669703 shutdown_connections 2023-10-08T13:52:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.726+0000 7f72bd7a7700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f72a40a15f0 0x7f72a40a3ab0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.726+0000 7f72bd7a7700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72b8106770 0x7f72b8196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.726+0000 7f72bd7a7700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f72b8068ab0 0x7f72b819c8d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.726+0000 7f72bd7a7700 1 --2- 172.21.15.57:0/1089669703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b8069470 0x7f72b819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.726+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1089669703 >> 172.21.15.57:0/1089669703 conn(0x7f72b8076790 msgr2=0x7f72b80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.726+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1089669703 shutdown_connections 2023-10-08T13:52:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.726+0000 7f72bd7a7700 1 -- 172.21.15.57:0/1089669703 wait complete. 2023-10-08T13:52:34.739 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-08T13:52:34.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.868+0000 7efccecf9700 1 -- 172.21.15.57:0/2821107307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcc8105b80 msgr2=0x7efcc810d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.868+0000 7efccecf9700 1 --2- 172.21.15.57:0/2821107307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcc8105b80 0x7efcc810d590 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efcc000b0d0 tx=0x7efcc000b3e0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.868+0000 7efccecf9700 1 -- 172.21.15.57:0/2821107307 shutdown_connections 2023-10-08T13:52:34.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.868+0000 7efccecf9700 1 --2- 172.21.15.57:0/2821107307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcc8105b80 0x7efcc810d590 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.868+0000 7efccecf9700 1 --2- 172.21.15.57:0/2821107307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8067ce0 0x7efcc80680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.868+0000 7efccecf9700 1 --2- 172.21.15.57:0/2821107307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc81051c0 0x7efcc8105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.868+0000 7efccecf9700 1 -- 172.21.15.57:0/2821107307 >> 172.21.15.57:0/2821107307 conn(0x7efcc8076780 msgr2=0x7efcc8076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.868+0000 7efccecf9700 1 -- 172.21.15.57:0/2821107307 shutdown_connections 2023-10-08T13:52:34.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.868+0000 7efccecf9700 1 -- 172.21.15.57:0/2821107307 wait complete. 2023-10-08T13:52:34.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efccecf9700 1 Processor -- start 2023-10-08T13:52:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efccecf9700 1 -- start start 2023-10-08T13:52:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efccecf9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8067ce0 0x7efcc80ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efccecf9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcc81051c0 0x7efcc80ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efccecf9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc8105b80 0x7efcc8100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efccecf9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81a50a0 con 0x7efcc81051c0 2023-10-08T13:52:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efccecf9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81a5210 con 0x7efcc8105b80 2023-10-08T13:52:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efccecf9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81a53d0 con 0x7efcc8067ce0 2023-10-08T13:52:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcc81051c0 0x7efcc80ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcc81051c0 0x7efcc80ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57090/0 (socket says 172.21.15.57:57090) 2023-10-08T13:52:34.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efcc7fff700 1 -- 172.21.15.57:0/86343263 learned_addr learned my addr 172.21.15.57:0/86343263 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efccd296700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc8105b80 0x7efcc8100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efccca95700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8067ce0 0x7efcc80ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efcc7fff700 1 -- 172.21.15.57:0/86343263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8067ce0 msgr2=0x7efcc80ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efcc7fff700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8067ce0 0x7efcc80ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efcc7fff700 1 -- 172.21.15.57:0/86343263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc8105b80 msgr2=0x7efcc8100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efcc7fff700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc8105b80 0x7efcc8100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.869+0000 7efcc7fff700 1 -- 172.21.15.57:0/86343263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0009d00 con 0x7efcc81051c0 2023-10-08T13:52:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.870+0000 7efcc7fff700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcc81051c0 0x7efcc80ffbf0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c920 tx=0x7efcb400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.870+0000 7efcc5ffb700 1 -- 172.21.15.57:0/86343263 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4007b00 con 0x7efcc81051c0 2023-10-08T13:52:34.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.870+0000 7efcc5ffb700 1 -- 172.21.15.57:0/86343263 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcb4004cf0 con 0x7efcc81051c0 2023-10-08T13:52:34.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.870+0000 7efccecf9700 1 -- 172.21.15.57:0/86343263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc81a56b0 con 0x7efcc81051c0 2023-10-08T13:52:34.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.870+0000 7efcc5ffb700 1 -- 172.21.15.57:0/86343263 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4005610 con 0x7efcc81051c0 2023-10-08T13:52:34.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.870+0000 7efccecf9700 1 -- 172.21.15.57:0/86343263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc81a5c10 con 0x7efcc81051c0 2023-10-08T13:52:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.872+0000 7efcc5ffb700 1 -- 172.21.15.57:0/86343263 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7efcb4004750 con 0x7efcc81051c0 2023-10-08T13:52:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.872+0000 7efcc5ffb700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efcb80a1670 0x7efcb80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.873+0000 7efcc5ffb700 1 -- 172.21.15.57:0/86343263 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7efcb40cd340 con 0x7efcc81051c0 2023-10-08T13:52:34.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.873+0000 7efccca95700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efcb80a1670 0x7efcb80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.873+0000 7efccca95700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efcb80a1670 0x7efcb80a3b30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efcbc005950 tx=0x7efcbc0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efcdaee2b80 1 Processor -- start 2023-10-08T13:52:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efcdaee2b80 1 -- start start 2023-10-08T13:52:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf94e80 0x56305cf95260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf957a0 0x56305cfa0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cfa0650 0x56305cfa2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305cefe460 con 0x56305cfa0650 2023-10-08T13:52:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305ceefc20 con 0x56305cf957a0 2023-10-08T13:52:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305ceed540 con 0x56305cf94e80 2023-10-08T13:52:34.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efccd296700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cfa0650 0x56305cfa2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efccd296700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cfa0650 0x56305cfa2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57100/0 (socket says 172.21.15.57:57100) 2023-10-08T13:52:34.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efccd296700 1 -- 172.21.15.57:0/2714590243 learned_addr learned my addr 172.21.15.57:0/2714590243 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efcc7fff700 1 --2- 172.21.15.57:0/2714590243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf957a0 0x56305cfa0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efccca95700 1 --2- 172.21.15.57:0/2714590243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf94e80 0x56305cf95260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efccd296700 1 -- 172.21.15.57:0/2714590243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf94e80 msgr2=0x56305cf95260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efccd296700 1 --2- 172.21.15.57:0/2714590243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf94e80 0x56305cf95260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efccd296700 1 -- 172.21.15.57:0/2714590243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf957a0 msgr2=0x56305cfa0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efccd296700 1 --2- 172.21.15.57:0/2714590243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf957a0 0x56305cfa0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efccd296700 1 -- 172.21.15.57:0/2714590243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56305cd13de0 con 0x56305cfa0650 2023-10-08T13:52:34.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efcc7fff700 1 --2- 172.21.15.57:0/2714590243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf957a0 0x56305cfa0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:34.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.874+0000 7efccd296700 1 --2- 172.21.15.57:0/2714590243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cfa0650 0x56305cfa2a40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efcc000cda0 tx=0x7efcc000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a0b4ceb2ff256 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcb2ffd700 1 -- 172.21.15.57:0/2714590243 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc000e050 con 0x56305cfa0650 2023-10-08T13:52:34.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcb2ffd700 1 -- 172.21.15.57:0/2714590243 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc0007df0 con 0x56305cfa0650 2023-10-08T13:52:34.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcb2ffd700 1 -- 172.21.15.57:0/2714590243 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0007980 con 0x56305cfa0650 2023-10-08T13:52:34.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcdaee2b80 1 -- 172.21.15.57:0/2714590243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cfa0650 msgr2=0x56305cfa2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/2714590243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cfa0650 0x56305cfa2a40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efcc000cda0 tx=0x7efcc000cdd0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcdaee2b80 1 -- 172.21.15.57:0/2714590243 shutdown_connections 2023-10-08T13:52:34.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/2714590243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cfa0650 0x56305cfa2a40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/2714590243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf94e80 0x56305cf95260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/2714590243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf957a0 0x56305cfa0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcdaee2b80 1 -- 172.21.15.57:0/2714590243 >> 172.21.15.57:0/2714590243 conn(0x56305cf0a260 msgr2=0x56305cf912d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcdaee2b80 1 -- 172.21.15.57:0/2714590243 shutdown_connections 2023-10-08T13:52:34.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.875+0000 7efcdaee2b80 1 -- 172.21.15.57:0/2714590243 wait complete. 2023-10-08T13:52:34.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcdaee2b80 1 -- start start 2023-10-08T13:52:34.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcdaee2b80 1 Processor -- start 2023-10-08T13:52:34.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf94e80 0x56305cf9ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf957a0 0x56305cf9d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cfa0650 0x56305cf9d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305cee8e30 con 0x56305cf957a0 2023-10-08T13:52:34.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305ce85710 con 0x56305cf94e80 2023-10-08T13:52:34.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305ce853b0 con 0x56305cfa0650 2023-10-08T13:52:34.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcdaee2b80 10 client.0 _ll_register_callbacks cb 0x56305cce2800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:34.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf957a0 0x56305cf9d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.876+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf957a0 0x56305cf9d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57104/0 (socket says 172.21.15.57:57104) 2023-10-08T13:52:34.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efcc7fff700 1 -- 172.21.15.57:0/4032652175 learned_addr learned my addr 172.21.15.57:0/4032652175 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efccd296700 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cfa0650 0x56305cf9d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efccca95700 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf94e80 0x56305cf9ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efcc7fff700 1 -- 172.21.15.57:0/4032652175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cfa0650 msgr2=0x56305cf9d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efcc7fff700 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cfa0650 0x56305cf9d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efcc7fff700 1 -- 172.21.15.57:0/4032652175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf94e80 msgr2=0x56305cf9ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efcc7fff700 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf94e80 0x56305cf9ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efcc7fff700 1 -- 172.21.15.57:0/4032652175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc810dec0 con 0x56305cf957a0 2023-10-08T13:52:34.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efcc7fff700 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf957a0 0x56305cf9d2b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efcb4096a30 tx=0x7efcb409a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa1dac305bc68ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efca7fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:34.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efca7fff700 1 -- 172.21.15.57:0/4032652175 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40cd150 con 0x56305cf957a0 2023-10-08T13:52:34.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efca7fff700 1 -- 172.21.15.57:0/4032652175 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7efcb40a2350 con 0x56305cf957a0 2023-10-08T13:52:34.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efcdaee2b80 1 -- 172.21.15.57:0/4032652175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56305ceb0eb0 con 0x56305cf957a0 2023-10-08T13:52:34.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efca7fff700 1 -- 172.21.15.57:0/4032652175 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcb40b8170 con 0x56305cf957a0 2023-10-08T13:52:34.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efcdaee2b80 10 client.16173 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:34.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efcdaee2b80 1 -- 172.21.15.57:0/4032652175 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56305ccfcc60 con 0x56305cf957a0 2023-10-08T13:52:34.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.877+0000 7efca7fff700 1 -- 172.21.15.57:0/4032652175 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcb407c080 con 0x56305cf957a0 2023-10-08T13:52:34.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.878+0000 7efca7fff700 1 -- 172.21.15.57:0/4032652175 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efcb409a220 con 0x56305cf957a0 2023-10-08T13:52:34.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.878+0000 7efcdaee2b80 10 client.16173 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:34.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.878+0000 7efcdaee2b80 10 client.16173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T13:52:34.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.878+0000 7efcdaee2b80 20 client.16173 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:34.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.878+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56305cfaff50 0x56305cfb2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.878+0000 7efcdaee2b80 4 client.16173 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:34.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.878+0000 7efcdaee2b80 1 -- 172.21.15.57:0/4032652175 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56305cee1e10 con 0x56305cfaff50 2023-10-08T13:52:34.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.878+0000 7efccca95700 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56305cfaff50 0x56305cfb2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efca7fff700 10 client.16173 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:52:34.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efccca95700 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56305cfaff50 0x56305cfb2370 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa59bff7cb31389f server_cookie=f01302e41ffd4a4c in_seq=0 out_seq=0 2023-10-08T13:52:34.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efca7fff700 1 -- 172.21.15.57:0/4032652175 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56305cee1e10 con 0x56305cfaff50 2023-10-08T13:52:34.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efca7fff700 10 client.16173 handle_command_reply: tid=0 2023-10-08T13:52:34.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 client.16173 shutdown 2023-10-08T13:52:34.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 -- 172.21.15.57:0/4032652175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf957a0 msgr2=0x56305cf9d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf957a0 0x56305cf9d2b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efcb4096a30 tx=0x7efcb409a770 comp rx=0 tx=0).stop 2023-10-08T13:52:34.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 -- 172.21.15.57:0/4032652175 shutdown_connections 2023-10-08T13:52:34.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56305cfaff50 0x56305cfb2370 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cfa0650 0x56305cf9d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efca7fff700 0 client.16173 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:52:34.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cf94e80 0x56305cf9ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/4032652175 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf957a0 0x56305cf9d2b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 -- 172.21.15.57:0/4032652175 >> 172.21.15.57:0/4032652175 conn(0x56305cf0a260 msgr2=0x56305cf90ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 -- 172.21.15.57:0/4032652175 shutdown_connections 2023-10-08T13:52:34.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.879+0000 7efcdaee2b80 1 -- 172.21.15.57:0/4032652175 wait complete. 2023-10-08T13:52:34.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.880+0000 7efcdaee2b80 20 client.16173 trim_cache size 0 max 16384 2023-10-08T13:52:34.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.892+0000 7efcdaee2b80 1 Processor -- start 2023-10-08T13:52:34.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.892+0000 7efcdaee2b80 1 -- start start 2023-10-08T13:52:34.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf957a0 0x56305cf9bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cfa0650 0x56305cf9c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305cd69920 con 0x56305cf94e80 2023-10-08T13:52:34.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305cf9c600 con 0x56305cfa0650 2023-10-08T13:52:34.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305cf8b5a0 con 0x56305cf957a0 2023-10-08T13:52:34.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efccca95700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efccca95700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57110/0 (socket says 172.21.15.57:57110) 2023-10-08T13:52:34.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efccca95700 1 -- 172.21.15.57:0/558741443 learned_addr learned my addr 172.21.15.57:0/558741443 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efccd296700 1 --2- 172.21.15.57:0/558741443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cfa0650 0x56305cf9c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efcc7fff700 1 --2- 172.21.15.57:0/558741443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf957a0 0x56305cf9bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efccca95700 1 -- 172.21.15.57:0/558741443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf957a0 msgr2=0x56305cf9bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efccca95700 1 --2- 172.21.15.57:0/558741443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf957a0 0x56305cf9bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efccca95700 1 -- 172.21.15.57:0/558741443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cfa0650 msgr2=0x56305cf9c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efccca95700 1 --2- 172.21.15.57:0/558741443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cfa0650 0x56305cf9c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efccca95700 1 -- 172.21.15.57:0/558741443 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56305cd3a250 con 0x56305cf94e80 2023-10-08T13:52:34.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efccca95700 1 --2- 172.21.15.57:0/558741443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa7140 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x56305cfb2cc0 tx=0x7efcbc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d842c202ac9c2d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efcb37fe700 1 -- 172.21.15.57:0/558741443 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc024040 con 0x56305cf94e80 2023-10-08T13:52:34.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efcb37fe700 1 -- 172.21.15.57:0/558741443 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcbc025500 con 0x56305cf94e80 2023-10-08T13:52:34.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.893+0000 7efcb37fe700 1 -- 172.21.15.57:0/558741443 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc005020 con 0x56305cf94e80 2023-10-08T13:52:34.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 -- 172.21.15.57:0/558741443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 msgr2=0x56305cfa7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/558741443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa7140 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x56305cfb2cc0 tx=0x7efcbc013000 comp rx=0 tx=0).stop 2023-10-08T13:52:34.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 -- 172.21.15.57:0/558741443 shutdown_connections 2023-10-08T13:52:34.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/558741443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cfa0650 0x56305cf9c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/558741443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa7140 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/558741443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf957a0 0x56305cf9bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 -- 172.21.15.57:0/558741443 >> 172.21.15.57:0/558741443 conn(0x56305cf0a260 msgr2=0x56305cf985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 -- 172.21.15.57:0/558741443 shutdown_connections 2023-10-08T13:52:34.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 -- 172.21.15.57:0/558741443 wait complete. 2023-10-08T13:52:34.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 -- start start 2023-10-08T13:52:34.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.894+0000 7efcdaee2b80 1 Processor -- start 2023-10-08T13:52:34.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf957a0 0x56305cfa5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efcdaee2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cfa0650 0x56305cfa5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305cfa6350 con 0x56305cf94e80 2023-10-08T13:52:34.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305d003dd0 con 0x56305cfa0650 2023-10-08T13:52:34.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efcdaee2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56305d003f10 con 0x56305cf957a0 2023-10-08T13:52:34.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efcdaee2b80 10 client.0 _ll_register_callbacks cb 0x7efcb4099b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:34.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efccca95700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efccca95700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57116/0 (socket says 172.21.15.57:57116) 2023-10-08T13:52:34.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efccca95700 1 -- 172.21.15.57:0/1085938623 learned_addr learned my addr 172.21.15.57:0/1085938623 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:34.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efcc7fff700 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf957a0 0x56305cfa5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efccd296700 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cfa0650 0x56305cfa5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efccca95700 1 -- 172.21.15.57:0/1085938623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf957a0 msgr2=0x56305cfa5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efccca95700 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf957a0 0x56305cfa5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efccca95700 1 -- 172.21.15.57:0/1085938623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cfa0650 msgr2=0x56305cfa5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:34.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efccca95700 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cfa0650 0x56305cfa5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.895+0000 7efccca95700 1 -- 172.21.15.57:0/1085938623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc00e7c0 con 0x56305cf94e80 2023-10-08T13:52:34.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efccca95700 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa70c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009800 tx=0x7efcbc00efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923f90244d24b99c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:34.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efca7fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:34.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efca7fff700 1 -- 172.21.15.57:0/1085938623 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc024030 con 0x56305cf94e80 2023-10-08T13:52:34.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efca7fff700 1 -- 172.21.15.57:0/1085938623 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7efcbc011c60 con 0x56305cf94e80 2023-10-08T13:52:34.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efcdaee2b80 1 -- 172.21.15.57:0/1085938623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcc0007df0 con 0x56305cf94e80 2023-10-08T13:52:34.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efca7fff700 1 -- 172.21.15.57:0/1085938623 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcbc011e20 con 0x56305cf94e80 2023-10-08T13:52:34.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efcdaee2b80 10 client.16179 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:34.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efcdaee2b80 1 -- 172.21.15.57:0/1085938623 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56305cd122b0 con 0x56305cf94e80 2023-10-08T13:52:34.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efca7fff700 1 -- 172.21.15.57:0/1085938623 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc0007df0 con 0x56305cf94e80 2023-10-08T13:52:34.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efca7fff700 1 -- 172.21.15.57:0/1085938623 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efcbc02b020 con 0x56305cf94e80 2023-10-08T13:52:34.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efcdaee2b80 10 client.16179 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:34.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efcdaee2b80 10 client.16179 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T13:52:34.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efcdaee2b80 20 client.16179 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:34.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56305d0055e0 0x56305d007a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:34.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efcdaee2b80 4 client.16179 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:52:34.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.896+0000 7efcdaee2b80 1 -- 172.21.15.57:0/1085938623 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56305cee1b30 con 0x56305d0055e0 2023-10-08T13:52:34.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcc7fff700 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56305d0055e0 0x56305d007a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:34.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcc7fff700 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56305d0055e0 0x56305d007a00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb07e80f9c0d1842 server_cookie=d5553e0f49694a4c in_seq=0 out_seq=0 2023-10-08T13:52:34.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efca7fff700 10 client.16179 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:52:34.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efca7fff700 1 -- 172.21.15.57:0/1085938623 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56305cee1b30 con 0x56305d0055e0 2023-10-08T13:52:34.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efca7fff700 10 client.16179 handle_command_reply: tid=0 2023-10-08T13:52:34.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcdaee2b80 1 client.16179 shutdown 2023-10-08T13:52:34.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcdaee2b80 1 -- 172.21.15.57:0/1085938623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 msgr2=0x56305cfa70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa70c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009800 tx=0x7efcbc00efa0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcdaee2b80 1 -- 172.21.15.57:0/1085938623 shutdown_connections 2023-10-08T13:52:34.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56305d0055e0 0x56305d007a00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56305cfa0650 0x56305cfa5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56305cf94e80 0x56305cfa70c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcdaee2b80 1 --2- 172.21.15.57:0/1085938623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56305cf957a0 0x56305cfa5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efcdaee2b80 1 -- 172.21.15.57:0/1085938623 >> 172.21.15.57:0/1085938623 conn(0x56305cf0a260 msgr2=0x56305cf9b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.897+0000 7efca7fff700 0 client.16179 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:52:34.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efcdaee2b80 1 -- 172.21.15.57:0/1085938623 shutdown_connections 2023-10-08T13:52:34.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efcdaee2b80 1 -- 172.21.15.57:0/1085938623 wait complete. 2023-10-08T13:52:34.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efcdaee2b80 20 client.16179 trim_cache size 0 max 16384 2023-10-08T13:52:34.908 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:52:34.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 -- 172.21.15.57:0/86343263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efcb80a1670 msgr2=0x7efcb80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efcb80a1670 0x7efcb80a3b30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efcbc005950 tx=0x7efcbc0058e0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 -- 172.21.15.57:0/86343263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcc81051c0 msgr2=0x7efcc80ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:34.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcc81051c0 0x7efcc80ffbf0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c920 tx=0x7efcb400cce0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 -- 172.21.15.57:0/86343263 shutdown_connections 2023-10-08T13:52:34.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efcb80a1670 0x7efcb80a3b30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc8105b80 0x7efcc8100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8067ce0 0x7efcc80ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 --2- 172.21.15.57:0/86343263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcc81051c0 0x7efcc80ffbf0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:34.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 -- 172.21.15.57:0/86343263 >> 172.21.15.57:0/86343263 conn(0x7efcc8076780 msgr2=0x7efcc80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:34.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 -- 172.21.15.57:0/86343263 shutdown_connections 2023-10-08T13:52:34.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:34.898+0000 7efccecf9700 1 -- 172.21.15.57:0/86343263 wait complete. 2023-10-08T13:52:34.912 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-08T13:52:35.019 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-08T13:52:35.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.041+0000 7f7d89489700 1 -- 172.21.15.57:0/3660138883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d84105b30 msgr2=0x7f7d8410d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.041+0000 7f7d89489700 1 --2- 172.21.15.57:0/3660138883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d84105b30 0x7f7d8410d6d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00b0a0 tx=0x7f7d7c00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.041+0000 7f7d89489700 1 -- 172.21.15.57:0/3660138883 shutdown_connections 2023-10-08T13:52:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.041+0000 7f7d89489700 1 --2- 172.21.15.57:0/3660138883 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d84105b30 0x7f7d8410d6d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.041+0000 7f7d89489700 1 --2- 172.21.15.57:0/3660138883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d840ff320 0x7f7d840ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.041+0000 7f7d89489700 1 --2- 172.21.15.57:0/3660138883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d840ffcd0 0x7f7d84105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.041+0000 7f7d89489700 1 -- 172.21.15.57:0/3660138883 >> 172.21.15.57:0/3660138883 conn(0x7f7d840749a0 msgr2=0x7f7d84074db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.041+0000 7f7d89489700 1 -- 172.21.15.57:0/3660138883 shutdown_connections 2023-10-08T13:52:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.041+0000 7f7d89489700 1 -- 172.21.15.57:0/3660138883 wait complete. 2023-10-08T13:52:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.041+0000 7f7d89489700 1 Processor -- start 2023-10-08T13:52:35.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d89489700 1 -- start start 2023-10-08T13:52:35.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d89489700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d840ff320 0x7f7d84100290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d89489700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d84105b30 0x7f7d841007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d89489700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d84100d10 0x7f7d841a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d89489700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d841a55a0 con 0x7f7d84100d10 2023-10-08T13:52:35.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d89489700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d841a5710 con 0x7f7d84105b30 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d89489700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d841a58d0 con 0x7f7d840ff320 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d84100d10 0x7f7d841a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d84100d10 0x7f7d841a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57134/0 (socket says 172.21.15.57:57134) 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d837fe700 1 -- 172.21.15.57:0/1360869395 learned_addr learned my addr 172.21.15.57:0/1360869395 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d827fc700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d84105b30 0x7f7d841007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d840ff320 0x7f7d84100290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d837fe700 1 -- 172.21.15.57:0/1360869395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d840ff320 msgr2=0x7f7d84100290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d837fe700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d840ff320 0x7f7d84100290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d837fe700 1 -- 172.21.15.57:0/1360869395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d84105b30 msgr2=0x7f7d841007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d837fe700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d84105b30 0x7f7d841007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.042+0000 7f7d837fe700 1 -- 172.21.15.57:0/1360869395 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c009d00 con 0x7f7d84100d10 2023-10-08T13:52:35.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.043+0000 7f7d837fe700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d84100d10 0x7f7d841a4fd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00b070 tx=0x7f7d7c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:35.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.043+0000 7f7d6bfff700 1 -- 172.21.15.57:0/1360869395 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c00e040 con 0x7f7d84100d10 2023-10-08T13:52:35.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.043+0000 7f7d6bfff700 1 -- 172.21.15.57:0/1360869395 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d7c007930 con 0x7f7d84100d10 2023-10-08T13:52:35.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.043+0000 7f7d89489700 1 -- 172.21.15.57:0/1360869395 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d841a5b50 con 0x7f7d84100d10 2023-10-08T13:52:35.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.043+0000 7f7d6bfff700 1 -- 172.21.15.57:0/1360869395 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c0052a0 con 0x7f7d84100d10 2023-10-08T13:52:35.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.043+0000 7f7d89489700 1 -- 172.21.15.57:0/1360869395 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d841a5f20 con 0x7f7d84100d10 2023-10-08T13:52:35.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.046+0000 7f7d6bfff700 1 -- 172.21.15.57:0/1360869395 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7d7c01d480 con 0x7f7d84100d10 2023-10-08T13:52:35.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.046+0000 7f7d6bfff700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7d6c0a15f0 0x7f7d6c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.046+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7d6c0a15f0 0x7f7d6c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.046+0000 7f7d6bfff700 1 -- 172.21.15.57:0/1360869395 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7d7c0ce650 con 0x7f7d84100d10 2023-10-08T13:52:35.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.047+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7d6c0a15f0 0x7f7d6c0a3ab0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7d78005950 tx=0x7f7d780058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.047+0000 7f7d95672b80 1 Processor -- start 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.047+0000 7f7d95672b80 1 -- start start 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.047+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c3e80 0x55618a5c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.047+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.047+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 0x55618a5d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.047+0000 7f7d95672b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a52d460 con 0x55618a5c47a0 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.047+0000 7f7d95672b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a51ec20 con 0x55618a5cf650 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.047+0000 7f7d95672b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a51c540 con 0x55618a5c3e80 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.047+0000 7f7d827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57140/0 (socket says 172.21.15.57:57140) 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d827fc700 1 -- 172.21.15.57:0/4041624418 learned_addr learned my addr 172.21.15.57:0/4041624418 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:35.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/4041624418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c3e80 0x55618a5c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d837fe700 1 --2- 172.21.15.57:0/4041624418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 0x55618a5d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d827fc700 1 -- 172.21.15.57:0/4041624418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c3e80 msgr2=0x55618a5c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d827fc700 1 --2- 172.21.15.57:0/4041624418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c3e80 0x55618a5c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d827fc700 1 -- 172.21.15.57:0/4041624418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 msgr2=0x55618a5d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d827fc700 1 --2- 172.21.15.57:0/4041624418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 0x55618a5d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d827fc700 1 -- 172.21.15.57:0/4041624418 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618a342de0 con 0x55618a5c47a0 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d837fe700 1 --2- 172.21.15.57:0/4041624418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 0x55618a5d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d827fc700 1 --2- 172.21.15.57:0/4041624418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5cf110 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7d74007be0 tx=0x7f7d74007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab062f939a61be7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d697fa700 1 -- 172.21.15.57:0/4041624418 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74004310 con 0x55618a5c47a0 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d697fa700 1 -- 172.21.15.57:0/4041624418 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d74010e40 con 0x55618a5c47a0 2023-10-08T13:52:35.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.048+0000 7f7d697fa700 1 -- 172.21.15.57:0/4041624418 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74003e90 con 0x55618a5c47a0 2023-10-08T13:52:35.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 -- 172.21.15.57:0/4041624418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 msgr2=0x55618a5cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:35.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 --2- 172.21.15.57:0/4041624418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5cf110 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7d74007be0 tx=0x7f7d74007ef0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 -- 172.21.15.57:0/4041624418 shutdown_connections 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 --2- 172.21.15.57:0/4041624418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 0x55618a5d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 --2- 172.21.15.57:0/4041624418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c3e80 0x55618a5c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 --2- 172.21.15.57:0/4041624418 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5cf110 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 -- 172.21.15.57:0/4041624418 >> 172.21.15.57:0/4041624418 conn(0x55618a539260 msgr2=0x55618a5c02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 -- 172.21.15.57:0/4041624418 shutdown_connections 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 -- 172.21.15.57:0/4041624418 wait complete. 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 -- start start 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.049+0000 7f7d95672b80 1 Processor -- start 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c3e80 0x55618a5cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5c47a0 0x55618a5cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5cf650 0x55618a5cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d95672b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a517e30 con 0x55618a5cf650 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d95672b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a4b4710 con 0x55618a5c47a0 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d95672b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a4b43b0 con 0x55618a5c3e80 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d95672b80 10 client.0 _ll_register_callbacks cb 0x55618a311800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5cf650 0x55618a5cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5cf650 0x55618a5cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57148/0 (socket says 172.21.15.57:57148) 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d837fe700 1 -- 172.21.15.57:0/2687680859 learned_addr learned my addr 172.21.15.57:0/2687680859 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d827fc700 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5c47a0 0x55618a5cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c3e80 0x55618a5cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d837fe700 1 -- 172.21.15.57:0/2687680859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c3e80 msgr2=0x55618a5cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d837fe700 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c3e80 0x55618a5cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d837fe700 1 -- 172.21.15.57:0/2687680859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5c47a0 msgr2=0x55618a5cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d837fe700 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5c47a0 0x55618a5cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d837fe700 1 -- 172.21.15.57:0/2687680859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d74007840 con 0x55618a5cf650 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d837fe700 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5cf650 0x55618a5cca50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0978c0 tx=0x7f7d7c00cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acd1b4bd4433c1e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.050+0000 7f7d66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d66ff5700 1 -- 172.21.15.57:0/2687680859 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c09f320 con 0x55618a5cf650 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d95672b80 1 -- 172.21.15.57:0/2687680859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55618a4dfeb0 con 0x55618a5cf650 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d66ff5700 1 -- 172.21.15.57:0/2687680859 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7d7c09bac0 con 0x55618a5cf650 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d66ff5700 1 -- 172.21.15.57:0/2687680859 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d7c09b3c0 con 0x55618a5cf650 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d95672b80 10 client.16191 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d95672b80 1 -- 172.21.15.57:0/2687680859 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55618a32bc60 con 0x55618a5cf650 2023-10-08T13:52:35.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d66ff5700 1 -- 172.21.15.57:0/2687680859 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55618a4dfeb0 con 0x55618a5cf650 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d66ff5700 1 -- 172.21.15.57:0/2687680859 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7d7c09a8b0 con 0x55618a5cf650 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d95672b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d95672b80 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d95672b80 20 client.16191 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d95672b80 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55618a5dee90 0x55618a5e1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d95672b80 4 client.16191 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.051+0000 7f7d95672b80 1 -- 172.21.15.57:0/2687680859 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55618a510e10 con 0x55618a5dee90 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.052+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55618a5dee90 0x55618a5e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.052+0000 7f7d66ff5700 10 client.16191 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.052+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55618a5dee90 0x55618a5e1280 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73aee95571f7f21f server_cookie=7c925b2666f4d4a7 in_seq=0 out_seq=0 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d66ff5700 1 -- 172.21.15.57:0/2687680859 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55618a510e10 con 0x55618a5dee90 2023-10-08T13:52:35.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d66ff5700 10 client.16191 handle_command_reply: tid=0 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 client.16191 shutdown 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 -- 172.21.15.57:0/2687680859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5cf650 msgr2=0x55618a5cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5cf650 0x55618a5cca50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0978c0 tx=0x7f7d7c00cfa0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 -- 172.21.15.57:0/2687680859 shutdown_connections 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55618a5dee90 0x55618a5e1280 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5cf650 0x55618a5cca50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c3e80 0x55618a5cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 --2- 172.21.15.57:0/2687680859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5c47a0 0x55618a5cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d66ff5700 0 client.16191 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 -- 172.21.15.57:0/2687680859 >> 172.21.15.57:0/2687680859 conn(0x55618a539260 msgr2=0x55618a5bfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 -- 172.21.15.57:0/2687680859 shutdown_connections 2023-10-08T13:52:35.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 1 -- 172.21.15.57:0/2687680859 wait complete. 2023-10-08T13:52:35.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.053+0000 7f7d95672b80 20 client.16191 trim_cache size 0 max 16384 2023-10-08T13:52:35.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.066+0000 7f7d95672b80 1 Processor -- start 2023-10-08T13:52:35.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d95672b80 1 -- start start 2023-10-08T13:52:35.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c3e80 0x55618a5d6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c47a0 0x55618a5cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 0x55618a5cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d95672b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a5d1f00 con 0x55618a5c3e80 2023-10-08T13:52:35.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d95672b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a398920 con 0x55618a5cf650 2023-10-08T13:52:35.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d95672b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a5d66d0 con 0x55618a5c47a0 2023-10-08T13:52:35.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d82ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c3e80 0x55618a5d6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d82ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c3e80 0x55618a5d6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57152/0 (socket says 172.21.15.57:57152) 2023-10-08T13:52:35.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d82ffd700 1 -- 172.21.15.57:0/2336545615 learned_addr learned my addr 172.21.15.57:0/2336545615 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:35.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d837fe700 1 --2- 172.21.15.57:0/2336545615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 0x55618a5cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d827fc700 1 --2- 172.21.15.57:0/2336545615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c47a0 0x55618a5cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d82ffd700 1 -- 172.21.15.57:0/2336545615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c47a0 msgr2=0x55618a5cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/2336545615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c47a0 0x55618a5cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d82ffd700 1 -- 172.21.15.57:0/2336545615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 msgr2=0x55618a5cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.067+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/2336545615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 0x55618a5cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.068+0000 7f7d82ffd700 1 -- 172.21.15.57:0/2336545615 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618a369250 con 0x55618a5c3e80 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.068+0000 7f7d837fe700 1 --2- 172.21.15.57:0/2336545615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 0x55618a5cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.068+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/2336545615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c3e80 0x55618a5d6190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55618a5e1b30 tx=0x7f7d78012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bfb1b6775b0f5f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.068+0000 7f7d69ffb700 1 -- 172.21.15.57:0/2336545615 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78024040 con 0x55618a5c3e80 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.068+0000 7f7d69ffb700 1 -- 172.21.15.57:0/2336545615 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d78011a00 con 0x55618a5c3e80 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.068+0000 7f7d69ffb700 1 -- 172.21.15.57:0/2336545615 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7801fa20 con 0x55618a5c3e80 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 -- 172.21.15.57:0/2336545615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c3e80 msgr2=0x55618a5d6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:35.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 --2- 172.21.15.57:0/2336545615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c3e80 0x55618a5d6190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55618a5e1b30 tx=0x7f7d78012000 comp rx=0 tx=0).stop 2023-10-08T13:52:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 -- 172.21.15.57:0/2336545615 shutdown_connections 2023-10-08T13:52:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 --2- 172.21.15.57:0/2336545615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5cf650 0x55618a5cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 --2- 172.21.15.57:0/2336545615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c3e80 0x55618a5d6190 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 --2- 172.21.15.57:0/2336545615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5c47a0 0x55618a5cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 -- 172.21.15.57:0/2336545615 >> 172.21.15.57:0/2336545615 conn(0x55618a539260 msgr2=0x55618a5e5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 -- 172.21.15.57:0/2336545615 shutdown_connections 2023-10-08T13:52:35.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 -- 172.21.15.57:0/2336545615 wait complete. 2023-10-08T13:52:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 -- start start 2023-10-08T13:52:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 Processor -- start 2023-10-08T13:52:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5c3e80 0x55618a5d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5d6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5cf650 0x55618a5d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a5d2a80 con 0x55618a5c47a0 2023-10-08T13:52:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a5d2bf0 con 0x55618a5c3e80 2023-10-08T13:52:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.069+0000 7f7d95672b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618a5d2db0 con 0x55618a5cf650 2023-10-08T13:52:35.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d95672b80 10 client.0 _ll_register_callbacks cb 0x7f7d7c097700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:35.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5d6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5d6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57160/0 (socket says 172.21.15.57:57160) 2023-10-08T13:52:35.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d827fc700 1 -- 172.21.15.57:0/1528654313 learned_addr learned my addr 172.21.15.57:0/1528654313 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:35.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d837fe700 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5cf650 0x55618a5d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5c3e80 0x55618a5d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d827fc700 1 -- 172.21.15.57:0/1528654313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5cf650 msgr2=0x55618a5d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:35.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d827fc700 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5cf650 0x55618a5d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d827fc700 1 -- 172.21.15.57:0/1528654313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5c3e80 msgr2=0x55618a5d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d827fc700 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5c3e80 0x55618a5d5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d827fc700 1 -- 172.21.15.57:0/1528654313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55618a32bc60 con 0x55618a5c47a0 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d827fc700 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5d6120 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7d74007be0 tx=0x7f7d74003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aae83aae3d3e3a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d66ff5700 1 -- 172.21.15.57:0/1528654313 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7401f030 con 0x55618a5c47a0 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d66ff5700 1 -- 172.21.15.57:0/1528654313 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7d74020070 con 0x55618a5c47a0 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.070+0000 7f7d95672b80 1 -- 172.21.15.57:0/1528654313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d74010e40 con 0x55618a5c47a0 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d66ff5700 1 -- 172.21.15.57:0/1528654313 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d74010b90 con 0x55618a5c47a0 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d95672b80 10 client.16197 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d95672b80 1 -- 172.21.15.57:0/1528654313 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f7d7800fb60 con 0x55618a5c47a0 2023-10-08T13:52:35.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d66ff5700 1 -- 172.21.15.57:0/1528654313 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d74010e40 con 0x55618a5c47a0 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d66ff5700 1 -- 172.21.15.57:0/1528654313 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7d74025070 con 0x55618a5c47a0 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d95672b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d95672b80 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d95672b80 20 client.16197 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d95672b80 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55618a633f90 0x55618a6363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d95672b80 4 client.16197 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d95672b80 1 -- 172.21.15.57:0/1528654313 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55618a510b30 con 0x55618a633f90 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.071+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55618a633f90 0x55618a6363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d66ff5700 10 client.16197 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d82ffd700 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55618a633f90 0x55618a6363b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7a927de0673c6c server_cookie=15be5795afd76b51 in_seq=0 out_seq=0 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d66ff5700 1 -- 172.21.15.57:0/1528654313 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55618a510b30 con 0x55618a633f90 2023-10-08T13:52:35.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d66ff5700 10 client.16197 handle_command_reply: tid=0 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 client.16197 shutdown 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 -- 172.21.15.57:0/1528654313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 msgr2=0x55618a5d6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5d6120 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7d74007be0 tx=0x7f7d74003920 comp rx=0 tx=0).stop 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 -- 172.21.15.57:0/1528654313 shutdown_connections 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55618a633f90 0x55618a6363b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55618a5cf650 0x55618a5d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d66ff5700 0 client.16197 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618a5c3e80 0x55618a5d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 --2- 172.21.15.57:0/1528654313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55618a5c47a0 0x55618a5d6120 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 -- 172.21.15.57:0/1528654313 >> 172.21.15.57:0/1528654313 conn(0x55618a539260 msgr2=0x55618a5c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:35.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 -- 172.21.15.57:0/1528654313 shutdown_connections 2023-10-08T13:52:35.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 1 -- 172.21.15.57:0/1528654313 wait complete. 2023-10-08T13:52:35.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.072+0000 7f7d95672b80 20 client.16197 trim_cache size 0 max 16384 2023-10-08T13:52:35.081 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:52:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 -- 172.21.15.57:0/1360869395 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7d6c0a15f0 msgr2=0x7f7d6c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7d6c0a15f0 0x7f7d6c0a3ab0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7d78005950 tx=0x7f7d780058e0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 -- 172.21.15.57:0/1360869395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d84100d10 msgr2=0x7f7d841a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d84100d10 0x7f7d841a4fd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00b070 tx=0x7f7d7c009300 comp rx=0 tx=0).stop 2023-10-08T13:52:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 -- 172.21.15.57:0/1360869395 shutdown_connections 2023-10-08T13:52:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7d84100d10 0x7f7d841a4fd0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d840ff320 0x7f7d84100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7d6c0a15f0 0x7f7d6c0a3ab0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 --2- 172.21.15.57:0/1360869395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d84105b30 0x7f7d841007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:35.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 -- 172.21.15.57:0/1360869395 >> 172.21.15.57:0/1360869395 conn(0x7f7d840749a0 msgr2=0x7f7d840fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:35.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 -- 172.21.15.57:0/1360869395 shutdown_connections 2023-10-08T13:52:35.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:35.073+0000 7f7d89489700 1 -- 172.21.15.57:0/1360869395 wait complete. 2023-10-08T13:52:35.087 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-08T13:52:35.125 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-08T13:52:35.126 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-08T13:52:35.126 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-08T13:52:35.164 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-08T13:52:35.167 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T13:52:35.167 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-08T13:52:35.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:34 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1363793679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:52:35.225 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2023-10-08T13:52:35.226 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2023-10-08T13:52:35.226 DEBUG:teuthology.orchestra.run.smithi057:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2023-10-08T13:52:35.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:34 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1363793679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:52:35.260 INFO:tasks.workunit.client.0.smithi057.stdout:getting blogbench 2023-10-08T13:52:35.260 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'getting blogbench' 2023-10-08T13:52:35.260 INFO:tasks.workunit.client.0.smithi057.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-10-08T13:52:35.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:34 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1363793679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:52:35.293 INFO:tasks.workunit.client.0.smithi057.stderr:--2023-10-08 13:52:35-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-10-08T13:52:35.302 INFO:tasks.workunit.client.0.smithi057.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2023-10-08T13:52:35.323 INFO:tasks.workunit.client.0.smithi057.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2023-10-08T13:52:35.344 INFO:tasks.workunit.client.0.smithi057.stderr:HTTP request sent, awaiting response... 200 OK 2023-10-08T13:52:35.344 INFO:tasks.workunit.client.0.smithi057.stderr:Length: 103355 (101K) [application/octet-stream] 2023-10-08T13:52:35.345 INFO:tasks.workunit.client.0.smithi057.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2023-10-08T13:52:35.348 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-08T13:52:35.386 INFO:tasks.workunit.client.0.smithi057.stderr: 0K .......... .......... .......... .......... .......... 49% 1.22M 0s 2023-10-08T13:52:35.406 INFO:tasks.workunit.client.0.smithi057.stderr: 50K .......... .......... .......... .......... .......... 99% 2.42M 0s 2023-10-08T13:52:35.406 INFO:tasks.workunit.client.0.smithi057.stderr: 100K 100% 1779G=0.06s 2023-10-08T13:52:35.406 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-08T13:52:35.406 INFO:tasks.workunit.client.0.smithi057.stderr:2023-10-08 13:52:35 (1.64 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2023-10-08T13:52:35.406 INFO:tasks.workunit.client.0.smithi057.stderr: 2023-10-08T13:52:35.406 INFO:tasks.workunit.client.0.smithi057.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2023-10-08T13:52:35.433 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/ 2023-10-08T13:52:35.434 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/m4/ 2023-10-08T13:52:35.435 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/m4/acx_pthread.m4 2023-10-08T13:52:35.436 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/ 2023-10-08T13:52:35.437 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/blogbench.8 2023-10-08T13:52:35.438 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/Makefile.am 2023-10-08T13:52:35.439 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/man/Makefile.in 2023-10-08T13:52:35.439 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/ 2023-10-08T13:52:35.440 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/mysnprintf.c 2023-10-08T13:52:35.441 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/mysnprintf.h 2023-10-08T13:52:35.442 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/blogbench.c 2023-10-08T13:52:35.443 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/blogbench.h 2023-10-08T13:52:35.444 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/fakesnprintf.c 2023-10-08T13:52:35.445 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/fakesnprintf.h 2023-10-08T13:52:35.445 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/blogbench_p.h 2023-10-08T13:52:35.446 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/bsd-getopt_long.c 2023-10-08T13:52:35.447 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/bsd-getopt_long.h 2023-10-08T13:52:35.447 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/Makefile.am 2023-10-08T13:52:35.448 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/Makefile.in 2023-10-08T13:52:35.449 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/process.c 2023-10-08T13:52:35.450 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/globals.h 2023-10-08T13:52:35.450 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/commenter.c 2023-10-08T13:52:35.451 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/rewriter.c 2023-10-08T13:52:35.452 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/reader.c 2023-10-08T13:52:35.453 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/writer.c 2023-10-08T13:52:35.454 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/src/helpers.c 2023-10-08T13:52:35.455 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/NEWS 2023-10-08T13:52:35.456 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/compile 2023-10-08T13:52:35.457 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/depcomp 2023-10-08T13:52:35.458 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/aclocal.m4 2023-10-08T13:52:35.459 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/README 2023-10-08T13:52:35.460 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/configure 2023-10-08T13:52:35.463 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/configure.ac 2023-10-08T13:52:35.463 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/config.guess 2023-10-08T13:52:35.464 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/install-sh 2023-10-08T13:52:35.465 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/config.sub 2023-10-08T13:52:35.466 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/missing 2023-10-08T13:52:35.467 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/mkinstalldirs 2023-10-08T13:52:35.467 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/Makefile.am 2023-10-08T13:52:35.468 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/Makefile.in 2023-10-08T13:52:35.468 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/config.h.in 2023-10-08T13:52:35.469 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/AUTHORS 2023-10-08T13:52:35.470 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/INSTALL 2023-10-08T13:52:35.471 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/ChangeLog 2023-10-08T13:52:35.472 INFO:tasks.workunit.client.0.smithi057.stdout:blogbench-1.0/COPYING 2023-10-08T13:52:35.473 INFO:tasks.workunit.client.0.smithi057.stderr:+ cd blogbench-1.0/ 2023-10-08T13:52:35.473 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'making blogbench' 2023-10-08T13:52:35.473 INFO:tasks.workunit.client.0.smithi057.stdout:making blogbench 2023-10-08T13:52:35.473 INFO:tasks.workunit.client.0.smithi057.stderr:+ ./configure 2023-10-08T13:52:35.582 INFO:tasks.workunit.client.0.smithi057.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-10-08T13:52:36.090 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-08T13:52:36.091 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-08T13:52:36.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:35 smithi057 ceph-mon[99476]: pgmap v330: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2023-10-08T13:52:36.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:35 smithi115 ceph-mon[104408]: pgmap v330: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2023-10-08T13:52:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.229+0000 7f5cc91a2700 1 -- 172.21.15.57:0/1391144222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc40ff620 msgr2=0x7f5cc40ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.229+0000 7f5cc91a2700 1 --2- 172.21.15.57:0/1391144222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc40ff620 0x7f5cc40ffaa0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009990 tx=0x7f5cb8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.229+0000 7f5cc91a2700 1 -- 172.21.15.57:0/1391144222 shutdown_connections 2023-10-08T13:52:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.229+0000 7f5cc91a2700 1 --2- 172.21.15.57:0/1391144222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc40fffe0 0x7f5cc410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.229+0000 7f5cc91a2700 1 --2- 172.21.15.57:0/1391144222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc41056b0 0x7f5cc4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.229+0000 7f5cc91a2700 1 --2- 172.21.15.57:0/1391144222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc40ff620 0x7f5cc40ffaa0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.229+0000 7f5cc91a2700 1 -- 172.21.15.57:0/1391144222 >> 172.21.15.57:0/1391144222 conn(0x7f5cc4076780 msgr2=0x7f5cc4076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.230+0000 7f5cc91a2700 1 -- 172.21.15.57:0/1391144222 shutdown_connections 2023-10-08T13:52:36.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.230+0000 7f5cc91a2700 1 -- 172.21.15.57:0/1391144222 wait complete. 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.230+0000 7f5cc91a2700 1 Processor -- start 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.230+0000 7f5cc91a2700 1 -- start start 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc91a2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc40ff620 0x7f5cc4197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc91a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc40fffe0 0x7f5cc4197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc91a2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc41056b0 0x7f5cc41a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc91a2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4198710 con 0x7f5cc41056b0 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc91a2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc41a2f40 con 0x7f5cc40fffe0 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc91a2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc41a3080 con 0x7f5cc40ff620 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc41056b0 0x7f5cc41a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc41056b0 0x7f5cc41a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57180/0 (socket says 172.21.15.57:57180) 2023-10-08T13:52:36.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc359e700 1 -- 172.21.15.57:0/2852853672 learned_addr learned my addr 172.21.15.57:0/2852853672 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc40ff620 0x7f5cc4197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc259c700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc40fffe0 0x7f5cc4197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc359e700 1 -- 172.21.15.57:0/2852853672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc40ff620 msgr2=0x7f5cc4197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc359e700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc40ff620 0x7f5cc4197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc359e700 1 -- 172.21.15.57:0/2852853672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc40fffe0 msgr2=0x7f5cc4197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc359e700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc40fffe0 0x7f5cc4197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc359e700 1 -- 172.21.15.57:0/2852853672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb80095f0 con 0x7f5cc41056b0 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.231+0000 7f5cc359e700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc41056b0 0x7f5cc41a2a00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc007f10 tx=0x7f5cbc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.232+0000 7f5cabfff700 1 -- 172.21.15.57:0/2852853672 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc017070 con 0x7f5cc41056b0 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.232+0000 7f5cc91a2700 1 -- 172.21.15.57:0/2852853672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc41a32e0 con 0x7f5cc41056b0 2023-10-08T13:52:36.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.232+0000 7f5cabfff700 1 -- 172.21.15.57:0/2852853672 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cbc00f040 con 0x7f5cc41056b0 2023-10-08T13:52:36.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.232+0000 7f5cabfff700 1 -- 172.21.15.57:0/2852853672 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc005740 con 0x7f5cc41056b0 2023-10-08T13:52:36.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.232+0000 7f5cc91a2700 1 -- 172.21.15.57:0/2852853672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc41a37a0 con 0x7f5cc41056b0 2023-10-08T13:52:36.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.234+0000 7f5cabfff700 1 -- 172.21.15.57:0/2852853672 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5cbc0058a0 con 0x7f5cc41056b0 2023-10-08T13:52:36.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.234+0000 7f5cabfff700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cac0a15f0 0x7f5cac0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.234+0000 7f5cabfff700 1 -- 172.21.15.57:0/2852853672 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5cbc0ccbf0 con 0x7f5cc41056b0 2023-10-08T13:52:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.234+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cac0a15f0 0x7f5cac0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.235+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cac0a15f0 0x7f5cac0a3ab0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4077ab0 tx=0x7f5cb400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.235+0000 7f5cd538bb80 1 Processor -- start 2023-10-08T13:52:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.235+0000 7f5cd538bb80 1 -- start start 2023-10-08T13:52:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.235+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b41932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.235+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.235+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 0x5641b41a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.235+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b40fc4e0 con 0x5641b4193820 2023-10-08T13:52:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.235+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b40edca0 con 0x5641b419e6d0 2023-10-08T13:52:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.235+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b40eb5c0 con 0x5641b4192f00 2023-10-08T13:52:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.235+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57186/0 (socket says 172.21.15.57:57186) 2023-10-08T13:52:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc259c700 1 -- 172.21.15.57:0/1733021727 learned_addr learned my addr 172.21.15.57:0/1733021727 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/1733021727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b41932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc359e700 1 --2- 172.21.15.57:0/1733021727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 0x5641b41a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc259c700 1 -- 172.21.15.57:0/1733021727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 msgr2=0x5641b41932e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc259c700 1 --2- 172.21.15.57:0/1733021727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b41932e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc259c700 1 -- 172.21.15.57:0/1733021727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 msgr2=0x5641b41a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc259c700 1 --2- 172.21.15.57:0/1733021727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 0x5641b41a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc259c700 1 -- 172.21.15.57:0/1733021727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641b3f11e60 con 0x5641b4193820 2023-10-08T13:52:36.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc359e700 1 --2- 172.21.15.57:0/1733021727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 0x5641b41a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:36.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5cc259c700 1 --2- 172.21.15.57:0/1733021727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8015000 tx=0x7f5cb8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6544765fce5ec639 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5ca97fa700 1 -- 172.21.15.57:0/1733021727 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb801d040 con 0x5641b4193820 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5ca97fa700 1 -- 172.21.15.57:0/1733021727 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cb800bec0 con 0x5641b4193820 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.236+0000 7f5ca97fa700 1 -- 172.21.15.57:0/1733021727 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8003bf0 con 0x5641b4193820 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 -- 172.21.15.57:0/1733021727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 msgr2=0x5641b419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/1733021727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8015000 tx=0x7f5cb8009510 comp rx=0 tx=0).stop 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 -- 172.21.15.57:0/1733021727 shutdown_connections 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/1733021727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 0x5641b41a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/1733021727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b41932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/1733021727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e190 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 -- 172.21.15.57:0/1733021727 >> 172.21.15.57:0/1733021727 conn(0x5641b41082e0 msgr2=0x5641b418f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 -- 172.21.15.57:0/1733021727 shutdown_connections 2023-10-08T13:52:36.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 -- 172.21.15.57:0/1733021727 wait complete. 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 -- start start 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.237+0000 7f5cd538bb80 1 Processor -- start 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b419e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 0x5641b419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b40e6eb0 con 0x5641b4193820 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b4083790 con 0x5641b419e6d0 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b4083430 con 0x5641b4192f00 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cd538bb80 10 client.0 _ll_register_callbacks cb 0x5641b3ee0880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57192/0 (socket says 172.21.15.57:57192) 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc259c700 1 -- 172.21.15.57:0/3961623495 learned_addr learned my addr 172.21.15.57:0/3961623495 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc359e700 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 0x5641b419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc259c700 1 -- 172.21.15.57:0/3961623495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 msgr2=0x5641b419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc259c700 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc259c700 1 -- 172.21.15.57:0/3961623495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 msgr2=0x5641b419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc259c700 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 0x5641b419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc259c700 1 -- 172.21.15.57:0/3961623495 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc410cde0 con 0x5641b4193820 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc359e700 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 0x5641b419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5cc259c700 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419b390 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80037d0 tx=0x7f5cb800b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38801b2862ee5076 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.238+0000 7f5ca6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:36.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3961623495 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb801d070 con 0x5641b4193820 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3961623495 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5cb80054e0 con 0x5641b4193820 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3961623495 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641b40dfe90 con 0x5641b4193820 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3961623495 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cb8003830 con 0x5641b4193820 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5cd538bb80 10 client.16209 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3961623495 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5641b3eface0 con 0x5641b4193820 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3961623495 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5641b40dfe90 con 0x5641b4193820 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3961623495 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5cb800faf0 con 0x5641b4193820 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5cd538bb80 10 client.16209 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5cd538bb80 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5cd538bb80 20 client.16209 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:36.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5641b41adc80 0x5641b41b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5cd538bb80 4 client.16209 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.239+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3961623495 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641b40dfbb0 con 0x5641b41adc80 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.240+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5641b41adc80 0x5641b41b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.240+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5641b41adc80 0x5641b41b0070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8168f9d82068a2b server_cookie=64a9207ba48445b2 in_seq=0 out_seq=0 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.240+0000 7f5ca6ff5700 10 client.16209 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3961623495 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5641b40dfbb0 con 0x5641b41adc80 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5ca6ff5700 10 client.16209 handle_command_reply: tid=0 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 client.16209 shutdown 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3961623495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 msgr2=0x5641b419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419b390 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80037d0 tx=0x7f5cb800b9a0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3961623495 shutdown_connections 2023-10-08T13:52:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5641b41adc80 0x5641b41b0070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b419e6d0 0x5641b419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5ca6ff5700 0 client.16209 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3961623495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419b390 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3961623495 >> 172.21.15.57:0/3961623495 conn(0x5641b41082e0 msgr2=0x5641b418ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3961623495 shutdown_connections 2023-10-08T13:52:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3961623495 wait complete. 2023-10-08T13:52:36.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.241+0000 7f5cd538bb80 20 client.16209 trim_cache size 0 max 16384 2023-10-08T13:52:36.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cd538bb80 1 Processor -- start 2023-10-08T13:52:36.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cd538bb80 1 -- start start 2023-10-08T13:52:36.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4192f00 0x5641b419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b419e6d0 0x5641b419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b3f679a0 con 0x5641b4193820 2023-10-08T13:52:36.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b4193c00 con 0x5641b4192f00 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b418bb60 con 0x5641b419e6d0 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57200/0 (socket says 172.21.15.57:57200) 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cc259c700 1 -- 172.21.15.57:0/92835854 learned_addr learned my addr 172.21.15.57:0/92835854 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.258+0000 7f5cc359e700 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b419e6d0 0x5641b419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4192f00 0x5641b419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cc259c700 1 -- 172.21.15.57:0/92835854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b419e6d0 msgr2=0x5641b419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cc259c700 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b419e6d0 0x5641b419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cc259c700 1 -- 172.21.15.57:0/92835854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4192f00 msgr2=0x5641b419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cc259c700 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4192f00 0x5641b419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cc259c700 1 -- 172.21.15.57:0/92835854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641b41b4c30 con 0x5641b4193820 2023-10-08T13:52:36.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cc359e700 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b419e6d0 0x5641b419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4192f00 0x5641b419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cc259c700 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e180 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8005f50 tx=0x7f5cb8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48e9bebab055af8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5ca9ffb700 1 -- 172.21.15.57:0/92835854 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb801d070 con 0x5641b4193820 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5ca9ffb700 1 -- 172.21.15.57:0/92835854 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cb8036070 con 0x5641b4193820 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5ca9ffb700 1 -- 172.21.15.57:0/92835854 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80216d0 con 0x5641b4193820 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cd538bb80 1 -- 172.21.15.57:0/92835854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 msgr2=0x5641b419e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.259+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e180 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8005f50 tx=0x7f5cb8000bc0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 -- 172.21.15.57:0/92835854 shutdown_connections 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b419e6d0 0x5641b419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4192f00 0x5641b419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/92835854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b4193820 0x5641b419e180 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 -- 172.21.15.57:0/92835854 >> 172.21.15.57:0/92835854 conn(0x5641b41082e0 msgr2=0x5641b41966f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 -- 172.21.15.57:0/92835854 shutdown_connections 2023-10-08T13:52:36.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 -- 172.21.15.57:0/92835854 wait complete. 2023-10-08T13:52:36.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 -- start start 2023-10-08T13:52:36.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 Processor -- start 2023-10-08T13:52:36.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b41a5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4193820 0x5641b41a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b419e6d0 0x5641b41a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b41a24d0 con 0x5641b419e6d0 2023-10-08T13:52:36.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b41a2640 con 0x5641b4193820 2023-10-08T13:52:36.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.260+0000 7f5cd538bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b4205e70 con 0x5641b4192f00 2023-10-08T13:52:36.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cd538bb80 10 client.0 _ll_register_callbacks cb 0x7f5cb8009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:36.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b419e6d0 0x5641b41a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b419e6d0 0x5641b41a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57208/0 (socket says 172.21.15.57:57208) 2023-10-08T13:52:36.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc359e700 1 -- 172.21.15.57:0/3425009760 learned_addr learned my addr 172.21.15.57:0/3425009760 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:36.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b41a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc259c700 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4193820 0x5641b41a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc359e700 1 -- 172.21.15.57:0/3425009760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 msgr2=0x5641b41a5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc359e700 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b41a5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc359e700 1 -- 172.21.15.57:0/3425009760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4193820 msgr2=0x5641b41a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc359e700 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4193820 0x5641b41a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc359e700 1 -- 172.21.15.57:0/3425009760 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641b3eface0 con 0x5641b419e6d0 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cc359e700 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b419e6d0 0x5641b41a1dc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00d7b0 tx=0x7f5cbc00dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce37e101344df14e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5ca6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3425009760 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc099d30 con 0x5641b419e6d0 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3425009760 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641b40aef30 con 0x5641b419e6d0 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3425009760 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5cbc0998d0 con 0x5641b419e6d0 2023-10-08T13:52:36.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3425009760 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cbc0b92c0 con 0x5641b419e6d0 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cd538bb80 10 client.16215 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.261+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3425009760 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5641b3f382d0 con 0x5641b419e6d0 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3425009760 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cbc063020 con 0x5641b419e6d0 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3425009760 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5cbc0a17d0 con 0x5641b419e6d0 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5cd538bb80 10 client.16215 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5cd538bb80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5cd538bb80 20 client.16215 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5641b4207690 0x5641b4209ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5cd538bb80 4 client.16215 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3425009760 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5641b3f32ff0 con 0x5641b4207690 2023-10-08T13:52:36.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5641b4207690 0x5641b4209ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5ca6ff5700 10 client.16215 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.262+0000 7f5cc2d9d700 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5641b4207690 0x5641b4209ab0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad14ba79a3aa9fa server_cookie=fb527cdfdbc28cae in_seq=0 out_seq=0 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5ca6ff5700 1 -- 172.21.15.57:0/3425009760 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5641b3f32ff0 con 0x5641b4207690 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5ca6ff5700 10 client.16215 handle_command_reply: tid=0 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 client.16215 shutdown 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3425009760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b419e6d0 msgr2=0x5641b41a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b419e6d0 0x5641b41a1dc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00d7b0 tx=0x7f5cbc00dbd0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3425009760 shutdown_connections 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5641b4207690 0x5641b4209ab0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5641b419e6d0 0x5641b41a1dc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5641b4192f00 0x5641b41a5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 --2- 172.21.15.57:0/3425009760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641b4193820 0x5641b41a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3425009760 >> 172.21.15.57:0/3425009760 conn(0x5641b41082e0 msgr2=0x5641b419ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5ca6ff5700 0 client.16215 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:36.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3425009760 shutdown_connections 2023-10-08T13:52:36.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 1 -- 172.21.15.57:0/3425009760 wait complete. 2023-10-08T13:52:36.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cd538bb80 20 client.16215 trim_cache size 0 max 16384 2023-10-08T13:52:36.272 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:52:36.272 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-10-08T13:52:36.272 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "4ce47814-5a32-4902-bcd1-98402297b012", 2023-10-08T13:52:36.272 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-10-08T13:52:36.273 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:52:36.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cc91a2700 1 -- 172.21.15.57:0/2852853672 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cac0a15f0 msgr2=0x7f5cac0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cc91a2700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cac0a15f0 0x7f5cac0a3ab0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4077ab0 tx=0x7f5cb400b410 comp rx=0 tx=0).stop 2023-10-08T13:52:36.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cc91a2700 1 -- 172.21.15.57:0/2852853672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc41056b0 msgr2=0x7f5cc41a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cc91a2700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc41056b0 0x7f5cc41a2a00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc007f10 tx=0x7f5cbc00d3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cc91a2700 1 -- 172.21.15.57:0/2852853672 shutdown_connections 2023-10-08T13:52:36.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cc91a2700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cc41056b0 0x7f5cc41a2a00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cc91a2700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc40ff620 0x7f5cc4197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cc91a2700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5cac0a15f0 0x7f5cac0a3ab0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cc91a2700 1 --2- 172.21.15.57:0/2852853672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cc40fffe0 0x7f5cc4197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.264+0000 7f5cc91a2700 1 -- 172.21.15.57:0/2852853672 >> 172.21.15.57:0/2852853672 conn(0x7f5cc4076780 msgr2=0x7f5cc40fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.265+0000 7f5cc91a2700 1 -- 172.21.15.57:0/2852853672 shutdown_connections 2023-10-08T13:52:36.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.265+0000 7f5cc91a2700 1 -- 172.21.15.57:0/2852853672 wait complete. 2023-10-08T13:52:36.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:35 smithi099 ceph-mon[103849]: pgmap v330: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2023-10-08T13:52:36.278 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:52:36.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 -- 172.21.15.57:0/1650294269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe02c0ff620 msgr2=0x7fe02c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 --2- 172.21.15.57:0/1650294269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe02c0ff620 0x7fe02c0ffaa0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009990 tx=0x7fe01c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 -- 172.21.15.57:0/1650294269 shutdown_connections 2023-10-08T13:52:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 --2- 172.21.15.57:0/1650294269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02c0fffe0 0x7fe02c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 --2- 172.21.15.57:0/1650294269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02c1056b0 0x7fe02c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 --2- 172.21.15.57:0/1650294269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe02c0ff620 0x7fe02c0ffaa0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 -- 172.21.15.57:0/1650294269 >> 172.21.15.57:0/1650294269 conn(0x7fe02c076780 msgr2=0x7fe02c076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 -- 172.21.15.57:0/1650294269 shutdown_connections 2023-10-08T13:52:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 -- 172.21.15.57:0/1650294269 wait complete. 2023-10-08T13:52:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 Processor -- start 2023-10-08T13:52:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.446+0000 7fe03270c700 1 -- start start 2023-10-08T13:52:36.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe03270c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02c0ff620 0x7fe02c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe03270c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe02c0fffe0 0x7fe02c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe03270c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02c1056b0 0x7fe02c1a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe03270c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c198700 con 0x7fe02c0fffe0 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe03270c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1a2f30 con 0x7fe02c1056b0 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe03270c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1a3070 con 0x7fe02c0ff620 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe02b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe02c0fffe0 0x7fe02c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe02b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe02c0fffe0 0x7fe02c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57220/0 (socket says 172.21.15.57:57220) 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe02b7fe700 1 -- 172.21.15.57:0/3243305904 learned_addr learned my addr 172.21.15.57:0/3243305904 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe030ca9700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02c1056b0 0x7fe02c1a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe02bfff700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02c0ff620 0x7fe02c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe02b7fe700 1 -- 172.21.15.57:0/3243305904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02c0ff620 msgr2=0x7fe02c197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe02b7fe700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02c0ff620 0x7fe02c197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe02b7fe700 1 -- 172.21.15.57:0/3243305904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02c1056b0 msgr2=0x7fe02c1a29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe02b7fe700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02c1056b0 0x7fe02c1a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe02b7fe700 1 -- 172.21.15.57:0/3243305904 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c0095f0 con 0x7fe02c0fffe0 2023-10-08T13:52:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe030ca9700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02c1056b0 0x7fe02c1a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.447+0000 7fe02b7fe700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe02c0fffe0 0x7fe02c197eb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe01c004730 tx=0x7fe01c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.448+0000 7fe0297fa700 1 -- 172.21.15.57:0/3243305904 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c01d070 con 0x7fe02c0fffe0 2023-10-08T13:52:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.448+0000 7fe0297fa700 1 -- 172.21.15.57:0/3243305904 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe01c022470 con 0x7fe02c0fffe0 2023-10-08T13:52:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.448+0000 7fe03270c700 1 -- 172.21.15.57:0/3243305904 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c1a3270 con 0x7fe02c0fffe0 2023-10-08T13:52:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.448+0000 7fe0297fa700 1 -- 172.21.15.57:0/3243305904 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c017650 con 0x7fe02c0fffe0 2023-10-08T13:52:36.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.448+0000 7fe03270c700 1 -- 172.21.15.57:0/3243305904 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c1a3750 con 0x7fe02c0fffe0 2023-10-08T13:52:36.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.450+0000 7fe0297fa700 1 -- 172.21.15.57:0/3243305904 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe01c0225e0 con 0x7fe02c0fffe0 2023-10-08T13:52:36.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.451+0000 7fe0297fa700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0180a15f0 0x7fe0180a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.451+0000 7fe0297fa700 1 -- 172.21.15.57:0/3243305904 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe01c0ce8c0 con 0x7fe02c0fffe0 2023-10-08T13:52:36.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.451+0000 7fe02bfff700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0180a15f0 0x7fe0180a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe02bfff700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0180a15f0 0x7fe0180a3ab0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe02c077ab0 tx=0x7fe014009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe03e8f5b80 1 Processor -- start 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe03e8f5b80 1 -- start start 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe02260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe0d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe0fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfd6b460 con 0x55f9bfe01e80 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfd5cc20 con 0x55f9bfe0d650 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfd5a540 con 0x55f9bfe027a0 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe02bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe02260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe02bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe02260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57226/0 (socket says 172.21.15.57:57226) 2023-10-08T13:52:36.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe02bfff700 1 -- 172.21.15.57:0/1753767365 learned_addr learned my addr 172.21.15.57:0/1753767365 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe02bfff700 1 -- 172.21.15.57:0/1753767365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 msgr2=0x55f9bfe0d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe02bfff700 1 --2- 172.21.15.57:0/1753767365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe0d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe02bfff700 1 -- 172.21.15.57:0/1753767365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 msgr2=0x55f9bfe0fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe030ca9700 1 --2- 172.21.15.57:0/1753767365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe0fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe02bfff700 1 --2- 172.21.15.57:0/1753767365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe0fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.452+0000 7fe02bfff700 1 -- 172.21.15.57:0/1753767365 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bfb80de0 con 0x55f9bfe01e80 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe02bfff700 1 --2- 172.21.15.57:0/1753767365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe02260 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe014011e20 tx=0x7fe014014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba2ca4caa95fa63 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe0227fc700 1 -- 172.21.15.57:0/1753767365 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01401f620 con 0x55f9bfe01e80 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe0227fc700 1 -- 172.21.15.57:0/1753767365 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe014016070 con 0x55f9bfe01e80 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe0227fc700 1 -- 172.21.15.57:0/1753767365 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01401d9a0 con 0x55f9bfe01e80 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1753767365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 msgr2=0x55f9bfe02260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/1753767365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe02260 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe014011e20 tx=0x7fe014014000 comp rx=0 tx=0).stop 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1753767365 shutdown_connections 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/1753767365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe0fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/1753767365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe02260 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/1753767365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe0d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1753767365 >> 172.21.15.57:0/1753767365 conn(0x55f9bfd77260 msgr2=0x55f9bfdfe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.453+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1753767365 shutdown_connections 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1753767365 wait complete. 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe03e8f5b80 1 -- start start 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe03e8f5b80 1 Processor -- start 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe01e80 0x55f9bfe0d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe0a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe0d650 0x55f9bfe0ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfd55e30 con 0x55f9bfe0d650 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfcf2710 con 0x55f9bfe01e80 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfcf23b0 con 0x55f9bfe027a0 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe03e8f5b80 10 client.0 _ll_register_callbacks cb 0x55f9bfb4f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.454+0000 7fe030ca9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe0d650 0x55f9bfe0ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe030ca9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe0d650 0x55f9bfe0ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57236/0 (socket says 172.21.15.57:57236) 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe030ca9700 1 -- 172.21.15.57:0/2331118286 learned_addr learned my addr 172.21.15.57:0/2331118286 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe02bfff700 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe01e80 0x55f9bfe0d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe02b7fe700 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe0a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe030ca9700 1 -- 172.21.15.57:0/2331118286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 msgr2=0x55f9bfe0a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe030ca9700 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe0a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe030ca9700 1 -- 172.21.15.57:0/2331118286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe01e80 msgr2=0x55f9bfe0d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe030ca9700 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe01e80 0x55f9bfe0d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe030ca9700 1 -- 172.21.15.57:0/2331118286 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe014011ad0 con 0x55f9bfe0d650 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe030ca9700 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe0d650 0x55f9bfe0ab30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe024007ba0 tx=0x7fe024007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ff40a237def37d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe00b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe00b7fe700 1 -- 172.21.15.57:0/2331118286 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024009e90 con 0x55f9bfe0d650 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe00b7fe700 1 -- 172.21.15.57:0/2331118286 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe024010830 con 0x55f9bfe0d650 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/2331118286 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9bfd1deb0 con 0x55f9bfe0d650 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe00b7fe700 1 -- 172.21.15.57:0/2331118286 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0240109f0 con 0x55f9bfe0d650 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe03e8f5b80 10 client.16227 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.455+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/2331118286 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f9bfb69c60 con 0x55f9bfe0d650 2023-10-08T13:52:36.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.456+0000 7fe00b7fe700 1 -- 172.21.15.57:0/2331118286 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9bfd1deb0 con 0x55f9bfe0d650 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.456+0000 7fe00b7fe700 1 -- 172.21.15.57:0/2331118286 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe024015cf0 con 0x55f9bfe0d650 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.456+0000 7fe03e8f5b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.456+0000 7fe03e8f5b80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.456+0000 7fe03e8f5b80 20 client.16227 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.456+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9bfe1d330 0x55f9bfe1f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.456+0000 7fe03e8f5b80 4 client.16227 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.456+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/2331118286 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9bfd4ee10 con 0x55f9bfe1d330 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.456+0000 7fe02bfff700 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9bfe1d330 0x55f9bfe1f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.457+0000 7fe00b7fe700 10 client.16227 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.457+0000 7fe02bfff700 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9bfe1d330 0x55f9bfe1f750 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e9450e1fc4761f server_cookie=873836485a08df0e in_seq=0 out_seq=0 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.457+0000 7fe00b7fe700 1 -- 172.21.15.57:0/2331118286 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f9bfd4ee10 con 0x55f9bfe1d330 2023-10-08T13:52:36.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.457+0000 7fe00b7fe700 10 client.16227 handle_command_reply: tid=0 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.457+0000 7fe03e8f5b80 1 client.16227 shutdown 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/2331118286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe0d650 msgr2=0x55f9bfe0ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe0d650 0x55f9bfe0ab30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe024007ba0 tx=0x7fe024007eb0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/2331118286 shutdown_connections 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9bfe1d330 0x55f9bfe1f750 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe0d650 0x55f9bfe0ab30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe00b7fe700 0 client.16227 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe01e80 0x55f9bfe0d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/2331118286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe0a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/2331118286 >> 172.21.15.57:0/2331118286 conn(0x55f9bfd77260 msgr2=0x55f9bfdfdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/2331118286 shutdown_connections 2023-10-08T13:52:36.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/2331118286 wait complete. 2023-10-08T13:52:36.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.458+0000 7fe03e8f5b80 20 client.16227 trim_cache size 0 max 16384 2023-10-08T13:52:36.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.474+0000 7fe03e8f5b80 1 Processor -- start 2023-10-08T13:52:36.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.474+0000 7fe03e8f5b80 1 -- start start 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe01e80 0x55f9bfe04d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe027a0 0x55f9bfe141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe05750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfbd6920 con 0x55f9bfe027a0 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfe030d0 con 0x55f9bfe0d650 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfe05290 con 0x55f9bfe01e80 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe027a0 0x55f9bfe141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe027a0 0x55f9bfe141a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57242/0 (socket says 172.21.15.57:57242) 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02b7fe700 1 -- 172.21.15.57:0/598130808 learned_addr learned my addr 172.21.15.57:0/598130808 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe030ca9700 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe05750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02bfff700 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe01e80 0x55f9bfe04d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02b7fe700 1 -- 172.21.15.57:0/598130808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe01e80 msgr2=0x55f9bfe04d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02b7fe700 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe01e80 0x55f9bfe04d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02b7fe700 1 -- 172.21.15.57:0/598130808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 msgr2=0x55f9bfe05750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02b7fe700 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe05750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02b7fe700 1 -- 172.21.15.57:0/598130808 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bfba7250 con 0x55f9bfe027a0 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe030ca9700 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe05750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02bfff700 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe01e80 0x55f9bfe04d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe02b7fe700 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe027a0 0x55f9bfe141a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0973b0 tx=0x7fe01c022c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96073a6810fc7f7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.475+0000 7fe022ffd700 1 -- 172.21.15.57:0/598130808 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c09bd30 con 0x55f9bfe027a0 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe022ffd700 1 -- 172.21.15.57:0/598130808 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe01c09b630 con 0x55f9bfe027a0 2023-10-08T13:52:36.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe022ffd700 1 -- 172.21.15.57:0/598130808 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c09b890 con 0x55f9bfe027a0 2023-10-08T13:52:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/598130808 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe027a0 msgr2=0x55f9bfe141a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe027a0 0x55f9bfe141a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0973b0 tx=0x7fe01c022c60 comp rx=0 tx=0).stop 2023-10-08T13:52:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/598130808 shutdown_connections 2023-10-08T13:52:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe05750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe01e80 0x55f9bfe04d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/598130808 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe027a0 0x55f9bfe141a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/598130808 >> 172.21.15.57:0/598130808 conn(0x55f9bfd77260 msgr2=0x55f9bfe188f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/598130808 shutdown_connections 2023-10-08T13:52:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/598130808 wait complete. 2023-10-08T13:52:36.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.476+0000 7fe03e8f5b80 1 -- start start 2023-10-08T13:52:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe03e8f5b80 1 Processor -- start 2023-10-08T13:52:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe13ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe03e8f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe12820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfe12d60 con 0x55f9bfe01e80 2023-10-08T13:52:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfe12ed0 con 0x55f9bfe0d650 2023-10-08T13:52:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe03e8f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bfe13090 con 0x55f9bfe027a0 2023-10-08T13:52:36.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe03e8f5b80 10 client.0 _ll_register_callbacks cb 0x7fe024009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:36.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe02bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe13ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe02bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe13ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57246/0 (socket says 172.21.15.57:57246) 2023-10-08T13:52:36.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe02bfff700 1 -- 172.21.15.57:0/1297892531 learned_addr learned my addr 172.21.15.57:0/1297892531 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:36.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe02b7fe700 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe02bfff700 1 -- 172.21.15.57:0/1297892531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 msgr2=0x55f9bfe122e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe030ca9700 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe12820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe02bfff700 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe122e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe02bfff700 1 -- 172.21.15.57:0/1297892531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 msgr2=0x55f9bfe12820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:36.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe02bfff700 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe12820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe02bfff700 1 -- 172.21.15.57:0/1297892531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c1a3750 con 0x55f9bfe01e80 2023-10-08T13:52:36.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe030ca9700 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe12820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe02bfff700 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe13ef0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe014020f20 tx=0x7fe014005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75ce226e8960d23 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.477+0000 7fe00b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe00b7fe700 1 -- 172.21.15.57:0/1297892531 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01402e040 con 0x55f9bfe01e80 2023-10-08T13:52:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe00b7fe700 1 -- 172.21.15.57:0/1297892531 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe014018070 con 0x55f9bfe01e80 2023-10-08T13:52:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1297892531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe014016070 con 0x55f9bfe01e80 2023-10-08T13:52:36.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe00b7fe700 1 -- 172.21.15.57:0/1297892531 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0140051f0 con 0x55f9bfe01e80 2023-10-08T13:52:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe03e8f5b80 10 client.16233 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1297892531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f9bfb7f2b0 con 0x55f9bfe01e80 2023-10-08T13:52:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe00b7fe700 1 -- 172.21.15.57:0/1297892531 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe014016070 con 0x55f9bfe01e80 2023-10-08T13:52:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe00b7fe700 1 -- 172.21.15.57:0/1297892531 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe014036050 con 0x55f9bfe01e80 2023-10-08T13:52:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe03e8f5b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:36.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe03e8f5b80 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:36.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe03e8f5b80 20 client.16233 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:36.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9bfe768d0 0x55f9bfe78cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:36.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe03e8f5b80 4 client.16233 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:52:36.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.478+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1297892531 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f9bfd4eb30 con 0x55f9bfe768d0 2023-10-08T13:52:36.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe02b7fe700 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9bfe768d0 0x55f9bfe78cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:36.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe00b7fe700 10 client.16233 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:36.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe02b7fe700 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9bfe768d0 0x55f9bfe78cf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b95f3de152365b5 server_cookie=53d96d0d7fa8cb02 in_seq=0 out_seq=0 2023-10-08T13:52:36.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe00b7fe700 1 -- 172.21.15.57:0/1297892531 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f9bfd4eb30 con 0x55f9bfe768d0 2023-10-08T13:52:36.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe00b7fe700 10 client.16233 handle_command_reply: tid=0 2023-10-08T13:52:36.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 client.16233 shutdown 2023-10-08T13:52:36.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1297892531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 msgr2=0x55f9bfe13ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe13ef0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe014020f20 tx=0x7fe014005d90 comp rx=0 tx=0).stop 2023-10-08T13:52:36.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1297892531 shutdown_connections 2023-10-08T13:52:36.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9bfe768d0 0x55f9bfe78cf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9bfe0d650 0x55f9bfe12820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe00b7fe700 0 client.16233 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:36.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9bfe01e80 0x55f9bfe13ef0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 --2- 172.21.15.57:0/1297892531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9bfe027a0 0x55f9bfe122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1297892531 >> 172.21.15.57:0/1297892531 conn(0x55f9bfd77260 msgr2=0x55f9bfe05750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1297892531 shutdown_connections 2023-10-08T13:52:36.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.479+0000 7fe03e8f5b80 1 -- 172.21.15.57:0/1297892531 wait complete. 2023-10-08T13:52:36.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03e8f5b80 20 client.16233 trim_cache size 0 max 16384 2023-10-08T13:52:36.493 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:52:36.493 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T13:52:36.493 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:52:36.493 INFO:teuthology.orchestra.run.smithi057.stdout: "4ce47814-5a32-4902-bcd1-98402297b012": { 2023-10-08T13:52:36.494 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:52:36.494 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "4ce47814-5a32-4902-bcd1-98402297b012", 2023-10-08T13:52:36.494 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:52:36.494 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:52:36.494 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:52:36.494 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:52:36.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 -- 172.21.15.57:0/3243305904 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0180a15f0 msgr2=0x7fe0180a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0180a15f0 0x7fe0180a3ab0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe02c077ab0 tx=0x7fe014009260 comp rx=0 tx=0).stop 2023-10-08T13:52:36.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 -- 172.21.15.57:0/3243305904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe02c0fffe0 msgr2=0x7fe02c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:36.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe02c0fffe0 0x7fe02c197eb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe01c004730 tx=0x7fe01c005ef0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 -- 172.21.15.57:0/3243305904 shutdown_connections 2023-10-08T13:52:36.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe0180a15f0 0x7fe0180a3ab0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02c1056b0 0x7fe02c1a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02c0ff620 0x7fe02c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 --2- 172.21.15.57:0/3243305904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe02c0fffe0 0x7fe02c197eb0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:36.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 -- 172.21.15.57:0/3243305904 >> 172.21.15.57:0/3243305904 conn(0x7fe02c076780 msgr2=0x7fe02c0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:36.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 -- 172.21.15.57:0/3243305904 shutdown_connections 2023-10-08T13:52:36.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:52:36.480+0000 7fe03270c700 1 -- 172.21.15.57:0/3243305904 wait complete. 2023-10-08T13:52:36.497 INFO:tasks.cephfs.filesystem:scrub status for tag:4ce47814-5a32-4902-bcd1-98402297b012 - {'path': '/', 'tag': '4ce47814-5a32-4902-bcd1-98402297b012', 'options': 'recursive,force'} 2023-10-08T13:52:36.590 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether build environment is sane... yes 2023-10-08T13:52:36.598 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gawk... gawk 2023-10-08T13:52:36.625 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether make sets $(MAKE)... yes 2023-10-08T13:52:36.634 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gcc... gcc 2023-10-08T13:52:36.683 INFO:tasks.workunit.client.0.smithi057.stdout:checking for C compiler default output file name... a.out 2023-10-08T13:52:36.684 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether the C compiler works... yes 2023-10-08T13:52:36.687 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether we are cross compiling... no 2023-10-08T13:52:36.722 INFO:tasks.workunit.client.0.smithi057.stdout:checking for suffix of executables... 2023-10-08T13:52:36.817 INFO:tasks.workunit.client.0.smithi057.stdout:checking for suffix of object files... o 2023-10-08T13:52:36.849 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether we are using the GNU C compiler... yes 2023-10-08T13:52:36.881 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether gcc accepts -g... yes 2023-10-08T13:52:36.923 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gcc option to accept ANSI C... none needed 2023-10-08T13:52:36.965 INFO:tasks.workunit.client.0.smithi057.stdout:checking for style of include used by make... GNU 2023-10-08T13:52:37.056 INFO:tasks.workunit.client.0.smithi057.stdout:checking dependency style of gcc... gcc3 2023-10-08T13:52:37.057 INFO:tasks.workunit.client.0.smithi057.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-10-08T13:52:37.107 INFO:tasks.workunit.client.0.smithi057.stdout:checking how to run the C preprocessor... gcc -E 2023-10-08T13:52:37.147 INFO:tasks.workunit.client.0.smithi057.stdout:checking for egrep... grep -E 2023-10-08T13:52:37.154 INFO:tasks.workunit.client.0.smithi057.stdout:checking for AIX... no 2023-10-08T13:52:37.218 INFO:tasks.workunit.client.0.smithi057.stdout:checking for library containing strerror... none required 2023-10-08T13:52:37.358 INFO:tasks.workunit.client.0.smithi057.stdout:checking for ANSI C header files... yes 2023-10-08T13:52:37.400 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-10-08T13:52:37.449 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/types.h... yes 2023-10-08T13:52:37.497 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/stat.h... yes 2023-10-08T13:52:37.546 INFO:tasks.workunit.client.0.smithi057.stdout:checking for stdlib.h... yes 2023-10-08T13:52:37.597 INFO:tasks.workunit.client.0.smithi057.stdout:checking for string.h... yes 2023-10-08T13:52:37.645 INFO:tasks.workunit.client.0.smithi057.stdout:checking for memory.h... yes 2023-10-08T13:52:37.693 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strings.h... yes 2023-10-08T13:52:37.741 INFO:tasks.workunit.client.0.smithi057.stdout:checking for inttypes.h... yes 2023-10-08T13:52:37.794 INFO:tasks.workunit.client.0.smithi057.stdout:checking for stdint.h... yes 2023-10-08T13:52:37.846 INFO:tasks.workunit.client.0.smithi057.stdout:checking for unistd.h... yes 2023-10-08T13:52:37.859 INFO:tasks.workunit.client.0.smithi057.stdout:checking for memory.h... (cached) yes 2023-10-08T13:52:37.864 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strings.h... (cached) yes 2023-10-08T13:52:37.869 INFO:tasks.workunit.client.0.smithi057.stdout:checking for unistd.h... (cached) yes 2023-10-08T13:52:37.912 INFO:tasks.workunit.client.0.smithi057.stdout:checking locale.h usability... yes 2023-10-08T13:52:37.929 INFO:tasks.workunit.client.0.smithi057.stdout:checking locale.h presence... yes 2023-10-08T13:52:37.929 INFO:tasks.workunit.client.0.smithi057.stdout:checking for locale.h... yes 2023-10-08T13:52:37.944 INFO:tasks.workunit.client.0.smithi057.stdout:checking for stdlib.h... (cached) yes 2023-10-08T13:52:37.949 INFO:tasks.workunit.client.0.smithi057.stdout:checking for string.h... (cached) yes 2023-10-08T13:52:38.005 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/socket.h usability... yes 2023-10-08T13:52:38.031 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/socket.h presence... yes 2023-10-08T13:52:38.031 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/socket.h... yes 2023-10-08T13:52:38.083 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/time.h usability... yes 2023-10-08T13:52:38.102 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/time.h presence... yes 2023-10-08T13:52:38.103 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/time.h... yes 2023-10-08T13:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.102+0000 7fa55317b700 1 -- 172.21.15.57:0/297183064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa54c0fff90 msgr2=0x7fa54c10c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.102+0000 7fa55317b700 1 --2- 172.21.15.57:0/297183064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa54c0fff90 0x7fa54c10c490 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa54400b0a0 tx=0x7fa54400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.102+0000 7fa55317b700 1 -- 172.21.15.57:0/297183064 shutdown_connections 2023-10-08T13:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.102+0000 7fa55317b700 1 --2- 172.21.15.57:0/297183064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa54c0fff90 0x7fa54c10c490 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.102+0000 7fa55317b700 1 --2- 172.21.15.57:0/297183064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa54c1045e0 0x7fa54c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.102+0000 7fa55317b700 1 --2- 172.21.15.57:0/297183064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa54c0ff5d0 0x7fa54c0ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.102+0000 7fa55317b700 1 -- 172.21.15.57:0/297183064 >> 172.21.15.57:0/297183064 conn(0x7fa54c076790 msgr2=0x7fa54c076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.102+0000 7fa55317b700 1 -- 172.21.15.57:0/297183064 shutdown_connections 2023-10-08T13:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.102+0000 7fa55317b700 1 -- 172.21.15.57:0/297183064 wait complete. 2023-10-08T13:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa55317b700 1 Processor -- start 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa55317b700 1 -- start start 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa55317b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa54c0ff5d0 0x7fa54c101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa55317b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa54c0fff90 0x7fa54c101760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa55317b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa54c1045e0 0x7fa54c101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa55317b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1a4fa0 con 0x7fa54c0ff5d0 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa55317b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1a5110 con 0x7fa54c0fff90 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa55317b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1a52d0 con 0x7fa54c1045e0 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa550f17700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa54c0ff5d0 0x7fa54c101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa54bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa54c0fff90 0x7fa54c101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa550f17700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa54c0ff5d0 0x7fa54c101220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57264/0 (socket says 172.21.15.57:57264) 2023-10-08T13:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.103+0000 7fa550f17700 1 -- 172.21.15.57:0/624907543 learned_addr learned my addr 172.21.15.57:0/624907543 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa551718700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa54c1045e0 0x7fa54c101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa54bfff700 1 -- 172.21.15.57:0/624907543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa54c1045e0 msgr2=0x7fa54c101ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa54bfff700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa54c1045e0 0x7fa54c101ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa54bfff700 1 -- 172.21.15.57:0/624907543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa54c0ff5d0 msgr2=0x7fa54c101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa54bfff700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa54c0ff5d0 0x7fa54c101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa54bfff700 1 -- 172.21.15.57:0/624907543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa544009d00 con 0x7fa54c0fff90 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa551718700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa54c1045e0 0x7fa54c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa550f17700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa54c0ff5d0 0x7fa54c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa54bfff700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa54c0fff90 0x7fa54c101760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa53800e9f0 tx=0x7fa53800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa549ffb700 1 -- 172.21.15.57:0/624907543 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800eee0 con 0x7fa54c0fff90 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa55317b700 1 -- 172.21.15.57:0/624907543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54c1a55b0 con 0x7fa54c0fff90 2023-10-08T13:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa549ffb700 1 -- 172.21.15.57:0/624907543 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa53800cbd0 con 0x7fa54c0fff90 2023-10-08T13:52:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa549ffb700 1 -- 172.21.15.57:0/624907543 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538018680 con 0x7fa54c0fff90 2023-10-08T13:52:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.104+0000 7fa55317b700 1 -- 172.21.15.57:0/624907543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c1a5b10 con 0x7fa54c0fff90 2023-10-08T13:52:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.107+0000 7fa549ffb700 1 -- 172.21.15.57:0/624907543 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa53800cd40 con 0x7fa54c0fff90 2023-10-08T13:52:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.107+0000 7fa549ffb700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa53c0a15f0 0x7fa53c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.107+0000 7fa550f17700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa53c0a15f0 0x7fa53c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.107+0000 7fa549ffb700 1 -- 172.21.15.57:0/624907543 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa538014070 con 0x7fa54c0fff90 2023-10-08T13:52:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa55f364b80 1 Processor -- start 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa55f364b80 1 -- start start 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa550f17700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa53c0a15f0 0x7fa53c0a3ab0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa5400105c0 tx=0x7fa54000b4a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e1880 0x557ccd1e1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e21a0 0x557ccd1dbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 0x557ccd1dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa55f364b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd144400 con 0x557ccd1e21a0 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa55f364b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd135bc0 con 0x557ccd1dc0b0 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa55f364b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd1334e0 con 0x557ccd1e1880 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa54bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e21a0 0x557ccd1dbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa54bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e21a0 0x557ccd1dbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57274/0 (socket says 172.21.15.57:57274) 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa54bfff700 1 -- 172.21.15.57:0/3138506990 learned_addr learned my addr 172.21.15.57:0/3138506990 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa551718700 1 --2- 172.21.15.57:0/3138506990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 0x557ccd1dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa550f17700 1 --2- 172.21.15.57:0/3138506990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e1880 0x557ccd1e1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa54bfff700 1 -- 172.21.15.57:0/3138506990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e1880 msgr2=0x557ccd1e1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa54bfff700 1 --2- 172.21.15.57:0/3138506990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e1880 0x557ccd1e1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa54bfff700 1 -- 172.21.15.57:0/3138506990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 msgr2=0x557ccd1dc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa54bfff700 1 --2- 172.21.15.57:0/3138506990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 0x557ccd1dc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.108+0000 7fa54bfff700 1 -- 172.21.15.57:0/3138506990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cccf59d80 con 0x557ccd1e21a0 2023-10-08T13:52:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.109+0000 7fa54bfff700 1 --2- 172.21.15.57:0/3138506990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e21a0 0x557ccd1dbb70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa5380a20e0 tx=0x7fa538099c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c29c3a0a32e5e7e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.109+0000 7fa536ffd700 1 -- 172.21.15.57:0/3138506990 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53809e3d0 con 0x557ccd1e21a0 2023-10-08T13:52:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.109+0000 7fa536ffd700 1 -- 172.21.15.57:0/3138506990 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5380cc040 con 0x557ccd1e21a0 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.109+0000 7fa536ffd700 1 -- 172.21.15.57:0/3138506990 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53809a740 con 0x557ccd1e21a0 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.109+0000 7fa55f364b80 1 -- 172.21.15.57:0/3138506990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e21a0 msgr2=0x557ccd1dbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.109+0000 7fa55f364b80 1 --2- 172.21.15.57:0/3138506990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e21a0 0x557ccd1dbb70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa5380a20e0 tx=0x7fa538099c60 comp rx=0 tx=0).stop 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 -- 172.21.15.57:0/3138506990 shutdown_connections 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 --2- 172.21.15.57:0/3138506990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 0x557ccd1dc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 --2- 172.21.15.57:0/3138506990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e1880 0x557ccd1e1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 --2- 172.21.15.57:0/3138506990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e21a0 0x557ccd1dbb70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 -- 172.21.15.57:0/3138506990 >> 172.21.15.57:0/3138506990 conn(0x557ccd150200 msgr2=0x557ccd1d6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 -- 172.21.15.57:0/3138506990 shutdown_connections 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 -- 172.21.15.57:0/3138506990 wait complete. 2023-10-08T13:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 -- start start 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 Processor -- start 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1dc0b0 0x557ccd1e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e1880 0x557ccd1e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.110+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 0x557ccd1e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa55f364b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd12edd0 con 0x557ccd1dc0b0 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa55f364b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd0cb6b0 con 0x557ccd1e21a0 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa55f364b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd0cb350 con 0x557ccd1e1880 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa55f364b80 10 client.0 _ll_register_callbacks cb 0x557cccf287a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa550f17700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1dc0b0 0x557ccd1e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa550f17700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1dc0b0 0x557ccd1e54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57278/0 (socket says 172.21.15.57:57278) 2023-10-08T13:52:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa551718700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 0x557ccd1e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa550f17700 1 -- 172.21.15.57:0/2282304601 learned_addr learned my addr 172.21.15.57:0/2282304601 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa54bfff700 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e1880 0x557ccd1e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa551718700 1 -- 172.21.15.57:0/2282304601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e1880 msgr2=0x557ccd1e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa551718700 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e1880 0x557ccd1e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa551718700 1 -- 172.21.15.57:0/2282304601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1dc0b0 msgr2=0x557ccd1e54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa551718700 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1dc0b0 0x557ccd1e54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa551718700 1 -- 172.21.15.57:0/2282304601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c1a5b10 con 0x557ccd1e21a0 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa550f17700 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1dc0b0 0x557ccd1e54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa551718700 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 0x557ccd1e5f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa54400b070 tx=0x7fa544007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67448beded6ceef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa52bfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa52bfff700 1 -- 172.21.15.57:0/2282304601 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54400e040 con 0x557ccd1e21a0 2023-10-08T13:52:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa55f364b80 1 -- 172.21.15.57:0/2282304601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ccd0f6e50 con 0x557ccd1e21a0 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.111+0000 7fa52bfff700 1 -- 172.21.15.57:0/2282304601 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa544004c50 con 0x557ccd1e21a0 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa52bfff700 1 -- 172.21.15.57:0/2282304601 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa544004e10 con 0x557ccd1e21a0 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa55f364b80 10 client.25817 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa55f364b80 1 -- 172.21.15.57:0/2282304601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557cccf42c00 con 0x557ccd1e21a0 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa52bfff700 1 -- 172.21.15.57:0/2282304601 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ccd0f6e50 con 0x557ccd1e21a0 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa52bfff700 1 -- 172.21.15.57:0/2282304601 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa544019070 con 0x557ccd1e21a0 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa55f364b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa55f364b80 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa55f364b80 20 client.25817 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa55f364b80 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ccd1f5c80 0x557ccd1f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa55f364b80 4 client.25817 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa55f364b80 1 -- 172.21.15.57:0/2282304601 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ccd127db0 con 0x557ccd1f5c80 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.112+0000 7fa550f17700 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ccd1f5c80 0x557ccd1f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa52bfff700 10 client.25817 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa550f17700 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ccd1f5c80 0x557ccd1f80a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447457935274604 server_cookie=ae03bffeb8577ac0 in_seq=0 out_seq=0 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa52bfff700 1 -- 172.21.15.57:0/2282304601 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557ccd127db0 con 0x557ccd1f5c80 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa52bfff700 10 client.25817 handle_command_reply: tid=0 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa55f364b80 1 client.25817 shutdown 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa55f364b80 1 -- 172.21.15.57:0/2282304601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 msgr2=0x557ccd1e5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa55f364b80 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 0x557ccd1e5f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa54400b070 tx=0x7fa544007f30 comp rx=0 tx=0).stop 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa55f364b80 1 -- 172.21.15.57:0/2282304601 shutdown_connections 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa55f364b80 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ccd1f5c80 0x557ccd1f80a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa55f364b80 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 0x557ccd1e5f50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa52bfff700 0 client.25817 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa55f364b80 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1dc0b0 0x557ccd1e54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa55f364b80 1 --2- 172.21.15.57:0/2282304601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e1880 0x557ccd1e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.113+0000 7fa55f364b80 1 -- 172.21.15.57:0/2282304601 >> 172.21.15.57:0/2282304601 conn(0x557ccd150200 msgr2=0x557ccd1d6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.114+0000 7fa55f364b80 1 -- 172.21.15.57:0/2282304601 shutdown_connections 2023-10-08T13:52:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.114+0000 7fa55f364b80 1 -- 172.21.15.57:0/2282304601 wait complete. 2023-10-08T13:52:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.114+0000 7fa55f364b80 20 client.25817 trim_cache size 0 max 16384 2023-10-08T13:52:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa55f364b80 1 Processor -- start 2023-10-08T13:52:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa55f364b80 1 -- start start 2023-10-08T13:52:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 0x557ccd1dbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1e4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e21a0 0x557ccd1ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa55f364b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cccfaf8c0 con 0x557ccd1e1880 2023-10-08T13:52:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa55f364b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd1d8360 con 0x557ccd1dc0b0 2023-10-08T13:52:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa55f364b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd1d8100 con 0x557ccd1e21a0 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa54bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa54bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1e4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57288/0 (socket says 172.21.15.57:57288) 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa54bfff700 1 -- 172.21.15.57:0/2403025165 learned_addr learned my addr 172.21.15.57:0/2403025165 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa550f17700 1 --2- 172.21.15.57:0/2403025165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 0x557ccd1dbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.130+0000 7fa551718700 1 --2- 172.21.15.57:0/2403025165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e21a0 0x557ccd1ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.131+0000 7fa550f17700 1 -- 172.21.15.57:0/2403025165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e21a0 msgr2=0x557ccd1ecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.131+0000 7fa550f17700 1 --2- 172.21.15.57:0/2403025165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e21a0 0x557ccd1ecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.131+0000 7fa550f17700 1 -- 172.21.15.57:0/2403025165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 msgr2=0x557ccd1e4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.131+0000 7fa550f17700 1 --2- 172.21.15.57:0/2403025165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1e4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.131+0000 7fa550f17700 1 -- 172.21.15.57:0/2403025165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cccf801f0 con 0x557ccd1dc0b0 2023-10-08T13:52:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.131+0000 7fa550f17700 1 --2- 172.21.15.57:0/2403025165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 0x557ccd1dbb70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x557ccd1f8680 tx=0x7fa540013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79bf3e8f0e4d0776 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.131+0000 7fa5377fe700 1 -- 172.21.15.57:0/2403025165 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54002e070 con 0x557ccd1dc0b0 2023-10-08T13:52:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.131+0000 7fa5377fe700 1 -- 172.21.15.57:0/2403025165 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa540015070 con 0x557ccd1dc0b0 2023-10-08T13:52:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.131+0000 7fa5377fe700 1 -- 172.21.15.57:0/2403025165 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540024040 con 0x557ccd1dc0b0 2023-10-08T13:52:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 -- 172.21.15.57:0/2403025165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 msgr2=0x557ccd1dbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 --2- 172.21.15.57:0/2403025165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 0x557ccd1dbb70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x557ccd1f8680 tx=0x7fa540013000 comp rx=0 tx=0).stop 2023-10-08T13:52:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 -- 172.21.15.57:0/2403025165 shutdown_connections 2023-10-08T13:52:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 --2- 172.21.15.57:0/2403025165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1e21a0 0x557ccd1ecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 --2- 172.21.15.57:0/2403025165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1dc0b0 0x557ccd1dbb70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 --2- 172.21.15.57:0/2403025165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1e4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 -- 172.21.15.57:0/2403025165 >> 172.21.15.57:0/2403025165 conn(0x557ccd150200 msgr2=0x557ccd1e05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 -- 172.21.15.57:0/2403025165 shutdown_connections 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 -- 172.21.15.57:0/2403025165 wait complete. 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 -- start start 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.132+0000 7fa55f364b80 1 Processor -- start 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1dc0b0 0x557ccd1de6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1debe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa55f364b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 0x557ccd1dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa55f364b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd1ddb10 con 0x557ccd1e1880 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa55f364b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd1ddc80 con 0x557ccd1e21a0 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa55f364b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ccd1dde40 con 0x557ccd1dc0b0 2023-10-08T13:52:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa55f364b80 10 client.0 _ll_register_callbacks cb 0x7fa544009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa54bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1debe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa54bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1debe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57294/0 (socket says 172.21.15.57:57294) 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa54bfff700 1 -- 172.21.15.57:0/1569680290 learned_addr learned my addr 172.21.15.57:0/1569680290 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa551718700 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 0x557ccd1dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa550f17700 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1dc0b0 0x557ccd1de6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa54bfff700 1 -- 172.21.15.57:0/1569680290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1dc0b0 msgr2=0x557ccd1de6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa54bfff700 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1dc0b0 0x557ccd1de6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa54bfff700 1 -- 172.21.15.57:0/1569680290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 msgr2=0x557ccd1dd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa54bfff700 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 0x557ccd1dd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa54bfff700 1 -- 172.21.15.57:0/1569680290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa540011710 con 0x557ccd1e1880 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa551718700 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 0x557ccd1dd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa54bfff700 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1debe0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa5380985d0 tx=0x7fa5380cd580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf68a0addeb4c5b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.133+0000 7fa52bfff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa52bfff700 1 -- 172.21.15.57:0/1569680290 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538099970 con 0x557ccd1e1880 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa55f364b80 1 -- 172.21.15.57:0/1569680290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5380cc040 con 0x557ccd1e1880 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa52bfff700 1 -- 172.21.15.57:0/1569680290 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa5380a2610 con 0x557ccd1e1880 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa52bfff700 1 -- 172.21.15.57:0/1569680290 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa53809a4e0 con 0x557ccd1e1880 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa55f364b80 10 client.16251 fetch_fsmap learned FSMap version 27 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa55f364b80 1 -- 172.21.15.57:0/1569680290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557cccf58250 con 0x557ccd1e1880 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa52bfff700 1 -- 172.21.15.57:0/1569680290 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5380cc040 con 0x557ccd1e1880 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa52bfff700 1 -- 172.21.15.57:0/1569680290 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa538096070 con 0x557ccd1e1880 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa55f364b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 27 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa55f364b80 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa55f364b80 20 client.16251 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa55f364b80 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ccd24a9e0 0x557ccd24ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa55f364b80 4 client.16251 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:52:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa55f364b80 1 -- 172.21.15.57:0/1569680290 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557ccd127ad0 con 0x557ccd24a9e0 2023-10-08T13:52:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.134+0000 7fa550f17700 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ccd24a9e0 0x557ccd24ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.135+0000 7fa52bfff700 10 client.16251 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.135+0000 7fa550f17700 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ccd24a9e0 0x557ccd24ce00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7266e0f5b712c29 server_cookie=fb7574724a8e14ab in_seq=0 out_seq=0 2023-10-08T13:52:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:37 smithi115 ceph-mon[104408]: pgmap v331: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2023-10-08T13:52:38.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:37 smithi115 ceph-mon[104408]: scrub queued for path: / 2023-10-08T13:52:38.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:37 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:38.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:37 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:52:38.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:37 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa52bfff700 1 -- 172.21.15.57:0/1569680290 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557ccd127ad0 con 0x557ccd24a9e0 2023-10-08T13:52:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa52bfff700 10 client.16251 handle_command_reply: tid=0 2023-10-08T13:52:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 client.16251 shutdown 2023-10-08T13:52:38.216 INFO:tasks.workunit.client.0.smithi057.stdout:checking fcntl.h usability... yes 2023-10-08T13:52:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 -- 172.21.15.57:0/1569680290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 msgr2=0x557ccd1debe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1debe0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa5380985d0 tx=0x7fa5380cd580 comp rx=0 tx=0).stop 2023-10-08T13:52:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 -- 172.21.15.57:0/1569680290 shutdown_connections 2023-10-08T13:52:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ccd24a9e0 0x557ccd24ce00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ccd1e21a0 0x557ccd1dd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ccd1dc0b0 0x557ccd1de6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa52bfff700 0 client.16251 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 --2- 172.21.15.57:0/1569680290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ccd1e1880 0x557ccd1debe0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 -- 172.21.15.57:0/1569680290 >> 172.21.15.57:0/1569680290 conn(0x557ccd150200 msgr2=0x557ccd1e0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 -- 172.21.15.57:0/1569680290 shutdown_connections 2023-10-08T13:52:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 1 -- 172.21.15.57:0/1569680290 wait complete. 2023-10-08T13:52:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.215+0000 7fa55f364b80 20 client.16251 trim_cache size 0 max 16384 2023-10-08T13:52:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:52:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:52:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:52:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:52:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 -- 172.21.15.57:0/624907543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa53c0a15f0 msgr2=0x7fa53c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa53c0a15f0 0x7fa53c0a3ab0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa5400105c0 tx=0x7fa54000b4a0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 -- 172.21.15.57:0/624907543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa54c0fff90 msgr2=0x7fa54c101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa54c0fff90 0x7fa54c101760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa53800e9f0 tx=0x7fa53800ed00 comp rx=0 tx=0).stop 2023-10-08T13:52:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 -- 172.21.15.57:0/624907543 shutdown_connections 2023-10-08T13:52:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa54c1045e0 0x7fa54c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa54c0ff5d0 0x7fa54c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa53c0a15f0 0x7fa53c0a3ab0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 --2- 172.21.15.57:0/624907543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa54c0fff90 0x7fa54c101760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 -- 172.21.15.57:0/624907543 >> 172.21.15.57:0/624907543 conn(0x7fa54c076790 msgr2=0x7fa54c067e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 -- 172.21.15.57:0/624907543 shutdown_connections 2023-10-08T13:52:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:38.216+0000 7fa55317b700 1 -- 172.21.15.57:0/624907543 wait complete. 2023-10-08T13:52:38.233 INFO:tasks.workunit.client.0.smithi057.stdout:checking fcntl.h presence... yes 2023-10-08T13:52:38.234 INFO:tasks.workunit.client.0.smithi057.stdout:checking for fcntl.h... yes 2023-10-08T13:52:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:37 smithi057 ceph-mon[99476]: pgmap v331: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2023-10-08T13:52:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:37 smithi057 ceph-mon[99476]: scrub queued for path: / 2023-10-08T13:52:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:37 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:37 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:52:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:37 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:37 smithi099 ceph-mon[103849]: pgmap v331: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2023-10-08T13:52:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:37 smithi099 ceph-mon[103849]: scrub queued for path: / 2023-10-08T13:52:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:37 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:37 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:52:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:37 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:52:38.285 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/fcntl.h usability... yes 2023-10-08T13:52:38.303 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/fcntl.h presence... yes 2023-10-08T13:52:38.303 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/fcntl.h... yes 2023-10-08T13:52:38.407 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/param.h usability... yes 2023-10-08T13:52:38.427 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/param.h presence... yes 2023-10-08T13:52:38.428 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/param.h... yes 2023-10-08T13:52:38.474 INFO:tasks.workunit.client.0.smithi057.stdout:checking ioctl.h usability... no 2023-10-08T13:52:38.492 INFO:tasks.workunit.client.0.smithi057.stdout:checking ioctl.h presence... no 2023-10-08T13:52:38.492 INFO:tasks.workunit.client.0.smithi057.stdout:checking for ioctl.h... no 2023-10-08T13:52:38.540 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/ioctl.h usability... yes 2023-10-08T13:52:38.557 INFO:tasks.workunit.client.0.smithi057.stdout:checking sys/ioctl.h presence... yes 2023-10-08T13:52:38.558 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sys/ioctl.h... yes 2023-10-08T13:52:38.607 INFO:tasks.workunit.client.0.smithi057.stdout:checking return type of signal handlers... void 2023-10-08T13:52:38.652 INFO:tasks.workunit.client.0.smithi057.stdout:checking for size_t... yes 2023-10-08T13:52:38.689 INFO:tasks.workunit.client.0.smithi057.stdout:checking for dev_t... yes 2023-10-08T13:52:38.731 INFO:tasks.workunit.client.0.smithi057.stdout:checking for short... yes 2023-10-08T13:52:38.808 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of short... 2 2023-10-08T13:52:38.852 INFO:tasks.workunit.client.0.smithi057.stdout:checking for int... yes 2023-10-08T13:52:38.915 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of int... 4 2023-10-08T13:52:38.960 INFO:tasks.workunit.client.0.smithi057.stdout:checking for long... yes 2023-10-08T13:52:39.028 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of long... 8 2023-10-08T13:52:39.072 INFO:tasks.workunit.client.0.smithi057.stdout:checking for long long... yes 2023-10-08T13:52:39.141 INFO:tasks.workunit.client.0.smithi057.stdout:checking size of long long... 8 2023-10-08T13:52:39.189 INFO:tasks.workunit.client.0.smithi057.stdout:checking for ssize_t... yes 2023-10-08T13:52:39.243 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sig_atomic_t... yes 2023-10-08T13:52:39.287 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether gcc needs -traditional... no 2023-10-08T13:52:39.321 INFO:tasks.workunit.client.0.smithi057.stdout:checking for an ANSI C-conforming const... yes 2023-10-08T13:52:39.344 INFO:tasks.workunit.client.0.smithi057.stdout:checking for inline... inline 2023-10-08T13:52:39.401 INFO:tasks.workunit.client.0.smithi057.stdout:checking for working alloca.h... yes 2023-10-08T13:52:39.447 INFO:tasks.workunit.client.0.smithi057.stdout:checking for alloca... yes 2023-10-08T13:52:39.512 INFO:tasks.workunit.client.0.smithi057.stdout:checking for working memcmp... yes 2023-10-08T13:52:39.566 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strftime... yes 2023-10-08T13:52:39.636 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2023-10-08T13:52:39.694 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether stat accepts an empty string... no 2023-10-08T13:52:39.746 INFO:tasks.workunit.client.0.smithi057.stdout:checking for vprintf... yes 2023-10-08T13:52:39.802 INFO:tasks.workunit.client.0.smithi057.stdout:checking for _doprnt... no 2023-10-08T13:52:39.825 INFO:tasks.workunit.client.0.smithi057.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-10-08T13:52:39.838 INFO:tasks.workunit.client.0.smithi057.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-10-08T13:52:39.886 INFO:tasks.workunit.client.0.smithi057.stdout:checking for the pthreads library -lpthreads... no 2023-10-08T13:52:39.936 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether pthreads work without any flags... no 2023-10-08T13:52:39.952 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether pthreads work with -Kthread... no 2023-10-08T13:52:39.967 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether pthreads work with -kthread... no 2023-10-08T13:52:40.008 INFO:tasks.workunit.client.0.smithi057.stdout:checking for the pthreads library -llthread... no 2023-10-08T13:52:40.059 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether pthreads work with -pthread... yes 2023-10-08T13:52:40.104 INFO:tasks.workunit.client.0.smithi057.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2023-10-08T13:52:40.105 INFO:tasks.workunit.client.0.smithi057.stdout:checking if more special flags are required for pthreads... no 2023-10-08T13:52:40.105 INFO:tasks.workunit.client.0.smithi057.stdout:checking for cc_r... gcc 2023-10-08T13:52:40.184 INFO:tasks.workunit.client.0.smithi057.stdout:checking for setrlimit... yes 2023-10-08T13:52:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:39 smithi115 ceph-mon[104408]: pgmap v332: 129 pgs: 129 active+clean; 1.7 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 12 op/s 2023-10-08T13:52:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:39 smithi115 ceph-mon[104408]: scrub summary: idle 2023-10-08T13:52:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:39 smithi057 ceph-mon[99476]: pgmap v332: 129 pgs: 129 active+clean; 1.7 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 12 op/s 2023-10-08T13:52:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:39 smithi057 ceph-mon[99476]: scrub summary: idle 2023-10-08T13:52:40.242 INFO:tasks.workunit.client.0.smithi057.stdout:checking for setproctitle... no 2023-10-08T13:52:40.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:39 smithi099 ceph-mon[103849]: pgmap v332: 129 pgs: 129 active+clean; 1.7 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 12 op/s 2023-10-08T13:52:40.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:39 smithi099 ceph-mon[103849]: scrub summary: idle 2023-10-08T13:52:40.299 INFO:tasks.workunit.client.0.smithi057.stdout:checking for getopt_long... yes 2023-10-08T13:52:40.358 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strtoull... yes 2023-10-08T13:52:40.416 INFO:tasks.workunit.client.0.smithi057.stdout:checking for strtoq... yes 2023-10-08T13:52:40.474 INFO:tasks.workunit.client.0.smithi057.stdout:checking for snprintf... yes 2023-10-08T13:52:40.533 INFO:tasks.workunit.client.0.smithi057.stdout:checking for vsnprintf... yes 2023-10-08T13:52:40.592 INFO:tasks.workunit.client.0.smithi057.stdout:checking for vfprintf... yes 2023-10-08T13:52:40.651 INFO:tasks.workunit.client.0.smithi057.stdout:checking for gethostname... yes 2023-10-08T13:52:40.835 INFO:tasks.workunit.client.0.smithi057.stdout:checking for setlocale... yes 2023-10-08T13:52:40.948 INFO:tasks.workunit.client.0.smithi057.stdout:checking for sync... yes 2023-10-08T13:52:41.012 INFO:tasks.workunit.client.0.smithi057.stdout:checking whether snprintf is C99 conformant... done 2023-10-08T13:52:41.023 INFO:tasks.workunit.client.0.smithi057.stdout:configure: creating ./config.status 2023-10-08T13:52:41.185 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating Makefile 2023-10-08T13:52:41.202 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating src/Makefile 2023-10-08T13:52:41.232 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating man/Makefile 2023-10-08T13:52:41.259 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: creating config.h 2023-10-08T13:52:41.311 INFO:tasks.workunit.client.0.smithi057.stdout:config.status: executing depfiles commands 2023-10-08T13:52:41.440 INFO:tasks.workunit.client.0.smithi057.stderr:+ make 2023-10-08T13:52:41.449 INFO:tasks.workunit.client.0.smithi057.stdout:make all-recursive 2023-10-08T13:52:41.450 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-08T13:52:41.453 INFO:tasks.workunit.client.0.smithi057.stdout:Making all in src 2023-10-08T13:52:41.460 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-10-08T13:52:41.460 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2023-10-08T13:52:41.460 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2023-10-08T13:52:41.542 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2023-10-08T13:52:41.542 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2023-10-08T13:52:41.606 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2023-10-08T13:52:41.606 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2023-10-08T13:52:41.657 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2023-10-08T13:52:41.657 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2023-10-08T13:52:41.707 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2023-10-08T13:52:41.707 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2023-10-08T13:52:41.756 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2023-10-08T13:52:41.756 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2023-10-08T13:52:41.802 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2023-10-08T13:52:41.802 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2023-10-08T13:52:41.874 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2023-10-08T13:52:41.874 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2023-10-08T13:52:41.907 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2023-10-08T13:52:41.907 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2023-10-08T13:52:41.941 INFO:tasks.workunit.client.0.smithi057.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2023-10-08T13:52:41.941 INFO:tasks.workunit.client.0.smithi057.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2023-10-08T13:52:41.980 INFO:tasks.workunit.client.0.smithi057.stdout:gcc -g -O2 -pthread -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2023-10-08T13:52:42.000 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-10-08T13:52:42.000 INFO:tasks.workunit.client.0.smithi057.stdout:Making all in man 2023-10-08T13:52:42.021 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-10-08T13:52:42.021 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Nothing to be done for 'all'. 2023-10-08T13:52:42.022 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-10-08T13:52:42.023 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-08T13:52:42.024 INFO:tasks.workunit.client.0.smithi057.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-08T13:52:42.024 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-08T13:52:42.025 INFO:tasks.workunit.client.0.smithi057.stderr:+ cd src 2023-10-08T13:52:42.025 INFO:tasks.workunit.client.0.smithi057.stderr:+ mkdir blogtest_in 2023-10-08T13:52:42.026 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'running blogbench' 2023-10-08T13:52:42.027 INFO:tasks.workunit.client.0.smithi057.stdout:running blogbench 2023-10-08T13:52:42.027 INFO:tasks.workunit.client.0.smithi057.stderr:+ ./blogbench -d blogtest_in 2023-10-08T13:52:42.027 INFO:tasks.workunit.client.0.smithi057.stdout: 2023-10-08T13:52:42.027 INFO:tasks.workunit.client.0.smithi057.stdout:Frequency = 10 secs 2023-10-08T13:52:42.027 INFO:tasks.workunit.client.0.smithi057.stdout:Scratch dir = [blogtest_in] 2023-10-08T13:52:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:41 smithi115 ceph-mon[104408]: pgmap v333: 129 pgs: 129 active+clean; 1.7 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 48 op/s 2023-10-08T13:52:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:41 smithi057 ceph-mon[99476]: pgmap v333: 129 pgs: 129 active+clean; 1.7 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 48 op/s 2023-10-08T13:52:42.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:41 smithi099 ceph-mon[103849]: pgmap v333: 129 pgs: 129 active+clean; 1.7 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 41 KiB/s wr, 48 op/s 2023-10-08T13:52:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:43 smithi115 ceph-mon[104408]: pgmap v334: 129 pgs: 129 active+clean; 2.0 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 65 op/s 2023-10-08T13:52:44.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:43 smithi057 ceph-mon[99476]: pgmap v334: 129 pgs: 129 active+clean; 2.0 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 65 op/s 2023-10-08T13:52:44.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:43 smithi099 ceph-mon[103849]: pgmap v334: 129 pgs: 129 active+clean; 2.0 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 65 op/s 2023-10-08T13:52:44.362 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 3 writers... 2023-10-08T13:52:44.362 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 1 rewriters... 2023-10-08T13:52:44.362 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 5 commenters... 2023-10-08T13:52:44.362 INFO:tasks.workunit.client.0.smithi057.stdout:Spawning 100 readers... 2023-10-08T13:52:44.363 INFO:tasks.workunit.client.0.smithi057.stdout:Benchmarking for 30 iterations. 2023-10-08T13:52:44.364 INFO:tasks.workunit.client.0.smithi057.stdout:The test will run during 5 minutes. 2023-10-08T13:52:44.364 INFO:tasks.workunit.client.0.smithi057.stdout: 2023-10-08T13:52:44.364 INFO:tasks.workunit.client.0.smithi057.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2023-10-08T13:52:46.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:45 smithi115 ceph-mon[104408]: pgmap v335: 129 pgs: 129 active+clean; 5.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 392 KiB/s wr, 94 op/s 2023-10-08T13:52:46.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:45 smithi057 ceph-mon[99476]: pgmap v335: 129 pgs: 129 active+clean; 5.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 392 KiB/s wr, 94 op/s 2023-10-08T13:52:46.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:45 smithi099 ceph-mon[103849]: pgmap v335: 129 pgs: 129 active+clean; 5.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 392 KiB/s wr, 94 op/s 2023-10-08T13:52:48.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:47 smithi115 ceph-mon[104408]: pgmap v336: 129 pgs: 129 active+clean; 5.9 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 468 KiB/s wr, 116 op/s 2023-10-08T13:52:48.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:47 smithi057 ceph-mon[99476]: pgmap v336: 129 pgs: 129 active+clean; 5.9 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 468 KiB/s wr, 116 op/s 2023-10-08T13:52:48.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:47 smithi099 ceph-mon[103849]: pgmap v336: 129 pgs: 129 active+clean; 5.9 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 468 KiB/s wr, 116 op/s 2023-10-08T13:52:50.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:49 smithi115 ceph-mon[104408]: pgmap v337: 129 pgs: 129 active+clean; 6.8 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 570 KiB/s wr, 122 op/s 2023-10-08T13:52:50.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:49 smithi057 ceph-mon[99476]: pgmap v337: 129 pgs: 129 active+clean; 6.8 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 570 KiB/s wr, 122 op/s 2023-10-08T13:52:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:49 smithi099 ceph-mon[103849]: pgmap v337: 129 pgs: 129 active+clean; 6.8 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 570 KiB/s wr, 122 op/s 2023-10-08T13:52:51.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:50 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:52:51.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:50 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:52:51.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:50 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:52:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.333+0000 7f4a60d56700 1 -- 172.21.15.57:0/1424972172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a5c106730 msgr2=0x7f4a5c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.333+0000 7f4a60d56700 1 --2- 172.21.15.57:0/1424972172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a5c106730 0x7f4a5c106b10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009990 tx=0x7f4a4c009ca0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.333+0000 7f4a60d56700 1 -- 172.21.15.57:0/1424972172 shutdown_connections 2023-10-08T13:52:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.333+0000 7f4a60d56700 1 --2- 172.21.15.57:0/1424972172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a5c069350 0x7f4a5c10d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.333+0000 7f4a60d56700 1 --2- 172.21.15.57:0/1424972172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a5c106730 0x7f4a5c106b10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.333+0000 7f4a60d56700 1 --2- 172.21.15.57:0/1424972172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c068990 0x7f4a5c068e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.333+0000 7f4a60d56700 1 -- 172.21.15.57:0/1424972172 >> 172.21.15.57:0/1424972172 conn(0x7f4a5c076790 msgr2=0x7f4a5c076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.333+0000 7f4a60d56700 1 -- 172.21.15.57:0/1424972172 shutdown_connections 2023-10-08T13:52:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.333+0000 7f4a60d56700 1 -- 172.21.15.57:0/1424972172 wait complete. 2023-10-08T13:52:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a60d56700 1 Processor -- start 2023-10-08T13:52:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a60d56700 1 -- start start 2023-10-08T13:52:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a60d56700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a5c068990 0x7f4a5c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a60d56700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c069350 0x7f4a5c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a60d56700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a5c106730 0x7f4a5c1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a60d56700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c197010 con 0x7f4a5c068990 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a60d56700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c197180 con 0x7f4a5c069350 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a60d56700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c197340 con 0x7f4a5c106730 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a5a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a5c068990 0x7f4a5c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a5a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a5c068990 0x7f4a5c19c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57326/0 (socket says 172.21.15.57:57326) 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a5a59c700 1 -- 172.21.15.57:0/3407977281 learned_addr learned my addr 172.21.15.57:0/3407977281 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a51d9b700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c069350 0x7f4a5c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.334+0000 7f4a5ad9d700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a5c106730 0x7f4a5c1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a5a59c700 1 -- 172.21.15.57:0/3407977281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a5c106730 msgr2=0x7f4a5c1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a5c106730 0x7f4a5c1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a5a59c700 1 -- 172.21.15.57:0/3407977281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c069350 msgr2=0x7f4a5c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c069350 0x7f4a5c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a5a59c700 1 -- 172.21.15.57:0/3407977281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0095f0 con 0x7f4a5c068990 2023-10-08T13:52:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a51d9b700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c069350 0x7f4a5c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a5c068990 0x7f4a5c19c880 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000c00 tx=0x7f4a4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a53fff700 1 -- 172.21.15.57:0/3407977281 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c01d070 con 0x7f4a5c068990 2023-10-08T13:52:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a60d56700 1 -- 172.21.15.57:0/3407977281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c1975c0 con 0x7f4a5c068990 2023-10-08T13:52:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a53fff700 1 -- 172.21.15.57:0/3407977281 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c022470 con 0x7f4a5c068990 2023-10-08T13:52:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a53fff700 1 -- 172.21.15.57:0/3407977281 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c017650 con 0x7f4a5c068990 2023-10-08T13:52:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.335+0000 7f4a60d56700 1 -- 172.21.15.57:0/3407977281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c1a3870 con 0x7f4a5c068990 2023-10-08T13:52:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.337+0000 7f4a53fff700 1 -- 172.21.15.57:0/3407977281 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4a4c0225e0 con 0x7f4a5c068990 2023-10-08T13:52:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.338+0000 7f4a53fff700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4a480a15f0 0x7f4a480a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.338+0000 7f4a51d9b700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4a480a15f0 0x7f4a480a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.338+0000 7f4a53fff700 1 -- 172.21.15.57:0/3407977281 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4a4c0ce600 con 0x7f4a5c068990 2023-10-08T13:52:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.339+0000 7f4a51d9b700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4a480a15f0 0x7f4a480a3ab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c197b70 tx=0x7f4a44009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a6cf3fb80 1 Processor -- start 2023-10-08T13:52:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a6cf3fb80 1 -- start start 2023-10-08T13:52:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc313dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3137b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 0x558cc3138510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc30a0400 con 0x558cc313e1a0 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc3091bc0 con 0x558cc31380b0 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc308f4e0 con 0x558cc313d880 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a51d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3137b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a5a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc313dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a51d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3137b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57336/0 (socket says 172.21.15.57:57336) 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a5a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc313dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58146/0 (socket says 172.21.15.57:58146) 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a51d9b700 1 -- 172.21.15.57:0/880357436 learned_addr learned my addr 172.21.15.57:0/880357436 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.340+0000 7f4a5ad9d700 1 --2- 172.21.15.57:0/880357436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 0x558cc3138510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.341+0000 7f4a5ad9d700 1 -- 172.21.15.57:0/880357436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 msgr2=0x558cc313dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.341+0000 7f4a5ad9d700 1 --2- 172.21.15.57:0/880357436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc313dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.341+0000 7f4a5ad9d700 1 -- 172.21.15.57:0/880357436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 msgr2=0x558cc3137b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.341+0000 7f4a5ad9d700 1 --2- 172.21.15.57:0/880357436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3137b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.341+0000 7f4a5ad9d700 1 -- 172.21.15.57:0/880357436 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cc2eb5d80 con 0x558cc31380b0 2023-10-08T13:52:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.341+0000 7f4a51d9b700 1 --2- 172.21.15.57:0/880357436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3137b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.341+0000 7f4a5ad9d700 1 --2- 172.21.15.57:0/880357436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 0x558cc3138510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a54011580 tx=0x7f4a54009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3175983b791310d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.341+0000 7f4a50d99700 1 -- 172.21.15.57:0/880357436 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400e040 con 0x558cc31380b0 2023-10-08T13:52:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.341+0000 7f4a50d99700 1 -- 172.21.15.57:0/880357436 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a5400f040 con 0x558cc31380b0 2023-10-08T13:52:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.341+0000 7f4a50d99700 1 -- 172.21.15.57:0/880357436 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a540044f0 con 0x558cc31380b0 2023-10-08T13:52:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.342+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/880357436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 msgr2=0x558cc3138510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.342+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/880357436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 0x558cc3138510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a54011580 tx=0x7f4a54009920 comp rx=0 tx=0).stop 2023-10-08T13:52:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.342+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/880357436 shutdown_connections 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.342+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/880357436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 0x558cc3138510 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.342+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/880357436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc313dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.342+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/880357436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3137b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.342+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/880357436 >> 172.21.15.57:0/880357436 conn(0x558cc30ac200 msgr2=0x558cc3132b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.342+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/880357436 shutdown_connections 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.342+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/880357436 wait complete. 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.342+0000 7f4a6cf3fb80 1 -- start start 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a6cf3fb80 1 Processor -- start 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 0x558cc3144170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc31414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3141c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc308add0 con 0x558cc313e1a0 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc30276b0 con 0x558cc31380b0 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc3027350 con 0x558cc313d880 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a6cf3fb80 10 client.0 _ll_register_callbacks cb 0x558cc2e847a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a5ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3141c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a51d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc31414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a51d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc31414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58154/0 (socket says 172.21.15.57:58154) 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a5ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3141c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57344/0 (socket says 172.21.15.57:57344) 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a51d9b700 1 -- 172.21.15.57:0/2834609748 learned_addr learned my addr 172.21.15.57:0/2834609748 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 0x558cc3144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a5ad9d700 1 -- 172.21.15.57:0/2834609748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 msgr2=0x558cc31414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a5ad9d700 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc31414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a5ad9d700 1 -- 172.21.15.57:0/2834609748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 msgr2=0x558cc3144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a5ad9d700 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 0x558cc3144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a5ad9d700 1 -- 172.21.15.57:0/2834609748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a44017070 con 0x558cc313e1a0 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.343+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 0x558cc3144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a5ad9d700 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3141c10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a54000f80 tx=0x7f4a54011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce275085b59df4c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2834609748 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400e040 con 0x558cc313e1a0 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2834609748 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4a54003d00 con 0x558cc313e1a0 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2834609748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cc3052e50 con 0x558cc313e1a0 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2834609748 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a54003ec0 con 0x558cc313e1a0 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a6cf3fb80 10 client.16263 fetch_fsmap learned FSMap version 28 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2834609748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f4a54011230 con 0x558cc313e1a0 2023-10-08T13:52:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2834609748 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558cc3052e50 con 0x558cc313e1a0 2023-10-08T13:52:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2834609748 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4a5401b020 con 0x558cc313e1a0 2023-10-08T13:52:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a6cf3fb80 10 client.16263 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T13:52:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a6cf3fb80 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a6cf3fb80 20 client.16263 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558cc3151ca0 0x558cc31540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a6cf3fb80 4 client.16263 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:52:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.344+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2834609748 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cc3083db0 con 0x558cc3151ca0 2023-10-08T13:52:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.345+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558cc3151ca0 0x558cc31540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.345+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558cc3151ca0 0x558cc31540c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e3f74b8772cd08 server_cookie=5099468dbb6a26ac in_seq=0 out_seq=0 2023-10-08T13:52:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.345+0000 7f4a41ffb700 10 client.16263 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2834609748 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558cc3083db0 con 0x558cc3151ca0 2023-10-08T13:52:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a41ffb700 10 client.16263 handle_command_reply: tid=0 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 client.16263 shutdown 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2834609748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 msgr2=0x558cc3141c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3141c10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a54000f80 tx=0x7f4a54011f00 comp rx=0 tx=0).stop 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2834609748 shutdown_connections 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558cc3151ca0 0x558cc31540c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313e1a0 0x558cc3141c10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a41ffb700 0 client.16263 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc31380b0 0x558cc3144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2834609748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc31414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2834609748 >> 172.21.15.57:0/2834609748 conn(0x558cc30ac200 msgr2=0x558cc3132010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2834609748 shutdown_connections 2023-10-08T13:52:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2834609748 wait complete. 2023-10-08T13:52:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.353+0000 7f4a6cf3fb80 20 client.16263 trim_cache size 0 max 16384 2023-10-08T13:52:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.370+0000 7f4a6cf3fb80 1 Processor -- start 2023-10-08T13:52:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.370+0000 7f4a6cf3fb80 1 -- start start 2023-10-08T13:52:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.370+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc31380b0 0x558cc3137b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.370+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313d880 0x558cc3138490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.370+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 0x558cc3132790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.370+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc2f0b8c0 con 0x558cc313d880 2023-10-08T13:52:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.370+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc313dfa0 con 0x558cc313e1a0 2023-10-08T13:52:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.370+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc313fa20 con 0x558cc31380b0 2023-10-08T13:52:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.371+0000 7f4a5a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc31380b0 0x558cc3137b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.371+0000 7f4a5ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 0x558cc3132790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.371+0000 7f4a5ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 0x558cc3132790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:58776/0 (socket says 172.21.15.57:58776) 2023-10-08T13:52:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.371+0000 7f4a5ad9d700 1 -- 172.21.15.57:0/3981852383 learned_addr learned my addr 172.21.15.57:0/3981852383 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.371+0000 7f4a5a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc31380b0 0x558cc3137b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58160/0 (socket says 172.21.15.57:58160) 2023-10-08T13:52:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.371+0000 7f4a5a59c700 1 -- 172.21.15.57:0/3981852383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 msgr2=0x558cc3132790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.371+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/3981852383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 0x558cc3132790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.371+0000 7f4a5a59c700 1 -- 172.21.15.57:0/3981852383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313d880 msgr2=0x558cc3138490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:52:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.371+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/3981852383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313d880 0x558cc3138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.371+0000 7f4a5a59c700 1 -- 172.21.15.57:0/3981852383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cc2e9ec00 con 0x558cc31380b0 2023-10-08T13:52:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.372+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/3981852383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc31380b0 0x558cc3137b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0cb9f0 tx=0x7f4a4c022c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d464d86e7fea85cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.373+0000 7f4a5159a700 1 -- 172.21.15.57:0/3981852383 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c09b310 con 0x558cc31380b0 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.373+0000 7f4a5159a700 1 -- 172.21.15.57:0/3981852383 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c0cd040 con 0x558cc31380b0 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.373+0000 7f4a5159a700 1 -- 172.21.15.57:0/3981852383 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c09b570 con 0x558cc31380b0 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.374+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/3981852383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc31380b0 msgr2=0x558cc3137b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.374+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/3981852383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc31380b0 0x558cc3137b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0cb9f0 tx=0x7f4a4c022c80 comp rx=0 tx=0).stop 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.374+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/3981852383 shutdown_connections 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.374+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/3981852383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 0x558cc3132790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.374+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/3981852383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc31380b0 0x558cc3137b70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.374+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/3981852383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc313d880 0x558cc3138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.374+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/3981852383 >> 172.21.15.57:0/3981852383 conn(0x558cc30ac200 msgr2=0x558cc31589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.374+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/3981852383 shutdown_connections 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.374+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/3981852383 wait complete. 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.374+0000 7f4a6cf3fb80 1 -- start start 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a6cf3fb80 1 Processor -- start 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc31380b0 0x558cc3144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc3145130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a6cf3fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 0x558cc3145670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc3148f70 con 0x558cc31380b0 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc31490e0 con 0x558cc313e1a0 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a6cf3fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc31ad290 con 0x558cc313d880 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a6cf3fb80 10 client.0 _ll_register_callbacks cb 0x7f4a44004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a5a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc31380b0 0x558cc3144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a5a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc31380b0 0x558cc3144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57354/0 (socket says 172.21.15.57:57354) 2023-10-08T13:52:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a5a59c700 1 -- 172.21.15.57:0/2697548522 learned_addr learned my addr 172.21.15.57:0/2697548522 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:52:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a5ad9d700 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 0x558cc3145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.375+0000 7f4a51d9b700 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc3145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.376+0000 7f4a51d9b700 1 -- 172.21.15.57:0/2697548522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 msgr2=0x558cc3145670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.376+0000 7f4a51d9b700 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 0x558cc3145670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.376+0000 7f4a51d9b700 1 -- 172.21.15.57:0/2697548522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc31380b0 msgr2=0x558cc3144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:52:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.376+0000 7f4a51d9b700 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc31380b0 0x558cc3144bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.376+0000 7f4a51d9b700 1 -- 172.21.15.57:0/2697548522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c1a3870 con 0x558cc313d880 2023-10-08T13:52:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.377+0000 7f4a51d9b700 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc3145130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x558cc313d660 tx=0x7f4a44004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3047cfe0b45e6d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:52:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.377+0000 7f4a41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:52:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2697548522 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44024070 con 0x558cc313d880 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2697548522 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4a44003e30 con 0x558cc313d880 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2697548522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a5400f040 con 0x558cc313d880 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2697548522 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a4401f8f0 con 0x558cc313d880 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a6cf3fb80 10 client.25864 fetch_fsmap learned FSMap version 28 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2697548522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558cc2edc1f0 con 0x558cc313d880 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2697548522 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a5400f040 con 0x558cc313d880 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2697548522 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4a44028ba0 con 0x558cc313d880 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a6cf3fb80 10 client.25864 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a6cf3fb80 10 client.25864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a6cf3fb80 20 client.25864 populate_metadata read hostname 'smithi057' 2023-10-08T13:52:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558cc31aeaf0 0x558cc31b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:52:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a6cf3fb80 4 client.25864 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:52:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.378+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2697548522 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558cc3083ad0 con 0x558cc31aeaf0 2023-10-08T13:52:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.379+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558cc31aeaf0 0x558cc31b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:52:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.379+0000 7f4a41ffb700 10 client.25864 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.379+0000 7f4a5a59c700 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558cc31aeaf0 0x558cc31b0f10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=842c20834966c1fc server_cookie=4ad05686bc8129b7 in_seq=0 out_seq=0 2023-10-08T13:52:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.888+0000 7f4a41ffb700 1 -- 172.21.15.57:0/2697548522 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558cc3083ad0 con 0x558cc31aeaf0 2023-10-08T13:52:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.888+0000 7f4a41ffb700 10 client.25864 handle_command_reply: tid=0 2023-10-08T13:52:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.888+0000 7f4a6cf3fb80 1 client.25864 shutdown 2023-10-08T13:52:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2697548522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 msgr2=0x558cc3145130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc3145130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x558cc313d660 tx=0x7f4a44004750 comp rx=0 tx=0).stop 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2697548522 shutdown_connections 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558cc31aeaf0 0x558cc31b0f10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558cc313e1a0 0x558cc3145670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a41ffb700 0 client.25864 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cc31380b0 0x558cc3144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 1 --2- 172.21.15.57:0/2697548522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cc313d880 0x558cc3145130 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2697548522 >> 172.21.15.57:0/2697548522 conn(0x558cc30ac200 msgr2=0x558cc3138490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2697548522 shutdown_connections 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 1 -- 172.21.15.57:0/2697548522 wait complete. 2023-10-08T13:52:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a6cf3fb80 20 client.25864 trim_cache size 0 max 16384 2023-10-08T13:52:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:52:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:52:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:52:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:52:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a60d56700 1 -- 172.21.15.57:0/3407977281 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4a480a15f0 msgr2=0x7f4a480a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a60d56700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4a480a15f0 0x7f4a480a3ab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c197b70 tx=0x7f4a44009260 comp rx=0 tx=0).stop 2023-10-08T13:52:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a60d56700 1 -- 172.21.15.57:0/3407977281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a5c068990 msgr2=0x7f4a5c19c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:52:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.889+0000 7f4a60d56700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a5c068990 0x7f4a5c19c880 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000c00 tx=0x7f4a4c004730 comp rx=0 tx=0).stop 2023-10-08T13:52:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.890+0000 7f4a60d56700 1 -- 172.21.15.57:0/3407977281 shutdown_connections 2023-10-08T13:52:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.890+0000 7f4a60d56700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a5c106730 0x7f4a5c1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.890+0000 7f4a60d56700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a5c068990 0x7f4a5c19c880 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.890+0000 7f4a60d56700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4a480a15f0 0x7f4a480a3ab0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.890+0000 7f4a60d56700 1 --2- 172.21.15.57:0/3407977281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a5c069350 0x7f4a5c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:52:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.890+0000 7f4a60d56700 1 -- 172.21.15.57:0/3407977281 >> 172.21.15.57:0/3407977281 conn(0x7f4a5c076790 msgr2=0x7f4a5c0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:52:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.890+0000 7f4a60d56700 1 -- 172.21.15.57:0/3407977281 shutdown_connections 2023-10-08T13:52:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:52:51.890+0000 7f4a60d56700 1 -- 172.21.15.57:0/3407977281 wait complete. 2023-10-08T13:52:52.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:51 smithi115 ceph-mon[104408]: pgmap v338: 129 pgs: 129 active+clean; 12 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2023-10-08T13:52:52.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:51 smithi057 ceph-mon[99476]: pgmap v338: 129 pgs: 129 active+clean; 12 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2023-10-08T13:52:52.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:51 smithi099 ceph-mon[103849]: pgmap v338: 129 pgs: 129 active+clean; 12 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.0 MiB/s wr, 136 op/s 2023-10-08T13:52:54.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:53 smithi115 ceph-mon[104408]: pgmap v339: 129 pgs: 129 active+clean; 26 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.2 MiB/s wr, 154 op/s 2023-10-08T13:52:54.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:53 smithi057 ceph-mon[99476]: pgmap v339: 129 pgs: 129 active+clean; 26 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.2 MiB/s wr, 154 op/s 2023-10-08T13:52:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:53 smithi099 ceph-mon[103849]: pgmap v339: 129 pgs: 129 active+clean; 26 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.2 MiB/s wr, 154 op/s 2023-10-08T13:52:54.364 INFO:tasks.workunit.client.0.smithi057.stdout: 11 119644 778 67756 543 24815 855 2023-10-08T13:52:55.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:52:55.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:52:55.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:54 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:52:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:52:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:52:55.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:54 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:52:55.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:52:55.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:52:55.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:54 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:52:56.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:55 smithi115 ceph-mon[104408]: pgmap v340: 129 pgs: 129 active+clean; 32 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 230 op/s 2023-10-08T13:52:56.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:55 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:52:56.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:52:56.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:52:56.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:52:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:55 smithi057 ceph-mon[99476]: pgmap v340: 129 pgs: 129 active+clean; 32 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 230 op/s 2023-10-08T13:52:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:55 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:52:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:52:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:52:56.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:52:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:55 smithi099 ceph-mon[103849]: pgmap v340: 129 pgs: 129 active+clean; 32 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 230 op/s 2023-10-08T13:52:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:55 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:52:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:52:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:52:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:52:58.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:57 smithi115 ceph-mon[104408]: pgmap v341: 129 pgs: 129 active+clean; 38 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 275 op/s 2023-10-08T13:52:58.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:57 smithi057 ceph-mon[99476]: pgmap v341: 129 pgs: 129 active+clean; 38 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 275 op/s 2023-10-08T13:52:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:57 smithi099 ceph-mon[103849]: pgmap v341: 129 pgs: 129 active+clean; 38 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 275 op/s 2023-10-08T13:52:59.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:58 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:52:59.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:58 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:52:59.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:58 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:53:00.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:52:59 smithi115 ceph-mon[104408]: pgmap v342: 129 pgs: 129 active+clean; 38 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 258 op/s 2023-10-08T13:53:00.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:52:59 smithi057 ceph-mon[99476]: pgmap v342: 129 pgs: 129 active+clean; 38 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 258 op/s 2023-10-08T13:53:00.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:52:59 smithi099 ceph-mon[103849]: pgmap v342: 129 pgs: 129 active+clean; 38 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 258 op/s 2023-10-08T13:53:02.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:01 smithi115 ceph-mon[104408]: pgmap v343: 129 pgs: 129 active+clean; 37 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 262 op/s 2023-10-08T13:53:02.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:01 smithi057 ceph-mon[99476]: pgmap v343: 129 pgs: 129 active+clean; 37 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 262 op/s 2023-10-08T13:53:02.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:01 smithi099 ceph-mon[103849]: pgmap v343: 129 pgs: 129 active+clean; 37 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 262 op/s 2023-10-08T13:53:04.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:03 smithi115 ceph-mon[104408]: pgmap v344: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 254 op/s 2023-10-08T13:53:04.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:03 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:53:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:03 smithi057 ceph-mon[99476]: pgmap v344: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 254 op/s 2023-10-08T13:53:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:03 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:53:04.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:03 smithi099 ceph-mon[103849]: pgmap v344: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 254 op/s 2023-10-08T13:53:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:03 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:53:04.364 INFO:tasks.workunit.client.0.smithi057.stdout: 15 128195 199 72955 273 41655 411 2023-10-08T13:53:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.006+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2867327734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6cd01028d0 msgr2=0x7f6cd0102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.006+0000 7f6cd4b71700 1 --2- 172.21.15.57:0/2867327734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6cd01028d0 0x7f6cd0102d50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009990 tx=0x7f6cb8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.006+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2867327734 shutdown_connections 2023-10-08T13:53:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.006+0000 7f6cd4b71700 1 --2- 172.21.15.57:0/2867327734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd0103290 0x7f6cd010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.006+0000 7f6cd4b71700 1 --2- 172.21.15.57:0/2867327734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd01088e0 0x7f6cd0108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.006+0000 7f6cd4b71700 1 --2- 172.21.15.57:0/2867327734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6cd01028d0 0x7f6cd0102d50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.006+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2867327734 >> 172.21.15.57:0/2867327734 conn(0x7f6cd00fe3a0 msgr2=0x7f6cd01007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.006+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2867327734 shutdown_connections 2023-10-08T13:53:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.006+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2867327734 wait complete. 2023-10-08T13:53:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.007+0000 7f6cd4b71700 1 Processor -- start 2023-10-08T13:53:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.007+0000 7f6cd4b71700 1 -- start start 2023-10-08T13:53:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cd4b71700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd01028d0 0x7f6cd019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cd4b71700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd0103290 0x7f6cd019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cd4b71700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6cd01088e0 0x7f6cd0198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cd4b71700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd0199230 con 0x7f6cd01088e0 2023-10-08T13:53:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cd4b71700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd01993a0 con 0x7f6cd0103290 2023-10-08T13:53:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cd4b71700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd0199560 con 0x7f6cd01028d0 2023-10-08T13:53:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cce59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd01028d0 0x7f6cd019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cce59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd01028d0 0x7f6cd019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58182/0 (socket says 172.21.15.57:58182) 2023-10-08T13:53:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cced9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6cd01088e0 0x7f6cd0198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cce59c700 1 -- 172.21.15.57:0/2096300493 learned_addr learned my addr 172.21.15.57:0/2096300493 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd0103290 0x7f6cd019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cce59c700 1 -- 172.21.15.57:0/2096300493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd0103290 msgr2=0x7f6cd019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cce59c700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd0103290 0x7f6cd019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cce59c700 1 -- 172.21.15.57:0/2096300493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6cd01088e0 msgr2=0x7f6cd0198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cce59c700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6cd01088e0 0x7f6cd0198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cce59c700 1 -- 172.21.15.57:0/2096300493 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb80095f0 con 0x7f6cd01028d0 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.008+0000 7f6cce59c700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd01028d0 0x7f6cd019ea50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000e9a0 tx=0x7f6cc000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.009+0000 7f6cc77fe700 1 -- 172.21.15.57:0/2096300493 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0009790 con 0x7f6cd01028d0 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.009+0000 7f6cc77fe700 1 -- 172.21.15.57:0/2096300493 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc0010430 con 0x7f6cd01028d0 2023-10-08T13:53:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.009+0000 7f6cc77fe700 1 -- 172.21.15.57:0/2096300493 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0018680 con 0x7f6cd01028d0 2023-10-08T13:53:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.009+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2096300493 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd0199840 con 0x7f6cd01028d0 2023-10-08T13:53:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.009+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2096300493 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd01a5a40 con 0x7f6cd01028d0 2023-10-08T13:53:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.011+0000 7f6cc77fe700 1 -- 172.21.15.57:0/2096300493 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6cc00105a0 con 0x7f6cd01028d0 2023-10-08T13:53:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.011+0000 7f6cc77fe700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6cbc0a1670 0x7f6cbc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.012+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6cbc0a1670 0x7f6cbc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.012+0000 7f6cc77fe700 1 -- 172.21.15.57:0/2096300493 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6cc0014070 con 0x7f6cd01028d0 2023-10-08T13:53:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.012+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6cbc0a1670 0x7f6cbc0a3b30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cd019a220 tx=0x7f6cb8005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.012+0000 7f6ce0d5ab80 1 Processor -- start 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6ce0d5ab80 1 -- start start 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea0880 0x55e17dea0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17de9ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17de9b0b0 0x55e17de9b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17de03400 con 0x55e17de9b0b0 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17ddf4bc0 con 0x55e17dea11a0 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17ddf24e0 con 0x55e17dea0880 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6cced9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17de9b0b0 0x55e17de9b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6cce59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea0880 0x55e17dea0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6cced9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17de9b0b0 0x55e17de9b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57382/0 (socket says 172.21.15.57:57382) 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6cced9d700 1 -- 172.21.15.57:0/2079183662 learned_addr learned my addr 172.21.15.57:0/2079183662 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6cce59c700 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea0880 0x55e17dea0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58190/0 (socket says 172.21.15.57:58190) 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17de9ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6cce59c700 1 -- 172.21.15.57:0/2079183662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 msgr2=0x55e17de9ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6cce59c700 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17de9ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6cce59c700 1 -- 172.21.15.57:0/2079183662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17de9b0b0 msgr2=0x55e17de9b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6cce59c700 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17de9b0b0 0x55e17de9b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.013+0000 7f6cce59c700 1 -- 172.21.15.57:0/2079183662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e17dc18d80 con 0x55e17dea0880 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.014+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17de9ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.014+0000 7f6cced9d700 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17de9b0b0 0x55e17de9b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.014+0000 7f6cce59c700 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea0880 0x55e17dea0c60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00ca920 tx=0x7f6cc0099b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e03f837b55725d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.014+0000 7f6cc4ff9700 1 -- 172.21.15.57:0/2079183662 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc009e120 con 0x55e17dea0880 2023-10-08T13:53:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.014+0000 7f6cc4ff9700 1 -- 172.21.15.57:0/2079183662 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc009e300 con 0x55e17dea0880 2023-10-08T13:53:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.014+0000 7f6cc4ff9700 1 -- 172.21.15.57:0/2079183662 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00cd370 con 0x55e17dea0880 2023-10-08T13:53:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/2079183662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea0880 msgr2=0x55e17dea0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea0880 0x55e17dea0c60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00ca920 tx=0x7f6cc0099b90 comp rx=0 tx=0).stop 2023-10-08T13:53:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/2079183662 shutdown_connections 2023-10-08T13:53:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17de9b0b0 0x55e17de9b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea0880 0x55e17dea0c60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/2079183662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17de9ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/2079183662 >> 172.21.15.57:0/2079183662 conn(0x55e17de0f200 msgr2=0x55e17de95b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/2079183662 shutdown_connections 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/2079183662 wait complete. 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 -- start start 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.015+0000 7f6ce0d5ab80 1 Processor -- start 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17de9b0b0 0x55e17dea44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17dea4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17ddeddd0 con 0x55e17dea0880 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17dd8a6b0 con 0x55e17dea11a0 2023-10-08T13:53:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17dd8a350 con 0x55e17de9b0b0 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ce0d5ab80 10 client.0 _ll_register_callbacks cb 0x55e17dbe77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ccdd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ccdd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57386/0 (socket says 172.21.15.57:57386) 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/680641550 learned_addr learned my addr 172.21.15.57:0/680641550 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6cced9d700 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17dea4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6cce59c700 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17de9b0b0 0x55e17dea44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/680641550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17de9b0b0 msgr2=0x55e17dea44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17de9b0b0 0x55e17dea44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/680641550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 msgr2=0x55e17dea4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17dea4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/680641550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc8011330 con 0x55e17dea0880 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6cced9d700 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17dea4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea49f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009990 tx=0x7f6cb801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1257b3be2f8c17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.016+0000 7f6cb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/680641550 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80056f0 con 0x55e17dea0880 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/680641550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e17ddb5e50 con 0x55e17dea0880 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/680641550 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6cb8004c40 con 0x55e17dea0880 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/680641550 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cb8004e00 con 0x55e17dea0880 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6ce0d5ab80 10 client.16278 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/680641550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f6cd01a5a40 con 0x55e17dea0880 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/680641550 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e17ddb5e50 con 0x55e17dea0880 2023-10-08T13:53:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/680641550 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6cb80304b0 con 0x55e17dea0880 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6ce0d5ab80 10 client.16278 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6ce0d5ab80 10 client.16278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6ce0d5ab80 20 client.16278 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e17deb4d50 0x55e17deb7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6ce0d5ab80 4 client.16278 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/680641550 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e17dde6db0 con 0x55e17deb4d50 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.017+0000 7f6cce59c700 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e17deb4d50 0x55e17deb7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.018+0000 7f6cce59c700 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e17deb4d50 0x55e17deb7170 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867be25ec4a8204e server_cookie=28d82b7c6cbdf4af in_seq=0 out_seq=0 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.018+0000 7f6cb5ffb700 10 client.16278 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.018+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/680641550 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e17dde6db0 con 0x55e17deb4d50 2023-10-08T13:53:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.018+0000 7f6cb5ffb700 10 client.16278 handle_command_reply: tid=0 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.018+0000 7f6ce0d5ab80 1 client.16278 shutdown 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/680641550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 msgr2=0x55e17dea49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea49f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009990 tx=0x7f6cb801a050 comp rx=0 tx=0).stop 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/680641550 shutdown_connections 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e17deb4d50 0x55e17deb7170 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17dea4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6cb5ffb700 0 client.16278 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17de9b0b0 0x55e17dea44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/680641550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea49f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/680641550 >> 172.21.15.57:0/680641550 conn(0x55e17de0f200 msgr2=0x55e17de95010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/680641550 shutdown_connections 2023-10-08T13:53:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/680641550 wait complete. 2023-10-08T13:53:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.019+0000 7f6ce0d5ab80 20 client.16278 trim_cache size 0 max 16384 2023-10-08T13:53:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.036+0000 7f6ce0d5ab80 1 Processor -- start 2023-10-08T13:53:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.036+0000 7f6ce0d5ab80 1 -- start start 2023-10-08T13:53:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.036+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17de9b0b0 0x55e17deabbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.036+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.036+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea11a0 0x55e17de95790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.036+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17dc6e8c0 con 0x55e17dea0880 2023-10-08T13:53:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.036+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17de92830 con 0x55e17de9b0b0 2023-10-08T13:53:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.036+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17de97330 con 0x55e17dea11a0 2023-10-08T13:53:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6ccdd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6cce59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17de9b0b0 0x55e17deabbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6ccdd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57394/0 (socket says 172.21.15.57:57394) 2023-10-08T13:53:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/3049986964 learned_addr learned my addr 172.21.15.57:0/3049986964 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6cced9d700 1 --2- 172.21.15.57:0/3049986964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea11a0 0x55e17de95790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/3049986964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea11a0 msgr2=0x55e17de95790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/3049986964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea11a0 0x55e17de95790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/3049986964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17de9b0b0 msgr2=0x55e17deabbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/3049986964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17de9b0b0 0x55e17deabbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/3049986964 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e17dc01c00 con 0x55e17dea0880 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6cce59c700 1 --2- 172.21.15.57:0/3049986964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17de9b0b0 0x55e17deabbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/3049986964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea4d50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8030840 tx=0x7f6cb8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9653f7c46fa27f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6cc57fa700 1 -- 172.21.15.57:0/3049986964 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800b280 con 0x55e17dea0880 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6cc57fa700 1 -- 172.21.15.57:0/3049986964 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cb801e040 con 0x55e17dea0880 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.037+0000 7f6cc57fa700 1 -- 172.21.15.57:0/3049986964 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800f6c0 con 0x55e17dea0880 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/3049986964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 msgr2=0x55e17dea4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/3049986964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea4d50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8030840 tx=0x7f6cb8009310 comp rx=0 tx=0).stop 2023-10-08T13:53:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/3049986964 shutdown_connections 2023-10-08T13:53:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/3049986964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17dea11a0 0x55e17de95790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/3049986964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17de9b0b0 0x55e17deabbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/3049986964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17dea4d50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/3049986964 >> 172.21.15.57:0/3049986964 conn(0x55e17de0f200 msgr2=0x55e17dea2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/3049986964 shutdown_connections 2023-10-08T13:53:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/3049986964 wait complete. 2023-10-08T13:53:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 -- start start 2023-10-08T13:53:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.038+0000 7f6ce0d5ab80 1 Processor -- start 2023-10-08T13:53:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17de9b0b0 0x55e17deabbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17de9c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ce0d5ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17de9c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17de9ce50 con 0x55e17dea0880 2023-10-08T13:53:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17de9cfc0 con 0x55e17dea11a0 2023-10-08T13:53:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ce0d5ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17de9d180 con 0x55e17de9b0b0 2023-10-08T13:53:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ce0d5ab80 10 client.0 _ll_register_callbacks cb 0x7f6cb800ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ccdd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17de9c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ccdd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17de9c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57400/0 (socket says 172.21.15.57:57400) 2023-10-08T13:53:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/670471593 learned_addr learned my addr 172.21.15.57:0/670471593 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6cce59c700 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17de9b0b0 0x55e17deabbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6cced9d700 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17de9c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/670471593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17de9b0b0 msgr2=0x55e17deabbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17de9b0b0 0x55e17deabbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/670471593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 msgr2=0x55e17de9c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17de9c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ccdd9b700 1 -- 172.21.15.57:0/670471593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e17dc18d80 con 0x55e17dea0880 2023-10-08T13:53:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6ccdd9b700 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17de9c390 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009990 tx=0x7f6cb8030420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588784c4f01e915a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.039+0000 7f6cb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:05.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/670471593 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8004d90 con 0x55e17dea0880 2023-10-08T13:53:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/670471593 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6cb8030a30 con 0x55e17dea0880 2023-10-08T13:53:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/670471593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cc009e300 con 0x55e17dea0880 2023-10-08T13:53:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/670471593 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cb8030c30 con 0x55e17dea0880 2023-10-08T13:53:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6ce0d5ab80 10 client.16284 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/670471593 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f6cb8031e70 con 0x55e17dea0880 2023-10-08T13:53:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/670471593 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc009e300 con 0x55e17dea0880 2023-10-08T13:53:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/670471593 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6cb8038040 con 0x55e17dea0880 2023-10-08T13:53:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6ce0d5ab80 10 client.16284 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6ce0d5ab80 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6ce0d5ab80 20 client.16284 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e17df08e60 0x55e17df0b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6ce0d5ab80 4 client.16284 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:53:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/670471593 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e17dde6ad0 con 0x55e17df08e60 2023-10-08T13:53:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.040+0000 7f6cce59c700 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e17df08e60 0x55e17df0b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.041+0000 7f6cb5ffb700 10 client.16284 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.041+0000 7f6cce59c700 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e17df08e60 0x55e17df0b280 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8305128c77f7940d server_cookie=a2086c30a5cecaea in_seq=0 out_seq=0 2023-10-08T13:53:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6cb5ffb700 1 -- 172.21.15.57:0/670471593 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e17dde6ad0 con 0x55e17df08e60 2023-10-08T13:53:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6cb5ffb700 10 client.16284 handle_command_reply: tid=0 2023-10-08T13:53:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 client.16284 shutdown 2023-10-08T13:53:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/670471593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 msgr2=0x55e17de9c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17de9c390 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009990 tx=0x7f6cb8030420 comp rx=0 tx=0).stop 2023-10-08T13:53:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/670471593 shutdown_connections 2023-10-08T13:53:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e17df08e60 0x55e17df0b280 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e17dea11a0 0x55e17de9c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6cb5ffb700 0 client.16284 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e17de9b0b0 0x55e17deabbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 --2- 172.21.15.57:0/670471593 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e17dea0880 0x55e17de9c390 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/670471593 >> 172.21.15.57:0/670471593 conn(0x55e17de0f200 msgr2=0x55e17de9f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/670471593 shutdown_connections 2023-10-08T13:53:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 1 -- 172.21.15.57:0/670471593 wait complete. 2023-10-08T13:53:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.058+0000 7f6ce0d5ab80 20 client.16284 trim_cache size 0 max 16384 2023-10-08T13:53:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:53:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:53:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:53:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:53:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2096300493 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6cbc0a1670 msgr2=0x7f6cbc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6cbc0a1670 0x7f6cbc0a3b30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cd019a220 tx=0x7f6cb8005fd0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2096300493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd01028d0 msgr2=0x7f6cd019ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd01028d0 0x7f6cd019ea50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000e9a0 tx=0x7f6cc000ecb0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2096300493 shutdown_connections 2023-10-08T13:53:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6cd01088e0 0x7f6cd0198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd01028d0 0x7f6cd019ea50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f6cbc0a1670 0x7f6cbc0a3b30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 --2- 172.21.15.57:0/2096300493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd0103290 0x7f6cd019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2096300493 >> 172.21.15.57:0/2096300493 conn(0x7f6cd00fe3a0 msgr2=0x7f6cd00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2096300493 shutdown_connections 2023-10-08T13:53:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:05.059+0000 7f6cd4b71700 1 -- 172.21.15.57:0/2096300493 wait complete. 2023-10-08T13:53:06.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:05 smithi115 ceph-mon[104408]: pgmap v345: 129 pgs: 129 active+clean; 38 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 204 op/s 2023-10-08T13:53:06.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:05 smithi057 ceph-mon[99476]: pgmap v345: 129 pgs: 129 active+clean; 38 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 204 op/s 2023-10-08T13:53:06.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:05 smithi099 ceph-mon[103849]: pgmap v345: 129 pgs: 129 active+clean; 38 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 204 op/s 2023-10-08T13:53:06.498 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:53:06.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.628+0000 7fec5dd97700 1 -- 172.21.15.57:0/245432550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec581001c0 msgr2=0x7fec58100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.628+0000 7fec5dd97700 1 --2- 172.21.15.57:0/245432550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec581001c0 0x7fec58100640 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fec40009990 tx=0x7fec40009ca0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.628+0000 7fec5dd97700 1 -- 172.21.15.57:0/245432550 shutdown_connections 2023-10-08T13:53:06.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.628+0000 7fec5dd97700 1 --2- 172.21.15.57:0/245432550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec58100b80 0x7fec5810c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.628+0000 7fec5dd97700 1 --2- 172.21.15.57:0/245432550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec581046a0 0x7fec58104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.628+0000 7fec5dd97700 1 --2- 172.21.15.57:0/245432550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec581001c0 0x7fec58100640 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.629+0000 7fec5dd97700 1 -- 172.21.15.57:0/245432550 >> 172.21.15.57:0/245432550 conn(0x7fec58074940 msgr2=0x7fec58074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:06.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.629+0000 7fec5dd97700 1 -- 172.21.15.57:0/245432550 shutdown_connections 2023-10-08T13:53:06.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.629+0000 7fec5dd97700 1 -- 172.21.15.57:0/245432550 wait complete. 2023-10-08T13:53:06.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.629+0000 7fec5dd97700 1 Processor -- start 2023-10-08T13:53:06.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.629+0000 7fec5dd97700 1 -- start start 2023-10-08T13:53:06.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec5dd97700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec581001c0 0x7fec581979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec5dd97700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec58100b80 0x7fec58197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec5dd97700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec581046a0 0x7fec581a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec5dd97700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec58198770 con 0x7fec581046a0 2023-10-08T13:53:06.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec5dd97700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec581a2f80 con 0x7fec581001c0 2023-10-08T13:53:06.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec5dd97700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec581a30c0 con 0x7fec58100b80 2023-10-08T13:53:06.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec581046a0 0x7fec581a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec581046a0 0x7fec581a2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57414/0 (socket says 172.21.15.57:57414) 2023-10-08T13:53:06.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec57fff700 1 -- 172.21.15.57:0/3095434676 learned_addr learned my addr 172.21.15.57:0/3095434676 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:06.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec577fe700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec581001c0 0x7fec581979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec4ffff700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec58100b80 0x7fec58197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec57fff700 1 -- 172.21.15.57:0/3095434676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec58100b80 msgr2=0x7fec58197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec57fff700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec58100b80 0x7fec58197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec57fff700 1 -- 172.21.15.57:0/3095434676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec581001c0 msgr2=0x7fec581979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec57fff700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec581001c0 0x7fec581979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec57fff700 1 -- 172.21.15.57:0/3095434676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400095f0 con 0x7fec581046a0 2023-10-08T13:53:06.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec57fff700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec581046a0 0x7fec581a2a40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fec5000ef30 tx=0x7fec5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:06.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.630+0000 7fec557fa700 1 -- 172.21.15.57:0/3095434676 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50018070 con 0x7fec581046a0 2023-10-08T13:53:06.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.631+0000 7fec5dd97700 1 -- 172.21.15.57:0/3095434676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec581a32f0 con 0x7fec581046a0 2023-10-08T13:53:06.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.631+0000 7fec557fa700 1 -- 172.21.15.57:0/3095434676 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec5000f040 con 0x7fec581046a0 2023-10-08T13:53:06.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.631+0000 7fec557fa700 1 -- 172.21.15.57:0/3095434676 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5000cd00 con 0x7fec581046a0 2023-10-08T13:53:06.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.631+0000 7fec5dd97700 1 -- 172.21.15.57:0/3095434676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec581a3820 con 0x7fec581046a0 2023-10-08T13:53:06.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.633+0000 7fec557fa700 1 -- 172.21.15.57:0/3095434676 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fec50003a00 con 0x7fec581046a0 2023-10-08T13:53:06.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.633+0000 7fec557fa700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fec440a15f0 0x7fec440a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.633+0000 7fec557fa700 1 -- 172.21.15.57:0/3095434676 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fec500ccf30 con 0x7fec581046a0 2023-10-08T13:53:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.633+0000 7fec577fe700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fec440a15f0 0x7fec440a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.634+0000 7fec577fe700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fec440a15f0 0x7fec440a3ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fec58075e50 tx=0x7fec48009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.634+0000 7fec69f80b80 1 Processor -- start 2023-10-08T13:53:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.634+0000 7fec69f80b80 1 -- start start 2023-10-08T13:53:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.634+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ebffe80 0x56536ec00260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.634+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec007a0 0x56536ec0b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.634+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec0da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.634+0000 7fec69f80b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536eb69460 con 0x56536ec0b650 2023-10-08T13:53:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.634+0000 7fec69f80b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536eb5ac20 con 0x56536ebffe80 2023-10-08T13:53:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.634+0000 7fec69f80b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536eb58540 con 0x56536ec007a0 2023-10-08T13:53:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec0da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec007a0 0x56536ec0b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec0da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57424/0 (socket says 172.21.15.57:57424) 2023-10-08T13:53:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec007a0 0x56536ec0b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58234/0 (socket says 172.21.15.57:58234) 2023-10-08T13:53:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec57fff700 1 -- 172.21.15.57:0/3110936740 learned_addr learned my addr 172.21.15.57:0/3110936740 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec577fe700 1 --2- 172.21.15.57:0/3110936740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ebffe80 0x56536ec00260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4ffff700 1 -- 172.21.15.57:0/3110936740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ebffe80 msgr2=0x56536ec00260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4ffff700 1 --2- 172.21.15.57:0/3110936740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ebffe80 0x56536ec00260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4ffff700 1 -- 172.21.15.57:0/3110936740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 msgr2=0x56536ec0da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4ffff700 1 --2- 172.21.15.57:0/3110936740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec0da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4ffff700 1 -- 172.21.15.57:0/3110936740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56536e97ede0 con 0x56536ec007a0 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec57fff700 1 --2- 172.21.15.57:0/3110936740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec0da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4ffff700 1 --2- 172.21.15.57:0/3110936740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec007a0 0x56536ec0b110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fec400098e0 tx=0x7fec4000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17e1a3fdde24bb92 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4dffb700 1 -- 172.21.15.57:0/3110936740 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4001d070 con 0x56536ec007a0 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4dffb700 1 -- 172.21.15.57:0/3110936740 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec4001e040 con 0x56536ec007a0 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.635+0000 7fec4dffb700 1 -- 172.21.15.57:0/3110936740 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40004760 con 0x56536ec007a0 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.636+0000 7fec69f80b80 1 -- 172.21.15.57:0/3110936740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec007a0 msgr2=0x56536ec0b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.636+0000 7fec69f80b80 1 --2- 172.21.15.57:0/3110936740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec007a0 0x56536ec0b110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fec400098e0 tx=0x7fec4000b610 comp rx=0 tx=0).stop 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.636+0000 7fec69f80b80 1 -- 172.21.15.57:0/3110936740 shutdown_connections 2023-10-08T13:53:06.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.636+0000 7fec69f80b80 1 --2- 172.21.15.57:0/3110936740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec0da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.636+0000 7fec69f80b80 1 --2- 172.21.15.57:0/3110936740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ebffe80 0x56536ec00260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.636+0000 7fec69f80b80 1 --2- 172.21.15.57:0/3110936740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec007a0 0x56536ec0b110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.636+0000 7fec69f80b80 1 -- 172.21.15.57:0/3110936740 >> 172.21.15.57:0/3110936740 conn(0x56536eb75260 msgr2=0x56536ebfc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.636+0000 7fec69f80b80 1 -- 172.21.15.57:0/3110936740 shutdown_connections 2023-10-08T13:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.636+0000 7fec69f80b80 1 -- 172.21.15.57:0/3110936740 wait complete. 2023-10-08T13:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.636+0000 7fec69f80b80 1 -- start start 2023-10-08T13:53:06.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec69f80b80 1 Processor -- start 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ebffe80 0x56536ec0afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec007a0 0x56536ec08250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec0b650 0x56536ec087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec69f80b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536eb53e30 con 0x56536ebffe80 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec69f80b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536eaf0710 con 0x56536ec007a0 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec69f80b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536eaf03b0 con 0x56536ec0b650 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec69f80b80 10 client.0 _ll_register_callbacks cb 0x56536e94d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec577fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ebffe80 0x56536ec0afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec577fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ebffe80 0x56536ec0afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57426/0 (socket says 172.21.15.57:57426) 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec577fe700 1 -- 172.21.15.57:0/3101842071 learned_addr learned my addr 172.21.15.57:0/3101842071 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec57fff700 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec0b650 0x56536ec087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec4ffff700 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec007a0 0x56536ec08250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec577fe700 1 -- 172.21.15.57:0/3101842071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec0b650 msgr2=0x56536ec087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec577fe700 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec0b650 0x56536ec087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec577fe700 1 -- 172.21.15.57:0/3101842071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec007a0 msgr2=0x56536ec08250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec577fe700 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec007a0 0x56536ec08250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.637+0000 7fec577fe700 1 -- 172.21.15.57:0/3101842071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec581a3820 con 0x56536ebffe80 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec577fe700 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ebffe80 0x56536ec0afd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fec4801dc00 tx=0x7fec480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd84f246506bf210 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec36ffd700 1 -- 172.21.15.57:0/3101842071 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48022070 con 0x56536ebffe80 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec36ffd700 1 -- 172.21.15.57:0/3101842071 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fec4801c740 con 0x56536ebffe80 2023-10-08T13:53:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec69f80b80 1 -- 172.21.15.57:0/3101842071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56536eb1beb0 con 0x56536ebffe80 2023-10-08T13:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec36ffd700 1 -- 172.21.15.57:0/3101842071 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec4801c900 con 0x56536ebffe80 2023-10-08T13:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec69f80b80 10 client.16296 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec69f80b80 1 -- 172.21.15.57:0/3101842071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fec5810cda0 con 0x56536ebffe80 2023-10-08T13:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec36ffd700 1 -- 172.21.15.57:0/3101842071 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56536eb1beb0 con 0x56536ebffe80 2023-10-08T13:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec36ffd700 1 -- 172.21.15.57:0/3101842071 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fec48026b30 con 0x56536ebffe80 2023-10-08T13:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec69f80b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec69f80b80 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec69f80b80 20 client.16296 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec69f80b80 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56536ec1b1b0 0x56536ec1d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec69f80b80 4 client.16296 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.638+0000 7fec69f80b80 1 -- 172.21.15.57:0/3101842071 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56536eb4ce10 con 0x56536ec1b1b0 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.639+0000 7fec4ffff700 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56536ec1b1b0 0x56536ec1d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.639+0000 7fec36ffd700 10 client.16296 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.639+0000 7fec4ffff700 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56536ec1b1b0 0x56536ec1d5d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e84aade3dc956eab server_cookie=5cd52813e9223e4 in_seq=0 out_seq=0 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec36ffd700 1 -- 172.21.15.57:0/3101842071 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56536eb4ce10 con 0x56536ec1b1b0 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec36ffd700 10 client.16296 handle_command_reply: tid=0 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 client.16296 shutdown 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 -- 172.21.15.57:0/3101842071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ebffe80 msgr2=0x56536ec0afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ebffe80 0x56536ec0afd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fec4801dc00 tx=0x7fec480047c0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 -- 172.21.15.57:0/3101842071 shutdown_connections 2023-10-08T13:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56536ec1b1b0 0x56536ec1d5d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ec0b650 0x56536ec087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ebffe80 0x56536ec0afd0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 --2- 172.21.15.57:0/3101842071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec007a0 0x56536ec08250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 -- 172.21.15.57:0/3101842071 >> 172.21.15.57:0/3101842071 conn(0x56536eb75260 msgr2=0x56536ebfbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec36ffd700 0 client.16296 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 -- 172.21.15.57:0/3101842071 shutdown_connections 2023-10-08T13:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 1 -- 172.21.15.57:0/3101842071 wait complete. 2023-10-08T13:53:06.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.640+0000 7fec69f80b80 20 client.16296 trim_cache size 0 max 16384 2023-10-08T13:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.658+0000 7fec69f80b80 1 Processor -- start 2023-10-08T13:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.658+0000 7fec69f80b80 1 -- start start 2023-10-08T13:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.658+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 0x56536ec12140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.658+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec007a0 0x56536ec0ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.658+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec07ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.658+0000 7fec69f80b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536e9d4920 con 0x56536ec0b650 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.658+0000 7fec69f80b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536ec08df0 con 0x56536ec007a0 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.658+0000 7fec69f80b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536ec005e0 con 0x56536ebffe80 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec07ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec577fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 0x56536ec12140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec07ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57438/0 (socket says 172.21.15.57:57438) 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec577fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 0x56536ec12140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58246/0 (socket says 172.21.15.57:58246) 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec57fff700 1 -- 172.21.15.57:0/1563022960 learned_addr learned my addr 172.21.15.57:0/1563022960 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec4ffff700 1 --2- 172.21.15.57:0/1563022960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec007a0 0x56536ec0ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec57fff700 1 -- 172.21.15.57:0/1563022960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 msgr2=0x56536ec12140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec57fff700 1 --2- 172.21.15.57:0/1563022960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 0x56536ec12140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec57fff700 1 -- 172.21.15.57:0/1563022960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec007a0 msgr2=0x56536ec0ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec57fff700 1 --2- 172.21.15.57:0/1563022960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec007a0 0x56536ec0ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec57fff700 1 -- 172.21.15.57:0/1563022960 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56536e967c60 con 0x56536ec0b650 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec57fff700 1 --2- 172.21.15.57:0/1563022960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec07ae0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fec500999a0 tx=0x7fec50095940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47698e35a849cbd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec4e7fc700 1 -- 172.21.15.57:0/1563022960 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50004ad0 con 0x56536ec0b650 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec4e7fc700 1 -- 172.21.15.57:0/1563022960 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec5001b320 con 0x56536ec0b650 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.659+0000 7fec4e7fc700 1 -- 172.21.15.57:0/1563022960 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec500a1fa0 con 0x56536ec0b650 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.660+0000 7fec69f80b80 1 -- 172.21.15.57:0/1563022960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 msgr2=0x56536ec07ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.660+0000 7fec69f80b80 1 --2- 172.21.15.57:0/1563022960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec07ae0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fec500999a0 tx=0x7fec50095940 comp rx=0 tx=0).stop 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.660+0000 7fec69f80b80 1 -- 172.21.15.57:0/1563022960 shutdown_connections 2023-10-08T13:53:06.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.660+0000 7fec69f80b80 1 --2- 172.21.15.57:0/1563022960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec0b650 0x56536ec07ae0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.660+0000 7fec69f80b80 1 --2- 172.21.15.57:0/1563022960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 0x56536ec12140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.660+0000 7fec69f80b80 1 --2- 172.21.15.57:0/1563022960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec007a0 0x56536ec0ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.660+0000 7fec69f80b80 1 -- 172.21.15.57:0/1563022960 >> 172.21.15.57:0/1563022960 conn(0x56536eb75260 msgr2=0x56536ec06b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:06.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.660+0000 7fec69f80b80 1 -- 172.21.15.57:0/1563022960 shutdown_connections 2023-10-08T13:53:06.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.660+0000 7fec69f80b80 1 -- 172.21.15.57:0/1563022960 wait complete. 2023-10-08T13:53:06.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.660+0000 7fec69f80b80 1 -- start start 2023-10-08T13:53:06.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec69f80b80 1 Processor -- start 2023-10-08T13:53:06.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 0x56536ec118f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec007a0 0x56536ec11e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec69f80b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec0b650 0x56536ec10120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec69f80b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536ec106f0 con 0x56536ec007a0 2023-10-08T13:53:06.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec69f80b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536ec10860 con 0x56536ec0b650 2023-10-08T13:53:06.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec69f80b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536ec10a20 con 0x56536ebffe80 2023-10-08T13:53:06.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec69f80b80 10 client.0 _ll_register_callbacks cb 0x7fec4801d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:06.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec4ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec007a0 0x56536ec11e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec577fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 0x56536ec118f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec577fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 0x56536ec118f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58252/0 (socket says 172.21.15.57:58252) 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec4ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec007a0 0x56536ec11e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57442/0 (socket says 172.21.15.57:57442) 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec577fe700 1 -- 172.21.15.57:0/1097381164 learned_addr learned my addr 172.21.15.57:0/1097381164 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec57fff700 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec0b650 0x56536ec10120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec4ffff700 1 -- 172.21.15.57:0/1097381164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 msgr2=0x56536ec118f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec4ffff700 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 0x56536ec118f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec4ffff700 1 -- 172.21.15.57:0/1097381164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec0b650 msgr2=0x56536ec10120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec4ffff700 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec0b650 0x56536ec10120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.661+0000 7fec4ffff700 1 -- 172.21.15.57:0/1097381164 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec581a32f0 con 0x56536ec007a0 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec4ffff700 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec007a0 0x56536ec11e30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fec40009990 tx=0x7fec40005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796fcb5c4f62e30c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:06.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec36ffd700 1 -- 172.21.15.57:0/1097381164 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40034070 con 0x56536ec007a0 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec36ffd700 1 -- 172.21.15.57:0/1097381164 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fec400053e0 con 0x56536ec007a0 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec69f80b80 1 -- 172.21.15.57:0/1097381164 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec4001e040 con 0x56536ec007a0 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec36ffd700 1 -- 172.21.15.57:0/1097381164 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec4001d040 con 0x56536ec007a0 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec69f80b80 10 client.16302 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec69f80b80 1 -- 172.21.15.57:0/1097381164 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56536e9a5250 con 0x56536ec007a0 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec36ffd700 1 -- 172.21.15.57:0/1097381164 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec4001e040 con 0x56536ec007a0 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec36ffd700 1 -- 172.21.15.57:0/1097381164 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fec4000bac0 con 0x56536ec007a0 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec69f80b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec69f80b80 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec69f80b80 20 client.16302 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:06.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec69f80b80 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56536ec6ff60 0x56536ec72380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec69f80b80 4 client.16302 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.662+0000 7fec69f80b80 1 -- 172.21.15.57:0/1097381164 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56536eb4cb30 con 0x56536ec6ff60 2023-10-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec577fe700 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56536ec6ff60 0x56536ec72380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec36ffd700 10 client.16302 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec577fe700 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56536ec6ff60 0x56536ec72380 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5beaebc2c88e5440 server_cookie=2d6ec306cfd1a6ed in_seq=0 out_seq=0 2023-10-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec36ffd700 1 -- 172.21.15.57:0/1097381164 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56536eb4cb30 con 0x56536ec6ff60 2023-10-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec36ffd700 10 client.16302 handle_command_reply: tid=0 2023-10-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec69f80b80 1 client.16302 shutdown 2023-10-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec69f80b80 1 -- 172.21.15.57:0/1097381164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec007a0 msgr2=0x56536ec11e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec69f80b80 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec007a0 0x56536ec11e30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fec40009990 tx=0x7fec40005810 comp rx=0 tx=0).stop 2023-10-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec69f80b80 1 -- 172.21.15.57:0/1097381164 shutdown_connections 2023-10-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec69f80b80 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56536ec6ff60 0x56536ec72380 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec69f80b80 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56536ec0b650 0x56536ec10120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec69f80b80 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56536ebffe80 0x56536ec118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec69f80b80 1 --2- 172.21.15.57:0/1097381164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56536ec007a0 0x56536ec11e30 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec36ffd700 0 client.16302 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.663+0000 7fec69f80b80 1 -- 172.21.15.57:0/1097381164 >> 172.21.15.57:0/1097381164 conn(0x56536eb75260 msgr2=0x56536ec07a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec69f80b80 1 -- 172.21.15.57:0/1097381164 shutdown_connections 2023-10-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec69f80b80 1 -- 172.21.15.57:0/1097381164 wait complete. 2023-10-08T13:53:06.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec69f80b80 20 client.16302 trim_cache size 0 max 16384 2023-10-08T13:53:06.673 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:53:06.673 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-10-08T13:53:06.673 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-10-08T13:53:06.673 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:53:06.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 -- 172.21.15.57:0/3095434676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fec440a15f0 msgr2=0x7fec440a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fec440a15f0 0x7fec440a3ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fec58075e50 tx=0x7fec48009260 comp rx=0 tx=0).stop 2023-10-08T13:53:06.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 -- 172.21.15.57:0/3095434676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec581046a0 msgr2=0x7fec581a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec581046a0 0x7fec581a2a40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fec5000ef30 tx=0x7fec5000c3a0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 -- 172.21.15.57:0/3095434676 shutdown_connections 2023-10-08T13:53:06.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec581046a0 0x7fec581a2a40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fec440a15f0 0x7fec440a3ab0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec581001c0 0x7fec581979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 --2- 172.21.15.57:0/3095434676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec58100b80 0x7fec58197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 -- 172.21.15.57:0/3095434676 >> 172.21.15.57:0/3095434676 conn(0x7fec58074940 msgr2=0x7fec580fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:06.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 -- 172.21.15.57:0/3095434676 shutdown_connections 2023-10-08T13:53:06.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.664+0000 7fec5dd97700 1 -- 172.21.15.57:0/3095434676 wait complete. 2023-10-08T13:53:06.677 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-08T13:53:06.677 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:53:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.806+0000 7fd94e770700 1 -- 172.21.15.57:0/628995776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd948105600 msgr2=0x7fd9481059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.806+0000 7fd94e770700 1 --2- 172.21.15.57:0/628995776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd948105600 0x7fd9481059e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd9380099c0 tx=0x7fd938009cd0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.806+0000 7fd94e770700 1 -- 172.21.15.57:0/628995776 shutdown_connections 2023-10-08T13:53:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.806+0000 7fd94e770700 1 --2- 172.21.15.57:0/628995776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9480fff00 0x7fd94810c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.806+0000 7fd94e770700 1 --2- 172.21.15.57:0/628995776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd948105600 0x7fd9481059e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.806+0000 7fd94e770700 1 --2- 172.21.15.57:0/628995776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480ff540 0x7fd9480ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.806+0000 7fd94e770700 1 -- 172.21.15.57:0/628995776 >> 172.21.15.57:0/628995776 conn(0x7fd9480fb030 msgr2=0x7fd9480fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.806+0000 7fd94e770700 1 -- 172.21.15.57:0/628995776 shutdown_connections 2023-10-08T13:53:06.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.806+0000 7fd94e770700 1 -- 172.21.15.57:0/628995776 wait complete. 2023-10-08T13:53:06.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.807+0000 7fd94e770700 1 Processor -- start 2023-10-08T13:53:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.807+0000 7fd94e770700 1 -- start start 2023-10-08T13:53:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd94e770700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480ff540 0x7fd9481978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd94e770700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9480fff00 0x7fd948197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd94e770700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd948105600 0x7fd9481a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd94e770700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd948198660 con 0x7fd9480fff00 2023-10-08T13:53:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd94e770700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9481a2e70 con 0x7fd948105600 2023-10-08T13:53:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd94e770700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9481a2fb0 con 0x7fd9480ff540 2023-10-08T13:53:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9477fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9480fff00 0x7fd948197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9477fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9480fff00 0x7fd948197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57454/0 (socket says 172.21.15.57:57454) 2023-10-08T13:53:06.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd94cd0d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd948105600 0x7fd9481a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9477fe700 1 -- 172.21.15.57:0/2637517139 learned_addr learned my addr 172.21.15.57:0/2637517139 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd947fff700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480ff540 0x7fd9481978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9477fe700 1 -- 172.21.15.57:0/2637517139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480ff540 msgr2=0x7fd9481978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9477fe700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480ff540 0x7fd9481978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9477fe700 1 -- 172.21.15.57:0/2637517139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd948105600 msgr2=0x7fd9481a2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9477fe700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd948105600 0x7fd9481a2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9477fe700 1 -- 172.21.15.57:0/2637517139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380095f0 con 0x7fd9480fff00 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9477fe700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9480fff00 0x7fd948197df0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9a0 tx=0x7fd93c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9457fa700 1 -- 172.21.15.57:0/2637517139 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c009790 con 0x7fd9480fff00 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9457fa700 1 -- 172.21.15.57:0/2637517139 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd93c010430 con 0x7fd9480fff00 2023-10-08T13:53:06.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd94e770700 1 -- 172.21.15.57:0/2637517139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9481a3240 con 0x7fd9480fff00 2023-10-08T13:53:06.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.808+0000 7fd9457fa700 1 -- 172.21.15.57:0/2637517139 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c018680 con 0x7fd9480fff00 2023-10-08T13:53:06.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.809+0000 7fd94e770700 1 -- 172.21.15.57:0/2637517139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9481a37a0 con 0x7fd9480fff00 2023-10-08T13:53:06.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.810+0000 7fd94e770700 1 -- 172.21.15.57:0/2637517139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd948107a80 con 0x7fd9480fff00 2023-10-08T13:53:06.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.811+0000 7fd9457fa700 1 -- 172.21.15.57:0/2637517139 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd93c0105a0 con 0x7fd9480fff00 2023-10-08T13:53:06.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.811+0000 7fd9457fa700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9300a1670 0x7fd9300a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:06.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.811+0000 7fd947fff700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9300a1670 0x7fd9300a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:06.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.811+0000 7fd9457fa700 1 -- 172.21.15.57:0/2637517139 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fd93c014070 con 0x7fd9480fff00 2023-10-08T13:53:06.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.812+0000 7fd947fff700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9300a1670 0x7fd9300a3b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd938009990 tx=0x7fd938019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:06.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:06.816+0000 7fd9457fa700 1 -- 172.21.15.57:0/2637517139 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd93c096580 con 0x7fd9480fff00 2023-10-08T13:53:07.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.073+0000 7fd94e770700 1 -- 172.21.15.57:0/2637517139 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd948067e40 con 0x7fd9480fff00 2023-10-08T13:53:07.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.074+0000 7fd9457fa700 1 -- 172.21.15.57:0/2637517139 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11373 (secure 0 0 0) 0x7fd93c099ba0 con 0x7fd9480fff00 2023-10-08T13:53:07.075 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:53:07.075 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:53:02.074692+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:53:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.076+0000 7fd94e770700 1 -- 172.21.15.57:0/2637517139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9300a1670 msgr2=0x7fd9300a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.076+0000 7fd94e770700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9300a1670 0x7fd9300a3b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd938009990 tx=0x7fd938019040 comp rx=0 tx=0).stop 2023-10-08T13:53:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.076+0000 7fd94e770700 1 -- 172.21.15.57:0/2637517139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9480fff00 msgr2=0x7fd948197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.076+0000 7fd94e770700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9480fff00 0x7fd948197df0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9a0 tx=0x7fd93c00ecb0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.077+0000 7fd94e770700 1 -- 172.21.15.57:0/2637517139 shutdown_connections 2023-10-08T13:53:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.077+0000 7fd94e770700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9300a1670 0x7fd9300a3b30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.077+0000 7fd94e770700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd948105600 0x7fd9481a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.077+0000 7fd94e770700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480ff540 0x7fd9481978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.077+0000 7fd94e770700 1 --2- 172.21.15.57:0/2637517139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd9480fff00 0x7fd948197df0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.077+0000 7fd94e770700 1 -- 172.21.15.57:0/2637517139 >> 172.21.15.57:0/2637517139 conn(0x7fd9480fb030 msgr2=0x7fd9480fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.077+0000 7fd94e770700 1 -- 172.21.15.57:0/2637517139 shutdown_connections 2023-10-08T13:53:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.077+0000 7fd94e770700 1 -- 172.21.15.57:0/2637517139 wait complete. 2023-10-08T13:53:07.079 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2023-10-08T13:53:07.092 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-08T13:53:07.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.221+0000 7fa28995a700 1 -- 172.21.15.57:0/785055616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2841046a0 msgr2=0x7fa284104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.221+0000 7fa28995a700 1 --2- 172.21.15.57:0/785055616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2841046a0 0x7fa284104a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009f90 tx=0x7fa27c00b330 comp rx=0 tx=0).stop 2023-10-08T13:53:07.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.221+0000 7fa28995a700 1 -- 172.21.15.57:0/785055616 shutdown_connections 2023-10-08T13:53:07.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.221+0000 7fa28995a700 1 --2- 172.21.15.57:0/785055616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa284100b80 0x7fa28410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.221+0000 7fa28995a700 1 --2- 172.21.15.57:0/785055616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2841046a0 0x7fa284104a80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.221+0000 7fa28995a700 1 --2- 172.21.15.57:0/785055616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2841001c0 0x7fa284100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.221+0000 7fa28995a700 1 -- 172.21.15.57:0/785055616 >> 172.21.15.57:0/785055616 conn(0x7fa2840749a0 msgr2=0x7fa284074db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.221+0000 7fa28995a700 1 -- 172.21.15.57:0/785055616 shutdown_connections 2023-10-08T13:53:07.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.221+0000 7fa28995a700 1 -- 172.21.15.57:0/785055616 wait complete. 2023-10-08T13:53:07.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.222+0000 7fa28995a700 1 Processor -- start 2023-10-08T13:53:07.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.222+0000 7fa28995a700 1 -- start start 2023-10-08T13:53:07.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa28995a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2841001c0 0x7fa284197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa28995a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa284100b80 0x7fa2841980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa28995a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2841046a0 0x7fa2841a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa28995a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa284198910 con 0x7fa284100b80 2023-10-08T13:53:07.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa28995a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2841a3120 con 0x7fa2841001c0 2023-10-08T13:53:07.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa28995a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2841a3260 con 0x7fa2841046a0 2023-10-08T13:53:07.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa2827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa284100b80 0x7fa2841980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa282ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2841001c0 0x7fa284197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa2827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa284100b80 0x7fa2841980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57468/0 (socket says 172.21.15.57:57468) 2023-10-08T13:53:07.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa2827fc700 1 -- 172.21.15.57:0/1960856497 learned_addr learned my addr 172.21.15.57:0/1960856497 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa2837fe700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2841046a0 0x7fa2841a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa2827fc700 1 -- 172.21.15.57:0/1960856497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2841046a0 msgr2=0x7fa2841a2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa2827fc700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2841046a0 0x7fa2841a2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa2827fc700 1 -- 172.21.15.57:0/1960856497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2841001c0 msgr2=0x7fa284197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa2827fc700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2841001c0 0x7fa284197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa2827fc700 1 -- 172.21.15.57:0/1960856497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2780095f0 con 0x7fa284100b80 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa282ffd700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2841001c0 0x7fa284197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa2827fc700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa284100b80 0x7fa2841980a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa27800ec10 tx=0x7fa27800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.223+0000 7fa288958700 1 -- 172.21.15.57:0/1960856497 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27800cc90 con 0x7fa284100b80 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.224+0000 7fa288958700 1 -- 172.21.15.57:0/1960856497 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2780044e0 con 0x7fa284100b80 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.224+0000 7fa28995a700 1 -- 172.21.15.57:0/1960856497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c009c40 con 0x7fa284100b80 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.224+0000 7fa288958700 1 -- 172.21.15.57:0/1960856497 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278010430 con 0x7fa284100b80 2023-10-08T13:53:07.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.224+0000 7fa28995a700 1 -- 172.21.15.57:0/1960856497 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2841a3870 con 0x7fa284100b80 2023-10-08T13:53:07.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.226+0000 7fa288958700 1 -- 172.21.15.57:0/1960856497 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa278003fb0 con 0x7fa284100b80 2023-10-08T13:53:07.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.226+0000 7fa288958700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa26c0a15f0 0x7fa26c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.226+0000 7fa288958700 1 -- 172.21.15.57:0/1960856497 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa278014070 con 0x7fa284100b80 2023-10-08T13:53:07.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.226+0000 7fa282ffd700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa26c0a15f0 0x7fa26c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa282ffd700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa26c0a15f0 0x7fa26c0a3ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009790 tx=0x7fa27c0096c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa295b43b80 1 Processor -- start 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa295b43b80 1 -- start start 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f1f8e80 0x55572f1f9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f204110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f204650 0x55572f206a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa295b43b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f162460 con 0x55572f1f97a0 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa295b43b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f153c20 con 0x55572f204650 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa295b43b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f151540 con 0x55572f1f8e80 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa2827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f204110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa2827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f204110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57476/0 (socket says 172.21.15.57:57476) 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa2827fc700 1 -- 172.21.15.57:0/2509390961 learned_addr learned my addr 172.21.15.57:0/2509390961 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.227+0000 7fa282ffd700 1 --2- 172.21.15.57:0/2509390961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f1f8e80 0x55572f1f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.228+0000 7fa2827fc700 1 -- 172.21.15.57:0/2509390961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f1f8e80 msgr2=0x55572f1f9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.228+0000 7fa2837fe700 1 --2- 172.21.15.57:0/2509390961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f204650 0x55572f206a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.228+0000 7fa2827fc700 1 --2- 172.21.15.57:0/2509390961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f1f8e80 0x55572f1f9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.228+0000 7fa2827fc700 1 -- 172.21.15.57:0/2509390961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f204650 msgr2=0x55572f206a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.228+0000 7fa2827fc700 1 --2- 172.21.15.57:0/2509390961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f204650 0x55572f206a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.228+0000 7fa2827fc700 1 -- 172.21.15.57:0/2509390961 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55572ef77de0 con 0x55572f1f97a0 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.228+0000 7fa2827fc700 1 --2- 172.21.15.57:0/2509390961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f204110 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa2780a24c0 tx=0x7fa27809a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a08fa531eba2b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.228+0000 7fa269ffb700 1 -- 172.21.15.57:0/2509390961 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27809ad60 con 0x55572f1f97a0 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.228+0000 7fa269ffb700 1 -- 172.21.15.57:0/2509390961 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2780850c0 con 0x55572f1f97a0 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.228+0000 7fa269ffb700 1 -- 172.21.15.57:0/2509390961 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278098a90 con 0x55572f1f97a0 2023-10-08T13:53:07.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 -- 172.21.15.57:0/2509390961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 msgr2=0x55572f204110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 --2- 172.21.15.57:0/2509390961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f204110 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa2780a24c0 tx=0x7fa27809a1a0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 -- 172.21.15.57:0/2509390961 shutdown_connections 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 --2- 172.21.15.57:0/2509390961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f204650 0x55572f206a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 --2- 172.21.15.57:0/2509390961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f1f8e80 0x55572f1f9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 --2- 172.21.15.57:0/2509390961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f204110 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 -- 172.21.15.57:0/2509390961 >> 172.21.15.57:0/2509390961 conn(0x55572f16e260 msgr2=0x55572f1f52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 -- 172.21.15.57:0/2509390961 shutdown_connections 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 -- 172.21.15.57:0/2509390961 wait complete. 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 -- start start 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.229+0000 7fa295b43b80 1 Processor -- start 2023-10-08T13:53:07.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f2014b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f201f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa295b43b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f14ce30 con 0x55572f1f97a0 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa295b43b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f0e9710 con 0x55572f1f8e80 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa295b43b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f0e93b0 con 0x55572f204650 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa295b43b80 10 client.0 _ll_register_callbacks cb 0x55572ef46800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2019f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57482/0 (socket says 172.21.15.57:57482) 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2827fc700 1 -- 172.21.15.57:0/570434425 learned_addr learned my addr 172.21.15.57:0/570434425 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2837fe700 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f201f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa282ffd700 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f2014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2827fc700 1 -- 172.21.15.57:0/570434425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 msgr2=0x55572f201f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2827fc700 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f201f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2827fc700 1 -- 172.21.15.57:0/570434425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 msgr2=0x55572f2014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2827fc700 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f2014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2827fc700 1 -- 172.21.15.57:0/570434425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2841a3870 con 0x55572f1f97a0 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2837fe700 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f201f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa2827fc700 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2019f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa2780cdaa0 tx=0x7fa278003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec4efc70ba17beb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa262ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.230+0000 7fa262ffd700 1 -- 172.21.15.57:0/570434425 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27809a660 con 0x55572f1f97a0 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa262ffd700 1 -- 172.21.15.57:0/570434425 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa27809e7b0 con 0x55572f1f97a0 2023-10-08T13:53:07.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa295b43b80 1 -- 172.21.15.57:0/570434425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55572f114eb0 con 0x55572f1f97a0 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa262ffd700 1 -- 172.21.15.57:0/570434425 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa27809a8c0 con 0x55572f1f97a0 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa295b43b80 10 client.16320 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa295b43b80 1 -- 172.21.15.57:0/570434425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55572ef60c60 con 0x55572f1f97a0 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa262ffd700 1 -- 172.21.15.57:0/570434425 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa27807d080 con 0x55572f1f97a0 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa262ffd700 1 -- 172.21.15.57:0/570434425 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa2780a2660 con 0x55572f1f97a0 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa295b43b80 10 client.16320 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa295b43b80 10 client.16320 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa295b43b80 20 client.16320 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa295b43b80 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55572f213e10 0x55572f216200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa295b43b80 4 client.16320 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:07.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa295b43b80 1 -- 172.21.15.57:0/570434425 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55572f145e10 con 0x55572f213e10 2023-10-08T13:53:07.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.231+0000 7fa282ffd700 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55572f213e10 0x55572f216200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.232+0000 7fa282ffd700 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55572f213e10 0x55572f216200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d156edf67ee953f9 server_cookie=4fa275d4a7432535 in_seq=0 out_seq=0 2023-10-08T13:53:07.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.232+0000 7fa262ffd700 10 client.16320 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:53:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.232+0000 7fa262ffd700 1 -- 172.21.15.57:0/570434425 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55572f145e10 con 0x55572f213e10 2023-10-08T13:53:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.232+0000 7fa262ffd700 10 client.16320 handle_command_reply: tid=0 2023-10-08T13:53:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.232+0000 7fa295b43b80 1 client.16320 shutdown 2023-10-08T13:53:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 1 -- 172.21.15.57:0/570434425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 msgr2=0x55572f2019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2019f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa2780cdaa0 tx=0x7fa278003a10 comp rx=0 tx=0).stop 2023-10-08T13:53:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 1 -- 172.21.15.57:0/570434425 shutdown_connections 2023-10-08T13:53:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55572f213e10 0x55572f216200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f201f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f2014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 1 --2- 172.21.15.57:0/570434425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2019f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa262ffd700 0 client.16320 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:53:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 1 -- 172.21.15.57:0/570434425 >> 172.21.15.57:0/570434425 conn(0x55572f16e260 msgr2=0x55572f1f4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 1 -- 172.21.15.57:0/570434425 shutdown_connections 2023-10-08T13:53:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 1 -- 172.21.15.57:0/570434425 wait complete. 2023-10-08T13:53:07.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.233+0000 7fa295b43b80 20 client.16320 trim_cache size 0 max 16384 2023-10-08T13:53:07.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.247+0000 7fa295b43b80 1 Processor -- start 2023-10-08T13:53:07.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa295b43b80 1 -- start start 2023-10-08T13:53:07.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f200d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f20b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f210000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa295b43b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572efcd920 con 0x55572f1f97a0 2023-10-08T13:53:07.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa295b43b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f201350 con 0x55572f1f8e80 2023-10-08T13:53:07.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa295b43b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f20b750 con 0x55572f204650 2023-10-08T13:53:07.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa2827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f20b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa2827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f20b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57490/0 (socket says 172.21.15.57:57490) 2023-10-08T13:53:07.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa2827fc700 1 -- 172.21.15.57:0/1535959561 learned_addr learned my addr 172.21.15.57:0/1535959561 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa2837fe700 1 --2- 172.21.15.57:0/1535959561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f210000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa282ffd700 1 --2- 172.21.15.57:0/1535959561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f200d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa2827fc700 1 -- 172.21.15.57:0/1535959561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 msgr2=0x55572f210000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa2827fc700 1 --2- 172.21.15.57:0/1535959561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f210000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa2827fc700 1 -- 172.21.15.57:0/1535959561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 msgr2=0x55572f200d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa2827fc700 1 --2- 172.21.15.57:0/1535959561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f200d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa2827fc700 1 -- 172.21.15.57:0/1535959561 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55572ef9e250 con 0x55572f1f97a0 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa2827fc700 1 --2- 172.21.15.57:0/1535959561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f20b190 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa27809a360 tx=0x7fa278004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1a3e000152494e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.248+0000 7fa26a7fc700 1 -- 172.21.15.57:0/1535959561 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278009a10 con 0x55572f1f97a0 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa26a7fc700 1 -- 172.21.15.57:0/1535959561 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55572f114eb0 con 0x55572f1f97a0 2023-10-08T13:53:07.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa26a7fc700 1 -- 172.21.15.57:0/1535959561 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27809a6f0 con 0x55572f1f97a0 2023-10-08T13:53:07.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa295b43b80 1 -- 172.21.15.57:0/1535959561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 msgr2=0x55572f20b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa295b43b80 1 --2- 172.21.15.57:0/1535959561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f20b190 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa27809a360 tx=0x7fa278004470 comp rx=0 tx=0).stop 2023-10-08T13:53:07.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa295b43b80 1 -- 172.21.15.57:0/1535959561 shutdown_connections 2023-10-08T13:53:07.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa295b43b80 1 --2- 172.21.15.57:0/1535959561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f210000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa295b43b80 1 --2- 172.21.15.57:0/1535959561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f200d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa295b43b80 1 --2- 172.21.15.57:0/1535959561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f20b190 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa295b43b80 1 -- 172.21.15.57:0/1535959561 >> 172.21.15.57:0/1535959561 conn(0x55572f16e260 msgr2=0x55572f20f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa295b43b80 1 -- 172.21.15.57:0/1535959561 shutdown_connections 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.249+0000 7fa295b43b80 1 -- 172.21.15.57:0/1535959561 wait complete. 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa295b43b80 1 -- start start 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa295b43b80 1 Processor -- start 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f209180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa295b43b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f209c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa295b43b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f26b0b0 con 0x55572f1f97a0 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa295b43b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f26b220 con 0x55572f1f8e80 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa295b43b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572f26b3e0 con 0x55572f204650 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa295b43b80 10 client.0 _ll_register_callbacks cb 0x7fa27c006000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa2827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa2827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57496/0 (socket says 172.21.15.57:57496) 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa2837fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f209c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.250+0000 7fa2827fc700 1 -- 172.21.15.57:0/980833414 learned_addr learned my addr 172.21.15.57:0/980833414 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa282ffd700 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f209180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa2837fe700 1 -- 172.21.15.57:0/980833414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 msgr2=0x55572f209180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa2837fe700 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f209180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa2837fe700 1 -- 172.21.15.57:0/980833414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 msgr2=0x55572f2096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa2837fe700 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa2837fe700 1 -- 172.21.15.57:0/980833414 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55572ef60c60 con 0x55572f204650 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa2827fc700 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa2837fe700 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f209c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55572f1f9c30 tx=0x7fa274009de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238efb65f1fa8542 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa262ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa262ffd700 1 -- 172.21.15.57:0/980833414 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274011cd0 con 0x55572f204650 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa295b43b80 1 -- 172.21.15.57:0/980833414 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2780850c0 con 0x55572f204650 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa262ffd700 1 -- 172.21.15.57:0/980833414 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa274011e30 con 0x55572f204650 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa262ffd700 1 -- 172.21.15.57:0/980833414 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa27401d3f0 con 0x55572f204650 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa295b43b80 10 client.25924 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa295b43b80 1 -- 172.21.15.57:0/980833414 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55572ef762b0 con 0x55572f204650 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.251+0000 7fa262ffd700 1 -- 172.21.15.57:0/980833414 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2780850c0 con 0x55572f204650 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa262ffd700 1 -- 172.21.15.57:0/980833414 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa27401c070 con 0x55572f204650 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa295b43b80 10 client.25924 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa295b43b80 10 client.25924 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa295b43b80 20 client.25924 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:07.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa295b43b80 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55572f26ccc0 0x55572f26f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa295b43b80 4 client.25924 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa295b43b80 1 -- 172.21.15.57:0/980833414 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55572f145b30 con 0x55572f26ccc0 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa282ffd700 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55572f26ccc0 0x55572f26f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa282ffd700 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55572f26ccc0 0x55572f26f0e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=efb474c521ee57f5 server_cookie=a65ffab4350b4be1 in_seq=0 out_seq=0 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa262ffd700 10 client.25924 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa262ffd700 1 -- 172.21.15.57:0/980833414 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55572f145b30 con 0x55572f26ccc0 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa262ffd700 10 client.25924 handle_command_reply: tid=0 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.252+0000 7fa295b43b80 1 client.25924 shutdown 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 1 -- 172.21.15.57:0/980833414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 msgr2=0x55572f209c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f209c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55572f1f9c30 tx=0x7fa274009de0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 1 -- 172.21.15.57:0/980833414 shutdown_connections 2023-10-08T13:53:07.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55572f26ccc0 0x55572f26f0e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55572f204650 0x55572f209c00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa262ffd700 0 client.25924 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:53:07.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55572f1f8e80 0x55572f209180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 1 --2- 172.21.15.57:0/980833414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55572f1f97a0 0x55572f2096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 1 -- 172.21.15.57:0/980833414 >> 172.21.15.57:0/980833414 conn(0x55572f16e260 msgr2=0x55572f210180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 1 -- 172.21.15.57:0/980833414 shutdown_connections 2023-10-08T13:53:07.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 1 -- 172.21.15.57:0/980833414 wait complete. 2023-10-08T13:53:07.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa295b43b80 20 client.25924 trim_cache size 0 max 16384 2023-10-08T13:53:07.261 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:53:07.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 -- 172.21.15.57:0/1960856497 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa26c0a15f0 msgr2=0x7fa26c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa26c0a15f0 0x7fa26c0a3ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009790 tx=0x7fa27c0096c0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 -- 172.21.15.57:0/1960856497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa284100b80 msgr2=0x7fa2841980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa284100b80 0x7fa2841980a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa27800ec10 tx=0x7fa27800ef20 comp rx=0 tx=0).stop 2023-10-08T13:53:07.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 -- 172.21.15.57:0/1960856497 shutdown_connections 2023-10-08T13:53:07.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2841046a0 0x7fa2841a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa26c0a15f0 0x7fa26c0a3ab0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2841001c0 0x7fa284197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 --2- 172.21.15.57:0/1960856497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa284100b80 0x7fa2841980a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 -- 172.21.15.57:0/1960856497 >> 172.21.15.57:0/1960856497 conn(0x7fa2840749a0 msgr2=0x7fa2840fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 -- 172.21.15.57:0/1960856497 shutdown_connections 2023-10-08T13:53:07.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.253+0000 7fa28995a700 1 -- 172.21.15.57:0/1960856497 wait complete. 2023-10-08T13:53:07.267 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-08T13:53:07.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.422+0000 7f05f8068700 1 -- 172.21.15.57:0/1183273388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f00ff640 msgr2=0x7f05f00ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.422+0000 7f05f8068700 1 --2- 172.21.15.57:0/1183273388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f00ff640 0x7f05f00ffac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009990 tx=0x7f05e0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.427+0000 7f05f8068700 1 -- 172.21.15.57:0/1183273388 shutdown_connections 2023-10-08T13:53:07.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.427+0000 7f05f8068700 1 --2- 172.21.15.57:0/1183273388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f0100000 0x7f05f0068970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.427+0000 7f05f8068700 1 --2- 172.21.15.57:0/1183273388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f0105650 0x7f05f0105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.427+0000 7f05f8068700 1 --2- 172.21.15.57:0/1183273388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f00ff640 0x7f05f00ffac0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.427+0000 7f05f8068700 1 -- 172.21.15.57:0/1183273388 >> 172.21.15.57:0/1183273388 conn(0x7f05f00767c0 msgr2=0x7f05f0076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.427+0000 7f05f8068700 1 -- 172.21.15.57:0/1183273388 shutdown_connections 2023-10-08T13:53:07.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.427+0000 7f05f8068700 1 -- 172.21.15.57:0/1183273388 wait complete. 2023-10-08T13:53:07.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.428+0000 7f05f8068700 1 Processor -- start 2023-10-08T13:53:07.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.428+0000 7f05f8068700 1 -- start start 2023-10-08T13:53:07.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.428+0000 7f05f8068700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f00ff640 0x7f05f0198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.428+0000 7f05f8068700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f0100000 0x7f05f0199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.428+0000 7f05f8068700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f0105650 0x7f05f019d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.428+0000 7f05f8068700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f019dde0 con 0x7f05f00ff640 2023-10-08T13:53:07.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.428+0000 7f05f8068700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f019df50 con 0x7f05f0105650 2023-10-08T13:53:07.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.428+0000 7f05f8068700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f019e110 con 0x7f05f0100000 2023-10-08T13:53:07.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f5603700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f0100000 0x7f05f0199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f5603700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f0100000 0x7f05f0199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58320/0 (socket says 172.21.15.57:58320) 2023-10-08T13:53:07.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f5603700 1 -- 172.21.15.57:0/2304975631 learned_addr learned my addr 172.21.15.57:0/2304975631 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f5e04700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f00ff640 0x7f05f0198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f6605700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f0105650 0x7f05f019d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f5603700 1 -- 172.21.15.57:0/2304975631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f0105650 msgr2=0x7f05f019d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f5603700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f0105650 0x7f05f019d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f5603700 1 -- 172.21.15.57:0/2304975631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f00ff640 msgr2=0x7f05f0198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f5603700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f00ff640 0x7f05f0198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f5603700 1 -- 172.21.15.57:0/2304975631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00095f0 con 0x7f05f0100000 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05f5603700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f0100000 0x7f05f0199400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05e0005770 tx=0x7f05e00057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.429+0000 7f05e6ffd700 1 -- 172.21.15.57:0/2304975631 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e001d070 con 0x7f05f0100000 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.430+0000 7f05e6ffd700 1 -- 172.21.15.57:0/2304975631 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05e0005da0 con 0x7f05f0100000 2023-10-08T13:53:07.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.430+0000 7f05f8068700 1 -- 172.21.15.57:0/2304975631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f019e390 con 0x7f05f0100000 2023-10-08T13:53:07.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.430+0000 7f05e6ffd700 1 -- 172.21.15.57:0/2304975631 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0022e70 con 0x7f05f0100000 2023-10-08T13:53:07.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.430+0000 7f05f8068700 1 -- 172.21.15.57:0/2304975631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f01a5c30 con 0x7f05f0100000 2023-10-08T13:53:07.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.432+0000 7f05e6ffd700 1 -- 172.21.15.57:0/2304975631 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f05e00173f0 con 0x7f05f0100000 2023-10-08T13:53:07.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.433+0000 7f05e6ffd700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f05dc0a15f0 0x7f05dc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.433+0000 7f05e6ffd700 1 -- 172.21.15.57:0/2304975631 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f05e00ce650 con 0x7f05f0100000 2023-10-08T13:53:07.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.433+0000 7f05f5e04700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f05dc0a15f0 0x7f05dc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.433+0000 7f05f5e04700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f05dc0a15f0 0x7f05dc0a3ab0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009be0 tx=0x7f05ec009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f0604251b80 1 Processor -- start 2023-10-08T13:53:07.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f0604251b80 1 -- start start 2023-10-08T13:53:07.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f0604251b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322924b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f0604251b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x563229256110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f0604251b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x563229258a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f0604251b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632291b4460 con 0x56322924b7a0 2023-10-08T13:53:07.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f0604251b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632291a5c20 con 0x563229256650 2023-10-08T13:53:07.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f0604251b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632291a3540 con 0x56322924ae80 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f05f5e04700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322924b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f05f5603700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x563229256110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f05f5e04700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322924b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58328/0 (socket says 172.21.15.57:58328) 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.434+0000 7f05f5e04700 1 -- 172.21.15.57:0/3411263028 learned_addr learned my addr 172.21.15.57:0/3411263028 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05f6605700 1 --2- 172.21.15.57:0/3411263028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x563229258a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05f5603700 1 -- 172.21.15.57:0/3411263028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 msgr2=0x56322924b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05f5603700 1 --2- 172.21.15.57:0/3411263028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322924b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05f5603700 1 -- 172.21.15.57:0/3411263028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 msgr2=0x563229258a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05f5603700 1 --2- 172.21.15.57:0/3411263028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x563229258a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05f5603700 1 -- 172.21.15.57:0/3411263028 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563228fc9de0 con 0x56322924b7a0 2023-10-08T13:53:07.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05f5e04700 1 --2- 172.21.15.57:0/3411263028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322924b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05f6605700 1 --2- 172.21.15.57:0/3411263028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x563229258a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05f5603700 1 --2- 172.21.15.57:0/3411263028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x563229256110 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f05e00cba40 tx=0x7f05e0005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af12248c63f536e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05dbfff700 1 -- 172.21.15.57:0/3411263028 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e009b360 con 0x56322924b7a0 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05dbfff700 1 -- 172.21.15.57:0/3411263028 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05e009f4b0 con 0x56322924b7a0 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.435+0000 7f05dbfff700 1 -- 172.21.15.57:0/3411263028 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0022570 con 0x56322924b7a0 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.436+0000 7f0604251b80 1 -- 172.21.15.57:0/3411263028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 msgr2=0x563229256110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.436+0000 7f0604251b80 1 --2- 172.21.15.57:0/3411263028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x563229256110 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f05e00cba40 tx=0x7f05e0005f10 comp rx=0 tx=0).stop 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.436+0000 7f0604251b80 1 -- 172.21.15.57:0/3411263028 shutdown_connections 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.436+0000 7f0604251b80 1 --2- 172.21.15.57:0/3411263028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x563229258a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.436+0000 7f0604251b80 1 --2- 172.21.15.57:0/3411263028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322924b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.436+0000 7f0604251b80 1 --2- 172.21.15.57:0/3411263028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x563229256110 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.436+0000 7f0604251b80 1 -- 172.21.15.57:0/3411263028 >> 172.21.15.57:0/3411263028 conn(0x5632291c0260 msgr2=0x5632292472d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.436+0000 7f0604251b80 1 -- 172.21.15.57:0/3411263028 shutdown_connections 2023-10-08T13:53:07.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.436+0000 7f0604251b80 1 -- 172.21.15.57:0/3411263028 wait complete. 2023-10-08T13:53:07.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f0604251b80 1 -- start start 2023-10-08T13:53:07.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f0604251b80 1 Processor -- start 2023-10-08T13:53:07.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f0604251b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56322924ae80 0x563229256a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f0604251b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x563229256f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f0604251b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563229256650 0x563229261ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f0604251b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322919ee30 con 0x56322924b7a0 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f0604251b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322913b710 con 0x56322924ae80 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f0604251b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322913b3b0 con 0x563229256650 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f0604251b80 10 client.0 _ll_register_callbacks cb 0x563228f98800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f05f5603700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x563229256f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f05f6605700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563229256650 0x563229261ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f05f6605700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563229256650 0x563229261ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58338/0 (socket says 172.21.15.57:58338) 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f05f6605700 1 -- 172.21.15.57:0/1831232346 learned_addr learned my addr 172.21.15.57:0/1831232346 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.437+0000 7f05f5e04700 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56322924ae80 0x563229256a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05f6605700 1 -- 172.21.15.57:0/1831232346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56322924ae80 msgr2=0x563229256a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05f6605700 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56322924ae80 0x563229256a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05f6605700 1 -- 172.21.15.57:0/1831232346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 msgr2=0x563229256f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05f6605700 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x563229256f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05f6605700 1 -- 172.21.15.57:0/1831232346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec017070 con 0x563229256650 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05f5603700 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x563229256f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05f6605700 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563229256650 0x563229261ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f05e8004090 tx=0x7f05e80043a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf1abe346d12a9e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05d97fa700 1 -- 172.21.15.57:0/1831232346 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8007710 con 0x563229256650 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f0604251b80 1 -- 172.21.15.57:0/1831232346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563229166eb0 con 0x563229256650 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05d97fa700 1 -- 172.21.15.57:0/1831232346 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f05e8007870 con 0x563229256650 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05d97fa700 1 -- 172.21.15.57:0/1831232346 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05e8007a30 con 0x563229256650 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f0604251b80 10 client.25936 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:07.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f0604251b80 1 -- 172.21.15.57:0/1831232346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f05f01a5c30 con 0x563229256650 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05d97fa700 1 -- 172.21.15.57:0/1831232346 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563229166eb0 con 0x563229256650 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.438+0000 7f05d97fa700 1 -- 172.21.15.57:0/1831232346 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f05e8014070 con 0x563229256650 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.439+0000 7f0604251b80 10 client.25936 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.439+0000 7f0604251b80 10 client.25936 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.439+0000 7f0604251b80 20 client.25936 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.439+0000 7f0604251b80 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x563229265e30 0x563229268250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.439+0000 7f0604251b80 4 client.25936 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.439+0000 7f0604251b80 1 -- 172.21.15.57:0/1831232346 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563229197e10 con 0x563229265e30 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.439+0000 7f05f5e04700 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x563229265e30 0x563229268250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.439+0000 7f05d97fa700 10 client.25936 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.439+0000 7f05f5e04700 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x563229265e30 0x563229268250 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77c5a4d9427eb679 server_cookie=a560e4a8e3070908 in_seq=0 out_seq=0 2023-10-08T13:53:07.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f05d97fa700 1 -- 172.21.15.57:0/1831232346 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563229197e10 con 0x563229265e30 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f05d97fa700 10 client.25936 handle_command_reply: tid=0 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 client.25936 shutdown 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 -- 172.21.15.57:0/1831232346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563229256650 msgr2=0x563229261ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563229256650 0x563229261ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f05e8004090 tx=0x7f05e80043a0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 -- 172.21.15.57:0/1831232346 shutdown_connections 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x563229265e30 0x563229268250 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563229256650 0x563229261ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f05d97fa700 0 client.25936 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56322924ae80 0x563229256a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 --2- 172.21.15.57:0/1831232346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x563229256f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 -- 172.21.15.57:0/1831232346 >> 172.21.15.57:0/1831232346 conn(0x5632291c0260 msgr2=0x563229246ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 -- 172.21.15.57:0/1831232346 shutdown_connections 2023-10-08T13:53:07.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 1 -- 172.21.15.57:0/1831232346 wait complete. 2023-10-08T13:53:07.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.440+0000 7f0604251b80 20 client.25936 trim_cache size 0 max 16384 2023-10-08T13:53:07.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.454+0000 7f0604251b80 1 Processor -- start 2023-10-08T13:53:07.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.454+0000 7f0604251b80 1 -- start start 2023-10-08T13:53:07.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.454+0000 7f0604251b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322925b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.454+0000 7f0604251b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322924fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.454+0000 7f0604251b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x5632292502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.454+0000 7f0604251b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56322901f920 con 0x56322924b7a0 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.454+0000 7f0604251b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563229262980 con 0x563229256650 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.454+0000 7f0604251b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563229241570 con 0x56322924ae80 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f5603700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322924fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f5603700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322924fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57532/0 (socket says 172.21.15.57:57532) 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f5e04700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322925b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f5603700 1 -- 172.21.15.57:0/3272892451 learned_addr learned my addr 172.21.15.57:0/3272892451 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f6605700 1 --2- 172.21.15.57:0/3272892451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x5632292502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f5603700 1 -- 172.21.15.57:0/3272892451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 msgr2=0x56322925b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f5603700 1 --2- 172.21.15.57:0/3272892451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322925b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f5603700 1 -- 172.21.15.57:0/3272892451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 msgr2=0x5632292502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f5603700 1 --2- 172.21.15.57:0/3272892451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x5632292502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f5603700 1 -- 172.21.15.57:0/3272892451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563228fb2c60 con 0x56322924b7a0 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f6605700 1 --2- 172.21.15.57:0/3272892451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x5632292502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05f5603700 1 --2- 172.21.15.57:0/3272892451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322924fd60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f05e00cba70 tx=0x7f05e001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8c66f663474d4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05e4ff9700 1 -- 172.21.15.57:0/3272892451 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e009b360 con 0x56322924b7a0 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05e4ff9700 1 -- 172.21.15.57:0/3272892451 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05e00a3660 con 0x56322924b7a0 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.455+0000 7f05e4ff9700 1 -- 172.21.15.57:0/3272892451 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e009b5c0 con 0x56322924b7a0 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 -- 172.21.15.57:0/3272892451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 msgr2=0x56322924fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 --2- 172.21.15.57:0/3272892451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322924fd60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f05e00cba70 tx=0x7f05e001a040 comp rx=0 tx=0).stop 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 -- 172.21.15.57:0/3272892451 shutdown_connections 2023-10-08T13:53:07.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 --2- 172.21.15.57:0/3272892451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x5632292502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 --2- 172.21.15.57:0/3272892451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322925b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 --2- 172.21.15.57:0/3272892451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322924fd60 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 -- 172.21.15.57:0/3272892451 >> 172.21.15.57:0/3272892451 conn(0x5632291c0260 msgr2=0x563229256a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 -- 172.21.15.57:0/3272892451 shutdown_connections 2023-10-08T13:53:07.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 -- 172.21.15.57:0/3272892451 wait complete. 2023-10-08T13:53:07.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 -- start start 2023-10-08T13:53:07.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.456+0000 7f0604251b80 1 Processor -- start 2023-10-08T13:53:07.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f0604251b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322925b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f0604251b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322925b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f0604251b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x5632292b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f0604251b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632292b18d0 con 0x56322924b7a0 2023-10-08T13:53:07.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f0604251b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632292b1a40 con 0x563229256650 2023-10-08T13:53:07.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f0604251b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632292b1c00 con 0x56322924ae80 2023-10-08T13:53:07.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f0604251b80 10 client.0 _ll_register_callbacks cb 0x7f05e8009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:07.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f5603700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322925b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f5603700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322925b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57538/0 (socket says 172.21.15.57:57538) 2023-10-08T13:53:07.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f5603700 1 -- 172.21.15.57:0/720146763 learned_addr learned my addr 172.21.15.57:0/720146763 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f5e04700 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322925b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f6605700 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x5632292b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f5603700 1 -- 172.21.15.57:0/720146763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 msgr2=0x56322925b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f5603700 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322925b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f5603700 1 -- 172.21.15.57:0/720146763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 msgr2=0x5632292b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f5603700 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x5632292b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f5603700 1 -- 172.21.15.57:0/720146763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563228fc9de0 con 0x56322924b7a0 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f6605700 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x5632292b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05f5603700 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322925b6d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f05e0095fd0 tx=0x7f05e0005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11aec8240cf09a32 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.457+0000 7f05d97fa700 1 -- 172.21.15.57:0/720146763 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0022630 con 0x56322924b7a0 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f05d97fa700 1 -- 172.21.15.57:0/720146763 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f05e00cd030 con 0x56322924b7a0 2023-10-08T13:53:07.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f0604251b80 1 -- 172.21.15.57:0/720146763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05e009f4b0 con 0x56322924b7a0 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f05d97fa700 1 -- 172.21.15.57:0/720146763 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05e0022790 con 0x56322924b7a0 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f0604251b80 10 client.16344 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f0604251b80 1 -- 172.21.15.57:0/720146763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563228ff0250 con 0x56322924b7a0 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f05d97fa700 1 -- 172.21.15.57:0/720146763 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05e009f4b0 con 0x56322924b7a0 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f05d97fa700 1 -- 172.21.15.57:0/720146763 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f05e00a31d0 con 0x56322924b7a0 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f0604251b80 10 client.16344 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f0604251b80 10 client.16344 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f0604251b80 20 client.16344 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f0604251b80 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x5632292ba170 0x5632292bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f0604251b80 4 client.16344 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:53:07.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f0604251b80 1 -- 172.21.15.57:0/720146763 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563229197b30 con 0x5632292ba170 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.458+0000 7f05f5e04700 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x5632292ba170 0x5632292bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f05d97fa700 10 client.16344 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f05f5e04700 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x5632292ba170 0x5632292bc560 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec1350769fc5aa0b server_cookie=ed0b378d42aefb0a in_seq=0 out_seq=0 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f05d97fa700 1 -- 172.21.15.57:0/720146763 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563229197b30 con 0x5632292ba170 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f05d97fa700 10 client.16344 handle_command_reply: tid=0 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 client.16344 shutdown 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 -- 172.21.15.57:0/720146763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 msgr2=0x56322925b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322925b6d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f05e0095fd0 tx=0x7f05e0005f10 comp rx=0 tx=0).stop 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 -- 172.21.15.57:0/720146763 shutdown_connections 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x5632292ba170 0x5632292bc560 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229256650 0x5632292b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56322924ae80 0x56322925b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 --2- 172.21.15.57:0/720146763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56322924b7a0 0x56322925b6d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f05d97fa700 0 client.16344 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:53:07.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 -- 172.21.15.57:0/720146763 >> 172.21.15.57:0/720146763 conn(0x5632291c0260 msgr2=0x563229250b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 -- 172.21.15.57:0/720146763 shutdown_connections 2023-10-08T13:53:07.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 1 -- 172.21.15.57:0/720146763 wait complete. 2023-10-08T13:53:07.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.459+0000 7f0604251b80 20 client.16344 trim_cache size 0 max 16384 2023-10-08T13:53:07.468 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:53:07.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 -- 172.21.15.57:0/2304975631 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f05dc0a15f0 msgr2=0x7f05dc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f05dc0a15f0 0x7f05dc0a3ab0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009be0 tx=0x7f05ec009260 comp rx=0 tx=0).stop 2023-10-08T13:53:07.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 -- 172.21.15.57:0/2304975631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f0100000 msgr2=0x7f05f0199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f0100000 0x7f05f0199400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05e0005770 tx=0x7f05e00057a0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 -- 172.21.15.57:0/2304975631 shutdown_connections 2023-10-08T13:53:07.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f05dc0a15f0 0x7f05dc0a3ab0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f0105650 0x7f05f019d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05f00ff640 0x7f05f0198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 --2- 172.21.15.57:0/2304975631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f0100000 0x7f05f0199400 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 -- 172.21.15.57:0/2304975631 >> 172.21.15.57:0/2304975631 conn(0x7f05f00767c0 msgr2=0x7f05f0103150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 -- 172.21.15.57:0/2304975631 shutdown_connections 2023-10-08T13:53:07.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.460+0000 7f05f8068700 1 -- 172.21.15.57:0/2304975631 wait complete. 2023-10-08T13:53:07.473 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-08T13:53:07.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.613+0000 7f59e9fa7700 1 -- 172.21.15.57:0/286756257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e41051c0 msgr2=0x7f59e4105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.613+0000 7f59e9fa7700 1 --2- 172.21.15.57:0/286756257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e41051c0 0x7f59e4105640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009920 tx=0x7f59cc009c30 comp rx=0 tx=0).stop 2023-10-08T13:53:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.613+0000 7f59e9fa7700 1 -- 172.21.15.57:0/286756257 shutdown_connections 2023-10-08T13:53:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.613+0000 7f59e9fa7700 1 --2- 172.21.15.57:0/286756257 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59e4105b80 0x7f59e410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.613+0000 7f59e9fa7700 1 --2- 172.21.15.57:0/286756257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59e4067ce0 0x7f59e40680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.613+0000 7f59e9fa7700 1 --2- 172.21.15.57:0/286756257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e41051c0 0x7f59e4105640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.613+0000 7f59e9fa7700 1 -- 172.21.15.57:0/286756257 >> 172.21.15.57:0/286756257 conn(0x7f59e4076780 msgr2=0x7f59e4076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.613+0000 7f59e9fa7700 1 -- 172.21.15.57:0/286756257 shutdown_connections 2023-10-08T13:53:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.613+0000 7f59e9fa7700 1 -- 172.21.15.57:0/286756257 wait complete. 2023-10-08T13:53:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.613+0000 7f59e9fa7700 1 Processor -- start 2023-10-08T13:53:07.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e9fa7700 1 -- start start 2023-10-08T13:53:07.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e9fa7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4067ce0 0x7f59e40ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e9fa7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59e41051c0 0x7f59e40ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e9fa7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59e4105b80 0x7f59e4100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e9fa7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e41a50c0 con 0x7f59e4105b80 2023-10-08T13:53:07.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e9fa7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e41a5230 con 0x7f59e41051c0 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e9fa7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e41a53f0 con 0x7f59e4067ce0 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59e4105b80 0x7f59e4100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4067ce0 0x7f59e40ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59e4105b80 0x7f59e4100130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57554/0 (socket says 172.21.15.57:57554) 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4067ce0 0x7f59e40ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58362/0 (socket says 172.21.15.57:58362) 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e3fff700 1 -- 172.21.15.57:0/2970941075 learned_addr learned my addr 172.21.15.57:0/2970941075 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e2ffd700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59e41051c0 0x7f59e40ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e3fff700 1 -- 172.21.15.57:0/2970941075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4067ce0 msgr2=0x7f59e40ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e3fff700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4067ce0 0x7f59e40ff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e3fff700 1 -- 172.21.15.57:0/2970941075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59e41051c0 msgr2=0x7f59e40ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e3fff700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59e41051c0 0x7f59e40ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.614+0000 7f59e3fff700 1 -- 172.21.15.57:0/2970941075 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc009520 con 0x7f59e4105b80 2023-10-08T13:53:07.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.615+0000 7f59e3fff700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59e4105b80 0x7f59e4100130 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c110 tx=0x7f59dc00c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.615+0000 7f59e0ff9700 1 -- 172.21.15.57:0/2970941075 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc017070 con 0x7f59e4105b80 2023-10-08T13:53:07.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.615+0000 7f59e9fa7700 1 -- 172.21.15.57:0/2970941075 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e41a5670 con 0x7f59e4105b80 2023-10-08T13:53:07.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.615+0000 7f59e0ff9700 1 -- 172.21.15.57:0/2970941075 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59dc00e050 con 0x7f59e4105b80 2023-10-08T13:53:07.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.615+0000 7f59e0ff9700 1 -- 172.21.15.57:0/2970941075 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0078f0 con 0x7f59e4105b80 2023-10-08T13:53:07.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.615+0000 7f59e9fa7700 1 -- 172.21.15.57:0/2970941075 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e41a5bd0 con 0x7f59e4105b80 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.617+0000 7f59e0ff9700 1 -- 172.21.15.57:0/2970941075 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f59dc007a50 con 0x7f59e4105b80 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.618+0000 7f59e0ff9700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f59d00a15f0 0x7f59d00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.618+0000 7f59e0ff9700 1 -- 172.21.15.57:0/2970941075 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f59dc0ccdb0 con 0x7f59e4105b80 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.618+0000 7f59e37fe700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f59d00a15f0 0x7f59d00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.618+0000 7f59e37fe700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f59d00a15f0 0x7f59d00a3ab0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009c60 tx=0x7f59d4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.618+0000 7f59f6190b80 1 Processor -- start 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59f6190b80 1 -- start start 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409653260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 0x55e40965e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e409660a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59f6190b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4095bc460 con 0x55e40965e650 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59f6190b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4095adc20 con 0x55e409652e80 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59f6190b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4095ab540 con 0x55e4096537a0 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e409660a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409653260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 0x55e40965e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409653260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:58984/0 (socket says 172.21.15.57:58984) 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e409660a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57562/0 (socket says 172.21.15.57:57562) 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 0x55e40965e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58372/0 (socket says 172.21.15.57:58372) 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e37fe700 1 -- 172.21.15.57:0/2071716763 learned_addr learned my addr 172.21.15.57:0/2071716763 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e37fe700 1 -- 172.21.15.57:0/2071716763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 msgr2=0x55e40965e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e37fe700 1 --2- 172.21.15.57:0/2071716763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 0x55e40965e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e37fe700 1 -- 172.21.15.57:0/2071716763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 msgr2=0x55e409660a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e37fe700 1 --2- 172.21.15.57:0/2071716763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e409660a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e37fe700 1 -- 172.21.15.57:0/2071716763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4093d1de0 con 0x55e409652e80 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e3fff700 1 --2- 172.21.15.57:0/2071716763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e409660a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.619+0000 7f59e37fe700 1 --2- 172.21.15.57:0/2071716763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409653260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f59d400e510 tx=0x7f59d4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20bc822c6d1ac998 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59d9ffb700 1 -- 172.21.15.57:0/2071716763 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d400ee30 con 0x55e409652e80 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59d9ffb700 1 -- 172.21.15.57:0/2071716763 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59d4021070 con 0x55e409652e80 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59d9ffb700 1 -- 172.21.15.57:0/2071716763 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4025830 con 0x55e409652e80 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59f6190b80 1 -- 172.21.15.57:0/2071716763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 msgr2=0x55e409653260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59f6190b80 1 --2- 172.21.15.57:0/2071716763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409653260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f59d400e510 tx=0x7f59d4014000 comp rx=0 tx=0).stop 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59f6190b80 1 -- 172.21.15.57:0/2071716763 shutdown_connections 2023-10-08T13:53:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59f6190b80 1 --2- 172.21.15.57:0/2071716763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e409660a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59f6190b80 1 --2- 172.21.15.57:0/2071716763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409653260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59f6190b80 1 --2- 172.21.15.57:0/2071716763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 0x55e40965e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59f6190b80 1 -- 172.21.15.57:0/2071716763 >> 172.21.15.57:0/2071716763 conn(0x55e4095c8260 msgr2=0x55e40964f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59f6190b80 1 -- 172.21.15.57:0/2071716763 shutdown_connections 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.620+0000 7f59f6190b80 1 -- 172.21.15.57:0/2071716763 wait complete. 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59f6190b80 1 -- start start 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59f6190b80 1 Processor -- start 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e409652e80 0x55e40965dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4096537a0 0x55e40965b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e40965e650 0x55e40965b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59f6190b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4095a6e30 con 0x55e409652e80 2023-10-08T13:53:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59f6190b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409543710 con 0x55e4096537a0 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59f6190b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4095433b0 con 0x55e40965e650 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59f6190b80 10 client.0 _ll_register_callbacks cb 0x55e4093a0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e409652e80 0x55e40965dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e409652e80 0x55e40965dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57564/0 (socket says 172.21.15.57:57564) 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59e37fe700 1 -- 172.21.15.57:0/3537792643 learned_addr learned my addr 172.21.15.57:0/3537792643 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59e2ffd700 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4096537a0 0x55e40965b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.621+0000 7f59e3fff700 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e40965e650 0x55e40965b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59e37fe700 1 -- 172.21.15.57:0/3537792643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e40965e650 msgr2=0x55e40965b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59e37fe700 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e40965e650 0x55e40965b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59e37fe700 1 -- 172.21.15.57:0/3537792643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4096537a0 msgr2=0x55e40965b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59e37fe700 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4096537a0 0x55e40965b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59e37fe700 1 -- 172.21.15.57:0/3537792643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e41a5bd0 con 0x55e409652e80 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59e37fe700 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e409652e80 0x55e40965dfc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f59d4005950 tx=0x7f59d401dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a59026929f81245 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59c2ffd700 1 -- 172.21.15.57:0/3537792643 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d400ee70 con 0x55e409652e80 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59f6190b80 1 -- 172.21.15.57:0/3537792643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e40956eeb0 con 0x55e409652e80 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59c2ffd700 1 -- 172.21.15.57:0/3537792643 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f59d4005250 con 0x55e409652e80 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59c2ffd700 1 -- 172.21.15.57:0/3537792643 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59d4005410 con 0x55e409652e80 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59f6190b80 10 client.16356 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59f6190b80 1 -- 172.21.15.57:0/3537792643 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f59d4017070 con 0x55e409652e80 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.622+0000 7f59c2ffd700 1 -- 172.21.15.57:0/3537792643 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e40956eeb0 con 0x55e409652e80 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.623+0000 7f59c2ffd700 1 -- 172.21.15.57:0/3537792643 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f59d4004b60 con 0x55e409652e80 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.623+0000 7f59f6190b80 10 client.16356 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.623+0000 7f59f6190b80 10 client.16356 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.623+0000 7f59f6190b80 20 client.16356 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.623+0000 7f59f6190b80 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e40966e0a0 0x55e4096704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.623+0000 7f59f6190b80 4 client.16356 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.623+0000 7f59f6190b80 1 -- 172.21.15.57:0/3537792643 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e40959fe10 con 0x55e40966e0a0 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.623+0000 7f59e2ffd700 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e40966e0a0 0x55e4096704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.623+0000 7f59e2ffd700 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e40966e0a0 0x55e4096704c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16a966173c35e231 server_cookie=b3a1375a6bf0151b in_seq=0 out_seq=0 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.623+0000 7f59c2ffd700 10 client.16356 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.624+0000 7f59c2ffd700 1 -- 172.21.15.57:0/3537792643 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e40959fe10 con 0x55e40966e0a0 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.624+0000 7f59c2ffd700 10 client.16356 handle_command_reply: tid=0 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.624+0000 7f59f6190b80 1 client.16356 shutdown 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.624+0000 7f59f6190b80 1 -- 172.21.15.57:0/3537792643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e409652e80 msgr2=0x55e40965dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.624+0000 7f59f6190b80 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e409652e80 0x55e40965dfc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f59d4005950 tx=0x7f59d401dde0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.624+0000 7f59f6190b80 1 -- 172.21.15.57:0/3537792643 shutdown_connections 2023-10-08T13:53:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.625+0000 7f59f6190b80 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e40966e0a0 0x55e4096704c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.625+0000 7f59f6190b80 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e40965e650 0x55e40965b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.625+0000 7f59c2ffd700 0 client.16356 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:53:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.625+0000 7f59f6190b80 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e409652e80 0x55e40965dfc0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.625+0000 7f59f6190b80 1 --2- 172.21.15.57:0/3537792643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4096537a0 0x55e40965b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.625+0000 7f59f6190b80 1 -- 172.21.15.57:0/3537792643 >> 172.21.15.57:0/3537792643 conn(0x55e4095c8260 msgr2=0x55e40964eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.625+0000 7f59f6190b80 1 -- 172.21.15.57:0/3537792643 shutdown_connections 2023-10-08T13:53:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.625+0000 7f59f6190b80 1 -- 172.21.15.57:0/3537792643 wait complete. 2023-10-08T13:53:07.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.625+0000 7f59f6190b80 20 client.16356 trim_cache size 0 max 16384 2023-10-08T13:53:07.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.639+0000 7f59f6190b80 1 Processor -- start 2023-10-08T13:53:07.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.639+0000 7f59f6190b80 1 -- start start 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.639+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e409652e80 0x55e40965b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.639+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4096537a0 0x55e4096651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.639+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e40965e650 0x55e409656650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.639+0000 7f59f6190b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409427920 con 0x55e4096537a0 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59f6190b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409649570 con 0x55e40965e650 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59f6190b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409654970 con 0x55e409652e80 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4096537a0 0x55e4096651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e409652e80 0x55e40965b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4096537a0 0x55e4096651a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57574/0 (socket says 172.21.15.57:57574) 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e2ffd700 1 -- 172.21.15.57:0/1157822972 learned_addr learned my addr 172.21.15.57:0/1157822972 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e3fff700 1 --2- 172.21.15.57:0/1157822972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e40965e650 0x55e409656650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e2ffd700 1 -- 172.21.15.57:0/1157822972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e409652e80 msgr2=0x55e40965b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e2ffd700 1 --2- 172.21.15.57:0/1157822972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e409652e80 0x55e40965b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e2ffd700 1 -- 172.21.15.57:0/1157822972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e40965e650 msgr2=0x55e409656650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e2ffd700 1 --2- 172.21.15.57:0/1157822972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e40965e650 0x55e409656650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e2ffd700 1 -- 172.21.15.57:0/1157822972 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4093bac60 con 0x55e4096537a0 2023-10-08T13:53:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e3fff700 1 --2- 172.21.15.57:0/1157822972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e40965e650 0x55e409656650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59e2ffd700 1 --2- 172.21.15.57:0/1157822972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4096537a0 0x55e4096651a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009230 tx=0x7f59cc005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ece77be4ef6e2e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59da7fc700 1 -- 172.21.15.57:0/1157822972 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc028db0 con 0x55e4096537a0 2023-10-08T13:53:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59da7fc700 1 -- 172.21.15.57:0/1157822972 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59cc01d040 con 0x55e4096537a0 2023-10-08T13:53:07.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.640+0000 7f59da7fc700 1 -- 172.21.15.57:0/1157822972 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00b7c0 con 0x55e4096537a0 2023-10-08T13:53:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.641+0000 7f59f6190b80 1 -- 172.21.15.57:0/1157822972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4096537a0 msgr2=0x55e4096651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.641+0000 7f59f6190b80 1 --2- 172.21.15.57:0/1157822972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4096537a0 0x55e4096651a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009230 tx=0x7f59cc005e00 comp rx=0 tx=0).stop 2023-10-08T13:53:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.641+0000 7f59f6190b80 1 -- 172.21.15.57:0/1157822972 shutdown_connections 2023-10-08T13:53:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.641+0000 7f59f6190b80 1 --2- 172.21.15.57:0/1157822972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e40965e650 0x55e409656650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.641+0000 7f59f6190b80 1 --2- 172.21.15.57:0/1157822972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e409652e80 0x55e40965b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.641+0000 7f59f6190b80 1 --2- 172.21.15.57:0/1157822972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4096537a0 0x55e4096651a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.641+0000 7f59f6190b80 1 -- 172.21.15.57:0/1157822972 >> 172.21.15.57:0/1157822972 conn(0x55e4095c8260 msgr2=0x55e40964e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.641+0000 7f59f6190b80 1 -- 172.21.15.57:0/1157822972 shutdown_connections 2023-10-08T13:53:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.641+0000 7f59f6190b80 1 -- 172.21.15.57:0/1157822972 wait complete. 2023-10-08T13:53:07.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.641+0000 7f59f6190b80 1 -- start start 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59f6190b80 1 Processor -- start 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409662fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 0x55e4096634e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59f6190b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e4096baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59f6190b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4096bb3e0 con 0x55e40965e650 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59f6190b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4096bb550 con 0x55e409652e80 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59f6190b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4096bb710 con 0x55e4096537a0 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59f6190b80 10 client.0 _ll_register_callbacks cb 0x7f59dc0092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e4096baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59e2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 0x55e4096634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e4096baea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57582/0 (socket says 172.21.15.57:57582) 2023-10-08T13:53:07.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59e3fff700 1 -- 172.21.15.57:0/2629371305 learned_addr learned my addr 172.21.15.57:0/2629371305 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.642+0000 7f59e37fe700 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409662fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59e3fff700 1 -- 172.21.15.57:0/2629371305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 msgr2=0x55e4096634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59e3fff700 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 0x55e4096634e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59e3fff700 1 -- 172.21.15.57:0/2629371305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 msgr2=0x55e409662fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59e3fff700 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409662fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59e3fff700 1 -- 172.21.15.57:0/2629371305 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc021070 con 0x55e40965e650 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59e2ffd700 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 0x55e4096634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59e37fe700 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409662fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59e3fff700 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e4096baea0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0062a0 tx=0x7f59dc01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73830d8f74096fa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59c2ffd700 1 -- 172.21.15.57:0/2629371305 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc09dbe0 con 0x55e40965e650 2023-10-08T13:53:07.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59c2ffd700 1 -- 172.21.15.57:0/2629371305 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f59dc0a1e20 con 0x55e40965e650 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59f6190b80 1 -- 172.21.15.57:0/2629371305 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59d4021070 con 0x55e40965e650 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59c2ffd700 1 -- 172.21.15.57:0/2629371305 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59dc0192c0 con 0x55e40965e650 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59f6190b80 10 client.16362 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59f6190b80 1 -- 172.21.15.57:0/2629371305 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f59cc00fe70 con 0x55e40965e650 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59c2ffd700 1 -- 172.21.15.57:0/2629371305 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59dc063020 con 0x55e40965e650 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.643+0000 7f59c2ffd700 1 -- 172.21.15.57:0/2629371305 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f59dc098f80 con 0x55e40965e650 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 10 client.16362 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 10 client.16362 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 20 client.16362 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e4096c2f80 0x55e4096c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 4 client.16362 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:53:07.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 1 -- 172.21.15.57:0/2629371305 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e40959fb30 con 0x55e4096c2f80 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59e37fe700 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e4096c2f80 0x55e4096c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59c2ffd700 10 client.16362 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59e37fe700 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e4096c2f80 0x55e4096c53a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9ac49bbb58c0486 server_cookie=e12399a9cbefe091 in_seq=0 out_seq=0 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59c2ffd700 1 -- 172.21.15.57:0/2629371305 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e40959fb30 con 0x55e4096c2f80 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59c2ffd700 10 client.16362 handle_command_reply: tid=0 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 1 client.16362 shutdown 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 1 -- 172.21.15.57:0/2629371305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 msgr2=0x55e4096baea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e4096baea0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0062a0 tx=0x7f59dc01e040 comp rx=0 tx=0).stop 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 1 -- 172.21.15.57:0/2629371305 shutdown_connections 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.644+0000 7f59f6190b80 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e4096c2f80 0x55e4096c53a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59f6190b80 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e40965e650 0x55e4096baea0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59f6190b80 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e409652e80 0x55e409662fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59f6190b80 1 --2- 172.21.15.57:0/2629371305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4096537a0 0x55e4096634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59c2ffd700 0 client.16362 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:53:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59f6190b80 1 -- 172.21.15.57:0/2629371305 >> 172.21.15.57:0/2629371305 conn(0x55e4095c8260 msgr2=0x55e409656650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59f6190b80 1 -- 172.21.15.57:0/2629371305 shutdown_connections 2023-10-08T13:53:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59f6190b80 1 -- 172.21.15.57:0/2629371305 wait complete. 2023-10-08T13:53:07.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59f6190b80 20 client.16362 trim_cache size 0 max 16384 2023-10-08T13:53:07.653 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:53:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 -- 172.21.15.57:0/2970941075 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f59d00a15f0 msgr2=0x7f59d00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f59d00a15f0 0x7f59d00a3ab0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009c60 tx=0x7f59d4009260 comp rx=0 tx=0).stop 2023-10-08T13:53:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 -- 172.21.15.57:0/2970941075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59e4105b80 msgr2=0x7f59e4100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59e4105b80 0x7f59e4100130 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c110 tx=0x7f59dc00c420 comp rx=0 tx=0).stop 2023-10-08T13:53:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 -- 172.21.15.57:0/2970941075 shutdown_connections 2023-10-08T13:53:07.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59e4105b80 0x7f59e4100130 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4067ce0 0x7f59e40ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f59d00a15f0 0x7f59d00a3ab0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 --2- 172.21.15.57:0/2970941075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59e41051c0 0x7f59e40ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 -- 172.21.15.57:0/2970941075 >> 172.21.15.57:0/2970941075 conn(0x7f59e4076780 msgr2=0x7f59e40fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 -- 172.21.15.57:0/2970941075 shutdown_connections 2023-10-08T13:53:07.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.645+0000 7f59e9fa7700 1 -- 172.21.15.57:0/2970941075 wait complete. 2023-10-08T13:53:07.658 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-08T13:53:07.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.789+0000 7f3b30ed4700 1 -- 172.21.15.57:0/1676634179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b2c0ff620 msgr2=0x7f3b2c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.789+0000 7f3b30ed4700 1 --2- 172.21.15.57:0/1676634179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b2c0ff620 0x7f3b2c0ffaa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3b140098f0 tx=0x7f3b14009c00 comp rx=0 tx=0).stop 2023-10-08T13:53:07.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.789+0000 7f3b30ed4700 1 -- 172.21.15.57:0/1676634179 shutdown_connections 2023-10-08T13:53:07.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.789+0000 7f3b30ed4700 1 --2- 172.21.15.57:0/1676634179 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c0fffe0 0x7f3b2c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.789+0000 7f3b30ed4700 1 --2- 172.21.15.57:0/1676634179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b2c1056b0 0x7f3b2c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.789+0000 7f3b30ed4700 1 --2- 172.21.15.57:0/1676634179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b2c0ff620 0x7f3b2c0ffaa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.789+0000 7f3b30ed4700 1 -- 172.21.15.57:0/1676634179 >> 172.21.15.57:0/1676634179 conn(0x7f3b2c076780 msgr2=0x7f3b2c076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.789+0000 7f3b30ed4700 1 -- 172.21.15.57:0/1676634179 shutdown_connections 2023-10-08T13:53:07.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.789+0000 7f3b30ed4700 1 -- 172.21.15.57:0/1676634179 wait complete. 2023-10-08T13:53:07.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.790+0000 7f3b30ed4700 1 Processor -- start 2023-10-08T13:53:07.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.790+0000 7f3b30ed4700 1 -- start start 2023-10-08T13:53:07.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b30ed4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b2c0ff620 0x7f3b2c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b30ed4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c0fffe0 0x7f3b2c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b30ed4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b2c1056b0 0x7f3b2c1a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b30ed4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c198840 con 0x7f3b2c0fffe0 2023-10-08T13:53:07.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b30ed4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c1a3000 con 0x7f3b2c0ff620 2023-10-08T13:53:07.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b30ed4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c1a3140 con 0x7f3b2c1056b0 2023-10-08T13:53:07.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b21d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c0fffe0 0x7f3b2c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b2a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b2c0ff620 0x7f3b2c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b2ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b2c1056b0 0x7f3b2c1a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b21d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c0fffe0 0x7f3b2c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57594/0 (socket says 172.21.15.57:57594) 2023-10-08T13:53:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b21d9b700 1 -- 172.21.15.57:0/3254059151 learned_addr learned my addr 172.21.15.57:0/3254059151 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b21d9b700 1 -- 172.21.15.57:0/3254059151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b2c1056b0 msgr2=0x7f3b2c1a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b2c1056b0 0x7f3b2c1a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b21d9b700 1 -- 172.21.15.57:0/3254059151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b2c0ff620 msgr2=0x7f3b2c197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b2c0ff620 0x7f3b2c197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b21d9b700 1 -- 172.21.15.57:0/3254059151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b14009520 con 0x7f3b2c0fffe0 2023-10-08T13:53:07.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b2a59c700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b2c0ff620 0x7f3b2c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c0fffe0 0x7f3b2c197f80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005950 tx=0x7f3b1400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b23fff700 1 -- 172.21.15.57:0/3254059151 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1401c070 con 0x7f3b2c0fffe0 2023-10-08T13:53:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.791+0000 7f3b30ed4700 1 -- 172.21.15.57:0/3254059151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c1a33c0 con 0x7f3b2c0fffe0 2023-10-08T13:53:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.792+0000 7f3b23fff700 1 -- 172.21.15.57:0/3254059151 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b1400baa0 con 0x7f3b2c0fffe0 2023-10-08T13:53:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.792+0000 7f3b23fff700 1 -- 172.21.15.57:0/3254059151 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140038e0 con 0x7f3b2c0fffe0 2023-10-08T13:53:07.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.792+0000 7f3b30ed4700 1 -- 172.21.15.57:0/3254059151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c1a38f0 con 0x7f3b2c0fffe0 2023-10-08T13:53:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.795+0000 7f3b23fff700 1 -- 172.21.15.57:0/3254059151 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3b14020030 con 0x7f3b2c0fffe0 2023-10-08T13:53:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.795+0000 7f3b23fff700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3b180a15f0 0x7f3b180a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.795+0000 7f3b23fff700 1 -- 172.21.15.57:0/3254059151 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3b140ce110 con 0x7f3b2c0fffe0 2023-10-08T13:53:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.795+0000 7f3b2a59c700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3b180a15f0 0x7f3b180a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b2a59c700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3b180a15f0 0x7f3b180a3ab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c077ab0 tx=0x7f3b1c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b3d0bdb80 1 Processor -- start 2023-10-08T13:53:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b3d0bdb80 1 -- start start 2023-10-08T13:53:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 0x56108d1be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d11c460 con 0x56108d1be650 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d10dc20 con 0x56108d1b37a0 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d10b540 con 0x56108d1b2e80 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b2ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b2a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b2ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1c0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57604/0 (socket says 172.21.15.57:57604) 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b2ad9d700 1 -- 172.21.15.57:0/1078067075 learned_addr learned my addr 172.21.15.57:0/1078067075 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.796+0000 7f3b2a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58412/0 (socket says 172.21.15.57:58412) 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/1078067075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 0x56108d1be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b2ad9d700 1 -- 172.21.15.57:0/1078067075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 msgr2=0x56108d1b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b2ad9d700 1 --2- 172.21.15.57:0/1078067075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b2ad9d700 1 -- 172.21.15.57:0/1078067075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 msgr2=0x56108d1be110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b2ad9d700 1 --2- 172.21.15.57:0/1078067075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 0x56108d1be110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b2ad9d700 1 -- 172.21.15.57:0/1078067075 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56108cf31de0 con 0x56108d1be650 2023-10-08T13:53:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/1078067075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 0x56108d1be110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:53:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b2a59c700 1 --2- 172.21.15.57:0/1078067075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b2ad9d700 1 --2- 172.21.15.57:0/1078067075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1c0a40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3b24011670 tx=0x7f3b240099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7db32a9504d31df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b20d99700 1 -- 172.21.15.57:0/1078067075 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400e040 con 0x56108d1be650 2023-10-08T13:53:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b20d99700 1 -- 172.21.15.57:0/1078067075 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b2400f040 con 0x56108d1be650 2023-10-08T13:53:07.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.797+0000 7f3b20d99700 1 -- 172.21.15.57:0/1078067075 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b240045e0 con 0x56108d1be650 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.798+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/1078067075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 msgr2=0x56108d1c0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.798+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/1078067075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1c0a40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3b24011670 tx=0x7f3b240099e0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.798+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/1078067075 shutdown_connections 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.798+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/1078067075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1c0a40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.798+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/1078067075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.798+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/1078067075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 0x56108d1be110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.798+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/1078067075 >> 172.21.15.57:0/1078067075 conn(0x56108d128260 msgr2=0x56108d1af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.798+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/1078067075 shutdown_connections 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.798+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/1078067075 wait complete. 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.798+0000 7f3b3d0bdb80 1 -- start start 2023-10-08T13:53:07.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b3d0bdb80 1 Processor -- start 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1bdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 0x56108d1bb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d106e30 con 0x56108d1be650 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d0a3710 con 0x56108d1b37a0 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d0a33b0 con 0x56108d1b2e80 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b3d0bdb80 10 client.0 _ll_register_callbacks cb 0x56108cf00800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b2ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b2a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1bdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b2ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1bb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57610/0 (socket says 172.21.15.57:57610) 2023-10-08T13:53:07.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b2ad9d700 1 -- 172.21.15.57:0/2237723936 learned_addr learned my addr 172.21.15.57:0/2237723936 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 0x56108d1bb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b2ad9d700 1 -- 172.21.15.57:0/2237723936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 msgr2=0x56108d1bdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b2ad9d700 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1bdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b2ad9d700 1 -- 172.21.15.57:0/2237723936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 msgr2=0x56108d1bb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b2ad9d700 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 0x56108d1bb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b2ad9d700 1 -- 172.21.15.57:0/2237723936 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c017070 con 0x56108d1be650 2023-10-08T13:53:07.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 0x56108d1bb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:53:07.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b2ad9d700 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1bb940 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009580 tx=0x7f3b240112c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c7d7b8d9285f7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.799+0000 7f3b11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:07.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2237723936 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400e070 con 0x56108d1be650 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2237723936 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3b24004470 con 0x56108d1be650 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2237723936 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56108d0ceeb0 con 0x56108d1be650 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2237723936 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b24004630 con 0x56108d1be650 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b3d0bdb80 10 client.16374 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2237723936 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f3b24011350 con 0x56108d1be650 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2237723936 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56108d0ceeb0 con 0x56108d1be650 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2237723936 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3b24027460 con 0x56108d1be650 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b3d0bdb80 10 client.16374 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b3d0bdb80 10 client.16374 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b3d0bdb80 20 client.16374 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:07.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56108d1ce130 0x56108d1d0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b3d0bdb80 4 client.16374 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:07.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2237723936 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56108d0ffe10 con 0x56108d1ce130 2023-10-08T13:53:07.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.800+0000 7f3b2a59c700 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56108d1ce130 0x56108d1d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.801+0000 7f3b11ffb700 10 client.16374 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:53:07.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.801+0000 7f3b2a59c700 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56108d1ce130 0x56108d1d0550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=edf9d54f2262dc13 server_cookie=55462b578e77abdc in_seq=0 out_seq=0 2023-10-08T13:53:07.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.803+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2237723936 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56108d0ffe10 con 0x56108d1ce130 2023-10-08T13:53:07.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.803+0000 7f3b11ffb700 10 client.16374 handle_command_reply: tid=0 2023-10-08T13:53:07.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.803+0000 7f3b3d0bdb80 1 client.16374 shutdown 2023-10-08T13:53:07.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.803+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2237723936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 msgr2=0x56108d1bb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.803+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1bb940 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009580 tx=0x7f3b240112c0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.804+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2237723936 shutdown_connections 2023-10-08T13:53:07.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.804+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56108d1ce130 0x56108d1d0550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.804+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1be650 0x56108d1bb940 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.804+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1bdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.804+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2237723936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b37a0 0x56108d1bb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.804+0000 7f3b11ffb700 0 client.16374 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:53:07.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.804+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2237723936 >> 172.21.15.57:0/2237723936 conn(0x56108d128260 msgr2=0x56108d1aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.804+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2237723936 shutdown_connections 2023-10-08T13:53:07.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.804+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2237723936 wait complete. 2023-10-08T13:53:07.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.804+0000 7f3b3d0bdb80 20 client.16374 trim_cache size 0 max 16384 2023-10-08T13:53:07.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.817+0000 7f3b3d0bdb80 1 Processor -- start 2023-10-08T13:53:07.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.817+0000 7f3b3d0bdb80 1 -- start start 2023-10-08T13:53:07.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b2e80 0x56108d1bb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1ca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1be650 0x56108d1b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108cf87920 con 0x56108d1b37a0 2023-10-08T13:53:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d1b3bb0 con 0x56108d1b2e80 2023-10-08T13:53:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d1cb790 con 0x56108d1be650 2023-10-08T13:53:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b21d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b21d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57616/0 (socket says 172.21.15.57:57616) 2023-10-08T13:53:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b21d9b700 1 -- 172.21.15.57:0/2386253728 learned_addr learned my addr 172.21.15.57:0/2386253728 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b2ad9d700 1 --2- 172.21.15.57:0/2386253728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1be650 0x56108d1b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b2a59c700 1 --2- 172.21.15.57:0/2386253728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b2e80 0x56108d1bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b21d9b700 1 -- 172.21.15.57:0/2386253728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1be650 msgr2=0x56108d1b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/2386253728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1be650 0x56108d1b85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b21d9b700 1 -- 172.21.15.57:0/2386253728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b2e80 msgr2=0x56108d1bb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/2386253728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b2e80 0x56108d1bb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b21d9b700 1 -- 172.21.15.57:0/2386253728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56108d1ab4f0 con 0x56108d1b37a0 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/2386253728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1ca000 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005e90 tx=0x7f3b14005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5eea09ef341ada server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b2159a700 1 -- 172.21.15.57:0/2386253728 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140a10c0 con 0x56108d1b37a0 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.818+0000 7f3b2159a700 1 -- 172.21.15.57:0/2386253728 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b1409b580 con 0x56108d1b37a0 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b2159a700 1 -- 172.21.15.57:0/2386253728 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140050d0 con 0x56108d1b37a0 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2386253728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 msgr2=0x56108d1ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2386253728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1ca000 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005e90 tx=0x7f3b14005040 comp rx=0 tx=0).stop 2023-10-08T13:53:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2386253728 shutdown_connections 2023-10-08T13:53:07.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2386253728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1be650 0x56108d1b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2386253728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1b2e80 0x56108d1bb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2386253728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1ca000 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2386253728 >> 172.21.15.57:0/2386253728 conn(0x56108d128260 msgr2=0x56108d1c98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2386253728 shutdown_connections 2023-10-08T13:53:07.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2386253728 wait complete. 2023-10-08T13:53:07.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.819+0000 7f3b3d0bdb80 1 -- start start 2023-10-08T13:53:07.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b3d0bdb80 1 Processor -- start 2023-10-08T13:53:07.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b3d0bdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1be650 0x56108d1c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d1c4030 con 0x56108d1b37a0 2023-10-08T13:53:07.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d1c41a0 con 0x56108d1be650 2023-10-08T13:53:07.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b3d0bdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108d1c4360 con 0x56108d1b2e80 2023-10-08T13:53:07.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b3d0bdb80 10 client.0 _ll_register_callbacks cb 0x7f3b1c00e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:07.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b21d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b21d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57622/0 (socket says 172.21.15.57:57622) 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b21d9b700 1 -- 172.21.15.57:0/2017983105 learned_addr learned my addr 172.21.15.57:0/2017983105 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b2a59c700 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b2ad9d700 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1be650 0x56108d1c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b21d9b700 1 -- 172.21.15.57:0/2017983105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 msgr2=0x56108d1c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b21d9b700 1 -- 172.21.15.57:0/2017983105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1be650 msgr2=0x56108d1c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1be650 0x56108d1c3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b21d9b700 1 -- 172.21.15.57:0/2017983105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c1a38f0 con 0x56108d1b37a0 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b2ad9d700 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1be650 0x56108d1c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b21d9b700 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1c35b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3b140cb610 tx=0x7f3b1409af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22adf9430cd7fd3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.820+0000 7f3b11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2017983105 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140b9070 con 0x56108d1b37a0 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2017983105 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3b140a3180 con 0x56108d1b37a0 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2017983105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b2400f040 con 0x56108d1b37a0 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2017983105 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b14099220 con 0x56108d1b37a0 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b3d0bdb80 10 client.16380 fetch_fsmap learned FSMap version 31 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2017983105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56108cf1ac60 con 0x56108d1b37a0 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2017983105 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b1407d070 con 0x56108d1b37a0 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2017983105 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 31) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3b14021380 con 0x56108d1b37a0 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b3d0bdb80 10 client.16380 fetch_fsmap finished waiting for FSMap version 31 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b3d0bdb80 10 client.16380 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b3d0bdb80 20 client.16380 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:07.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56108d222f60 0x56108d225380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b3d0bdb80 4 client.16380 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2017983105 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56108d0ffb30 con 0x56108d222f60 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.821+0000 7f3b2a59c700 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56108d222f60 0x56108d225380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.822+0000 7f3b2a59c700 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56108d222f60 0x56108d225380 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77c7ad6dcec97 server_cookie=7b045ed64bccf3ef in_seq=0 out_seq=0 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.822+0000 7f3b11ffb700 10 client.16380 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.824+0000 7f3b11ffb700 1 -- 172.21.15.57:0/2017983105 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56108d0ffb30 con 0x56108d222f60 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.824+0000 7f3b11ffb700 10 client.16380 handle_command_reply: tid=0 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.824+0000 7f3b3d0bdb80 1 client.16380 shutdown 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2017983105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 msgr2=0x56108d1c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1c35b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3b140cb610 tx=0x7f3b1409af30 comp rx=0 tx=0).stop 2023-10-08T13:53:07.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2017983105 shutdown_connections 2023-10-08T13:53:07.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x56108d222f60 0x56108d225380 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56108d1be650 0x56108d1c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56108d1b2e80 0x56108d1c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 1 --2- 172.21.15.57:0/2017983105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56108d1b37a0 0x56108d1c35b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b11ffb700 0 client.16380 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:53:07.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2017983105 >> 172.21.15.57:0/2017983105 conn(0x56108d128260 msgr2=0x56108d1ca180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2017983105 shutdown_connections 2023-10-08T13:53:07.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 1 -- 172.21.15.57:0/2017983105 wait complete. 2023-10-08T13:53:07.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b3d0bdb80 20 client.16380 trim_cache size 0 max 16384 2023-10-08T13:53:07.833 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:53:07.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 -- 172.21.15.57:0/3254059151 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3b180a15f0 msgr2=0x7f3b180a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3b180a15f0 0x7f3b180a3ab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c077ab0 tx=0x7f3b1c009260 comp rx=0 tx=0).stop 2023-10-08T13:53:07.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 -- 172.21.15.57:0/3254059151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c0fffe0 msgr2=0x7f3b2c197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c0fffe0 0x7f3b2c197f80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005950 tx=0x7f3b1400fa00 comp rx=0 tx=0).stop 2023-10-08T13:53:07.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 -- 172.21.15.57:0/3254059151 shutdown_connections 2023-10-08T13:53:07.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b2c1056b0 0x7f3b2c1a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3b180a15f0 0x7f3b180a3ab0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b2c0ff620 0x7f3b2c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 --2- 172.21.15.57:0/3254059151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b2c0fffe0 0x7f3b2c197f80 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 -- 172.21.15.57:0/3254059151 >> 172.21.15.57:0/3254059151 conn(0x7f3b2c076780 msgr2=0x7f3b2c0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 -- 172.21.15.57:0/3254059151 shutdown_connections 2023-10-08T13:53:07.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.825+0000 7f3b30ed4700 1 -- 172.21.15.57:0/3254059151 wait complete. 2023-10-08T13:53:07.838 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-08T13:53:07.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.970+0000 7f77c4614700 1 -- 172.21.15.57:0/40008044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77bc101100 msgr2=0x7f77bc10d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.970+0000 7f77c4614700 1 --2- 172.21.15.57:0/40008044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77bc101100 0x7f77bc10d610 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f77b400b0a0 tx=0x7f77b400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.970+0000 7f77c4614700 1 -- 172.21.15.57:0/40008044 shutdown_connections 2023-10-08T13:53:07.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.970+0000 7f77c4614700 1 --2- 172.21.15.57:0/40008044 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77bc101100 0x7f77bc10d610 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.970+0000 7f77c4614700 1 --2- 172.21.15.57:0/40008044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77bc106750 0x7f77bc106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.970+0000 7f77c4614700 1 --2- 172.21.15.57:0/40008044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77bc100740 0x7f77bc100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.970+0000 7f77c4614700 1 -- 172.21.15.57:0/40008044 >> 172.21.15.57:0/40008044 conn(0x7f77bc0fc230 msgr2=0x7f77bc0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.970+0000 7f77c4614700 1 -- 172.21.15.57:0/40008044 shutdown_connections 2023-10-08T13:53:07.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.970+0000 7f77c4614700 1 -- 172.21.15.57:0/40008044 wait complete. 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c4614700 1 Processor -- start 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c4614700 1 -- start start 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c4614700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77bc100740 0x7f77bc19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c4614700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77bc101100 0x7f77bc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c4614700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77bc106750 0x7f77bc196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c4614700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc1970d0 con 0x7f77bc100740 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c4614700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc197240 con 0x7f77bc106750 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c4614700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc197400 con 0x7f77bc101100 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c23b0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77bc100740 0x7f77bc19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c1baf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77bc101100 0x7f77bc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c23b0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77bc100740 0x7f77bc19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57634/0 (socket says 172.21.15.57:57634) 2023-10-08T13:53:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.971+0000 7f77c23b0700 1 -- 172.21.15.57:0/3099637383 learned_addr learned my addr 172.21.15.57:0/3099637383 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c2bb1700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77bc106750 0x7f77bc196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c1baf700 1 -- 172.21.15.57:0/3099637383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77bc106750 msgr2=0x7f77bc196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c1baf700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77bc106750 0x7f77bc196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c1baf700 1 -- 172.21.15.57:0/3099637383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77bc100740 msgr2=0x7f77bc19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c1baf700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77bc100740 0x7f77bc19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c1baf700 1 -- 172.21.15.57:0/3099637383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b4009d00 con 0x7f77bc101100 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c2bb1700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77bc106750 0x7f77bc196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c23b0700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77bc100740 0x7f77bc19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c1baf700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77bc101100 0x7f77bc19cda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00e9f0 tx=0x7f77ac00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77b37fe700 1 -- 172.21.15.57:0/3099637383 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac00eee0 con 0x7f77bc101100 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c4614700 1 -- 172.21.15.57:0/3099637383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc1976e0 con 0x7f77bc101100 2023-10-08T13:53:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77b37fe700 1 -- 172.21.15.57:0/3099637383 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac00cbd0 con 0x7f77bc101100 2023-10-08T13:53:07.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77b37fe700 1 -- 172.21.15.57:0/3099637383 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac018680 con 0x7f77bc101100 2023-10-08T13:53:07.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.972+0000 7f77c4614700 1 -- 172.21.15.57:0/3099637383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc1a3830 con 0x7f77bc101100 2023-10-08T13:53:07.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.974+0000 7f77b37fe700 1 -- 172.21.15.57:0/3099637383 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f77ac00cd40 con 0x7f77bc101100 2023-10-08T13:53:07.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.975+0000 7f77b37fe700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77a80a15f0 0x7f77a80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.975+0000 7f77c23b0700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77a80a15f0 0x7f77a80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.975+0000 7f77b37fe700 1 -- 172.21.15.57:0/3099637383 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f77ac014070 con 0x7f77bc101100 2023-10-08T13:53:07.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77d07fdb80 1 Processor -- start 2023-10-08T13:53:07.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77c23b0700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77a80a15f0 0x7f77a80a3ab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f77b80103d0 tx=0x7f77b8007480 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77d07fdb80 1 -- start start 2023-10-08T13:53:07.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a9695e80 0x55b7a9696260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a96967a0 0x55b7a96a1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96a1650 0x55b7a96a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a95ff460 con 0x55b7a96a1650 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a95f0c20 con 0x55b7a9695e80 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a95ee540 con 0x55b7a96967a0 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77c2bb1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96a1650 0x55b7a96a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77c1baf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a96967a0 0x55b7a96a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77c1baf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a96967a0 0x55b7a96a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58456/0 (socket says 172.21.15.57:58456) 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77c2bb1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96a1650 0x55b7a96a3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57646/0 (socket says 172.21.15.57:57646) 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77c1baf700 1 -- 172.21.15.57:0/2849618419 learned_addr learned my addr 172.21.15.57:0/2849618419 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77c23b0700 1 --2- 172.21.15.57:0/2849618419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a9695e80 0x55b7a9696260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.976+0000 7f77c1baf700 1 -- 172.21.15.57:0/2849618419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a9695e80 msgr2=0x55b7a9696260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.977+0000 7f77c1baf700 1 --2- 172.21.15.57:0/2849618419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a9695e80 0x55b7a9696260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.977+0000 7f77c1baf700 1 -- 172.21.15.57:0/2849618419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96a1650 msgr2=0x55b7a96a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.977+0000 7f77c1baf700 1 --2- 172.21.15.57:0/2849618419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96a1650 0x55b7a96a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.977+0000 7f77c1baf700 1 -- 172.21.15.57:0/2849618419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7a9414de0 con 0x55b7a96967a0 2023-10-08T13:53:07.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.977+0000 7f77c2bb1700 1 --2- 172.21.15.57:0/2849618419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96a1650 0x55b7a96a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.977+0000 7f77c1baf700 1 --2- 172.21.15.57:0/2849618419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a96967a0 0x55b7a96a1110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c5b0 tx=0x7f77ac099c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f75c12961e23f139 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.977+0000 7f77b0ff9700 1 -- 172.21.15.57:0/2849618419 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac09a2a0 con 0x55b7a96967a0 2023-10-08T13:53:07.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.977+0000 7f77b0ff9700 1 -- 172.21.15.57:0/2849618419 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac09e3f0 con 0x55b7a96967a0 2023-10-08T13:53:07.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.977+0000 7f77b0ff9700 1 -- 172.21.15.57:0/2849618419 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac09a700 con 0x55b7a96967a0 2023-10-08T13:53:07.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 -- 172.21.15.57:0/2849618419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a96967a0 msgr2=0x55b7a96a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/2849618419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a96967a0 0x55b7a96a1110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c5b0 tx=0x7f77ac099c80 comp rx=0 tx=0).stop 2023-10-08T13:53:07.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 -- 172.21.15.57:0/2849618419 shutdown_connections 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/2849618419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96a1650 0x55b7a96a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/2849618419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a9695e80 0x55b7a9696260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/2849618419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a96967a0 0x55b7a96a1110 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 -- 172.21.15.57:0/2849618419 >> 172.21.15.57:0/2849618419 conn(0x55b7a960b260 msgr2=0x55b7a96922d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 -- 172.21.15.57:0/2849618419 shutdown_connections 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 -- 172.21.15.57:0/2849618419 wait complete. 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 -- start start 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 Processor -- start 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a96a1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a969e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a969ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a95e9e30 con 0x55b7a96967a0 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a9586710 con 0x55b7a96a1650 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.978+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a95863b0 con 0x55b7a9695e80 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77d07fdb80 10 client.0 _ll_register_callbacks cb 0x55b7a93e3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c23b0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a96a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c1baf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a969e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c23b0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a96a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58460/0 (socket says 172.21.15.57:58460) 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c1baf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a969e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57650/0 (socket says 172.21.15.57:57650) 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c23b0700 1 -- 172.21.15.57:0/713305904 learned_addr learned my addr 172.21.15.57:0/713305904 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c2bb1700 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a969ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c23b0700 1 -- 172.21.15.57:0/713305904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 msgr2=0x55b7a969ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c23b0700 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a969ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c23b0700 1 -- 172.21.15.57:0/713305904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 msgr2=0x55b7a969e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c23b0700 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a969e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c23b0700 1 -- 172.21.15.57:0/713305904 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc10df10 con 0x55b7a9695e80 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c1baf700 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a969e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77c23b0700 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a96a1100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f77b8011920 tx=0x7f77b80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=715b646088ca21ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77a5ffb700 1 -- 172.21.15.57:0/713305904 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b801e6c0 con 0x55b7a9695e80 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77d07fdb80 1 -- 172.21.15.57:0/713305904 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7a95b1eb0 con 0x55b7a9695e80 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.979+0000 7f77a5ffb700 1 -- 172.21.15.57:0/713305904 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f77b8017070 con 0x55b7a9695e80 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77a5ffb700 1 -- 172.21.15.57:0/713305904 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77b801e820 con 0x55b7a9695e80 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77d07fdb80 10 client.25990 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:07.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77d07fdb80 1 -- 172.21.15.57:0/713305904 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f77bc1a3830 con 0x55b7a9695e80 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77a5ffb700 1 -- 172.21.15.57:0/713305904 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7a95b1eb0 con 0x55b7a9695e80 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77a5ffb700 1 -- 172.21.15.57:0/713305904 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f77b8028b30 con 0x55b7a9695e80 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77d07fdb80 10 client.25990 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77d07fdb80 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77d07fdb80 20 client.25990 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b7a96b1210 0x55b7a96b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77d07fdb80 4 client.25990 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77d07fdb80 1 -- 172.21.15.57:0/713305904 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7a95e2e10 con 0x55b7a96b1210 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.980+0000 7f77c1baf700 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b7a96b1210 0x55b7a96b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.981+0000 7f77a5ffb700 10 client.25990 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:07.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.981+0000 7f77c1baf700 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b7a96b1210 0x55b7a96b3630 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9557a7ed07a6cb server_cookie=faf8f5bb71b9ce45 in_seq=0 out_seq=0 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.981+0000 7f77a5ffb700 1 -- 172.21.15.57:0/713305904 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b7a95e2e10 con 0x55b7a96b1210 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.981+0000 7f77a5ffb700 10 client.25990 handle_command_reply: tid=0 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.981+0000 7f77d07fdb80 1 client.25990 shutdown 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.981+0000 7f77d07fdb80 1 -- 172.21.15.57:0/713305904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 msgr2=0x55b7a96a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.981+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a96a1100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f77b8011920 tx=0x7f77b80047c0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.981+0000 7f77d07fdb80 1 -- 172.21.15.57:0/713305904 shutdown_connections 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.981+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b7a96b1210 0x55b7a96b3630 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.982+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a969ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.982+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a96a1100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.982+0000 7f77a5ffb700 0 client.25990 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:07.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.982+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/713305904 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a969e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.982+0000 7f77d07fdb80 1 -- 172.21.15.57:0/713305904 >> 172.21.15.57:0/713305904 conn(0x55b7a960b260 msgr2=0x55b7a9691ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.982+0000 7f77d07fdb80 1 -- 172.21.15.57:0/713305904 shutdown_connections 2023-10-08T13:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.982+0000 7f77d07fdb80 1 -- 172.21.15.57:0/713305904 wait complete. 2023-10-08T13:53:07.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.982+0000 7f77d07fdb80 20 client.25990 trim_cache size 0 max 16384 2023-10-08T13:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77d07fdb80 1 Processor -- start 2023-10-08T13:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77d07fdb80 1 -- start start 2023-10-08T13:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a969ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a96997f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a946a920 con 0x55b7a96967a0 2023-10-08T13:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a9693060 con 0x55b7a96a1650 2023-10-08T13:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a96a86e0 con 0x55b7a9695e80 2023-10-08T13:53:07.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77c1baf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77c23b0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a969ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77c1baf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57658/0 (socket says 172.21.15.57:57658) 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77c1baf700 1 -- 172.21.15.57:0/728181315 learned_addr learned my addr 172.21.15.57:0/728181315 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77c1baf700 1 -- 172.21.15.57:0/728181315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 msgr2=0x55b7a969ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77c1baf700 1 --2- 172.21.15.57:0/728181315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a969ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77c1baf700 1 -- 172.21.15.57:0/728181315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 msgr2=0x55b7a96997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.995+0000 7f77c2bb1700 1 --2- 172.21.15.57:0/728181315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a96997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77c1baf700 1 --2- 172.21.15.57:0/728181315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a96997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77c1baf700 1 -- 172.21.15.57:0/728181315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7a93fdc60 con 0x55b7a96967a0 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77c1baf700 1 --2- 172.21.15.57:0/728181315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a81a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f77ac096d10 tx=0x7f77ac010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35ab2f041cc96c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:07.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77b17fa700 1 -- 172.21.15.57:0/728181315 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac098640 con 0x55b7a96967a0 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77b17fa700 1 -- 172.21.15.57:0/728181315 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac099790 con 0x55b7a96967a0 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77b17fa700 1 -- 172.21.15.57:0/728181315 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0a2400 con 0x55b7a96967a0 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77d07fdb80 1 -- 172.21.15.57:0/728181315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 msgr2=0x55b7a96a81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/728181315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a81a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f77ac096d10 tx=0x7f77ac010b90 comp rx=0 tx=0).stop 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77d07fdb80 1 -- 172.21.15.57:0/728181315 shutdown_connections 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/728181315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a96997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/728181315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a969ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/728181315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a81a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77d07fdb80 1 -- 172.21.15.57:0/728181315 >> 172.21.15.57:0/728181315 conn(0x55b7a960b260 msgr2=0x55b7a96914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77d07fdb80 1 -- 172.21.15.57:0/728181315 shutdown_connections 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.996+0000 7f77d07fdb80 1 -- 172.21.15.57:0/728181315 wait complete. 2023-10-08T13:53:08.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77d07fdb80 1 -- start start 2023-10-08T13:53:08.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77d07fdb80 1 Processor -- start 2023-10-08T13:53:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a969c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77d07fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a96a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a96a6d80 con 0x55b7a96967a0 2023-10-08T13:53:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a96a6ef0 con 0x55b7a96a1650 2023-10-08T13:53:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77d07fdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a9704040 con 0x55b7a9695e80 2023-10-08T13:53:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77d07fdb80 10 client.0 _ll_register_callbacks cb 0x7f77b8011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:08.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77c23b0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a969c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77c1baf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77c23b0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a969c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58474/0 (socket says 172.21.15.57:58474) 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.997+0000 7f77c23b0700 1 -- 172.21.15.57:0/1199377082 learned_addr learned my addr 172.21.15.57:0/1199377082 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77c2bb1700 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a96a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77c1baf700 1 -- 172.21.15.57:0/1199377082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 msgr2=0x55b7a969c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77c1baf700 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a969c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77c1baf700 1 -- 172.21.15.57:0/1199377082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 msgr2=0x55b7a96a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77c1baf700 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a96a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77c1baf700 1 -- 172.21.15.57:0/1199377082 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7a9414de0 con 0x55b7a96967a0 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77c2bb1700 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a96a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77c1baf700 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a6180 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x55b7a9693a10 tx=0x7f77ac099c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38a998f72a82b0c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:08.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77a5ffb700 1 -- 172.21.15.57:0/1199377082 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac099900 con 0x55b7a96967a0 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77a5ffb700 1 -- 172.21.15.57:0/1199377082 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f77ac0a21a0 con 0x55b7a96967a0 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77d07fdb80 1 -- 172.21.15.57:0/1199377082 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77ac09e3f0 con 0x55b7a96967a0 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77a5ffb700 1 -- 172.21.15.57:0/1199377082 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77ac099a60 con 0x55b7a96967a0 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77d07fdb80 10 client.16398 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77d07fdb80 1 -- 172.21.15.57:0/1199377082 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b7a943b250 con 0x55b7a96967a0 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.998+0000 7f77a5ffb700 1 -- 172.21.15.57:0/1199377082 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac09e3f0 con 0x55b7a96967a0 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77a5ffb700 1 -- 172.21.15.57:0/1199377082 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f77ac09f130 con 0x55b7a96967a0 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 10 client.16398 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:08.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 20 client.16398 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b7a9705370 0x55b7a9707790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 4 client.16398 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 1 -- 172.21.15.57:0/1199377082 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b7a95e2b30 con 0x55b7a9705370 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77c23b0700 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b7a9705370 0x55b7a9707790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77a5ffb700 10 client.16398 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77c23b0700 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b7a9705370 0x55b7a9707790 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5337b6db910d2b server_cookie=cbfd7c1dadcf2cf in_seq=0 out_seq=0 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77a5ffb700 1 -- 172.21.15.57:0/1199377082 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b7a95e2b30 con 0x55b7a9705370 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77a5ffb700 10 client.16398 handle_command_reply: tid=0 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 1 client.16398 shutdown 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 1 -- 172.21.15.57:0/1199377082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 msgr2=0x55b7a96a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:08.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a6180 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x55b7a9693a10 tx=0x7f77ac099c80 comp rx=0 tx=0).stop 2023-10-08T13:53:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 1 -- 172.21.15.57:0/1199377082 shutdown_connections 2023-10-08T13:53:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:07.999+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b7a9705370 0x55b7a9707790 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77a5ffb700 0 client.16398 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b7a96a1650 0x55b7a96a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b7a9695e80 0x55b7a969c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77d07fdb80 1 --2- 172.21.15.57:0/1199377082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b7a96967a0 0x55b7a96a6180 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77d07fdb80 1 -- 172.21.15.57:0/1199377082 >> 172.21.15.57:0/1199377082 conn(0x55b7a960b260 msgr2=0x55b7a96997f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77d07fdb80 1 -- 172.21.15.57:0/1199377082 shutdown_connections 2023-10-08T13:53:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77d07fdb80 1 -- 172.21.15.57:0/1199377082 wait complete. 2023-10-08T13:53:08.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77d07fdb80 20 client.16398 trim_cache size 0 max 16384 2023-10-08T13:53:08.009 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:53:08.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 -- 172.21.15.57:0/3099637383 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77a80a15f0 msgr2=0x7f77a80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:08.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77a80a15f0 0x7f77a80a3ab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f77b80103d0 tx=0x7f77b8007480 comp rx=0 tx=0).stop 2023-10-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 -- 172.21.15.57:0/3099637383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77bc101100 msgr2=0x7f77bc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77bc101100 0x7f77bc19cda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00e9f0 tx=0x7f77ac00ed00 comp rx=0 tx=0).stop 2023-10-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 -- 172.21.15.57:0/3099637383 shutdown_connections 2023-10-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f77a80a15f0 0x7f77a80a3ab0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77bc106750 0x7f77bc196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77bc100740 0x7f77bc19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 --2- 172.21.15.57:0/3099637383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77bc101100 0x7f77bc19cda0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 -- 172.21.15.57:0/3099637383 >> 172.21.15.57:0/3099637383 conn(0x7f77bc0fc230 msgr2=0x7f77bc0fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 -- 172.21.15.57:0/3099637383 shutdown_connections 2023-10-08T13:53:08.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:08.000+0000 7f77c4614700 1 -- 172.21.15.57:0/3099637383 wait complete. 2023-10-08T13:53:08.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:07 smithi057 ceph-mon[99476]: pgmap v346: 129 pgs: 129 active+clean; 43 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 132 op/s 2023-10-08T13:53:08.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:07 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2637517139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:53:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:07 smithi099 ceph-mon[103849]: pgmap v346: 129 pgs: 129 active+clean; 43 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 132 op/s 2023-10-08T13:53:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:07 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2637517139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:53:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:07 smithi115 ceph-mon[104408]: pgmap v346: 129 pgs: 129 active+clean; 43 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 132 op/s 2023-10-08T13:53:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:07 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2637517139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:53:09.014 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-08T13:53:09.014 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-08T13:53:09.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.146+0000 7f53a5bed700 1 -- 172.21.15.57:0/3907983343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53a00ff640 msgr2=0x7f53a00ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.146+0000 7f53a5bed700 1 --2- 172.21.15.57:0/3907983343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53a00ff640 0x7f53a00ffac0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5394009990 tx=0x7f5394009ca0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.146+0000 7f53a5bed700 1 -- 172.21.15.57:0/3907983343 shutdown_connections 2023-10-08T13:53:09.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.146+0000 7f53a5bed700 1 --2- 172.21.15.57:0/3907983343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53a0100000 0x7f53a010c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.146+0000 7f53a5bed700 1 --2- 172.21.15.57:0/3907983343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a01056e0 0x7f53a0105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.146+0000 7f53a5bed700 1 --2- 172.21.15.57:0/3907983343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53a00ff640 0x7f53a00ffac0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.146+0000 7f53a5bed700 1 -- 172.21.15.57:0/3907983343 >> 172.21.15.57:0/3907983343 conn(0x7f53a0076760 msgr2=0x7f53a0076b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.146+0000 7f53a5bed700 1 -- 172.21.15.57:0/3907983343 shutdown_connections 2023-10-08T13:53:09.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.146+0000 7f53a5bed700 1 -- 172.21.15.57:0/3907983343 wait complete. 2023-10-08T13:53:09.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.146+0000 7f53a5bed700 1 Processor -- start 2023-10-08T13:53:09.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f53a5bed700 1 -- start start 2023-10-08T13:53:09.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f53a5bed700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53a00ff640 0x7f53a019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f53a5bed700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a0100000 0x7f53a019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f53a5bed700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53a01056e0 0x7f53a0196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f53a5bed700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a0197110 con 0x7f53a00ff640 2023-10-08T13:53:09.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f53a5bed700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a0197280 con 0x7f53a0100000 2023-10-08T13:53:09.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f53a5bed700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a0197440 con 0x7f53a01056e0 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53a00ff640 0x7f53a019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53a00ff640 0x7f53a019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57676/0 (socket says 172.21.15.57:57676) 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539f7fe700 1 -- 172.21.15.57:0/562150659 learned_addr learned my addr 172.21.15.57:0/562150659 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539ffff700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53a01056e0 0x7f53a0196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539effd700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a0100000 0x7f53a019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539f7fe700 1 -- 172.21.15.57:0/562150659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53a01056e0 msgr2=0x7f53a0196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539f7fe700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53a01056e0 0x7f53a0196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539f7fe700 1 -- 172.21.15.57:0/562150659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a0100000 msgr2=0x7f53a019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539f7fe700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a0100000 0x7f53a019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539f7fe700 1 -- 172.21.15.57:0/562150659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940095f0 con 0x7f53a00ff640 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.147+0000 7f539effd700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a0100000 0x7f53a019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:53:09.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.148+0000 7f539f7fe700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53a00ff640 0x7f53a019c980 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f539000e9a0 tx=0x7f539000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.148+0000 7f539cff9700 1 -- 172.21.15.57:0/562150659 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390009790 con 0x7f53a00ff640 2023-10-08T13:53:09.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.148+0000 7f53a5bed700 1 -- 172.21.15.57:0/562150659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a0197720 con 0x7f53a00ff640 2023-10-08T13:53:09.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.148+0000 7f539cff9700 1 -- 172.21.15.57:0/562150659 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f539000cbd0 con 0x7f53a00ff640 2023-10-08T13:53:09.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.148+0000 7f539cff9700 1 -- 172.21.15.57:0/562150659 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390018680 con 0x7f53a00ff640 2023-10-08T13:53:09.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.148+0000 7f53a5bed700 1 -- 172.21.15.57:0/562150659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a01a3970 con 0x7f53a00ff640 2023-10-08T13:53:09.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.150+0000 7f539cff9700 1 -- 172.21.15.57:0/562150659 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f539000cd40 con 0x7f53a00ff640 2023-10-08T13:53:09.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.150+0000 7f539cff9700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f53880a15f0 0x7f53880a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.151+0000 7f539effd700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f53880a15f0 0x7f53880a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.151+0000 7f539cff9700 1 -- 172.21.15.57:0/562150659 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5390014070 con 0x7f53a00ff640 2023-10-08T13:53:09.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.151+0000 7f539effd700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f53880a15f0 0x7f53880a3ab0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f539400b2e0 tx=0x7f539400b650 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.151+0000 7f53b1dd6b80 1 Processor -- start 2023-10-08T13:53:09.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.151+0000 7f53b1dd6b80 1 -- start start 2023-10-08T13:53:09.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61bea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bea820 0x564d61bf5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bf56d0 0x564d61bf7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61b534e0 con 0x564d61bf56d0 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61b44ca0 con 0x564d61be9f00 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61b425c0 con 0x564d61bea820 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bf56d0 0x564d61bf7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bea820 0x564d61bf5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bf56d0 0x564d61bf7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57688/0 (socket says 172.21.15.57:57688) 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539ffff700 1 -- 172.21.15.57:0/651178274 learned_addr learned my addr 172.21.15.57:0/651178274 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539f7fe700 1 --2- 172.21.15.57:0/651178274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61bea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539ffff700 1 -- 172.21.15.57:0/651178274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bea820 msgr2=0x564d61bf5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539ffff700 1 --2- 172.21.15.57:0/651178274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bea820 0x564d61bf5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539ffff700 1 -- 172.21.15.57:0/651178274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 msgr2=0x564d61bea2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539ffff700 1 --2- 172.21.15.57:0/651178274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61bea2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539ffff700 1 -- 172.21.15.57:0/651178274 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d61968e60 con 0x564d61bf56d0 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539f7fe700 1 --2- 172.21.15.57:0/651178274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61bea2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f539ffff700 1 --2- 172.21.15.57:0/651178274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bf56d0 0x564d61bf7ac0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5398011680 tx=0x7f5398009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60347bbeaa8382a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.152+0000 7f5385ffb700 1 -- 172.21.15.57:0/651178274 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539800e040 con 0x564d61bf56d0 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f5385ffb700 1 -- 172.21.15.57:0/651178274 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f539800f040 con 0x564d61bf56d0 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f5385ffb700 1 -- 172.21.15.57:0/651178274 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398004850 con 0x564d61bf56d0 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/651178274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bf56d0 msgr2=0x564d61bf7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/651178274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bf56d0 0x564d61bf7ac0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5398011680 tx=0x7f5398009920 comp rx=0 tx=0).stop 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/651178274 shutdown_connections 2023-10-08T13:53:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/651178274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bf56d0 0x564d61bf7ac0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/651178274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61bea2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/651178274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bea820 0x564d61bf5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/651178274 >> 172.21.15.57:0/651178274 conn(0x564d61b5f2e0 msgr2=0x564d61be6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/651178274 shutdown_connections 2023-10-08T13:53:09.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.153+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/651178274 wait complete. 2023-10-08T13:53:09.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f53b1dd6b80 1 -- start start 2023-10-08T13:53:09.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f53b1dd6b80 1 Processor -- start 2023-10-08T13:53:09.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61bf50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61bf23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bf2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61b3deb0 con 0x564d61bea820 2023-10-08T13:53:09.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61ada790 con 0x564d61be9f00 2023-10-08T13:53:09.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61ada430 con 0x564d61bf56d0 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f53b1dd6b80 10 client.0 _ll_register_callbacks cb 0x564d61937880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f539effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61bf23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f539effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61bf23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57692/0 (socket says 172.21.15.57:57692) 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f539effd700 1 -- 172.21.15.57:0/2525076766 learned_addr learned my addr 172.21.15.57:0/2525076766 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f539ffff700 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bf2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.154+0000 7f539f7fe700 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61bf50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f539effd700 1 -- 172.21.15.57:0/2525076766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 msgr2=0x564d61bf2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f539effd700 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bf2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f539effd700 1 -- 172.21.15.57:0/2525076766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 msgr2=0x564d61bf50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f539effd700 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61bf50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f539effd700 1 -- 172.21.15.57:0/2525076766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5398011330 con 0x564d61bea820 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f539f7fe700 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61bf50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f539effd700 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61bf23b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5394015000 tx=0x7f5394009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ea994cc243c3ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f537effd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f537effd700 1 -- 172.21.15.57:0/2525076766 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539402c040 con 0x564d61bea820 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f537effd700 1 -- 172.21.15.57:0/2525076766 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5394005030 con 0x564d61bea820 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/2525076766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d61b05f30 con 0x564d61bea820 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f537effd700 1 -- 172.21.15.57:0/2525076766 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f539401e070 con 0x564d61bea820 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f53b1dd6b80 10 client.16410 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/2525076766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564d61951ce0 con 0x564d61bea820 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f537effd700 1 -- 172.21.15.57:0/2525076766 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d61b05f30 con 0x564d61bea820 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.155+0000 7f537effd700 1 -- 172.21.15.57:0/2525076766 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5394004b70 con 0x564d61bea820 2023-10-08T13:53:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.156+0000 7f53b1dd6b80 10 client.16410 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.156+0000 7f53b1dd6b80 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.156+0000 7f53b1dd6b80 20 client.16410 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.156+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564d61c096f0 0x564d61c0bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.156+0000 7f53b1dd6b80 4 client.16410 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.156+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/2525076766 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d61b36e90 con 0x564d61c096f0 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.156+0000 7f539f7fe700 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564d61c096f0 0x564d61c0bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.156+0000 7f539f7fe700 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564d61c096f0 0x564d61c0bb10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76c8d6cf9bfc0891 server_cookie=f1f1faea435c5199 in_seq=0 out_seq=0 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.156+0000 7f537effd700 10 client.16410 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f537effd700 1 -- 172.21.15.57:0/2525076766 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564d61b36e90 con 0x564d61c096f0 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f537effd700 10 client.16410 handle_command_reply: tid=0 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 client.16410 shutdown 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/2525076766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 msgr2=0x564d61bf23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61bf23b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5394015000 tx=0x7f5394009ca0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/2525076766 shutdown_connections 2023-10-08T13:53:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564d61c096f0 0x564d61c0bb10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bf2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61bf50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/2525076766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61bf23b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f537effd700 0 client.16410 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/2525076766 >> 172.21.15.57:0/2525076766 conn(0x564d61b5f2e0 msgr2=0x564d61be5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/2525076766 shutdown_connections 2023-10-08T13:53:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.157+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/2525076766 wait complete. 2023-10-08T13:53:09.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.158+0000 7f53b1dd6b80 20 client.16410 trim_cache size 0 max 16384 2023-10-08T13:53:09.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f53b1dd6b80 1 Processor -- start 2023-10-08T13:53:09.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f53b1dd6b80 1 -- start start 2023-10-08T13:53:09.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61be57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61becdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bfc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d619be9a0 con 0x564d61bea820 2023-10-08T13:53:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61c09500 con 0x564d61be9f00 2023-10-08T13:53:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61b05f30 con 0x564d61bf56d0 2023-10-08T13:53:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f539effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61becdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f539effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61becdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57700/0 (socket says 172.21.15.57:57700) 2023-10-08T13:53:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f539effd700 1 -- 172.21.15.57:0/1176396038 learned_addr learned my addr 172.21.15.57:0/1176396038 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.176+0000 7f539ffff700 1 --2- 172.21.15.57:0/1176396038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bfc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f539f7fe700 1 --2- 172.21.15.57:0/1176396038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61be57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f539effd700 1 -- 172.21.15.57:0/1176396038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 msgr2=0x564d61bfc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f539effd700 1 --2- 172.21.15.57:0/1176396038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bfc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f539effd700 1 -- 172.21.15.57:0/1176396038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 msgr2=0x564d61be57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f539effd700 1 --2- 172.21.15.57:0/1176396038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61be57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f539effd700 1 -- 172.21.15.57:0/1176396038 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d61c10760 con 0x564d61bea820 2023-10-08T13:53:09.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f539f7fe700 1 --2- 172.21.15.57:0/1176396038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61be57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:53:09.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f539effd700 1 --2- 172.21.15.57:0/1176396038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61becdd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5394020040 tx=0x7f539400fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22faae5c63b021c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f53867fc700 1 -- 172.21.15.57:0/1176396038 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539402c040 con 0x564d61bea820 2023-10-08T13:53:09.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f53867fc700 1 -- 172.21.15.57:0/1176396038 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5394004ce0 con 0x564d61bea820 2023-10-08T13:53:09.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.177+0000 7f53867fc700 1 -- 172.21.15.57:0/1176396038 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394031820 con 0x564d61bea820 2023-10-08T13:53:09.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1176396038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 msgr2=0x564d61becdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/1176396038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61becdd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5394020040 tx=0x7f539400fad0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1176396038 shutdown_connections 2023-10-08T13:53:09.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/1176396038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bfc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/1176396038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61be9f00 0x564d61be57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/1176396038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61bea820 0x564d61becdd0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1176396038 >> 172.21.15.57:0/1176396038 conn(0x564d61b5f2e0 msgr2=0x564d61bed730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1176396038 shutdown_connections 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1176396038 wait complete. 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 -- start start 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.178+0000 7f53b1dd6b80 1 Processor -- start 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61be9f00 0x564d61bfabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61bea820 0x564d61bfb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f53b1dd6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bfb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61c5d6d0 con 0x564d61be9f00 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61c5d840 con 0x564d61bea820 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f53b1dd6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d61c5da00 con 0x564d61bf56d0 2023-10-08T13:53:09.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f53b1dd6b80 10 client.0 _ll_register_callbacks cb 0x7f539400bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61be9f00 0x564d61bfabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61be9f00 0x564d61bfabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57704/0 (socket says 172.21.15.57:57704) 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539f7fe700 1 -- 172.21.15.57:0/1156442680 learned_addr learned my addr 172.21.15.57:0/1156442680 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539ffff700 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bfb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539effd700 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61bea820 0x564d61bfb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539ffff700 1 -- 172.21.15.57:0/1156442680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61bea820 msgr2=0x564d61bfb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539ffff700 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61bea820 0x564d61bfb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539ffff700 1 -- 172.21.15.57:0/1156442680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61be9f00 msgr2=0x564d61bfabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539ffff700 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61be9f00 0x564d61bfabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539ffff700 1 -- 172.21.15.57:0/1156442680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d61951ce0 con 0x564d61bf56d0 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f539ffff700 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bfb660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5398004430 tx=0x7f5398004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69f4402a9d7200b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.179+0000 7f537effd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f537effd700 1 -- 172.21.15.57:0/1156442680 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539800e040 con 0x564d61bf56d0 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f537effd700 1 -- 172.21.15.57:0/1156442680 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5398004460 con 0x564d61bf56d0 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1156442680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f539800f040 con 0x564d61bf56d0 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f537effd700 1 -- 172.21.15.57:0/1156442680 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5398004620 con 0x564d61bf56d0 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f53b1dd6b80 10 client.26014 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1156442680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564d6198f2d0 con 0x564d61bf56d0 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f537effd700 1 -- 172.21.15.57:0/1156442680 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f539800f040 con 0x564d61bf56d0 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f537effd700 1 -- 172.21.15.57:0/1156442680 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f539802d080 con 0x564d61bf56d0 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f53b1dd6b80 10 client.26014 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f53b1dd6b80 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f53b1dd6b80 20 client.26014 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:09.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564d61c5f260 0x564d61c61680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f53b1dd6b80 4 client.26014 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.180+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1156442680 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564d61b36bb0 con 0x564d61c5f260 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.181+0000 7f539f7fe700 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564d61c5f260 0x564d61c61680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.181+0000 7f539f7fe700 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564d61c5f260 0x564d61c61680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec4f37e4d88d157 server_cookie=201984af56468bcf in_seq=0 out_seq=0 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.181+0000 7f537effd700 10 client.26014 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f537effd700 1 -- 172.21.15.57:0/1156442680 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564d61b36bb0 con 0x564d61c5f260 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f537effd700 10 client.26014 handle_command_reply: tid=0 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 client.26014 shutdown 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1156442680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 msgr2=0x564d61bfb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bfb660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5398004430 tx=0x7f5398004260 comp rx=0 tx=0).stop 2023-10-08T13:53:09.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1156442680 shutdown_connections 2023-10-08T13:53:09.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564d61c5f260 0x564d61c61680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564d61bf56d0 0x564d61bfb660 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f537effd700 0 client.26014 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:09.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564d61be9f00 0x564d61bfabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 --2- 172.21.15.57:0/1156442680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d61bea820 0x564d61bfb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1156442680 >> 172.21.15.57:0/1156442680 conn(0x564d61b5f2e0 msgr2=0x564d61bf0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1156442680 shutdown_connections 2023-10-08T13:53:09.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 1 -- 172.21.15.57:0/1156442680 wait complete. 2023-10-08T13:53:09.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.182+0000 7f53b1dd6b80 20 client.26014 trim_cache size 0 max 16384 2023-10-08T13:53:09.189 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:53:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "return_code": 0, 2023-10-08T13:53:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_tag": "6db03f06-e888-4427-a580-5fa3643c8e0d", 2023-10-08T13:53:09.190 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": "asynchronous" 2023-10-08T13:53:09.190 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:53:09.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 -- 172.21.15.57:0/562150659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f53880a15f0 msgr2=0x7f53880a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f53880a15f0 0x7f53880a3ab0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f539400b2e0 tx=0x7f539400b650 comp rx=0 tx=0).stop 2023-10-08T13:53:09.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 -- 172.21.15.57:0/562150659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53a00ff640 msgr2=0x7f53a019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53a00ff640 0x7f53a019c980 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f539000e9a0 tx=0x7f539000ecb0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 -- 172.21.15.57:0/562150659 shutdown_connections 2023-10-08T13:53:09.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53a01056e0 0x7f53a0196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53a00ff640 0x7f53a019c980 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f53880a15f0 0x7f53880a3ab0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 --2- 172.21.15.57:0/562150659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a0100000 0x7f53a019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 -- 172.21.15.57:0/562150659 >> 172.21.15.57:0/562150659 conn(0x7f53a0076760 msgr2=0x7f53a01039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 -- 172.21.15.57:0/562150659 shutdown_connections 2023-10-08T13:53:09.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.183+0000 7f53a5bed700 1 -- 172.21.15.57:0/562150659 wait complete. 2023-10-08T13:53:09.197 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:53:09.222 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:08 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:53:09.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:08 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:53:09.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.327+0000 7f491ba56700 1 -- 172.21.15.57:0/1652545265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49140ff640 msgr2=0x7f49140ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.327+0000 7f491ba56700 1 --2- 172.21.15.57:0/1652545265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49140ff640 0x7f49140ffac0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4904009990 tx=0x7f4904009ca0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.327+0000 7f491ba56700 1 -- 172.21.15.57:0/1652545265 shutdown_connections 2023-10-08T13:53:09.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.327+0000 7f491ba56700 1 --2- 172.21.15.57:0/1652545265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4914100000 0x7f491410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.327+0000 7f491ba56700 1 --2- 172.21.15.57:0/1652545265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4914105650 0x7f4914105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.327+0000 7f491ba56700 1 --2- 172.21.15.57:0/1652545265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49140ff640 0x7f49140ffac0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.327+0000 7f491ba56700 1 -- 172.21.15.57:0/1652545265 >> 172.21.15.57:0/1652545265 conn(0x7f49140767c0 msgr2=0x7f4914076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.327+0000 7f491ba56700 1 -- 172.21.15.57:0/1652545265 shutdown_connections 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.327+0000 7f491ba56700 1 -- 172.21.15.57:0/1652545265 wait complete. 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f491ba56700 1 Processor -- start 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f491ba56700 1 -- start start 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f491ba56700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49140ff640 0x7f4914195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f491ba56700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4914100000 0x7f4914195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f491ba56700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4914105650 0x7f4914196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f491ba56700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49141a4f20 con 0x7f49140ff640 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f49197f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49140ff640 0x7f4914195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f49197f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49140ff640 0x7f4914195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57718/0 (socket says 172.21.15.57:57718) 2023-10-08T13:53:09.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f49197f2700 1 -- 172.21.15.57:0/1292256161 learned_addr learned my addr 172.21.15.57:0/1292256161 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f491ba56700 1 -- 172.21.15.57:0/1292256161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49141a5090 con 0x7f4914105650 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f491ba56700 1 -- 172.21.15.57:0/1292256161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49141a5250 con 0x7f4914100000 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f4918ff1700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4914100000 0x7f4914195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f49197f2700 1 -- 172.21.15.57:0/1292256161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4914100000 msgr2=0x7f4914195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f49197f2700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4914100000 0x7f4914195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f49197f2700 1 -- 172.21.15.57:0/1292256161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4914105650 msgr2=0x7f4914196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f4919ff3700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4914105650 0x7f4914196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f49197f2700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4914105650 0x7f4914196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f49197f2700 1 -- 172.21.15.57:0/1292256161 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49040095f0 con 0x7f49140ff640 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.328+0000 7f49197f2700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49140ff640 0x7f4914195810 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f491000e9a0 tx=0x7f491000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.329+0000 7f490a7fc700 1 -- 172.21.15.57:0/1292256161 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910009790 con 0x7f49140ff640 2023-10-08T13:53:09.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.329+0000 7f491ba56700 1 -- 172.21.15.57:0/1292256161 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49141a5530 con 0x7f49140ff640 2023-10-08T13:53:09.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.329+0000 7f490a7fc700 1 -- 172.21.15.57:0/1292256161 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f491000cbd0 con 0x7f49140ff640 2023-10-08T13:53:09.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.329+0000 7f490a7fc700 1 -- 172.21.15.57:0/1292256161 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910018680 con 0x7f49140ff640 2023-10-08T13:53:09.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.329+0000 7f491ba56700 1 -- 172.21.15.57:0/1292256161 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49141a5a10 con 0x7f49140ff640 2023-10-08T13:53:09.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.332+0000 7f490a7fc700 1 -- 172.21.15.57:0/1292256161 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f491000cd40 con 0x7f49140ff640 2023-10-08T13:53:09.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.332+0000 7f4927c3fb80 1 Processor -- start 2023-10-08T13:53:09.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4927c3fb80 1 -- start start 2023-10-08T13:53:09.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd1692260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd16927a0 0x561cd169d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd15fb460 con 0x561cd169d650 2023-10-08T13:53:09.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd15ecc20 con 0x561cd16927a0 2023-10-08T13:53:09.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd15ea540 con 0x561cd1691e80 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4919ff3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd1692260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4918ff1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4918ff1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57728/0 (socket says 172.21.15.57:57728) 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4919ff3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd1692260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58536/0 (socket says 172.21.15.57:58536) 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4918ff1700 1 -- 172.21.15.57:0/4179563419 learned_addr learned my addr 172.21.15.57:0/4179563419 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f49197f2700 1 --2- 172.21.15.57:0/4179563419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd16927a0 0x561cd169d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f490a7fc700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f49000a15f0 0x7f49000a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4918ff1700 1 -- 172.21.15.57:0/4179563419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 msgr2=0x561cd1692260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4918ff1700 1 --2- 172.21.15.57:0/4179563419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd1692260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4918ff1700 1 -- 172.21.15.57:0/4179563419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd16927a0 msgr2=0x561cd169d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4918ff1700 1 --2- 172.21.15.57:0/4179563419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd16927a0 0x561cd169d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4918ff1700 1 -- 172.21.15.57:0/4179563419 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cd1410de0 con 0x561cd169d650 2023-10-08T13:53:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.333+0000 7f4919ff3700 1 --2- 172.21.15.57:0/4179563419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd1692260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.334+0000 7f4919ff3700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f49000a15f0 0x7f49000a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.334+0000 7f490a7fc700 1 -- 172.21.15.57:0/1292256161 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f4910014070 con 0x7f49140ff640 2023-10-08T13:53:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.334+0000 7f4918ff1700 1 --2- 172.21.15.57:0/4179563419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169fa40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4904009990 tx=0x7f490400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70a87be0c84e3fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.334+0000 7f48ff7fe700 1 -- 172.21.15.57:0/4179563419 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490401d070 con 0x561cd169d650 2023-10-08T13:53:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.334+0000 7f48ff7fe700 1 -- 172.21.15.57:0/4179563419 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490401e040 con 0x561cd169d650 2023-10-08T13:53:09.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.334+0000 7f48ff7fe700 1 -- 172.21.15.57:0/4179563419 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904004920 con 0x561cd169d650 2023-10-08T13:53:09.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4919ff3700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f49000a15f0 0x7f49000a3ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f490c007720 tx=0x7f490c00a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 -- 172.21.15.57:0/4179563419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 msgr2=0x561cd169fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/4179563419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169fa40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4904009990 tx=0x7f490400b690 comp rx=0 tx=0).stop 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 -- 172.21.15.57:0/4179563419 shutdown_connections 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/4179563419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169fa40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/4179563419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd1692260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/4179563419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd16927a0 0x561cd169d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 -- 172.21.15.57:0/4179563419 >> 172.21.15.57:0/4179563419 conn(0x561cd1607260 msgr2=0x561cd168e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 -- 172.21.15.57:0/4179563419 shutdown_connections 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 -- 172.21.15.57:0/4179563419 wait complete. 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 -- start start 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.335+0000 7f4927c3fb80 1 Processor -- start 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd169cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd16927a0 0x561cd169a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd15e5e30 con 0x561cd169d650 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd1582710 con 0x561cd16927a0 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd15823b0 con 0x561cd1691e80 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4927c3fb80 10 client.0 _ll_register_callbacks cb 0x561cd13df800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4919ff3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f49197f2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd169cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4919ff3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57736/0 (socket says 172.21.15.57:57736) 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4919ff3700 1 -- 172.21.15.57:0/3395716451 learned_addr learned my addr 172.21.15.57:0/3395716451 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f4918ff1700 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd16927a0 0x561cd169a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f49197f2700 1 -- 172.21.15.57:0/3395716451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd16927a0 msgr2=0x561cd169a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f49197f2700 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd16927a0 0x561cd169a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f49197f2700 1 -- 172.21.15.57:0/3395716451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 msgr2=0x561cd169a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f49197f2700 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.336+0000 7f49197f2700 1 -- 172.21.15.57:0/3395716451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c008c10 con 0x561cd1691e80 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f49197f2700 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd169cf80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49100caa70 tx=0x7f4910099e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ce8548f1eaaf59 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f48fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f48fcff9700 1 -- 172.21.15.57:0/3395716451 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491009a760 con 0x561cd1691e80 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f48fcff9700 1 -- 172.21.15.57:0/3395716451 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f49100cd4d0 con 0x561cd1691e80 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3395716451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cd15adeb0 con 0x561cd1691e80 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f48fcff9700 1 -- 172.21.15.57:0/3395716451 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f491009aa00 con 0x561cd1691e80 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f4927c3fb80 10 client.26026 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3395716451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f491410ce10 con 0x561cd1691e80 2023-10-08T13:53:09.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f48fcff9700 1 -- 172.21.15.57:0/3395716451 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561cd15adeb0 con 0x561cd1691e80 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f48fcff9700 1 -- 172.21.15.57:0/3395716451 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f49100997f0 con 0x561cd1691e80 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f4927c3fb80 10 client.26026 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f4927c3fb80 10 client.26026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f4927c3fb80 20 client.26026 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561cd16ad090 0x561cd16af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f4927c3fb80 4 client.26026 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.337+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3395716451 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cd15dee10 con 0x561cd16ad090 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.338+0000 7f4918ff1700 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561cd16ad090 0x561cd16af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.338+0000 7f4918ff1700 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561cd16ad090 0x561cd16af4b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa059189ff186b7b server_cookie=ccd8ec8237be4630 in_seq=0 out_seq=0 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.338+0000 7f48fcff9700 10 client.26026 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f48fcff9700 1 -- 172.21.15.57:0/3395716451 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561cd15dee10 con 0x561cd16ad090 2023-10-08T13:53:09.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f48fcff9700 10 client.26026 handle_command_reply: tid=0 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 client.26026 shutdown 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3395716451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 msgr2=0x561cd169cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd169cf80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49100caa70 tx=0x7f4910099e40 comp rx=0 tx=0).stop 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3395716451 shutdown_connections 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561cd16ad090 0x561cd16af4b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd169a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd1691e80 0x561cd169cf80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f48fcff9700 0 client.26026 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3395716451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd16927a0 0x561cd169a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3395716451 >> 172.21.15.57:0/3395716451 conn(0x561cd1607260 msgr2=0x561cd168dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3395716451 shutdown_connections 2023-10-08T13:53:09.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3395716451 wait complete. 2023-10-08T13:53:09.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.339+0000 7f4927c3fb80 20 client.26026 trim_cache size 0 max 16384 2023-10-08T13:53:09.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.356+0000 7f4927c3fb80 1 Processor -- start 2023-10-08T13:53:09.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.356+0000 7f4927c3fb80 1 -- start start 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd1691e80 0x561cd16a4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd169cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd1698c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd1466920 con 0x561cd169d650 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd1692b80 con 0x561cd1691e80 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd16930d0 con 0x561cd16927a0 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4918ff1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd169cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4919ff3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd1698c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4918ff1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd169cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58554/0 (socket says 172.21.15.57:58554) 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4918ff1700 1 -- 172.21.15.57:0/3809665618 learned_addr learned my addr 172.21.15.57:0/3809665618 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:09.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f49197f2700 1 --2- 172.21.15.57:0/3809665618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd1691e80 0x561cd16a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4918ff1700 1 -- 172.21.15.57:0/3809665618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd1691e80 msgr2=0x561cd16a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4918ff1700 1 --2- 172.21.15.57:0/3809665618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd1691e80 0x561cd16a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4918ff1700 1 -- 172.21.15.57:0/3809665618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 msgr2=0x561cd1698c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4918ff1700 1 --2- 172.21.15.57:0/3809665618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd1698c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4918ff1700 1 -- 172.21.15.57:0/3809665618 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cd13f9c60 con 0x561cd16927a0 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f49197f2700 1 --2- 172.21.15.57:0/3809665618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd1691e80 0x561cd16a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4919ff3700 1 --2- 172.21.15.57:0/3809665618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd1698c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f4918ff1700 1 --2- 172.21.15.57:0/3809665618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd169cc20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4904009990 tx=0x7f490401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d609df7eed9bda server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f48fffff700 1 -- 172.21.15.57:0/3809665618 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904031070 con 0x561cd16927a0 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.357+0000 7f48fffff700 1 -- 172.21.15.57:0/3809665618 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490400bc10 con 0x561cd16927a0 2023-10-08T13:53:09.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f48fffff700 1 -- 172.21.15.57:0/3809665618 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490401d040 con 0x561cd16927a0 2023-10-08T13:53:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3809665618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 msgr2=0x561cd169cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3809665618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd169cc20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4904009990 tx=0x7f490401a040 comp rx=0 tx=0).stop 2023-10-08T13:53:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3809665618 shutdown_connections 2023-10-08T13:53:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3809665618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd169d650 0x561cd1698c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3809665618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd1691e80 0x561cd16a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3809665618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd169cc20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3809665618 >> 172.21.15.57:0/3809665618 conn(0x561cd1607260 msgr2=0x561cd168d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3809665618 shutdown_connections 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3809665618 wait complete. 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.358+0000 7f4927c3fb80 1 -- start start 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f4927c3fb80 1 Processor -- start 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd1691e80 0x561cd16a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd16a2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f4927c3fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd169d650 0x561cd16a3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd16a3e30 con 0x561cd1691e80 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd16a3fa0 con 0x561cd169d650 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f4927c3fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cd16a4160 con 0x561cd16927a0 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f4927c3fb80 10 client.0 _ll_register_callbacks cb 0x7f4910096640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:09.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f49197f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd1691e80 0x561cd16a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f49197f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd1691e80 0x561cd16a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57746/0 (socket says 172.21.15.57:57746) 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f49197f2700 1 -- 172.21.15.57:0/3124573943 learned_addr learned my addr 172.21.15.57:0/3124573943 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.359+0000 7f4918ff1700 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd16a2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f4919ff3700 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd169d650 0x561cd16a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f49197f2700 1 -- 172.21.15.57:0/3124573943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 msgr2=0x561cd16a2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f49197f2700 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd16a2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f49197f2700 1 -- 172.21.15.57:0/3124573943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd169d650 msgr2=0x561cd16a3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f49197f2700 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd169d650 0x561cd16a3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f49197f2700 1 -- 172.21.15.57:0/3124573943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c015570 con 0x561cd1691e80 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f4918ff1700 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd16a2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f49197f2700 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd1691e80 0x561cd16a2430 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4910098650 tx=0x7f49100092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c01548866a5fffd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f48fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f48fcff9700 1 -- 172.21.15.57:0/3124573943 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491009aa00 con 0x561cd1691e80 2023-10-08T13:53:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f48fcff9700 1 -- 172.21.15.57:0/3124573943 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f491009a300 con 0x561cd1691e80 2023-10-08T13:53:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3124573943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f490401e040 con 0x561cd1691e80 2023-10-08T13:53:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f48fcff9700 1 -- 172.21.15.57:0/3124573943 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f491009e450 con 0x561cd1691e80 2023-10-08T13:53:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f4927c3fb80 10 client.16434 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3124573943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561cd1410de0 con 0x561cd1691e80 2023-10-08T13:53:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.360+0000 7f48fcff9700 1 -- 172.21.15.57:0/3124573943 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490401e040 con 0x561cd1691e80 2023-10-08T13:53:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f48fcff9700 1 -- 172.21.15.57:0/3124573943 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f4910099420 con 0x561cd1691e80 2023-10-08T13:53:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4927c3fb80 10 client.16434 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4927c3fb80 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4927c3fb80 20 client.16434 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561cd1701d90 0x561cd17041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4927c3fb80 4 client.16434 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3124573943 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561cd15deb30 con 0x561cd1701d90 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4918ff1700 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561cd1701d90 0x561cd17041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4918ff1700 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561cd1701d90 0x561cd17041b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6ac9d06624ca11 server_cookie=3214d69523d4fcbd in_seq=0 out_seq=0 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f48fcff9700 10 client.16434 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f48fcff9700 1 -- 172.21.15.57:0/3124573943 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561cd15deb30 con 0x561cd1701d90 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f48fcff9700 10 client.16434 handle_command_reply: tid=0 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4927c3fb80 1 client.16434 shutdown 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3124573943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd1691e80 msgr2=0x561cd16a2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.361+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd1691e80 0x561cd16a2430 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4910098650 tx=0x7f49100092d0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3124573943 shutdown_connections 2023-10-08T13:53:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561cd1701d90 0x561cd17041b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cd169d650 0x561cd16a3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f48fcff9700 0 client.16434 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:09.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561cd1691e80 0x561cd16a2430 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f4927c3fb80 1 --2- 172.21.15.57:0/3124573943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd16927a0 0x561cd16a2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3124573943 >> 172.21.15.57:0/3124573943 conn(0x561cd1607260 msgr2=0x561cd169cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3124573943 shutdown_connections 2023-10-08T13:53:09.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f4927c3fb80 1 -- 172.21.15.57:0/3124573943 wait complete. 2023-10-08T13:53:09.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f4927c3fb80 20 client.16434 trim_cache size 0 max 16384 2023-10-08T13:53:09.370 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:53:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (0 inodes in the stack)", 2023-10-08T13:53:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:53:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "6db03f06-e888-4427-a580-5fa3643c8e0d": { 2023-10-08T13:53:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:53:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "6db03f06-e888-4427-a580-5fa3643c8e0d", 2023-10-08T13:53:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:53:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:53:09.371 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:53:09.371 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:53:09.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 -- 172.21.15.57:0/1292256161 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f49000a15f0 msgr2=0x7f49000a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f49000a15f0 0x7f49000a3ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f490c007720 tx=0x7f490c00a040 comp rx=0 tx=0).stop 2023-10-08T13:53:09.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 -- 172.21.15.57:0/1292256161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49140ff640 msgr2=0x7f4914195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:09.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49140ff640 0x7f4914195810 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f491000e9a0 tx=0x7f491000ecb0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 -- 172.21.15.57:0/1292256161 shutdown_connections 2023-10-08T13:53:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f49000a15f0 0x7f49000a3ab0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4914105650 0x7f4914196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49140ff640 0x7f4914195810 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 --2- 172.21.15.57:0/1292256161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4914100000 0x7f4914195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 -- 172.21.15.57:0/1292256161 >> 172.21.15.57:0/1292256161 conn(0x7f49140767c0 msgr2=0x7f49140fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 -- 172.21.15.57:0/1292256161 shutdown_connections 2023-10-08T13:53:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:09.362+0000 7f491ba56700 1 -- 172.21.15.57:0/1292256161 wait complete. 2023-10-08T13:53:09.375 INFO:tasks.cephfs.filesystem:scrub status for tag:6db03f06-e888-4427-a580-5fa3643c8e0d - {'path': '/', 'tag': '6db03f06-e888-4427-a580-5fa3643c8e0d', 'options': 'recursive,force'} 2023-10-08T13:53:09.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:08 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:53:10.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:09 smithi057 ceph-mon[99476]: pgmap v347: 129 pgs: 129 active+clean; 43 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 679 KiB/s wr, 64 op/s 2023-10-08T13:53:10.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:09 smithi057 ceph-mon[99476]: scrub queued for path: / 2023-10-08T13:53:10.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:09 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:10.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:09 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:09 smithi099 ceph-mon[103849]: pgmap v347: 129 pgs: 129 active+clean; 43 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 679 KiB/s wr, 64 op/s 2023-10-08T13:53:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:09 smithi099 ceph-mon[103849]: scrub queued for path: / 2023-10-08T13:53:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:09 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:09 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:09 smithi115 ceph-mon[104408]: pgmap v347: 129 pgs: 129 active+clean; 43 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 679 KiB/s wr, 64 op/s 2023-10-08T13:53:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:09 smithi115 ceph-mon[104408]: scrub queued for path: / 2023-10-08T13:53:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:09 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:10.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:09 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:11.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:10 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:11.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:10 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:11.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:10 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:11.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:10 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:11.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:10 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:11.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:10 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:11.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:10 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:11.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:10 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:11.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:10 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:11.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:10 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:11.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:10 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:11.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:10 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:11.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:10 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:11.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:10 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:11.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:10 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:11.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:10 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:11.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:10 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:11.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:10 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:12.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:11 smithi057 ceph-mon[99476]: pgmap v348: 129 pgs: 129 active+clean; 43 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 679 KiB/s wr, 64 op/s 2023-10-08T13:53:12.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:11 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:12.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:11 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:12.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:11 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:12.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:11 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:11 smithi099 ceph-mon[103849]: pgmap v348: 129 pgs: 129 active+clean; 43 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 679 KiB/s wr, 64 op/s 2023-10-08T13:53:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:11 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:11 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:11 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:11 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:12.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:11 smithi115 ceph-mon[104408]: pgmap v348: 129 pgs: 129 active+clean; 43 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 679 KiB/s wr, 64 op/s 2023-10-08T13:53:12.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:11 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:12.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:11 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:12.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:11 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:12.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:11 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.254 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.256 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.258 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.259 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.260 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.260 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.260 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.260 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.260 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.260 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.260 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.260 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.260 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:13.260 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:12 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:13.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:12 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:13.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:12 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: pgmap v349: 129 pgs: 129 active+clean; 52 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 62 op/s 2023-10-08T13:53:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.249 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.251 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:14.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:13 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: pgmap v349: 129 pgs: 129 active+clean; 52 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 62 op/s 2023-10-08T13:53:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.283 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.284 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.285 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.286 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.287 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.288 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.289 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.291 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.293 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:14.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:13 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.364 INFO:tasks.workunit.client.0.smithi057.stdout: 33 94377 958 59651 975 27550 705 2023-10-08T13:53:14.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: pgmap v349: 129 pgs: 129 active+clean; 52 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 62 op/s 2023-10-08T13:53:14.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:14.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:13 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: pgmap v350: 129 pgs: 129 active+clean; 55 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 71 op/s 2023-10-08T13:53:16.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.444 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.445 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.446 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.447 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.448 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.449 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.450 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.451 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.452 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.453 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.454 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.455 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.456 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.457 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:15 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.458 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.459 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.460 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.462 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.463 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.464 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.465 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.467 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.468 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.469 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.470 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.470 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.470 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.470 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.470 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.470 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.470 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.470 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.470 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.470 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.473 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.474 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.475 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.476 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.477 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.478 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.478 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.478 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.478 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.478 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.478 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.478 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.478 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.478 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.478 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.479 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.480 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.481 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.482 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.483 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.484 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.485 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.486 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:53:16.487 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:16 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: pgmap v350: 129 pgs: 129 active+clean; 55 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 71 op/s 2023-10-08T13:53:16.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.499 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.501 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.507 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.511 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.516 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.522 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.523 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.524 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.525 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:15 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:53:16.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:16 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.538 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: pgmap v350: 129 pgs: 129 active+clean; 55 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 71 op/s 2023-10-08T13:53:16.538 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.538 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.538 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.538 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.538 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.538 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.539 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.540 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.541 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.543 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.551 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.552 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.553 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.554 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.555 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.556 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.558 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.563 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.568 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.569 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.570 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.571 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.574 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.575 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.576 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.577 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.580 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:16.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:53:16.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:16 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:53:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.171+0000 7fd00e90b700 1 -- 172.21.15.57:0/922878027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd008105ad0 msgr2=0x7fd00810d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.171+0000 7fd00e90b700 1 --2- 172.21.15.57:0/922878027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd008105ad0 0x7fd00810d4e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd000009fc0 tx=0x7fd00000b330 comp rx=0 tx=0).stop 2023-10-08T13:53:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.171+0000 7fd00e90b700 1 -- 172.21.15.57:0/922878027 shutdown_connections 2023-10-08T13:53:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.171+0000 7fd00e90b700 1 --2- 172.21.15.57:0/922878027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd008105ad0 0x7fd00810d4e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.171+0000 7fd00e90b700 1 --2- 172.21.15.57:0/922878027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd008067fd0 0x7fd0080683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.171+0000 7fd00e90b700 1 --2- 172.21.15.57:0/922878027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd008105110 0x7fd008105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.171+0000 7fd00e90b700 1 -- 172.21.15.57:0/922878027 >> 172.21.15.57:0/922878027 conn(0x7fd008076790 msgr2=0x7fd008076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.172+0000 7fd00e90b700 1 -- 172.21.15.57:0/922878027 shutdown_connections 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.172+0000 7fd00e90b700 1 -- 172.21.15.57:0/922878027 wait complete. 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.172+0000 7fd00e90b700 1 Processor -- start 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.172+0000 7fd00e90b700 1 -- start start 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd00e90b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd008067fd0 0x7fd00819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd00e90b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd008105110 0x7fd00819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd00e90b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd008105ad0 0x7fd008196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd00e90b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd008197100 con 0x7fd008105110 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd00e90b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd008197270 con 0x7fd008105ad0 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd00e90b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd008197430 con 0x7fd008067fd0 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd007fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd008067fd0 0x7fd00819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd0077fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd008105110 0x7fd00819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd007fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd008067fd0 0x7fd00819c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58572/0 (socket says 172.21.15.57:58572) 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd007fff700 1 -- 172.21.15.57:0/975906756 learned_addr learned my addr 172.21.15.57:0/975906756 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd00cea8700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd008105ad0 0x7fd008196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd007fff700 1 -- 172.21.15.57:0/975906756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd008105ad0 msgr2=0x7fd008196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd007fff700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd008105ad0 0x7fd008196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd007fff700 1 -- 172.21.15.57:0/975906756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd008105110 msgr2=0x7fd00819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd007fff700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd008105110 0x7fd00819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd007fff700 1 -- 172.21.15.57:0/975906756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd000009c40 con 0x7fd008067fd0 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd00cea8700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd008105ad0 0x7fd008196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd0077fe700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd008105110 0x7fd00819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd007fff700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd008067fd0 0x7fd00819c920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcff800e9a0 tx=0x7fcff800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd0057fa700 1 -- 172.21.15.57:0/975906756 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8009790 con 0x7fd008067fd0 2023-10-08T13:53:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.173+0000 7fd00e90b700 1 -- 172.21.15.57:0/975906756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd008197710 con 0x7fd008067fd0 2023-10-08T13:53:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.174+0000 7fd00e90b700 1 -- 172.21.15.57:0/975906756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0081a3910 con 0x7fd008067fd0 2023-10-08T13:53:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.174+0000 7fd0057fa700 1 -- 172.21.15.57:0/975906756 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcff800ce80 con 0x7fd008067fd0 2023-10-08T13:53:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.174+0000 7fd0057fa700 1 -- 172.21.15.57:0/975906756 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8018420 con 0x7fd008067fd0 2023-10-08T13:53:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.176+0000 7fd0057fa700 1 -- 172.21.15.57:0/975906756 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcff80106a0 con 0x7fd008067fd0 2023-10-08T13:53:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.176+0000 7fd0057fa700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcff00a15f0 0x7fcff00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.176+0000 7fd0057fa700 1 -- 172.21.15.57:0/975906756 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcff8014070 con 0x7fd008067fd0 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.176+0000 7fd0077fe700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcff00a15f0 0x7fcff00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd01aaf4b80 1 Processor -- start 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd01aaf4b80 1 -- start start 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd0077fe700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcff00a15f0 0x7fcff00a3ab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd008197c10 tx=0x7fcffc009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb86a880 0x55a7bb86ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb864b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb8650b0 0x55a7bb865510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb7cd400 con 0x55a7bb86b1a0 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb7bebc0 con 0x55a7bb86a880 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb7bc4e0 con 0x55a7bb8650b0 2023-10-08T13:53:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd0077fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb864b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd0077fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb864b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57770/0 (socket says 172.21.15.57:57770) 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd0077fe700 1 -- 172.21.15.57:0/1261065832 learned_addr learned my addr 172.21.15.57:0/1261065832 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd007fff700 1 --2- 172.21.15.57:0/1261065832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb86a880 0x55a7bb86ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd00cea8700 1 --2- 172.21.15.57:0/1261065832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb8650b0 0x55a7bb865510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.177+0000 7fd0077fe700 1 -- 172.21.15.57:0/1261065832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb8650b0 msgr2=0x55a7bb865510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.178+0000 7fd0077fe700 1 --2- 172.21.15.57:0/1261065832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb8650b0 0x55a7bb865510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.178+0000 7fd0077fe700 1 -- 172.21.15.57:0/1261065832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb86a880 msgr2=0x55a7bb86ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.178+0000 7fd0077fe700 1 --2- 172.21.15.57:0/1261065832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb86a880 0x55a7bb86ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.178+0000 7fd0077fe700 1 -- 172.21.15.57:0/1261065832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7bb5e2d80 con 0x55a7bb86b1a0 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.178+0000 7fd0077fe700 1 --2- 172.21.15.57:0/1261065832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb864b70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcffc004380 tx=0x7fcffc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec2515d93d17009 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.178+0000 7fcfee7fc700 1 -- 172.21.15.57:0/1261065832 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc003c90 con 0x55a7bb86b1a0 2023-10-08T13:53:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.178+0000 7fcfee7fc700 1 -- 172.21.15.57:0/1261065832 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcffc003df0 con 0x55a7bb86b1a0 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.178+0000 7fcfee7fc700 1 -- 172.21.15.57:0/1261065832 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc00e6c0 con 0x55a7bb86b1a0 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.178+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/1261065832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 msgr2=0x55a7bb864b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.178+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/1261065832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb864b70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcffc004380 tx=0x7fcffc014000 comp rx=0 tx=0).stop 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/1261065832 shutdown_connections 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/1261065832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb8650b0 0x55a7bb865510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/1261065832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb86a880 0x55a7bb86ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/1261065832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb864b70 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/1261065832 >> 172.21.15.57:0/1261065832 conn(0x55a7bb7d9200 msgr2=0x55a7bb85fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/1261065832 shutdown_connections 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/1261065832 wait complete. 2023-10-08T13:53:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 -- start start 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 Processor -- start 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb8711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86a880 0x55a7bb86e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86b1a0 0x55a7bb86eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb7b7dd0 con 0x55a7bb86a880 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.179+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb7546b0 con 0x55a7bb8650b0 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb754350 con 0x55a7bb86b1a0 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd01aaf4b80 10 client.0 _ll_register_callbacks cb 0x55a7bb5b17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd007fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb8711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd0077fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86a880 0x55a7bb86e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd00cea8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86b1a0 0x55a7bb86eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd007fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb8711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59200/0 (socket says 172.21.15.57:59200) 2023-10-08T13:53:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd007fff700 1 -- 172.21.15.57:0/3877011028 learned_addr learned my addr 172.21.15.57:0/3877011028 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd007fff700 1 -- 172.21.15.57:0/3877011028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86b1a0 msgr2=0x55a7bb86eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd007fff700 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86b1a0 0x55a7bb86eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd007fff700 1 -- 172.21.15.57:0/3877011028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86a880 msgr2=0x55a7bb86e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd007fff700 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86a880 0x55a7bb86e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd007fff700 1 -- 172.21.15.57:0/3877011028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc017070 con 0x55a7bb8650b0 2023-10-08T13:53:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd007fff700 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb8711c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcff80967f0 tx=0x7fcff809a3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54aff4c0851c9e9c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fcfe77fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:53:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3877011028 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff80b0070 con 0x55a7bb8650b0 2023-10-08T13:53:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3877011028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7bb77fe50 con 0x55a7bb8650b0 2023-10-08T13:53:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3877011028 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcff809e0d0 con 0x55a7bb8650b0 2023-10-08T13:53:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3877011028 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcff809a550 con 0x55a7bb8650b0 2023-10-08T13:53:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd01aaf4b80 10 client.25985 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.180+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3877011028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a7bb5cbc00 con 0x55a7bb8650b0 2023-10-08T13:53:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3877011028 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7bb77fe50 con 0x55a7bb8650b0 2023-10-08T13:53:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3877011028 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fcff80a21f0 con 0x55a7bb8650b0 2023-10-08T13:53:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fd01aaf4b80 10 client.25985 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fd01aaf4b80 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fd01aaf4b80 20 client.25985 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a7bb87ecb0 0x55a7bb8810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fd01aaf4b80 4 client.25985 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3877011028 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7bb7b0db0 con 0x55a7bb87ecb0 2023-10-08T13:53:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fd0077fe700 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a7bb87ecb0 0x55a7bb8810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fcfe77fe700 10 client.25985 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.181+0000 7fd0077fe700 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a7bb87ecb0 0x55a7bb8810d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2dc1862541cfda server_cookie=bee82b7cfd605a25 in_seq=0 out_seq=0 2023-10-08T13:53:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3877011028 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a7bb7b0db0 con 0x55a7bb87ecb0 2023-10-08T13:53:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fcfe77fe700 10 client.25985 handle_command_reply: tid=0 2023-10-08T13:53:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 client.25985 shutdown 2023-10-08T13:53:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3877011028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 msgr2=0x55a7bb8711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb8711c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcff80967f0 tx=0x7fcff809a3c0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3877011028 shutdown_connections 2023-10-08T13:53:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a7bb87ecb0 0x55a7bb8810d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86b1a0 0x55a7bb86eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb8711c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3877011028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86a880 0x55a7bb86e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fcfe77fe700 0 client.25985 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3877011028 >> 172.21.15.57:0/3877011028 conn(0x55a7bb7d9200 msgr2=0x55a7bb85f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3877011028 shutdown_connections 2023-10-08T13:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3877011028 wait complete. 2023-10-08T13:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.182+0000 7fd01aaf4b80 20 client.25985 trim_cache size 0 max 16384 2023-10-08T13:53:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.197+0000 7fd01aaf4b80 1 Processor -- start 2023-10-08T13:53:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.197+0000 7fd01aaf4b80 1 -- start start 2023-10-08T13:53:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.197+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb864b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.197+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 0x55a7bb85f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.197+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb85fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.197+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb6388c0 con 0x55a7bb86b1a0 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.197+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb85a2b0 con 0x55a7bb8650b0 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.197+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb87c310 con 0x55a7bb86a880 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd00cea8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb85fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd00cea8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb85fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57786/0 (socket says 172.21.15.57:57786) 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd0077fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 0x55a7bb85f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd00cea8700 1 -- 172.21.15.57:0/1881092249 learned_addr learned my addr 172.21.15.57:0/1881092249 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd007fff700 1 --2- 172.21.15.57:0/1881092249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb864b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd00cea8700 1 -- 172.21.15.57:0/1881092249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 msgr2=0x55a7bb85f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd00cea8700 1 --2- 172.21.15.57:0/1881092249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 0x55a7bb85f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd00cea8700 1 -- 172.21.15.57:0/1881092249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 msgr2=0x55a7bb864b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd00cea8700 1 --2- 172.21.15.57:0/1881092249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb864b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd00cea8700 1 -- 172.21.15.57:0/1881092249 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7bb6091f0 con 0x55a7bb86b1a0 2023-10-08T13:53:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd007fff700 1 --2- 172.21.15.57:0/1881092249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb864b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fd00cea8700 1 --2- 172.21.15.57:0/1881092249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb85fcd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd0000093c0 tx=0x7fd0000093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0eb2c521bac1c14 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fcfeeffd700 1 -- 172.21.15.57:0/1881092249 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000022070 con 0x55a7bb86b1a0 2023-10-08T13:53:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fcfeeffd700 1 -- 172.21.15.57:0/1881092249 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd00000b9f0 con 0x55a7bb86b1a0 2023-10-08T13:53:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.198+0000 7fcfeeffd700 1 -- 172.21.15.57:0/1881092249 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000e040 con 0x55a7bb86b1a0 2023-10-08T13:53:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/1881092249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 msgr2=0x55a7bb85fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/1881092249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb85fcd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd0000093c0 tx=0x7fd0000093f0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/1881092249 shutdown_connections 2023-10-08T13:53:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/1881092249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb86b1a0 0x55a7bb85fcd0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/1881092249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb8650b0 0x55a7bb864b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/1881092249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 0x55a7bb85f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/1881092249 >> 172.21.15.57:0/1881092249 conn(0x55a7bb7d9200 msgr2=0x55a7bb86cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/1881092249 shutdown_connections 2023-10-08T13:53:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/1881092249 wait complete. 2023-10-08T13:53:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 -- start start 2023-10-08T13:53:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.199+0000 7fd01aaf4b80 1 Processor -- start 2023-10-08T13:53:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb8650b0 0x55a7bb873e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 0x55a7bb874360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd01aaf4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb86b1a0 0x55a7bb8d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb874a30 con 0x55a7bb8650b0 2023-10-08T13:53:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb8d6450 con 0x55a7bb86b1a0 2023-10-08T13:53:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd01aaf4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7bb8d6590 con 0x55a7bb86a880 2023-10-08T13:53:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd007fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb8650b0 0x55a7bb873e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd01aaf4b80 10 client.0 _ll_register_callbacks cb 0x7fcff8099c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd007fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb8650b0 0x55a7bb873e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57788/0 (socket says 172.21.15.57:57788) 2023-10-08T13:53:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd007fff700 1 -- 172.21.15.57:0/3377060054 learned_addr learned my addr 172.21.15.57:0/3377060054 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd00cea8700 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb86b1a0 0x55a7bb8d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd0077fe700 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 0x55a7bb874360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd007fff700 1 -- 172.21.15.57:0/3377060054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 msgr2=0x55a7bb874360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd007fff700 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 0x55a7bb874360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd007fff700 1 -- 172.21.15.57:0/3377060054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb86b1a0 msgr2=0x55a7bb8d5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd007fff700 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb86b1a0 0x55a7bb8d5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd007fff700 1 -- 172.21.15.57:0/3377060054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7bb5cbc00 con 0x55a7bb8650b0 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd0077fe700 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 0x55a7bb874360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd007fff700 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb8650b0 0x55a7bb873e20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcff809a280 tx=0x7fcff8010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5099c1bcfa9710e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fcfe77fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3377060054 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff80b0030 con 0x55a7bb8650b0 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3377060054 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fcff80b1070 con 0x55a7bb8650b0 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3377060054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcffc003df0 con 0x55a7bb8650b0 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.200+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3377060054 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcff80997e0 con 0x55a7bb8650b0 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fd01aaf4b80 10 client.16452 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3377060054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fd00810de10 con 0x55a7bb8650b0 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3377060054 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcff80a6080 con 0x55a7bb8650b0 2023-10-08T13:53:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3377060054 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fcff807c020 con 0x55a7bb8650b0 2023-10-08T13:53:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fd01aaf4b80 10 client.16452 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fd01aaf4b80 10 client.16452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fd01aaf4b80 20 client.16452 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a7bb8d7ad0 0x55a7bb8d9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fd01aaf4b80 4 client.16452 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:53:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3377060054 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a7bb7b0ad0 con 0x55a7bb8d7ad0 2023-10-08T13:53:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fd0077fe700 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a7bb8d7ad0 0x55a7bb8d9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fcfe77fe700 10 client.16452 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.201+0000 7fd0077fe700 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a7bb8d7ad0 0x55a7bb8d9ef0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42555a2ada0b50a server_cookie=597e9eb1a9d057da in_seq=0 out_seq=0 2023-10-08T13:53:18.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:17 smithi099 ceph-mon[103849]: pgmap v351: 129 pgs: 129 active+clean; 59 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 95 op/s 2023-10-08T13:53:18.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:17 smithi115 ceph-mon[104408]: pgmap v351: 129 pgs: 129 active+clean; 59 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 95 op/s 2023-10-08T13:53:18.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:17 smithi057 ceph-mon[99476]: pgmap v351: 129 pgs: 129 active+clean; 59 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 95 op/s 2023-10-08T13:53:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fcfe77fe700 1 -- 172.21.15.57:0/3377060054 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a7bb7b0ad0 con 0x55a7bb8d7ad0 2023-10-08T13:53:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fcfe77fe700 10 client.16452 handle_command_reply: tid=0 2023-10-08T13:53:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 client.16452 shutdown 2023-10-08T13:53:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3377060054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb8650b0 msgr2=0x55a7bb873e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb8650b0 0x55a7bb873e20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcff809a280 tx=0x7fcff8010a40 comp rx=0 tx=0).stop 2023-10-08T13:53:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3377060054 shutdown_connections 2023-10-08T13:53:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a7bb8d7ad0 0x55a7bb8d9ef0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bb86b1a0 0x55a7bb8d5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fcfe77fe700 0 client.16452 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a7bb8650b0 0x55a7bb873e20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 --2- 172.21.15.57:0/3377060054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7bb86a880 0x55a7bb874360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3377060054 >> 172.21.15.57:0/3377060054 conn(0x55a7bb7d9200 msgr2=0x55a7bb87a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3377060054 shutdown_connections 2023-10-08T13:53:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 1 -- 172.21.15.57:0/3377060054 wait complete. 2023-10-08T13:53:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.775+0000 7fd01aaf4b80 20 client.16452 trim_cache size 0 max 16384 2023-10-08T13:53:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:53:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:53:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:53:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:53:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 -- 172.21.15.57:0/975906756 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcff00a15f0 msgr2=0x7fcff00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcff00a15f0 0x7fcff00a3ab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd008197c10 tx=0x7fcffc009310 comp rx=0 tx=0).stop 2023-10-08T13:53:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 -- 172.21.15.57:0/975906756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd008067fd0 msgr2=0x7fd00819c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd008067fd0 0x7fd00819c920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcff800e9a0 tx=0x7fcff800ecb0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 -- 172.21.15.57:0/975906756 shutdown_connections 2023-10-08T13:53:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fcff00a15f0 0x7fcff00a3ab0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd008105ad0 0x7fd008196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd008067fd0 0x7fd00819c920 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 --2- 172.21.15.57:0/975906756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd008105110 0x7fd00819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 -- 172.21.15.57:0/975906756 >> 172.21.15.57:0/975906756 conn(0x7fd008076790 msgr2=0x7fd0080fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 -- 172.21.15.57:0/975906756 shutdown_connections 2023-10-08T13:53:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:18.776+0000 7fd00e90b700 1 -- 172.21.15.57:0/975906756 wait complete. 2023-10-08T13:53:20.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:20 smithi099 ceph-mon[103849]: pgmap v352: 129 pgs: 129 active+clean; 76 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 138 op/s 2023-10-08T13:53:20.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:19 smithi115 ceph-mon[104408]: pgmap v352: 129 pgs: 129 active+clean; 76 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 138 op/s 2023-10-08T13:53:20.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:19 smithi057 ceph-mon[99476]: pgmap v352: 129 pgs: 129 active+clean; 76 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 138 op/s 2023-10-08T13:53:22.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:22 smithi099 ceph-mon[103849]: pgmap v353: 129 pgs: 129 active+clean; 131 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 430 op/s 2023-10-08T13:53:22.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:22 smithi115 ceph-mon[104408]: pgmap v353: 129 pgs: 129 active+clean; 131 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 430 op/s 2023-10-08T13:53:22.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:22 smithi057 ceph-mon[99476]: pgmap v353: 129 pgs: 129 active+clean; 131 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 430 op/s 2023-10-08T13:53:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:24 smithi099 ceph-mon[103849]: pgmap v354: 129 pgs: 129 active+clean; 147 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.2 MiB/s wr, 544 op/s 2023-10-08T13:53:24.364 INFO:tasks.workunit.client.0.smithi057.stdout: 44 120715 424 82167 597 32224 369 2023-10-08T13:53:24.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:24 smithi115 ceph-mon[104408]: pgmap v354: 129 pgs: 129 active+clean; 147 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.2 MiB/s wr, 544 op/s 2023-10-08T13:53:24.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:24 smithi057 ceph-mon[99476]: pgmap v354: 129 pgs: 129 active+clean; 147 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.2 MiB/s wr, 544 op/s 2023-10-08T13:53:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:26 smithi099 ceph-mon[103849]: pgmap v355: 129 pgs: 129 active+clean; 153 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.0 MiB/s wr, 595 op/s 2023-10-08T13:53:26.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:26 smithi115 ceph-mon[104408]: pgmap v355: 129 pgs: 129 active+clean; 153 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.0 MiB/s wr, 595 op/s 2023-10-08T13:53:26.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:26 smithi057 ceph-mon[99476]: pgmap v355: 129 pgs: 129 active+clean; 153 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.0 MiB/s wr, 595 op/s 2023-10-08T13:53:28.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:28 smithi099 ceph-mon[103849]: pgmap v356: 129 pgs: 129 active+clean; 154 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.8 MiB/s wr, 594 op/s 2023-10-08T13:53:28.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:28 smithi115 ceph-mon[104408]: pgmap v356: 129 pgs: 129 active+clean; 154 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.8 MiB/s wr, 594 op/s 2023-10-08T13:53:28.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:28 smithi057 ceph-mon[99476]: pgmap v356: 129 pgs: 129 active+clean; 154 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.8 MiB/s wr, 594 op/s 2023-10-08T13:53:30.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:30 smithi099 ceph-mon[103849]: pgmap v357: 129 pgs: 129 active+clean; 156 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.6 MiB/s wr, 574 op/s 2023-10-08T13:53:30.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:30 smithi115 ceph-mon[104408]: pgmap v357: 129 pgs: 129 active+clean; 156 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.6 MiB/s wr, 574 op/s 2023-10-08T13:53:30.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:30 smithi057 ceph-mon[99476]: pgmap v357: 129 pgs: 129 active+clean; 156 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.6 MiB/s wr, 574 op/s 2023-10-08T13:53:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.917+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1469984477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd84119fd0 msgr2=0x7fdd8411c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.917+0000 7fdd88fd3700 1 --2- 172.21.15.57:0/1469984477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd84119fd0 0x7fdd8411c3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009f90 tx=0x7fdd7c00b330 comp rx=0 tx=0).stop 2023-10-08T13:53:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.924+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1469984477 shutdown_connections 2023-10-08T13:53:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.924+0000 7fdd88fd3700 1 --2- 172.21.15.57:0/1469984477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd84119fd0 0x7fdd8411c3c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.924+0000 7fdd88fd3700 1 --2- 172.21.15.57:0/1469984477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8410ef10 0x7fdd841077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.924+0000 7fdd88fd3700 1 --2- 172.21.15.57:0/1469984477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd8410f380 0x7fdd84107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.925+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1469984477 >> 172.21.15.57:0/1469984477 conn(0x7fdd8406b7c0 msgr2=0x7fdd8406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.931+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1469984477 shutdown_connections 2023-10-08T13:53:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.933+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1469984477 wait complete. 2023-10-08T13:53:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.934+0000 7fdd88fd3700 1 Processor -- start 2023-10-08T13:53:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.934+0000 7fdd88fd3700 1 -- start start 2023-10-08T13:53:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.934+0000 7fdd88fd3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8410ef10 0x7fdd84077a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.934+0000 7fdd88fd3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8410f380 0x7fdd84077f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd88fd3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd84119fd0 0x7fdd84072b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd88fd3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840732e0 con 0x7fdd8410f380 2023-10-08T13:53:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd88fd3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84073450 con 0x7fdd8410ef10 2023-10-08T13:53:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd88fd3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84073610 con 0x7fdd84119fd0 2023-10-08T13:53:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd82d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd84119fd0 0x7fdd84072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd82d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd84119fd0 0x7fdd84072b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58616/0 (socket says 172.21.15.57:58616) 2023-10-08T13:53:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd82d9d700 1 -- 172.21.15.57:0/1689657104 learned_addr learned my addr 172.21.15.57:0/1689657104 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8410f380 0x7fdd84077f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd82d9d700 1 -- 172.21.15.57:0/1689657104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8410ef10 msgr2=0x7fdd84077a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T13:53:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd82d9d700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8410ef10 0x7fdd84077a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd82d9d700 1 -- 172.21.15.57:0/1689657104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8410f380 msgr2=0x7fdd84077f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd82d9d700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8410f380 0x7fdd84077f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.935+0000 7fdd82d9d700 1 -- 172.21.15.57:0/1689657104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd740095f0 con 0x7fdd84119fd0 2023-10-08T13:53:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.936+0000 7fdd82d9d700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd84119fd0 0x7fdd84072b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009400 tx=0x7fdd7c0094e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.944+0000 7fdd737fe700 1 -- 172.21.15.57:0/1689657104 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00e040 con 0x7fdd84119fd0 2023-10-08T13:53:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.944+0000 7fdd737fe700 1 -- 172.21.15.57:0/1689657104 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd7c00bbf0 con 0x7fdd84119fd0 2023-10-08T13:53:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.944+0000 7fdd737fe700 1 -- 172.21.15.57:0/1689657104 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c005040 con 0x7fdd84119fd0 2023-10-08T13:53:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.945+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1689657104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7c009c40 con 0x7fdd84119fd0 2023-10-08T13:53:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.945+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1689657104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8407f900 con 0x7fdd84119fd0 2023-10-08T13:53:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.951+0000 7fdd951bcb80 1 Processor -- start 2023-10-08T13:53:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.951+0000 7fdd951bcb80 1 -- start start 2023-10-08T13:53:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.951+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c089c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.951+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08a1a0 0x55560c083b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.951+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c084510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.951+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560bfec400 con 0x55560c0840b0 2023-10-08T13:53:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.951+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560bfddbc0 con 0x55560c08a1a0 2023-10-08T13:53:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.951+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560bfdb4e0 con 0x55560c089880 2023-10-08T13:53:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd81d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c089c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd81d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c089c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58620/0 (socket says 172.21.15.57:58620) 2023-10-08T13:53:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd81d9b700 1 -- 172.21.15.57:0/2182581663 learned_addr learned my addr 172.21.15.57:0/2182581663 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/2182581663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c084510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd81d9b700 1 -- 172.21.15.57:0/2182581663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08a1a0 msgr2=0x55560c083b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:53:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/2182581663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08a1a0 0x55560c083b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd81d9b700 1 -- 172.21.15.57:0/2182581663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 msgr2=0x55560c084510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/2182581663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c084510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd81d9b700 1 -- 172.21.15.57:0/2182581663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55560be01d80 con 0x55560c089880 2023-10-08T13:53:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/2182581663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c089c60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd780126c0 tx=0x7fdd78009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb70147b2910594 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd70ff9700 1 -- 172.21.15.57:0/2182581663 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78012ce0 con 0x55560c089880 2023-10-08T13:53:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd70ff9700 1 -- 172.21.15.57:0/2182581663 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd78012e40 con 0x55560c089880 2023-10-08T13:53:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.952+0000 7fdd70ff9700 1 -- 172.21.15.57:0/2182581663 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7801e920 con 0x55560c089880 2023-10-08T13:53:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.953+0000 7fdd951bcb80 1 -- 172.21.15.57:0/2182581663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 msgr2=0x55560c089c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.953+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/2182581663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c089c60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd780126c0 tx=0x7fdd78009670 comp rx=0 tx=0).stop 2023-10-08T13:53:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.954+0000 7fdd737fe700 1 -- 172.21.15.57:0/1689657104 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdd7c01b450 con 0x7fdd84119fd0 2023-10-08T13:53:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.954+0000 7fdd951bcb80 1 -- 172.21.15.57:0/2182581663 shutdown_connections 2023-10-08T13:53:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.954+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/2182581663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c084510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.954+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/2182581663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c089c60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.954+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/2182581663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08a1a0 0x55560c083b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.954+0000 7fdd951bcb80 1 -- 172.21.15.57:0/2182581663 >> 172.21.15.57:0/2182581663 conn(0x55560bff8200 msgr2=0x55560c07eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.955+0000 7fdd737fe700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd6c0a1650 0x7fdd6c0a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.955+0000 7fdd737fe700 1 -- 172.21.15.57:0/1689657104 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdd7c0ce580 con 0x7fdd84119fd0 2023-10-08T13:53:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.955+0000 7fdd8259c700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd6c0a1650 0x7fdd6c0a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.955+0000 7fdd951bcb80 1 -- 172.21.15.57:0/2182581663 shutdown_connections 2023-10-08T13:53:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.955+0000 7fdd951bcb80 1 -- 172.21.15.57:0/2182581663 wait complete. 2023-10-08T13:53:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.956+0000 7fdd951bcb80 1 -- start start 2023-10-08T13:53:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.956+0000 7fdd951bcb80 1 Processor -- start 2023-10-08T13:53:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.956+0000 7fdd8259c700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd6c0a1650 0x7fdd6c0a3b10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55560c07fc70 tx=0x7fdd74009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c08d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c08dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c099d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560bfd6dd0 con 0x55560c0840b0 2023-10-08T13:53:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560bf736b0 con 0x55560c08e1d0 2023-10-08T13:53:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560bf73350 con 0x55560c089880 2023-10-08T13:53:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd951bcb80 10 client.0 _ll_register_callbacks cb 0x55560bdd07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd8259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c08d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd8259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c08d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57814/0 (socket says 172.21.15.57:57814) 2023-10-08T13:53:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd8259c700 1 -- 172.21.15.57:0/1636818694 learned_addr learned my addr 172.21.15.57:0/1636818694 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd82d9d700 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd82d9d700 1 -- 172.21.15.57:0/1636818694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 msgr2=0x55560c08dc90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:53:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd82d9d700 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c08dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd82d9d700 1 -- 172.21.15.57:0/1636818694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 msgr2=0x55560c08d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.957+0000 7fdd82d9d700 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c08d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.958+0000 7fdd82d9d700 1 -- 172.21.15.57:0/1636818694 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7801a040 con 0x55560c08e1d0 2023-10-08T13:53:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.958+0000 7fdd8259c700 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c08d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.958+0000 7fdd82d9d700 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c099d30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0cb9d0 tx=0x7fdd7c09ae30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a29491eb05a5c73 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.958+0000 7fdd69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:53:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.958+0000 7fdd69ffb700 1 -- 172.21.15.57:0/1636818694 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c09f280 con 0x55560c08e1d0 2023-10-08T13:53:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.958+0000 7fdd69ffb700 1 -- 172.21.15.57:0/1636818694 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdd7c09b2f0 con 0x55560c08e1d0 2023-10-08T13:53:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.958+0000 7fdd951bcb80 1 -- 172.21.15.57:0/1636818694 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55560bf9ee50 con 0x55560c08e1d0 2023-10-08T13:53:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.958+0000 7fdd69ffb700 1 -- 172.21.15.57:0/1636818694 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd7c0ce420 con 0x55560c08e1d0 2023-10-08T13:53:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd951bcb80 10 client.25997 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd951bcb80 1 -- 172.21.15.57:0/1636818694 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55560bdeac00 con 0x55560c08e1d0 2023-10-08T13:53:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd69ffb700 1 -- 172.21.15.57:0/1636818694 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55560bf9ee50 con 0x55560c08e1d0 2023-10-08T13:53:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd69ffb700 1 -- 172.21.15.57:0/1636818694 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fdd7c0b3d20 con 0x55560c08e1d0 2023-10-08T13:53:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd951bcb80 10 client.25997 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd951bcb80 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd951bcb80 20 client.25997 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55560c09e280 0x55560c0a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd951bcb80 4 client.25997 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd951bcb80 1 -- 172.21.15.57:0/1636818694 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55560bfcfdb0 con 0x55560c09e280 2023-10-08T13:53:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.959+0000 7fdd8259c700 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55560c09e280 0x55560c0a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.960+0000 7fdd69ffb700 10 client.25997 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.960+0000 7fdd8259c700 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55560c09e280 0x55560c0a06a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=549282ba6b4096c2 server_cookie=a540d976f4097819 in_seq=0 out_seq=0 2023-10-08T13:53:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd69ffb700 1 -- 172.21.15.57:0/1636818694 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55560bfcfdb0 con 0x55560c09e280 2023-10-08T13:53:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd69ffb700 10 client.25997 handle_command_reply: tid=0 2023-10-08T13:53:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 client.25997 shutdown 2023-10-08T13:53:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 -- 172.21.15.57:0/1636818694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 msgr2=0x55560c099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c099d30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0cb9d0 tx=0x7fdd7c09ae30 comp rx=0 tx=0).stop 2023-10-08T13:53:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 -- 172.21.15.57:0/1636818694 shutdown_connections 2023-10-08T13:53:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55560c09e280 0x55560c0a06a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c099d30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd69ffb700 0 client.25997 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c08d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/1636818694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c08dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 -- 172.21.15.57:0/1636818694 >> 172.21.15.57:0/1636818694 conn(0x55560bff8200 msgr2=0x55560c07e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 -- 172.21.15.57:0/1636818694 shutdown_connections 2023-10-08T13:53:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.961+0000 7fdd951bcb80 1 -- 172.21.15.57:0/1636818694 wait complete. 2023-10-08T13:53:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.962+0000 7fdd951bcb80 20 client.25997 trim_cache size 0 max 16384 2023-10-08T13:53:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.981+0000 7fdd951bcb80 1 Processor -- start 2023-10-08T13:53:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.981+0000 7fdd951bcb80 1 -- start start 2023-10-08T13:53:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c07e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c07ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c08c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560be578c0 con 0x55560c0840b0 2023-10-08T13:53:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c0804e0 con 0x55560c08e1d0 2023-10-08T13:53:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c07f610 con 0x55560c089880 2023-10-08T13:53:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd8259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c07e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd8259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c07e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57822/0 (socket says 172.21.15.57:57822) 2023-10-08T13:53:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd8259c700 1 -- 172.21.15.57:0/2359376459 learned_addr learned my addr 172.21.15.57:0/2359376459 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/2359376459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c07ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd82d9d700 1 --2- 172.21.15.57:0/2359376459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c08c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd81d9b700 1 -- 172.21.15.57:0/2359376459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 msgr2=0x55560c08c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/2359376459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c08c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd81d9b700 1 -- 172.21.15.57:0/2359376459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 msgr2=0x55560c07e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/2359376459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c07e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.982+0000 7fdd81d9b700 1 -- 172.21.15.57:0/2359376459 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55560be281f0 con 0x55560c089880 2023-10-08T13:53:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.983+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/2359376459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c07ecd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdd78012690 tx=0x7fdd78005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5db798688a113015 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.983+0000 7fdd717fa700 1 -- 172.21.15.57:0/2359376459 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78012cb0 con 0x55560c089880 2023-10-08T13:53:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.983+0000 7fdd717fa700 1 -- 172.21.15.57:0/2359376459 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd780050e0 con 0x55560c089880 2023-10-08T13:53:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.983+0000 7fdd717fa700 1 -- 172.21.15.57:0/2359376459 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78003680 con 0x55560c089880 2023-10-08T13:53:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 -- 172.21.15.57:0/2359376459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 msgr2=0x55560c07ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/2359376459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c07ecd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdd78012690 tx=0x7fdd78005ce0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 -- 172.21.15.57:0/2359376459 shutdown_connections 2023-10-08T13:53:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/2359376459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c08c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/2359376459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c07e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/2359376459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c07ecd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 -- 172.21.15.57:0/2359376459 >> 172.21.15.57:0/2359376459 conn(0x55560bff8200 msgr2=0x55560c084490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 -- 172.21.15.57:0/2359376459 shutdown_connections 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 -- 172.21.15.57:0/2359376459 wait complete. 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 -- start start 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.984+0000 7fdd951bcb80 1 Processor -- start 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c094e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c092c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd951bcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c0933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c093ab0 con 0x55560c0840b0 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c0f1090 con 0x55560c08e1d0 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd951bcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560c0f11d0 con 0x55560c089880 2023-10-08T13:53:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd951bcb80 10 client.0 _ll_register_callbacks cb 0x7fdd7c097630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd8259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c094e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd8259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c094e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57828/0 (socket says 172.21.15.57:57828) 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd8259c700 1 -- 172.21.15.57:0/462257964 learned_addr learned my addr 172.21.15.57:0/462257964 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c092c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd82d9d700 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c0933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd81d9b700 1 -- 172.21.15.57:0/462257964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 msgr2=0x55560c0933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c0933a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd81d9b700 1 -- 172.21.15.57:0/462257964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 msgr2=0x55560c094e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c094e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.985+0000 7fdd81d9b700 1 -- 172.21.15.57:0/462257964 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd78016040 con 0x55560c089880 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.986+0000 7fdd81d9b700 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c092c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55560c08eeb0 tx=0x7fdd78005500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9db09524f19bbc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.986+0000 7fdd69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.986+0000 7fdd69ffb700 1 -- 172.21.15.57:0/462257964 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78009eb0 con 0x55560c089880 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.986+0000 7fdd69ffb700 1 -- 172.21.15.57:0/462257964 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fdd78009670 con 0x55560c089880 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.986+0000 7fdd951bcb80 1 -- 172.21.15.57:0/462257964 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd78012e40 con 0x55560c089880 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.986+0000 7fdd69ffb700 1 -- 172.21.15.57:0/462257964 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd78003890 con 0x55560c089880 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.986+0000 7fdd951bcb80 10 client.26065 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.986+0000 7fdd951bcb80 1 -- 172.21.15.57:0/462257964 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55560be00250 con 0x55560c089880 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.986+0000 7fdd69ffb700 1 -- 172.21.15.57:0/462257964 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd78012e40 con 0x55560c089880 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.986+0000 7fdd69ffb700 1 -- 172.21.15.57:0/462257964 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fdd78028980 con 0x55560c089880 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.987+0000 7fdd951bcb80 10 client.26065 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.987+0000 7fdd951bcb80 10 client.26065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.987+0000 7fdd951bcb80 20 client.26065 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.987+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55560c09e280 0x55560c0f4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.987+0000 7fdd951bcb80 4 client.26065 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:53:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.987+0000 7fdd951bcb80 1 -- 172.21.15.57:0/462257964 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55560bfcfad0 con 0x55560c09e280 2023-10-08T13:53:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.987+0000 7fdd8259c700 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55560c09e280 0x55560c0f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.987+0000 7fdd69ffb700 10 client.26065 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:31.987+0000 7fdd8259c700 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55560c09e280 0x55560c0f4530 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db970079beff176 server_cookie=79e1c23ac1a81b52 in_seq=0 out_seq=0 2023-10-08T13:53:32.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:32 smithi099 ceph-mon[103849]: pgmap v358: 129 pgs: 129 active+clean; 162 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 553 op/s 2023-10-08T13:53:32.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:32 smithi115 ceph-mon[104408]: pgmap v358: 129 pgs: 129 active+clean; 162 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 553 op/s 2023-10-08T13:53:32.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:32 smithi057 ceph-mon[99476]: pgmap v358: 129 pgs: 129 active+clean; 162 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 553 op/s 2023-10-08T13:53:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.521+0000 7fdd69ffb700 1 -- 172.21.15.57:0/462257964 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55560bfcfad0 con 0x55560c09e280 2023-10-08T13:53:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.521+0000 7fdd69ffb700 10 client.26065 handle_command_reply: tid=0 2023-10-08T13:53:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.521+0000 7fdd951bcb80 1 client.26065 shutdown 2023-10-08T13:53:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.525+0000 7fdd951bcb80 1 -- 172.21.15.57:0/462257964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 msgr2=0x55560c092c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.525+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c092c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55560c08eeb0 tx=0x7fdd78005500 comp rx=0 tx=0).stop 2023-10-08T13:53:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.525+0000 7fdd951bcb80 1 -- 172.21.15.57:0/462257964 shutdown_connections 2023-10-08T13:53:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.525+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55560c09e280 0x55560c0f4530 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.525+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55560c08e1d0 0x55560c0933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.525+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55560c0840b0 0x55560c094e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.525+0000 7fdd951bcb80 1 --2- 172.21.15.57:0/462257964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560c089880 0x55560c092c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.525+0000 7fdd951bcb80 1 -- 172.21.15.57:0/462257964 >> 172.21.15.57:0/462257964 conn(0x55560bff8200 msgr2=0x55560c0993d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.525+0000 7fdd69ffb700 0 client.26065 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.525+0000 7fdd951bcb80 1 -- 172.21.15.57:0/462257964 shutdown_connections 2023-10-08T13:53:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.526+0000 7fdd951bcb80 1 -- 172.21.15.57:0/462257964 wait complete. 2023-10-08T13:53:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.531+0000 7fdd951bcb80 20 client.26065 trim_cache size 0 max 16384 2023-10-08T13:53:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:53:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:53:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:53:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:53:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.539+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1689657104 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd6c0a1650 msgr2=0x7fdd6c0a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.539+0000 7fdd88fd3700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd6c0a1650 0x7fdd6c0a3b10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55560c07fc70 tx=0x7fdd74009310 comp rx=0 tx=0).stop 2023-10-08T13:53:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.539+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1689657104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd84119fd0 msgr2=0x7fdd84072b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.539+0000 7fdd88fd3700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd84119fd0 0x7fdd84072b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009400 tx=0x7fdd7c0094e0 comp rx=0 tx=0).stop 2023-10-08T13:53:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.540+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1689657104 shutdown_connections 2023-10-08T13:53:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.540+0000 7fdd88fd3700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd84119fd0 0x7fdd84072b10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.540+0000 7fdd88fd3700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fdd6c0a1650 0x7fdd6c0a3b10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.540+0000 7fdd88fd3700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd8410ef10 0x7fdd84077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.540+0000 7fdd88fd3700 1 --2- 172.21.15.57:0/1689657104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdd8410f380 0x7fdd84077f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.540+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1689657104 >> 172.21.15.57:0/1689657104 conn(0x7fdd8406b7c0 msgr2=0x7fdd84117fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.540+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1689657104 shutdown_connections 2023-10-08T13:53:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:33.540+0000 7fdd88fd3700 1 -- 172.21.15.57:0/1689657104 wait complete. 2023-10-08T13:53:34.364 INFO:tasks.workunit.client.0.smithi057.stdout: 75 423342 1596 297690 1402 116379 723 2023-10-08T13:53:34.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:34 smithi115 ceph-mon[104408]: pgmap v359: 129 pgs: 129 active+clean; 162 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2023-10-08T13:53:34.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:34 smithi057 ceph-mon[99476]: pgmap v359: 129 pgs: 129 active+clean; 162 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2023-10-08T13:53:34.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:34 smithi099 ceph-mon[103849]: pgmap v359: 129 pgs: 129 active+clean; 162 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2023-10-08T13:53:36.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:36 smithi115 ceph-mon[104408]: pgmap v360: 129 pgs: 129 active+clean; 169 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2023-10-08T13:53:36.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:36 smithi057 ceph-mon[99476]: pgmap v360: 129 pgs: 129 active+clean; 169 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2023-10-08T13:53:36.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:36 smithi099 ceph-mon[103849]: pgmap v360: 129 pgs: 129 active+clean; 169 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2023-10-08T13:53:38.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:38 smithi115 ceph-mon[104408]: pgmap v361: 129 pgs: 129 active+clean; 174 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 349 op/s 2023-10-08T13:53:38.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:38 smithi057 ceph-mon[99476]: pgmap v361: 129 pgs: 129 active+clean; 174 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 349 op/s 2023-10-08T13:53:38.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:38 smithi099 ceph-mon[103849]: pgmap v361: 129 pgs: 129 active+clean; 174 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 349 op/s 2023-10-08T13:53:39.376 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:53:39.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.508+0000 7f735c67d700 1 -- 172.21.15.57:0/2230738269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73541051c0 msgr2=0x7f7354105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:39.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.508+0000 7f735c67d700 1 --2- 172.21.15.57:0/2230738269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73541051c0 0x7f7354105640 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f73440099c0 tx=0x7f7344009cd0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.509+0000 7f735c67d700 1 -- 172.21.15.57:0/2230738269 shutdown_connections 2023-10-08T13:53:39.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.509+0000 7f735c67d700 1 --2- 172.21.15.57:0/2230738269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7354105b80 0x7f735410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.509+0000 7f735c67d700 1 --2- 172.21.15.57:0/2230738269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7354067ce0 0x7f73540680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.509+0000 7f735c67d700 1 --2- 172.21.15.57:0/2230738269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73541051c0 0x7f7354105640 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.509+0000 7f735c67d700 1 -- 172.21.15.57:0/2230738269 >> 172.21.15.57:0/2230738269 conn(0x7f7354076780 msgr2=0x7f7354076b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:39.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.509+0000 7f735c67d700 1 -- 172.21.15.57:0/2230738269 shutdown_connections 2023-10-08T13:53:39.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.509+0000 7f735c67d700 1 -- 172.21.15.57:0/2230738269 wait complete. 2023-10-08T13:53:39.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.509+0000 7f735c67d700 1 Processor -- start 2023-10-08T13:53:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.509+0000 7f735c67d700 1 -- start start 2023-10-08T13:53:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735c67d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7354067ce0 0x7f735419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735c67d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73541051c0 0x7f735419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735c67d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7354105b80 0x7f7354196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735c67d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73541970a0 con 0x7f7354105b80 2023-10-08T13:53:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735c67d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7354197210 con 0x7f7354067ce0 2023-10-08T13:53:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735c67d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73541973d0 con 0x7f73541051c0 2023-10-08T13:53:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735ac1a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7354105b80 0x7f7354196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f7359c18700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73541051c0 0x7f735419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735ac1a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7354105b80 0x7f7354196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57846/0 (socket says 172.21.15.57:57846) 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735ac1a700 1 -- 172.21.15.57:0/2510144596 learned_addr learned my addr 172.21.15.57:0/2510144596 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735a419700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7354067ce0 0x7f735419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735ac1a700 1 -- 172.21.15.57:0/2510144596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73541051c0 msgr2=0x7f735419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735ac1a700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73541051c0 0x7f735419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735ac1a700 1 -- 172.21.15.57:0/2510144596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7354067ce0 msgr2=0x7f735419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735ac1a700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7354067ce0 0x7f735419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735ac1a700 1 -- 172.21.15.57:0/2510144596 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73440095f0 con 0x7f7354105b80 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.510+0000 7f735ac1a700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7354105b80 0x7f7354196a40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f734c00ef30 tx=0x7f734c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.511+0000 7f734b7fe700 1 -- 172.21.15.57:0/2510144596 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c018070 con 0x7f7354105b80 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.511+0000 7f734b7fe700 1 -- 172.21.15.57:0/2510144596 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f734c00f040 con 0x7f7354105b80 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.511+0000 7f735c67d700 1 -- 172.21.15.57:0/2510144596 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7354197650 con 0x7f7354105b80 2023-10-08T13:53:39.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.511+0000 7f734b7fe700 1 -- 172.21.15.57:0/2510144596 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c00cd00 con 0x7f7354105b80 2023-10-08T13:53:39.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.511+0000 7f735c67d700 1 -- 172.21.15.57:0/2510144596 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73541a3900 con 0x7f7354105b80 2023-10-08T13:53:39.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.513+0000 7f734b7fe700 1 -- 172.21.15.57:0/2510144596 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f734c003a00 con 0x7f7354105b80 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.513+0000 7f734b7fe700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73400a15f0 0x7f73400a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.513+0000 7f734b7fe700 1 -- 172.21.15.57:0/2510144596 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f734c0cced0 con 0x7f7354105b80 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.513+0000 7f735a419700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73400a15f0 0x7f73400a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.514+0000 7f735a419700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73400a15f0 0x7f73400a3ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7350007be0 tx=0x7f73500073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.514+0000 7f7368866b80 1 Processor -- start 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.514+0000 7f7368866b80 1 -- start start 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7368866b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 0x561ce1394260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7368866b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13947a0 0x561ce13a3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7368866b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13a3a60 0x561ce13a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7368866b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce12fd460 con 0x561ce1393e80 2023-10-08T13:53:39.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7368866b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce12eec20 con 0x561ce13a3a60 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7368866b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce12ec540 con 0x561ce13947a0 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f735a419700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 0x561ce1394260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7359c18700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13947a0 0x561ce13a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f735ac1a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13a3a60 0x561ce13a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7359c18700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13947a0 0x561ce13a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58664/0 (socket says 172.21.15.57:58664) 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7359c18700 1 -- 172.21.15.57:0/668163723 learned_addr learned my addr 172.21.15.57:0/668163723 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7359c18700 1 -- 172.21.15.57:0/668163723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13a3a60 msgr2=0x561ce13a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7359c18700 1 --2- 172.21.15.57:0/668163723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13a3a60 0x561ce13a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7359c18700 1 -- 172.21.15.57:0/668163723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 msgr2=0x561ce1394260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7359c18700 1 --2- 172.21.15.57:0/668163723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 0x561ce1394260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7359c18700 1 -- 172.21.15.57:0/668163723 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ce1112de0 con 0x561ce13947a0 2023-10-08T13:53:39.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f735ac1a700 1 --2- 172.21.15.57:0/668163723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13a3a60 0x561ce13a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7359c18700 1 --2- 172.21.15.57:0/668163723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13947a0 0x561ce13a3520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f73440099c0 tx=0x7f7344005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82b35b878fed20 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7348ff9700 1 -- 172.21.15.57:0/668163723 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734401c050 con 0x561ce13947a0 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7348ff9700 1 -- 172.21.15.57:0/668163723 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f734401d040 con 0x561ce13947a0 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.515+0000 7f7348ff9700 1 -- 172.21.15.57:0/668163723 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73440038e0 con 0x561ce13947a0 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.516+0000 7f7368866b80 1 -- 172.21.15.57:0/668163723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13947a0 msgr2=0x561ce13a3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.516+0000 7f7368866b80 1 --2- 172.21.15.57:0/668163723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13947a0 0x561ce13a3520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f73440099c0 tx=0x7f7344005fb0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.516+0000 7f7368866b80 1 -- 172.21.15.57:0/668163723 shutdown_connections 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.516+0000 7f7368866b80 1 --2- 172.21.15.57:0/668163723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13a3a60 0x561ce13a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.516+0000 7f7368866b80 1 --2- 172.21.15.57:0/668163723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 0x561ce1394260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.516+0000 7f7368866b80 1 --2- 172.21.15.57:0/668163723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13947a0 0x561ce13a3520 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.516+0000 7f7368866b80 1 -- 172.21.15.57:0/668163723 >> 172.21.15.57:0/668163723 conn(0x561ce1309260 msgr2=0x561ce13902d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.516+0000 7f7368866b80 1 -- 172.21.15.57:0/668163723 shutdown_connections 2023-10-08T13:53:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.516+0000 7f7368866b80 1 -- 172.21.15.57:0/668163723 wait complete. 2023-10-08T13:53:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f7368866b80 1 -- start start 2023-10-08T13:53:39.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f7368866b80 1 Processor -- start 2023-10-08T13:53:39.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f7368866b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 0x561ce13a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f7368866b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 0x561ce13a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f7368866b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13a3a60 0x561ce13a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f7368866b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce12e7e30 con 0x561ce1393e80 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f7368866b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce1284710 con 0x561ce13947a0 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f7368866b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce12843b0 con 0x561ce13a3a60 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f7368866b80 10 client.0 _ll_register_callbacks cb 0x561ce10e1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f735a419700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 0x561ce13a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f735a419700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 0x561ce13a34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57856/0 (socket says 172.21.15.57:57856) 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f735a419700 1 -- 172.21.15.57:0/898552953 learned_addr learned my addr 172.21.15.57:0/898552953 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.517+0000 7f735ac1a700 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13a3a60 0x561ce13a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f7359c18700 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 0x561ce13a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f735a419700 1 -- 172.21.15.57:0/898552953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13a3a60 msgr2=0x561ce13a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f735a419700 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13a3a60 0x561ce13a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f735a419700 1 -- 172.21.15.57:0/898552953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 msgr2=0x561ce13a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f735a419700 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 0x561ce13a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f735a419700 1 -- 172.21.15.57:0/898552953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73541a3900 con 0x561ce1393e80 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f7359c18700 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 0x561ce13a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f735a419700 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 0x561ce13a34c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f73500042e0 tx=0x7f73500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1e1007c2e78877 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f733dffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f733dffb700 1 -- 172.21.15.57:0/898552953 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350003b90 con 0x561ce1393e80 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f7368866b80 1 -- 172.21.15.57:0/898552953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ce12afeb0 con 0x561ce1393e80 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f733dffb700 1 -- 172.21.15.57:0/898552953 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7350017070 con 0x561ce1393e80 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f733dffb700 1 -- 172.21.15.57:0/898552953 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7350003cf0 con 0x561ce1393e80 2023-10-08T13:53:39.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f7368866b80 10 client.16482 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f7368866b80 1 -- 172.21.15.57:0/898552953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f735410dec0 con 0x561ce1393e80 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.518+0000 7f733dffb700 1 -- 172.21.15.57:0/898552953 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561ce12afeb0 con 0x561ce1393e80 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.519+0000 7f733dffb700 1 -- 172.21.15.57:0/898552953 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f7350020480 con 0x561ce1393e80 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.519+0000 7f7368866b80 10 client.16482 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.519+0000 7f7368866b80 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.519+0000 7f7368866b80 20 client.16482 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.519+0000 7f7368866b80 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561ce13b3660 0x561ce13b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.519+0000 7f7368866b80 4 client.16482 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.519+0000 7f7368866b80 1 -- 172.21.15.57:0/898552953 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ce12e0e10 con 0x561ce13b3660 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.519+0000 7f7359c18700 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561ce13b3660 0x561ce13b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.519+0000 7f7359c18700 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561ce13b3660 0x561ce13b5a80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36729b0c6278f243 server_cookie=c4edaf238d572540 in_seq=0 out_seq=0 2023-10-08T13:53:39.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.519+0000 7f733dffb700 10 client.16482 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:39.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f733dffb700 1 -- 172.21.15.57:0/898552953 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561ce12e0e10 con 0x561ce13b3660 2023-10-08T13:53:39.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f733dffb700 10 client.16482 handle_command_reply: tid=0 2023-10-08T13:53:39.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 client.16482 shutdown 2023-10-08T13:53:39.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 -- 172.21.15.57:0/898552953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 msgr2=0x561ce13a34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:39.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 0x561ce13a34c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f73500042e0 tx=0x7f73500047c0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 -- 172.21.15.57:0/898552953 shutdown_connections 2023-10-08T13:53:39.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561ce13b3660 0x561ce13b5a80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13a3a60 0x561ce13a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce1393e80 0x561ce13a34c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 --2- 172.21.15.57:0/898552953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 0x561ce13a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 -- 172.21.15.57:0/898552953 >> 172.21.15.57:0/898552953 conn(0x561ce1309260 msgr2=0x561ce138fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:39.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f733dffb700 0 client.16482 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:39.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 -- 172.21.15.57:0/898552953 shutdown_connections 2023-10-08T13:53:39.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 1 -- 172.21.15.57:0/898552953 wait complete. 2023-10-08T13:53:39.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.520+0000 7f7368866b80 20 client.16482 trim_cache size 0 max 16384 2023-10-08T13:53:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f7368866b80 1 Processor -- start 2023-10-08T13:53:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f7368866b80 1 -- start start 2023-10-08T13:53:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f7368866b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce1393e80 0x561ce13aa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f7368866b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 0x561ce139bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f7368866b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13a3a60 0x561ce139c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f7368866b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce1168920 con 0x561ce13a3a60 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f7368866b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce138a600 con 0x561ce13947a0 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f7368866b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce13a6290 con 0x561ce1393e80 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f735ac1a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13a3a60 0x561ce139c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f735a419700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce1393e80 0x561ce13aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f735ac1a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13a3a60 0x561ce139c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57868/0 (socket says 172.21.15.57:57868) 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f735a419700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce1393e80 0x561ce13aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58676/0 (socket says 172.21.15.57:58676) 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.537+0000 7f735ac1a700 1 -- 172.21.15.57:0/2205963107 learned_addr learned my addr 172.21.15.57:0/2205963107 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f7359c18700 1 --2- 172.21.15.57:0/2205963107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 0x561ce139bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f735ac1a700 1 -- 172.21.15.57:0/2205963107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce1393e80 msgr2=0x561ce13aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f735ac1a700 1 --2- 172.21.15.57:0/2205963107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce1393e80 0x561ce13aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f735ac1a700 1 -- 172.21.15.57:0/2205963107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 msgr2=0x561ce139bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f735ac1a700 1 --2- 172.21.15.57:0/2205963107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 0x561ce139bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f735ac1a700 1 -- 172.21.15.57:0/2205963107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ce10fbc60 con 0x561ce13a3a60 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f735a419700 1 --2- 172.21.15.57:0/2205963107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce1393e80 0x561ce13aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f735ac1a700 1 --2- 172.21.15.57:0/2205963107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13a3a60 0x561ce139c060 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f734c0ca380 tx=0x7f734c095860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ece34876c7ef41b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f73497fa700 1 -- 172.21.15.57:0/2205963107 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c01b320 con 0x561ce13a3a60 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f73497fa700 1 -- 172.21.15.57:0/2205963107 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f734c09a340 con 0x561ce13a3a60 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.538+0000 7f73497fa700 1 -- 172.21.15.57:0/2205963107 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c0a1fd0 con 0x561ce13a3a60 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 -- 172.21.15.57:0/2205963107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13a3a60 msgr2=0x561ce139c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 --2- 172.21.15.57:0/2205963107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13a3a60 0x561ce139c060 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f734c0ca380 tx=0x7f734c095860 comp rx=0 tx=0).stop 2023-10-08T13:53:39.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 -- 172.21.15.57:0/2205963107 shutdown_connections 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 --2- 172.21.15.57:0/2205963107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13a3a60 0x561ce139c060 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 --2- 172.21.15.57:0/2205963107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce1393e80 0x561ce13aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 --2- 172.21.15.57:0/2205963107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce13947a0 0x561ce139bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 -- 172.21.15.57:0/2205963107 >> 172.21.15.57:0/2205963107 conn(0x561ce1309260 msgr2=0x561ce138f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 -- 172.21.15.57:0/2205963107 shutdown_connections 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 -- 172.21.15.57:0/2205963107 wait complete. 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 -- start start 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.539+0000 7f7368866b80 1 Processor -- start 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7368866b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce1393e80 0x561ce13aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7368866b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13947a0 0x561ce13a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7368866b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13a3a60 0x561ce13a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7368866b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce13a91b0 con 0x561ce13947a0 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7368866b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce13a9320 con 0x561ce1393e80 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7368866b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce13a94e0 con 0x561ce13a3a60 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7368866b80 10 client.0 _ll_register_callbacks cb 0x7f7350010e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7359c18700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13947a0 0x561ce13a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f735a419700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce1393e80 0x561ce13aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f735ac1a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13a3a60 0x561ce13a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7359c18700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13947a0 0x561ce13a86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57872/0 (socket says 172.21.15.57:57872) 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f735a419700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce1393e80 0x561ce13aa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59296/0 (socket says 172.21.15.57:59296) 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7359c18700 1 -- 172.21.15.57:0/317457564 learned_addr learned my addr 172.21.15.57:0/317457564 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7359c18700 1 -- 172.21.15.57:0/317457564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13a3a60 msgr2=0x561ce13a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:39.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7359c18700 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13a3a60 0x561ce13a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7359c18700 1 -- 172.21.15.57:0/317457564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce1393e80 msgr2=0x561ce13aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7359c18700 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce1393e80 0x561ce13aa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7359c18700 1 -- 172.21.15.57:0/317457564 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735410dec0 con 0x561ce13947a0 2023-10-08T13:53:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7359c18700 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13947a0 0x561ce13a86a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7344005380 tx=0x7f7344009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae333e5693b90861 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f733dffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f733dffb700 1 -- 172.21.15.57:0/317457564 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734401c040 con 0x561ce13947a0 2023-10-08T13:53:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f7368866b80 1 -- 172.21.15.57:0/317457564 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f734401d040 con 0x561ce13947a0 2023-10-08T13:53:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.540+0000 7f733dffb700 1 -- 172.21.15.57:0/317457564 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7344005150 con 0x561ce13947a0 2023-10-08T13:53:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f733dffb700 1 -- 172.21.15.57:0/317457564 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7344005940 con 0x561ce13947a0 2023-10-08T13:53:39.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f7368866b80 10 client.16488 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f7368866b80 1 -- 172.21.15.57:0/317457564 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f7354197650 con 0x561ce13947a0 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f733dffb700 1 -- 172.21.15.57:0/317457564 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f734401d040 con 0x561ce13947a0 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f733dffb700 1 -- 172.21.15.57:0/317457564 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f734403a070 con 0x561ce13947a0 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f7368866b80 10 client.16488 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f7368866b80 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f7368866b80 20 client.16488 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f7368866b80 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561ce1408780 0x561ce140aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f7368866b80 4 client.16488 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f7368866b80 1 -- 172.21.15.57:0/317457564 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561ce12e0b30 con 0x561ce1408780 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f735a419700 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561ce1408780 0x561ce140aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:39.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f733dffb700 10 client.16488 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.541+0000 7f735a419700 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561ce1408780 0x561ce140aba0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43fe0072a1a804f0 server_cookie=19e90870bcae9fe5 in_seq=0 out_seq=0 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f733dffb700 1 -- 172.21.15.57:0/317457564 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561ce12e0b30 con 0x561ce1408780 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f733dffb700 10 client.16488 handle_command_reply: tid=0 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 client.16488 shutdown 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 -- 172.21.15.57:0/317457564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13947a0 msgr2=0x561ce13a86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13947a0 0x561ce13a86a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7344005380 tx=0x7f7344009cd0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 -- 172.21.15.57:0/317457564 shutdown_connections 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561ce1408780 0x561ce140aba0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561ce13a3a60 0x561ce13a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f733dffb700 0 client.16488 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:39.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ce1393e80 0x561ce13aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 --2- 172.21.15.57:0/317457564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ce13947a0 0x561ce13a86a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 -- 172.21.15.57:0/317457564 >> 172.21.15.57:0/317457564 conn(0x561ce1309260 msgr2=0x561ce13a0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 -- 172.21.15.57:0/317457564 shutdown_connections 2023-10-08T13:53:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 1 -- 172.21.15.57:0/317457564 wait complete. 2023-10-08T13:53:39.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f7368866b80 20 client.16488 trim_cache size 0 max 16384 2023-10-08T13:53:39.551 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:53:39.551 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T13:53:39.551 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:53:39.551 INFO:teuthology.orchestra.run.smithi057.stdout: "6db03f06-e888-4427-a580-5fa3643c8e0d": { 2023-10-08T13:53:39.552 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:53:39.552 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "6db03f06-e888-4427-a580-5fa3643c8e0d", 2023-10-08T13:53:39.552 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:53:39.552 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:53:39.552 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:53:39.552 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:53:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f735c67d700 1 -- 172.21.15.57:0/2510144596 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73400a15f0 msgr2=0x7f73400a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f735c67d700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73400a15f0 0x7f73400a3ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7350007be0 tx=0x7f73500073d0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f735c67d700 1 -- 172.21.15.57:0/2510144596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7354105b80 msgr2=0x7f7354196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.542+0000 7f735c67d700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7354105b80 0x7f7354196a40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f734c00ef30 tx=0x7f734c00c3a0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.543+0000 7f735c67d700 1 -- 172.21.15.57:0/2510144596 shutdown_connections 2023-10-08T13:53:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.543+0000 7f735c67d700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7354105b80 0x7f7354196a40 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.543+0000 7f735c67d700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73400a15f0 0x7f73400a3ab0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.543+0000 7f735c67d700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7354067ce0 0x7f735419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.543+0000 7f735c67d700 1 --2- 172.21.15.57:0/2510144596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73541051c0 0x7f735419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:39.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.543+0000 7f735c67d700 1 -- 172.21.15.57:0/2510144596 >> 172.21.15.57:0/2510144596 conn(0x7f7354076780 msgr2=0x7f73540fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:39.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.543+0000 7f735c67d700 1 -- 172.21.15.57:0/2510144596 shutdown_connections 2023-10-08T13:53:39.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:53:39.543+0000 7f735c67d700 1 -- 172.21.15.57:0/2510144596 wait complete. 2023-10-08T13:53:39.556 INFO:tasks.cephfs.filesystem:scrub status for tag:6db03f06-e888-4427-a580-5fa3643c8e0d - {'path': '/', 'tag': '6db03f06-e888-4427-a580-5fa3643c8e0d', 'options': 'recursive,force'} 2023-10-08T13:53:40.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:40 smithi115 ceph-mon[104408]: pgmap v362: 129 pgs: 129 active+clean; 178 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 393 op/s 2023-10-08T13:53:40.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:40 smithi057 ceph-mon[99476]: pgmap v362: 129 pgs: 129 active+clean; 178 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 393 op/s 2023-10-08T13:53:40.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:40 smithi099 ceph-mon[103849]: pgmap v362: 129 pgs: 129 active+clean; 178 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 393 op/s 2023-10-08T13:53:42.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:42 smithi115 ceph-mon[104408]: pgmap v363: 129 pgs: 129 active+clean; 182 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 411 op/s 2023-10-08T13:53:42.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:42 smithi057 ceph-mon[99476]: pgmap v363: 129 pgs: 129 active+clean; 182 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 411 op/s 2023-10-08T13:53:42.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:42 smithi099 ceph-mon[103849]: pgmap v363: 129 pgs: 129 active+clean; 182 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 411 op/s 2023-10-08T13:53:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:43 smithi115 ceph-mon[104408]: pgmap v364: 129 pgs: 129 active+clean; 195 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 421 op/s 2023-10-08T13:53:44.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:43 smithi057 ceph-mon[99476]: pgmap v364: 129 pgs: 129 active+clean; 195 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 421 op/s 2023-10-08T13:53:44.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:43 smithi099 ceph-mon[103849]: pgmap v364: 129 pgs: 129 active+clean; 195 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 421 op/s 2023-10-08T13:53:44.364 INFO:tasks.workunit.client.0.smithi057.stdout: 93 319041 975 222101 1081 68867 57 2023-10-08T13:53:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:45 smithi115 ceph-mon[104408]: pgmap v365: 129 pgs: 129 active+clean; 241 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 597 op/s 2023-10-08T13:53:46.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:45 smithi057 ceph-mon[99476]: pgmap v365: 129 pgs: 129 active+clean; 241 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 597 op/s 2023-10-08T13:53:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:45 smithi099 ceph-mon[103849]: pgmap v365: 129 pgs: 129 active+clean; 241 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 597 op/s 2023-10-08T13:53:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.743+0000 7fe315ac7700 1 -- 172.21.15.57:0/337802323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe310111590 msgr2=0x7fe310113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.743+0000 7fe315ac7700 1 --2- 172.21.15.57:0/337802323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe310111590 0x7fe310113980 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe30800b0a0 tx=0x7fe30800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.743+0000 7fe315ac7700 1 -- 172.21.15.57:0/337802323 shutdown_connections 2023-10-08T13:53:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.743+0000 7fe315ac7700 1 --2- 172.21.15.57:0/337802323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe310111590 0x7fe310113980 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.743+0000 7fe315ac7700 1 --2- 172.21.15.57:0/337802323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe310074d20 0x7fe310075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.743+0000 7fe315ac7700 1 --2- 172.21.15.57:0/337802323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe310075640 0x7fe310111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.743+0000 7fe315ac7700 1 -- 172.21.15.57:0/337802323 >> 172.21.15.57:0/337802323 conn(0x7fe3100fe3a0 msgr2=0x7fe3101007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.743+0000 7fe315ac7700 1 -- 172.21.15.57:0/337802323 shutdown_connections 2023-10-08T13:53:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.743+0000 7fe315ac7700 1 -- 172.21.15.57:0/337802323 wait complete. 2023-10-08T13:53:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.744+0000 7fe315ac7700 1 Processor -- start 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.744+0000 7fe315ac7700 1 -- start start 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.744+0000 7fe315ac7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe310074d20 0x7fe31019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.744+0000 7fe315ac7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe310075640 0x7fe31019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.744+0000 7fe315ac7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe310111590 0x7fe3101a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.744+0000 7fe315ac7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3101a2220 con 0x7fe310075640 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.744+0000 7fe315ac7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3101a2390 con 0x7fe310074d20 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.744+0000 7fe315ac7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3101a2550 con 0x7fe310111590 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.744+0000 7fe30effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe310075640 0x7fe31019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe310075640 0x7fe31019d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57886/0 (socket says 172.21.15.57:57886) 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30effd700 1 -- 172.21.15.57:0/1234832630 learned_addr learned my addr 172.21.15.57:0/1234832630 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30ffff700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe310111590 0x7fe3101a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe310074d20 0x7fe31019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30effd700 1 -- 172.21.15.57:0/1234832630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe310111590 msgr2=0x7fe3101a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30effd700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe310111590 0x7fe3101a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30effd700 1 -- 172.21.15.57:0/1234832630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe310074d20 msgr2=0x7fe31019d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30effd700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe310074d20 0x7fe31019d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30effd700 1 -- 172.21.15.57:0/1234832630 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308009d00 con 0x7fe310075640 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30effd700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe310075640 0x7fe31019d890 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe30400d950 tx=0x7fe30400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30cff9700 1 -- 172.21.15.57:0/1234832630 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304009940 con 0x7fe310075640 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30cff9700 1 -- 172.21.15.57:0/1234832630 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe304010460 con 0x7fe310075640 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe30cff9700 1 -- 172.21.15.57:0/1234832630 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30400f620 con 0x7fe310075640 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe315ac7700 1 -- 172.21.15.57:0/1234832630 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3101a2830 con 0x7fe310075640 2023-10-08T13:53:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.745+0000 7fe315ac7700 1 -- 172.21.15.57:0/1234832630 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3101aa070 con 0x7fe310075640 2023-10-08T13:53:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.748+0000 7fe30cff9700 1 -- 172.21.15.57:0/1234832630 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe30400f780 con 0x7fe310075640 2023-10-08T13:53:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.749+0000 7fe30cff9700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe2f80a15f0 0x7fe2f80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.749+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe2f80a15f0 0x7fe2f80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.749+0000 7fe30cff9700 1 -- 172.21.15.57:0/1234832630 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe3040cd2b0 con 0x7fe310075640 2023-10-08T13:53:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe2f80a15f0 0x7fe2f80a3ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe300006fd0 tx=0x7fe300008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe321cb0b80 1 Processor -- start 2023-10-08T13:53:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe321cb0b80 1 -- start start 2023-10-08T13:53:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b1880 0x564c4e1b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e1abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1ac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e114400 con 0x564c4e1ac0b0 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e105bc0 con 0x564c4e1b21a0 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e1034e0 con 0x564c4e1b1880 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe30ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe30ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57894/0 (socket says 172.21.15.57:57894) 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.750+0000 7fe30ffff700 1 -- 172.21.15.57:0/1865788725 learned_addr learned my addr 172.21.15.57:0/1865788725 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.751+0000 7fe30ffff700 1 -- 172.21.15.57:0/1865788725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b1880 msgr2=0x564c4e1b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.751+0000 7fe30ffff700 1 --2- 172.21.15.57:0/1865788725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b1880 0x564c4e1b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.751+0000 7fe30ffff700 1 -- 172.21.15.57:0/1865788725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 msgr2=0x564c4e1abb70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.751+0000 7fe30ffff700 1 --2- 172.21.15.57:0/1865788725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e1abb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.751+0000 7fe30ffff700 1 -- 172.21.15.57:0/1865788725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c4df29d80 con 0x564c4e1ac0b0 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.751+0000 7fe30ffff700 1 --2- 172.21.15.57:0/1865788725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1ac510 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe308007cb0 tx=0x7fe308007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fdbf441c3c2346 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.751+0000 7fe2f5ffb700 1 -- 172.21.15.57:0/1865788725 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800e040 con 0x564c4e1ac0b0 2023-10-08T13:53:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.751+0000 7fe2f5ffb700 1 -- 172.21.15.57:0/1865788725 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3080092d0 con 0x564c4e1ac0b0 2023-10-08T13:53:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.751+0000 7fe2f5ffb700 1 -- 172.21.15.57:0/1865788725 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30801c6a0 con 0x564c4e1ac0b0 2023-10-08T13:53:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.752+0000 7fe321cb0b80 1 -- 172.21.15.57:0/1865788725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 msgr2=0x564c4e1ac510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.752+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/1865788725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1ac510 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe308007cb0 tx=0x7fe308007ce0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.752+0000 7fe321cb0b80 1 -- 172.21.15.57:0/1865788725 shutdown_connections 2023-10-08T13:53:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.752+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/1865788725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1ac510 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.752+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/1865788725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b1880 0x564c4e1b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.752+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/1865788725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e1abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.752+0000 7fe321cb0b80 1 -- 172.21.15.57:0/1865788725 >> 172.21.15.57:0/1865788725 conn(0x564c4e120200 msgr2=0x564c4e1a6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.753+0000 7fe321cb0b80 1 -- 172.21.15.57:0/1865788725 shutdown_connections 2023-10-08T13:53:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.753+0000 7fe321cb0b80 1 -- 172.21.15.57:0/1865788725 wait complete. 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.753+0000 7fe321cb0b80 1 -- start start 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.753+0000 7fe321cb0b80 1 Processor -- start 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b1880 0x564c4e1b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e1b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e0fedd0 con 0x564c4e1ac0b0 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e09b6b0 con 0x564c4e1b21a0 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e09b350 con 0x564c4e1b1880 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe321cb0b80 10 client.0 _ll_register_callbacks cb 0x564c4def87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe30f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe30f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57898/0 (socket says 172.21.15.57:57898) 2023-10-08T13:53:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe30f7fe700 1 -- 172.21.15.57:0/2459796544 learned_addr learned my addr 172.21.15.57:0/2459796544 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe30ffff700 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e1b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe30f7fe700 1 -- 172.21.15.57:0/2459796544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b1880 msgr2=0x564c4e1b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b1880 0x564c4e1b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe30f7fe700 1 -- 172.21.15.57:0/2459796544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 msgr2=0x564c4e1b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e1b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.754+0000 7fe30f7fe700 1 -- 172.21.15.57:0/2459796544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe310114470 con 0x564c4e1ac0b0 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1b5630 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe30000dd20 tx=0x7fe300014710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829b2a1c2768d19b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe2eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2459796544 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300014c40 con 0x564c4e1ac0b0 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2459796544 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe300014da0 con 0x564c4e1ac0b0 2023-10-08T13:53:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2459796544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c4e0c6e50 con 0x564c4e1ac0b0 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2459796544 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe30001f400 con 0x564c4e1ac0b0 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe321cb0b80 10 client.16500 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2459796544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564c4df12c00 con 0x564c4e1ac0b0 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2459796544 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c4e0c6e50 con 0x564c4e1ac0b0 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2459796544 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fe300012610 con 0x564c4e1ac0b0 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe321cb0b80 10 client.16500 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe321cb0b80 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe321cb0b80 20 client.16500 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.755+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564c4e1c5e70 0x564c4e1c8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.756+0000 7fe321cb0b80 4 client.16500 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.756+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2459796544 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c4e0f7db0 con 0x564c4e1c5e70 2023-10-08T13:53:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.756+0000 7fe30effd700 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564c4e1c5e70 0x564c4e1c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.757+0000 7fe30effd700 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564c4e1c5e70 0x564c4e1c8290 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dd7c1486848557f server_cookie=b4eadeac7bd96a21 in_seq=0 out_seq=0 2023-10-08T13:53:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.757+0000 7fe2eeffd700 10 client.16500 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2459796544 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564c4e0f7db0 con 0x564c4e1c5e70 2023-10-08T13:53:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe2eeffd700 10 client.16500 handle_command_reply: tid=0 2023-10-08T13:53:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe321cb0b80 1 client.16500 shutdown 2023-10-08T13:53:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2459796544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 msgr2=0x564c4e1b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1b5630 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe30000dd20 tx=0x7fe300014710 comp rx=0 tx=0).stop 2023-10-08T13:53:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2459796544 shutdown_connections 2023-10-08T13:53:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564c4e1c5e70 0x564c4e1c8290 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e1b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1ac0b0 0x564c4e1b5630 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2459796544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b1880 0x564c4e1b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2459796544 >> 172.21.15.57:0/2459796544 conn(0x564c4e120200 msgr2=0x564c4e1a6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.758+0000 7fe2eeffd700 0 client.16500 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.759+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2459796544 shutdown_connections 2023-10-08T13:53:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.759+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2459796544 wait complete. 2023-10-08T13:53:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.759+0000 7fe321cb0b80 20 client.16500 trim_cache size 0 max 16384 2023-10-08T13:53:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.784+0000 7fe321cb0b80 1 Processor -- start 2023-10-08T13:53:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.784+0000 7fe321cb0b80 1 -- start start 2023-10-08T13:53:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1ac0b0 0x564c4e1abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 0x564c4e1a6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b21a0 0x564c4e1a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4df7f8c0 con 0x564c4e1b1880 2023-10-08T13:53:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e1b3bc0 con 0x564c4e1ac0b0 2023-10-08T13:53:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e1b0330 con 0x564c4e1b21a0 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe30f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1ac0b0 0x564c4e1abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe30f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1ac0b0 0x564c4e1abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59332/0 (socket says 172.21.15.57:59332) 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe30f7fe700 1 -- 172.21.15.57:0/1925290517 learned_addr learned my addr 172.21.15.57:0/1925290517 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe30ffff700 1 --2- 172.21.15.57:0/1925290517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b21a0 0x564c4e1a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe30f7fe700 1 -- 172.21.15.57:0/1925290517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b21a0 msgr2=0x564c4e1a6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/1925290517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b21a0 0x564c4e1a6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe30f7fe700 1 -- 172.21.15.57:0/1925290517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 msgr2=0x564c4e1a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/1925290517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 0x564c4e1a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe30f7fe700 1 -- 172.21.15.57:0/1925290517 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c4df501f0 con 0x564c4e1ac0b0 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.785+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/1925290517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1ac0b0 0x564c4e1abb70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe300012fd0 tx=0x7fe300013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f66dab4dda603d75 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.786+0000 7fe2f67fc700 1 -- 172.21.15.57:0/1925290517 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000095e0 con 0x564c4e1ac0b0 2023-10-08T13:53:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.786+0000 7fe2f67fc700 1 -- 172.21.15.57:0/1925290517 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe300018050 con 0x564c4e1ac0b0 2023-10-08T13:53:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.786+0000 7fe2f67fc700 1 -- 172.21.15.57:0/1925290517 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30002f690 con 0x564c4e1ac0b0 2023-10-08T13:53:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.787+0000 7fe321cb0b80 1 -- 172.21.15.57:0/1925290517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1ac0b0 msgr2=0x564c4e1abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.787+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/1925290517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1ac0b0 0x564c4e1abb70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe300012fd0 tx=0x7fe300013000 comp rx=0 tx=0).stop 2023-10-08T13:53:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.787+0000 7fe321cb0b80 1 -- 172.21.15.57:0/1925290517 shutdown_connections 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.787+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/1925290517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1b21a0 0x564c4e1a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.787+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/1925290517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1ac0b0 0x564c4e1abb70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.787+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/1925290517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 0x564c4e1a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.787+0000 7fe321cb0b80 1 -- 172.21.15.57:0/1925290517 >> 172.21.15.57:0/1925290517 conn(0x564c4e120200 msgr2=0x564c4e1a2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.787+0000 7fe321cb0b80 1 -- 172.21.15.57:0/1925290517 shutdown_connections 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.787+0000 7fe321cb0b80 1 -- 172.21.15.57:0/1925290517 wait complete. 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe321cb0b80 1 -- start start 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe321cb0b80 1 Processor -- start 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1ac0b0 0x564c4e1baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 0x564c4e1bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe321cb0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e218ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e1bb9a0 con 0x564c4e1b1880 2023-10-08T13:53:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e2191e0 con 0x564c4e1b21a0 2023-10-08T13:53:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe321cb0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4e219320 con 0x564c4e1ac0b0 2023-10-08T13:53:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe321cb0b80 10 client.0 _ll_register_callbacks cb 0x7fe30000d690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:53:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe30effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 0x564c4e1bb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe30effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 0x564c4e1bb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57914/0 (socket says 172.21.15.57:57914) 2023-10-08T13:53:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe30effd700 1 -- 172.21.15.57:0/2469630102 learned_addr learned my addr 172.21.15.57:0/2469630102 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:53:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe30ffff700 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e218ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.788+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1ac0b0 0x564c4e1baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe30effd700 1 -- 172.21.15.57:0/2469630102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1ac0b0 msgr2=0x564c4e1baba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe30effd700 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1ac0b0 0x564c4e1baba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe30effd700 1 -- 172.21.15.57:0/2469630102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 msgr2=0x564c4e218ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:53:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe30effd700 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e218ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe30effd700 1 -- 172.21.15.57:0/2469630102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c4df12c00 con 0x564c4e1b1880 2023-10-08T13:53:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe30ffff700 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e218ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:53:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe30effd700 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 0x564c4e1bb0e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe3040ca760 tx=0x7fe30409e260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3243a533255101e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:53:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe2eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:53:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2469630102 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304096780 con 0x564c4e1b1880 2023-10-08T13:53:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2469630102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3080092d0 con 0x564c4e1b1880 2023-10-08T13:53:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2469630102 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe3040968e0 con 0x564c4e1b1880 2023-10-08T13:53:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2469630102 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe304099560 con 0x564c4e1b1880 2023-10-08T13:53:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe321cb0b80 10 client.16503 fetch_fsmap learned FSMap version 32 2023-10-08T13:53:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.789+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2469630102 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564c4df28250 con 0x564c4e1b1880 2023-10-08T13:53:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.790+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2469630102 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3080092d0 con 0x564c4e1b1880 2023-10-08T13:53:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.790+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2469630102 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fe3040990a0 con 0x564c4e1b1880 2023-10-08T13:53:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.790+0000 7fe321cb0b80 10 client.16503 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:53:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.790+0000 7fe321cb0b80 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:53:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.790+0000 7fe321cb0b80 20 client.16503 populate_metadata read hostname 'smithi057' 2023-10-08T13:53:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.790+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564c4e21abd0 0x564c4e21cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:53:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.790+0000 7fe321cb0b80 4 client.16503 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:53:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.790+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2469630102 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564c4e0f7ad0 con 0x564c4e21abd0 2023-10-08T13:53:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.790+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564c4e21abd0 0x564c4e21cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:53:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.791+0000 7fe2eeffd700 10 client.16503 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:46.791+0000 7fe30f7fe700 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564c4e21abd0 0x564c4e21cff0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d240e1f9fdd54 server_cookie=51949cb82128cf77 in_seq=0 out_seq=0 2023-10-08T13:53:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe2eeffd700 1 -- 172.21.15.57:0/2469630102 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564c4e0f7ad0 con 0x564c4e21abd0 2023-10-08T13:53:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe2eeffd700 10 client.16503 handle_command_reply: tid=0 2023-10-08T13:53:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 client.16503 shutdown 2023-10-08T13:53:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2469630102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 msgr2=0x564c4e1bb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 0x564c4e1bb0e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe3040ca760 tx=0x7fe30409e260 comp rx=0 tx=0).stop 2023-10-08T13:53:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2469630102 shutdown_connections 2023-10-08T13:53:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564c4e21abd0 0x564c4e21cff0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c4e1b21a0 0x564c4e218ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe2eeffd700 0 client.16503 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:53:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4e1ac0b0 0x564c4e1baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 --2- 172.21.15.57:0/2469630102 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564c4e1b1880 0x564c4e1bb0e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2469630102 >> 172.21.15.57:0/2469630102 conn(0x564c4e120200 msgr2=0x564c4e1b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2469630102 shutdown_connections 2023-10-08T13:53:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 1 -- 172.21.15.57:0/2469630102 wait complete. 2023-10-08T13:53:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.461+0000 7fe321cb0b80 20 client.16503 trim_cache size 0 max 16384 2023-10-08T13:53:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:53:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:53:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:53:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:53:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 -- 172.21.15.57:0/1234832630 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe2f80a15f0 msgr2=0x7fe2f80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe2f80a15f0 0x7fe2f80a3ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe300006fd0 tx=0x7fe300008040 comp rx=0 tx=0).stop 2023-10-08T13:53:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 -- 172.21.15.57:0/1234832630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe310075640 msgr2=0x7fe31019d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:53:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe310075640 0x7fe31019d890 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe30400d950 tx=0x7fe30400dc60 comp rx=0 tx=0).stop 2023-10-08T13:53:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 -- 172.21.15.57:0/1234832630 shutdown_connections 2023-10-08T13:53:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe310111590 0x7fe3101a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe2f80a15f0 0x7fe2f80a3ab0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe310074d20 0x7fe31019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 --2- 172.21.15.57:0/1234832630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe310075640 0x7fe31019d890 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:53:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 -- 172.21.15.57:0/1234832630 >> 172.21.15.57:0/1234832630 conn(0x7fe3100fe3a0 msgr2=0x7fe310112cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:53:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 -- 172.21.15.57:0/1234832630 shutdown_connections 2023-10-08T13:53:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:53:47.462+0000 7fe315ac7700 1 -- 172.21.15.57:0/1234832630 wait complete. 2023-10-08T13:53:48.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:47 smithi115 ceph-mon[104408]: pgmap v366: 129 pgs: 129 active+clean; 289 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 651 op/s 2023-10-08T13:53:48.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:47 smithi057 ceph-mon[99476]: pgmap v366: 129 pgs: 129 active+clean; 289 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 651 op/s 2023-10-08T13:53:48.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:47 smithi099 ceph-mon[103849]: pgmap v366: 129 pgs: 129 active+clean; 289 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 651 op/s 2023-10-08T13:53:50.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:49 smithi115 ceph-mon[104408]: pgmap v367: 129 pgs: 129 active+clean; 308 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 612 op/s 2023-10-08T13:53:50.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:49 smithi057 ceph-mon[99476]: pgmap v367: 129 pgs: 129 active+clean; 308 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 612 op/s 2023-10-08T13:53:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:49 smithi099 ceph-mon[103849]: pgmap v367: 129 pgs: 129 active+clean; 308 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 612 op/s 2023-10-08T13:53:52.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:51 smithi115 ceph-mon[104408]: pgmap v368: 129 pgs: 129 active+clean; 345 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 15 MiB/s wr, 778 op/s 2023-10-08T13:53:52.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:51 smithi057 ceph-mon[99476]: pgmap v368: 129 pgs: 129 active+clean; 345 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 15 MiB/s wr, 778 op/s 2023-10-08T13:53:52.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:51 smithi099 ceph-mon[103849]: pgmap v368: 129 pgs: 129 active+clean; 345 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 15 MiB/s wr, 778 op/s 2023-10-08T13:53:54.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:53 smithi115 ceph-mon[104408]: pgmap v369: 129 pgs: 129 active+clean; 356 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 15 MiB/s wr, 839 op/s 2023-10-08T13:53:54.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:53 smithi057 ceph-mon[99476]: pgmap v369: 129 pgs: 129 active+clean; 356 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 15 MiB/s wr, 839 op/s 2023-10-08T13:53:54.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:53 smithi099 ceph-mon[103849]: pgmap v369: 129 pgs: 129 active+clean; 356 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 15 MiB/s wr, 839 op/s 2023-10-08T13:53:54.364 INFO:tasks.workunit.client.0.smithi057.stdout: 101 489594 457 341305 366 85289 74 2023-10-08T13:53:56.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:55 smithi115 ceph-mon[104408]: pgmap v370: 129 pgs: 129 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 809 op/s 2023-10-08T13:53:56.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:55 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:53:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:55 smithi057 ceph-mon[99476]: pgmap v370: 129 pgs: 129 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 809 op/s 2023-10-08T13:53:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:55 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:53:56.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:55 smithi099 ceph-mon[103849]: pgmap v370: 129 pgs: 129 active+clean; 353 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 809 op/s 2023-10-08T13:53:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:55 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:53:57.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:56 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:56 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:56 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:56 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:56 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:53:57.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:56 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:53:57.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:56 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:56 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:56 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:56 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:56 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:56 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:53:57.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:56 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:53:57.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:56 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:53:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:53:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:56 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:53:58.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:57 smithi057 ceph-mon[99476]: pgmap v371: 129 pgs: 129 active+clean; 357 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 640 op/s 2023-10-08T13:53:58.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:57 smithi099 ceph-mon[103849]: pgmap v371: 129 pgs: 129 active+clean; 357 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 640 op/s 2023-10-08T13:53:58.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:57 smithi115 ceph-mon[104408]: pgmap v371: 129 pgs: 129 active+clean; 357 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 640 op/s 2023-10-08T13:54:00.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:53:59 smithi057 ceph-mon[99476]: pgmap v372: 129 pgs: 129 active+clean; 359 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 452 op/s 2023-10-08T13:54:00.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:53:59 smithi099 ceph-mon[103849]: pgmap v372: 129 pgs: 129 active+clean; 359 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 452 op/s 2023-10-08T13:54:00.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:53:59 smithi115 ceph-mon[104408]: pgmap v372: 129 pgs: 129 active+clean; 359 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 452 op/s 2023-10-08T13:54:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.040+0000 7f3a84693700 1 -- 172.21.15.57:0/758064975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c0fefc0 msgr2=0x7f3a7c0ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.040+0000 7f3a84693700 1 --2- 172.21.15.57:0/758064975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c0fefc0 0x7f3a7c0ff3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3a780098f0 tx=0x7f3a78009c00 comp rx=0 tx=0).stop 2023-10-08T13:54:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.040+0000 7f3a84693700 1 -- 172.21.15.57:0/758064975 shutdown_connections 2023-10-08T13:54:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.040+0000 7f3a84693700 1 --2- 172.21.15.57:0/758064975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a7c1040e0 0x7f3a7c10bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.040+0000 7f3a84693700 1 --2- 172.21.15.57:0/758064975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c0fefc0 0x7f3a7c0ff3a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.040+0000 7f3a84693700 1 --2- 172.21.15.57:0/758064975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c0ff970 0x7f3a7c1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.040+0000 7f3a84693700 1 -- 172.21.15.57:0/758064975 >> 172.21.15.57:0/758064975 conn(0x7f3a7c075bf0 msgr2=0x7f3a7c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.040+0000 7f3a84693700 1 -- 172.21.15.57:0/758064975 shutdown_connections 2023-10-08T13:54:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.040+0000 7f3a84693700 1 -- 172.21.15.57:0/758064975 wait complete. 2023-10-08T13:54:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a84693700 1 Processor -- start 2023-10-08T13:54:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a84693700 1 -- start start 2023-10-08T13:54:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a84693700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a7c0fefc0 0x7f3a7c10d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a84693700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c1040e0 0x7f3a7c10d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a84693700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c10de00 0x7f3a7c1a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a84693700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1a53e0 con 0x7f3a7c0fefc0 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a84693700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1a5550 con 0x7f3a7c10de00 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a84693700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1a5710 con 0x7f3a7c1040e0 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a8242f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a7c0fefc0 0x7f3a7c10d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a8242f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a7c0fefc0 0x7f3a7c10d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57926/0 (socket says 172.21.15.57:57926) 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a82c30700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c10de00 0x7f3a7c1a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.041+0000 7f3a81c2e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c1040e0 0x7f3a7c10d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a8242f700 1 -- 172.21.15.57:0/2509193264 learned_addr learned my addr 172.21.15.57:0/2509193264 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a81c2e700 1 -- 172.21.15.57:0/2509193264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c10de00 msgr2=0x7f3a7c1a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c10de00 0x7f3a7c1a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a81c2e700 1 -- 172.21.15.57:0/2509193264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a7c0fefc0 msgr2=0x7f3a7c10d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a7c0fefc0 0x7f3a7c10d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a81c2e700 1 -- 172.21.15.57:0/2509193264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a78009520 con 0x7f3a7c1040e0 2023-10-08T13:54:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a8242f700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a7c0fefc0 0x7f3a7c10d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c1040e0 0x7f3a7c10d8c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00e9c0 tx=0x7f3a6c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a737fe700 1 -- 172.21.15.57:0/2509193264 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00c9c0 con 0x7f3a7c1040e0 2023-10-08T13:54:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a737fe700 1 -- 172.21.15.57:0/2509193264 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c010430 con 0x7f3a7c1040e0 2023-10-08T13:54:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a84693700 1 -- 172.21.15.57:0/2509193264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c1a59f0 con 0x7f3a7c1040e0 2023-10-08T13:54:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a737fe700 1 -- 172.21.15.57:0/2509193264 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c018680 con 0x7f3a7c1040e0 2023-10-08T13:54:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.042+0000 7f3a84693700 1 -- 172.21.15.57:0/2509193264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c1a5f50 con 0x7f3a7c1040e0 2023-10-08T13:54:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.044+0000 7f3a737fe700 1 -- 172.21.15.57:0/2509193264 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3a6c00cb20 con 0x7f3a7c1040e0 2023-10-08T13:54:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.045+0000 7f3a737fe700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a680a1670 0x7f3a680a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.045+0000 7f3a737fe700 1 -- 172.21.15.57:0/2509193264 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3a6c014070 con 0x7f3a7c1040e0 2023-10-08T13:54:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.045+0000 7f3a8242f700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a680a1670 0x7f3a680a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.045+0000 7f3a8242f700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a680a1670 0x7f3a680a3b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a78019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.047+0000 7f3a9087cb80 1 Processor -- start 2023-10-08T13:54:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a9087cb80 1 -- start start 2023-10-08T13:54:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd1880 0x55cdffdd1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd21a0 0x55cdffdcbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdcc0b0 0x55cdffdcc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffd34400 con 0x55cdffdd1880 2023-10-08T13:54:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffd25bc0 con 0x55cdffdd21a0 2023-10-08T13:54:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffd234e0 con 0x55cdffdcc0b0 2023-10-08T13:54:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a81c2e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd21a0 0x55cdffdcbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a8242f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd1880 0x55cdffdd1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a81c2e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd21a0 0x55cdffdcbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59362/0 (socket says 172.21.15.57:59362) 2023-10-08T13:54:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a81c2e700 1 -- 172.21.15.57:0/3712094987 learned_addr learned my addr 172.21.15.57:0/3712094987 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a82c30700 1 --2- 172.21.15.57:0/3712094987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdcc0b0 0x55cdffdcc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a81c2e700 1 -- 172.21.15.57:0/3712094987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdcc0b0 msgr2=0x55cdffdcc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/3712094987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdcc0b0 0x55cdffdcc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a81c2e700 1 -- 172.21.15.57:0/3712094987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd1880 msgr2=0x55cdffdd1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/3712094987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd1880 0x55cdffdd1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a81c2e700 1 -- 172.21.15.57:0/3712094987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdffb49d80 con 0x55cdffdd21a0 2023-10-08T13:54:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.048+0000 7f3a8242f700 1 --2- 172.21.15.57:0/3712094987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd1880 0x55cdffdd1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.049+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/3712094987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd21a0 0x55cdffdcbb70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c096690 tx=0x7f3a6c096600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=283ee8b653451330 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.049+0000 7f3a70ff9700 1 -- 172.21.15.57:0/3712094987 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c09e120 con 0x55cdffdd21a0 2023-10-08T13:54:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.049+0000 7f3a70ff9700 1 -- 172.21.15.57:0/3712094987 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c09a190 con 0x55cdffdd21a0 2023-10-08T13:54:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.049+0000 7f3a70ff9700 1 -- 172.21.15.57:0/3712094987 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0106c0 con 0x55cdffdd21a0 2023-10-08T13:54:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.049+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3712094987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd21a0 msgr2=0x55cdffdcbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.049+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/3712094987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd21a0 0x55cdffdcbb70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c096690 tx=0x7f3a6c096600 comp rx=0 tx=0).stop 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3712094987 shutdown_connections 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/3712094987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdcc0b0 0x55cdffdcc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/3712094987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd1880 0x55cdffdd1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/3712094987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd21a0 0x55cdffdcbb70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3712094987 >> 172.21.15.57:0/3712094987 conn(0x55cdffd40200 msgr2=0x55cdffdc6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3712094987 shutdown_connections 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3712094987 wait complete. 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 -- start start 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 Processor -- start 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffdd55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffdd5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 0x55cdffdd6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffd1edd0 con 0x55cdffdcc0b0 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffcbb6b0 con 0x55cdffdd1880 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffcbb350 con 0x55cdffdd21a0 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.050+0000 7f3a9087cb80 10 client.0 _ll_register_callbacks cb 0x55cdffb187a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a81c2e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffdd5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a81c2e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffdd5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59368/0 (socket says 172.21.15.57:59368) 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a81c2e700 1 -- 172.21.15.57:0/722892066 learned_addr learned my addr 172.21.15.57:0/722892066 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a8242f700 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffdd55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a82c30700 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 0x55cdffdd6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a8242f700 1 -- 172.21.15.57:0/722892066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 msgr2=0x55cdffdd6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a8242f700 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 0x55cdffdd6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a8242f700 1 -- 172.21.15.57:0/722892066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 msgr2=0x55cdffdd5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a8242f700 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffdd5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a8242f700 1 -- 172.21.15.57:0/722892066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c0675b0 con 0x55cdffdcc0b0 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a82c30700 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 0x55cdffdd6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffdd5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a8242f700 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffdd55e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009810 tx=0x7f3a78009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a13e64e38d343a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a65ffb700 1 -- 172.21.15.57:0/722892066 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a78005340 con 0x55cdffdcc0b0 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a65ffb700 1 -- 172.21.15.57:0/722892066 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3a780054a0 con 0x55cdffdcc0b0 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.051+0000 7f3a9087cb80 1 -- 172.21.15.57:0/722892066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdffce6e50 con 0x55cdffdcc0b0 2023-10-08T13:54:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a65ffb700 1 -- 172.21.15.57:0/722892066 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a78005660 con 0x55cdffdcc0b0 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a9087cb80 10 client.16515 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a9087cb80 1 -- 172.21.15.57:0/722892066 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f3a7c1a5f50 con 0x55cdffdcc0b0 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a65ffb700 1 -- 172.21.15.57:0/722892066 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cdffce6e50 con 0x55cdffdcc0b0 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a65ffb700 1 -- 172.21.15.57:0/722892066 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f3a7802e6e0 con 0x55cdffdcc0b0 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a9087cb80 10 client.16515 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a9087cb80 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a9087cb80 20 client.16515 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cdffde5d30 0x55cdffde8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a9087cb80 4 client.16515 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a9087cb80 1 -- 172.21.15.57:0/722892066 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdffd17db0 con 0x55cdffde5d30 2023-10-08T13:54:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.052+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cdffde5d30 0x55cdffde8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.053+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cdffde5d30 0x55cdffde8150 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846b8f98d194def4 server_cookie=c6e057d1c9ccf38b in_seq=0 out_seq=0 2023-10-08T13:54:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.053+0000 7f3a65ffb700 10 client.16515 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.053+0000 7f3a65ffb700 1 -- 172.21.15.57:0/722892066 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cdffd17db0 con 0x55cdffde5d30 2023-10-08T13:54:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.053+0000 7f3a65ffb700 10 client.16515 handle_command_reply: tid=0 2023-10-08T13:54:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.053+0000 7f3a9087cb80 1 client.16515 shutdown 2023-10-08T13:54:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.053+0000 7f3a9087cb80 1 -- 172.21.15.57:0/722892066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 msgr2=0x55cdffdd55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.053+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffdd55e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009810 tx=0x7f3a78009260 comp rx=0 tx=0).stop 2023-10-08T13:54:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.053+0000 7f3a9087cb80 1 -- 172.21.15.57:0/722892066 shutdown_connections 2023-10-08T13:54:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.054+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cdffde5d30 0x55cdffde8150 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.054+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 0x55cdffdd6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.054+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffdd55e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.054+0000 7f3a65ffb700 0 client.16515 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.054+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/722892066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffdd5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.054+0000 7f3a9087cb80 1 -- 172.21.15.57:0/722892066 >> 172.21.15.57:0/722892066 conn(0x55cdffd40200 msgr2=0x55cdffdc6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.054+0000 7f3a9087cb80 1 -- 172.21.15.57:0/722892066 shutdown_connections 2023-10-08T13:54:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.054+0000 7f3a9087cb80 1 -- 172.21.15.57:0/722892066 wait complete. 2023-10-08T13:54:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.054+0000 7f3a9087cb80 20 client.16515 trim_cache size 0 max 16384 2023-10-08T13:54:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.069+0000 7f3a9087cb80 1 Processor -- start 2023-10-08T13:54:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.069+0000 7f3a9087cb80 1 -- start start 2023-10-08T13:54:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.069+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdcc0b0 0x55cdffdcbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.069+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd1880 0x55cdffdd0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.069+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd21a0 0x55cdffddcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.069+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffb9f8c0 con 0x55cdffdd21a0 2023-10-08T13:54:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.069+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffdd12f0 con 0x55cdffdcc0b0 2023-10-08T13:54:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.069+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffdc7940 con 0x55cdffdd1880 2023-10-08T13:54:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a8242f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdcc0b0 0x55cdffdcbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a82c30700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd21a0 0x55cdffddcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a8242f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdcc0b0 0x55cdffdcbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59374/0 (socket says 172.21.15.57:59374) 2023-10-08T13:54:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a8242f700 1 -- 172.21.15.57:0/1378073173 learned_addr learned my addr 172.21.15.57:0/1378073173 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a82c30700 1 --2- 172.21.15.57:0/1378073173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd21a0 0x55cdffddcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57952/0 (socket says 172.21.15.57:57952) 2023-10-08T13:54:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/1378073173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd1880 0x55cdffdd0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a8242f700 1 -- 172.21.15.57:0/1378073173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd1880 msgr2=0x55cdffdd0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a8242f700 1 --2- 172.21.15.57:0/1378073173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd1880 0x55cdffdd0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a8242f700 1 -- 172.21.15.57:0/1378073173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd21a0 msgr2=0x55cdffddcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a8242f700 1 --2- 172.21.15.57:0/1378073173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd21a0 0x55cdffddcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a8242f700 1 -- 172.21.15.57:0/1378073173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdffb32c00 con 0x55cdffdcc0b0 2023-10-08T13:54:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a8242f700 1 --2- 172.21.15.57:0/1378073173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdcc0b0 0x55cdffdcbb70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a7802ea10 tx=0x7f3a7800f560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde2e17c9cfb472a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a717fa700 1 -- 172.21.15.57:0/1378073173 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7802e610 con 0x55cdffdcc0b0 2023-10-08T13:54:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a717fa700 1 -- 172.21.15.57:0/1378073173 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a7801d040 con 0x55cdffdcc0b0 2023-10-08T13:54:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.070+0000 7f3a717fa700 1 -- 172.21.15.57:0/1378073173 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a78003a00 con 0x55cdffdcc0b0 2023-10-08T13:54:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.071+0000 7f3a9087cb80 1 -- 172.21.15.57:0/1378073173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdcc0b0 msgr2=0x55cdffdcbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.071+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/1378073173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdcc0b0 0x55cdffdcbb70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a7802ea10 tx=0x7f3a7800f560 comp rx=0 tx=0).stop 2023-10-08T13:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.071+0000 7f3a9087cb80 1 -- 172.21.15.57:0/1378073173 shutdown_connections 2023-10-08T13:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.071+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/1378073173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdd21a0 0x55cdffddcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.071+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/1378073173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdcc0b0 0x55cdffdcbb70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.071+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/1378073173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd1880 0x55cdffdd0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.071+0000 7f3a9087cb80 1 -- 172.21.15.57:0/1378073173 >> 172.21.15.57:0/1378073173 conn(0x55cdffd40200 msgr2=0x55cdffdd06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.071+0000 7f3a9087cb80 1 -- 172.21.15.57:0/1378073173 shutdown_connections 2023-10-08T13:54:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.071+0000 7f3a9087cb80 1 -- 172.21.15.57:0/1378073173 wait complete. 2023-10-08T13:54:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a9087cb80 1 -- start start 2023-10-08T13:54:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a9087cb80 1 Processor -- start 2023-10-08T13:54:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffe2daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffe2e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a9087cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 0x55cdffe2e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffe38cf0 con 0x55cdffdcc0b0 2023-10-08T13:54:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffe38e60 con 0x55cdffdd1880 2023-10-08T13:54:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a9087cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdffe39020 con 0x55cdffdd21a0 2023-10-08T13:54:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a9087cb80 10 client.0 _ll_register_callbacks cb 0x7f3a7800b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a8242f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffe2daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a81c2e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffe2e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a81c2e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffe2e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59382/0 (socket says 172.21.15.57:59382) 2023-10-08T13:54:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a8242f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffe2daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57954/0 (socket says 172.21.15.57:57954) 2023-10-08T13:54:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a81c2e700 1 -- 172.21.15.57:0/3444047321 learned_addr learned my addr 172.21.15.57:0/3444047321 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a82c30700 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 0x55cdffe2e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a81c2e700 1 -- 172.21.15.57:0/3444047321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 msgr2=0x55cdffe2e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 0x55cdffe2e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a81c2e700 1 -- 172.21.15.57:0/3444047321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 msgr2=0x55cdffe2daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffe2daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a81c2e700 1 -- 172.21.15.57:0/3444047321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c1a5f50 con 0x55cdffdd1880 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.072+0000 7f3a8242f700 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffe2daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a81c2e700 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffe2e030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c096690 tx=0x7f3a6c005b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba8c13631a6c51ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a65ffb700 1 -- 172.21.15.57:0/3444047321 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c09e120 con 0x55cdffdd1880 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a65ffb700 1 -- 172.21.15.57:0/3444047321 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3a6c098530 con 0x55cdffdd1880 2023-10-08T13:54:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3444047321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a6c09a190 con 0x55cdffdd1880 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a65ffb700 1 -- 172.21.15.57:0/3444047321 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a6c010860 con 0x55cdffdd1880 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a9087cb80 10 client.26054 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3444047321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cdffb701f0 con 0x55cdffdd1880 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a65ffb700 1 -- 172.21.15.57:0/3444047321 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c09a190 con 0x55cdffdd1880 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a65ffb700 1 -- 172.21.15.57:0/3444047321 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f3a6c09a3f0 con 0x55cdffdd1880 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a9087cb80 10 client.26054 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a9087cb80 10 client.26054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a9087cb80 20 client.26054 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cdffe3a640 0x55cdffe3ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a9087cb80 4 client.26054 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.073+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3444047321 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cdffd17ad0 con 0x55cdffe3a640 2023-10-08T13:54:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.074+0000 7f3a8242f700 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cdffe3a640 0x55cdffe3ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.074+0000 7f3a65ffb700 10 client.26054 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.074+0000 7f3a8242f700 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cdffe3a640 0x55cdffe3ca60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a5a37aae893961 server_cookie=858ebec9a6e69749 in_seq=0 out_seq=0 2023-10-08T13:54:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:00 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:54:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:00 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:54:01.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:00 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:54:01.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:00 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:54:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:00 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:54:01.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:00 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:54:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a65ffb700 1 -- 172.21.15.57:0/3444047321 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cdffd17ad0 con 0x55cdffe3a640 2023-10-08T13:54:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a65ffb700 10 client.26054 handle_command_reply: tid=0 2023-10-08T13:54:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 client.26054 shutdown 2023-10-08T13:54:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3444047321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 msgr2=0x55cdffe2e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffe2e030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c096690 tx=0x7f3a6c005b60 comp rx=0 tx=0).stop 2023-10-08T13:54:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3444047321 shutdown_connections 2023-10-08T13:54:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cdffe3a640 0x55cdffe3ca60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cdffdd21a0 0x55cdffe2e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a65ffb700 0 client.26054 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cdffdcc0b0 0x55cdffe2daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 --2- 172.21.15.57:0/3444047321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cdffdd1880 0x55cdffe2e030 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3444047321 >> 172.21.15.57:0/3444047321 conn(0x55cdffd40200 msgr2=0x55cdffdd2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3444047321 shutdown_connections 2023-10-08T13:54:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 1 -- 172.21.15.57:0/3444047321 wait complete. 2023-10-08T13:54:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a9087cb80 20 client.26054 trim_cache size 0 max 16384 2023-10-08T13:54:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:54:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:54:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:54:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:54:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a84693700 1 -- 172.21.15.57:0/2509193264 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a680a1670 msgr2=0x7f3a680a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a84693700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a680a1670 0x7f3a680a3b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a78019040 comp rx=0 tx=0).stop 2023-10-08T13:54:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.579+0000 7f3a84693700 1 -- 172.21.15.57:0/2509193264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c1040e0 msgr2=0x7f3a7c10d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.580+0000 7f3a84693700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c1040e0 0x7f3a7c10d8c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00e9c0 tx=0x7f3a6c00ecd0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.580+0000 7f3a84693700 1 -- 172.21.15.57:0/2509193264 shutdown_connections 2023-10-08T13:54:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.580+0000 7f3a84693700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a680a1670 0x7f3a680a3b30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.580+0000 7f3a84693700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c10de00 0x7f3a7c1a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.580+0000 7f3a84693700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a7c0fefc0 0x7f3a7c10d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.580+0000 7f3a84693700 1 --2- 172.21.15.57:0/2509193264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c1040e0 0x7f3a7c10d8c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.580+0000 7f3a84693700 1 -- 172.21.15.57:0/2509193264 >> 172.21.15.57:0/2509193264 conn(0x7f3a7c075bf0 msgr2=0x7f3a7c102440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.580+0000 7f3a84693700 1 -- 172.21.15.57:0/2509193264 shutdown_connections 2023-10-08T13:54:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:01.580+0000 7f3a84693700 1 -- 172.21.15.57:0/2509193264 wait complete. 2023-10-08T13:54:02.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:01 smithi099 ceph-mon[103849]: pgmap v373: 129 pgs: 129 active+clean; 359 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 4.9 MiB/s wr, 377 op/s 2023-10-08T13:54:02.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:01 smithi115 ceph-mon[104408]: pgmap v373: 129 pgs: 129 active+clean; 359 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 4.9 MiB/s wr, 377 op/s 2023-10-08T13:54:02.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:01 smithi057 ceph-mon[99476]: pgmap v373: 129 pgs: 129 active+clean; 359 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 4.9 MiB/s wr, 377 op/s 2023-10-08T13:54:04.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:03 smithi099 ceph-mon[103849]: pgmap v374: 129 pgs: 129 active+clean; 361 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2023-10-08T13:54:04.364 INFO:tasks.workunit.client.0.smithi057.stdout: 124 821467 1080 571083 1288 127417 40 2023-10-08T13:54:04.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:03 smithi115 ceph-mon[104408]: pgmap v374: 129 pgs: 129 active+clean; 361 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2023-10-08T13:54:04.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:03 smithi057 ceph-mon[99476]: pgmap v374: 129 pgs: 129 active+clean; 361 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.9 MiB/s wr, 175 op/s 2023-10-08T13:54:06.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:05 smithi099 ceph-mon[103849]: pgmap v375: 129 pgs: 129 active+clean; 367 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2023-10-08T13:54:06.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:05 smithi115 ceph-mon[104408]: pgmap v375: 129 pgs: 129 active+clean; 367 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2023-10-08T13:54:06.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:05 smithi057 ceph-mon[99476]: pgmap v375: 129 pgs: 129 active+clean; 367 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2023-10-08T13:54:08.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:08 smithi099 ceph-mon[103849]: pgmap v376: 129 pgs: 129 active+clean; 370 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 353 op/s 2023-10-08T13:54:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:08 smithi115 ceph-mon[104408]: pgmap v376: 129 pgs: 129 active+clean; 370 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 353 op/s 2023-10-08T13:54:08.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:08 smithi057 ceph-mon[99476]: pgmap v376: 129 pgs: 129 active+clean; 370 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 353 op/s 2023-10-08T13:54:09.557 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:54:09.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.704+0000 7fb4ffe04700 1 -- 172.21.15.57:0/4159358602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f81010e0 msgr2=0x7fb4f810d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:09.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.704+0000 7fb4ffe04700 1 --2- 172.21.15.57:0/4159358602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f81010e0 0x7fb4f810d5f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009f90 tx=0x7fb4f000b330 comp rx=0 tx=0).stop 2023-10-08T13:54:09.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.704+0000 7fb4ffe04700 1 -- 172.21.15.57:0/4159358602 shutdown_connections 2023-10-08T13:54:09.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.704+0000 7fb4ffe04700 1 --2- 172.21.15.57:0/4159358602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f81010e0 0x7fb4f810d5f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.704+0000 7fb4ffe04700 1 --2- 172.21.15.57:0/4159358602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f8106730 0x7fb4f8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.704+0000 7fb4ffe04700 1 --2- 172.21.15.57:0/4159358602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f8100720 0x7fb4f8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.704+0000 7fb4ffe04700 1 -- 172.21.15.57:0/4159358602 >> 172.21.15.57:0/4159358602 conn(0x7fb4f80fc230 msgr2=0x7fb4f80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:09.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.704+0000 7fb4ffe04700 1 -- 172.21.15.57:0/4159358602 shutdown_connections 2023-10-08T13:54:09.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.704+0000 7fb4ffe04700 1 -- 172.21.15.57:0/4159358602 wait complete. 2023-10-08T13:54:09.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4ffe04700 1 Processor -- start 2023-10-08T13:54:09.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4ffe04700 1 -- start start 2023-10-08T13:54:09.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4ffe04700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f8100720 0x7fb4f8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4ffe04700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f81010e0 0x7fb4f8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4ffe04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f8106730 0x7fb4f8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4ffe04700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81a4ea0 con 0x7fb4f8100720 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4ffe04700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81a5010 con 0x7fb4f8106730 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4ffe04700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81a51d0 con 0x7fb4f81010e0 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fdba0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f8100720 0x7fb4f8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fd39f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f81010e0 0x7fb4f8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fdba0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f8100720 0x7fb4f8072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57968/0 (socket says 172.21.15.57:57968) 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fdba0700 1 -- 172.21.15.57:0/2169857140 learned_addr learned my addr 172.21.15.57:0/2169857140 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fe3a1700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f8106730 0x7fb4f8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fdba0700 1 -- 172.21.15.57:0/2169857140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f81010e0 msgr2=0x7fb4f8073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f81010e0 0x7fb4f8073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fdba0700 1 -- 172.21.15.57:0/2169857140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f8106730 msgr2=0x7fb4f8073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:09.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f8106730 0x7fb4f8073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fdba0700 1 -- 172.21.15.57:0/2169857140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0009c40 con 0x7fb4f8100720 2023-10-08T13:54:09.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.705+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f81010e0 0x7fb4f8073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:09.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.706+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f8100720 0x7fb4f8072af0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400e9c0 tx=0x7fb4f400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:09.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.706+0000 7fb4eeffd700 1 -- 172.21.15.57:0/2169857140 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4004a30 con 0x7fb4f8100720 2023-10-08T13:54:09.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.706+0000 7fb4ffe04700 1 -- 172.21.15.57:0/2169857140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f81a54b0 con 0x7fb4f8100720 2023-10-08T13:54:09.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.706+0000 7fb4eeffd700 1 -- 172.21.15.57:0/2169857140 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f400cc80 con 0x7fb4f8100720 2023-10-08T13:54:09.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.706+0000 7fb4eeffd700 1 -- 172.21.15.57:0/2169857140 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40054e0 con 0x7fb4f8100720 2023-10-08T13:54:09.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.706+0000 7fb4ffe04700 1 -- 172.21.15.57:0/2169857140 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f81a59c0 con 0x7fb4f8100720 2023-10-08T13:54:09.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.709+0000 7fb4eeffd700 1 -- 172.21.15.57:0/2169857140 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb4f4004b90 con 0x7fb4f8100720 2023-10-08T13:54:09.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.709+0000 7fb4eeffd700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb4e40a15f0 0x7fb4e40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.709+0000 7fb4eeffd700 1 -- 172.21.15.57:0/2169857140 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb4f4014070 con 0x7fb4f8100720 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.709+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb4e40a15f0 0x7fb4e40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.710+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb4e40a15f0 0x7fb4e40a3ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80741d0 tx=0x7fb4e8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.710+0000 7fb50bfedb80 1 Processor -- start 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.710+0000 7fb50bfedb80 1 -- start start 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.710+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f500260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.710+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f5007a0 0x55a85f50b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f50b650 0x55a85f50da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f469460 con 0x55a85f5007a0 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f45ac20 con 0x55a85f50b650 2023-10-08T13:54:09.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f458540 con 0x55a85f4ffe80 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fd39f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f5007a0 0x55a85f50b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fdba0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f500260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fe3a1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f50b650 0x55a85f50da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fdba0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f500260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58788/0 (socket says 172.21.15.57:58788) 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fdba0700 1 -- 172.21.15.57:0/1459257573 learned_addr learned my addr 172.21.15.57:0/1459257573 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fd39f700 1 -- 172.21.15.57:0/1459257573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 msgr2=0x55a85f500260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/1459257573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f500260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fd39f700 1 -- 172.21.15.57:0/1459257573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f50b650 msgr2=0x55a85f50da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/1459257573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f50b650 0x55a85f50da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fd39f700 1 -- 172.21.15.57:0/1459257573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a85f27ede0 con 0x55a85f5007a0 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fe3a1700 1 --2- 172.21.15.57:0/1459257573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f50b650 0x55a85f50da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:09.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/1459257573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f500260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/1459257573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f5007a0 0x55a85f50b110 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80042f0 tx=0x7fb4e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bfacdf1ee1e3ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4e3fff700 1 -- 172.21.15.57:0/1459257573 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8003d10 con 0x55a85f5007a0 2023-10-08T13:54:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4e3fff700 1 -- 172.21.15.57:0/1459257573 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4e8003e70 con 0x55a85f5007a0 2023-10-08T13:54:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.711+0000 7fb4e3fff700 1 -- 172.21.15.57:0/1459257573 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e801e750 con 0x55a85f5007a0 2023-10-08T13:54:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.712+0000 7fb50bfedb80 1 -- 172.21.15.57:0/1459257573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f5007a0 msgr2=0x55a85f50b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.712+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/1459257573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f5007a0 0x55a85f50b110 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80042f0 tx=0x7fb4e8014000 comp rx=0 tx=0).stop 2023-10-08T13:54:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.712+0000 7fb50bfedb80 1 -- 172.21.15.57:0/1459257573 shutdown_connections 2023-10-08T13:54:09.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.712+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/1459257573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f50b650 0x55a85f50da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.712+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/1459257573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f500260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.712+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/1459257573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f5007a0 0x55a85f50b110 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.712+0000 7fb50bfedb80 1 -- 172.21.15.57:0/1459257573 >> 172.21.15.57:0/1459257573 conn(0x55a85f475260 msgr2=0x55a85f4fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:09.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.712+0000 7fb50bfedb80 1 -- 172.21.15.57:0/1459257573 shutdown_connections 2023-10-08T13:54:09.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.712+0000 7fb50bfedb80 1 -- 172.21.15.57:0/1459257573 wait complete. 2023-10-08T13:54:09.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb50bfedb80 1 -- start start 2023-10-08T13:54:09.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb50bfedb80 1 Processor -- start 2023-10-08T13:54:09.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f4ffe80 0x55a85f50b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f5007a0 0x55a85f5083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f508a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f453e30 con 0x55a85f50b650 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f3f0710 con 0x55a85f4ffe80 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f3f03b0 con 0x55a85f5007a0 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb50bfedb80 10 client.0 _ll_register_callbacks cb 0x55a85f24d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb4fe3a1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f508a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.713+0000 7fb4fdba0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f4ffe80 0x55a85f50b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fdba0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f4ffe80 0x55a85f50b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59408/0 (socket says 172.21.15.57:59408) 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fe3a1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f508a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57984/0 (socket says 172.21.15.57:57984) 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fdba0700 1 -- 172.21.15.57:0/216888500 learned_addr learned my addr 172.21.15.57:0/216888500 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f5007a0 0x55a85f5083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fd39f700 1 -- 172.21.15.57:0/216888500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f4ffe80 msgr2=0x55a85f50b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:09.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f4ffe80 0x55a85f50b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fd39f700 1 -- 172.21.15.57:0/216888500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 msgr2=0x55a85f508a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f508a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fd39f700 1 -- 172.21.15.57:0/216888500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f81a59c0 con 0x55a85f5007a0 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f4ffe80 0x55a85f50b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fe3a1700 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f508a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f5007a0 0x55a85f5083a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8003960 tx=0x7fb4e8026630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e37c97750388624d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4e17fa700 1 -- 172.21.15.57:0/216888500 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e802b070 con 0x55a85f5007a0 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4e17fa700 1 -- 172.21.15.57:0/216888500 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb4e8026a30 con 0x55a85f5007a0 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb50bfedb80 1 -- 172.21.15.57:0/216888500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a85f41beb0 con 0x55a85f5007a0 2023-10-08T13:54:09.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb4e17fa700 1 -- 172.21.15.57:0/216888500 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4e8005050 con 0x55a85f5007a0 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.714+0000 7fb50bfedb80 10 client.26122 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.715+0000 7fb50bfedb80 1 -- 172.21.15.57:0/216888500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fb4f810def0 con 0x55a85f5007a0 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.715+0000 7fb4e17fa700 1 -- 172.21.15.57:0/216888500 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a85f41beb0 con 0x55a85f5007a0 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.715+0000 7fb4e17fa700 1 -- 172.21.15.57:0/216888500 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb4e8024020 con 0x55a85f5007a0 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.715+0000 7fb50bfedb80 10 client.26122 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.715+0000 7fb50bfedb80 10 client.26122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.715+0000 7fb50bfedb80 20 client.26122 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.715+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a85f51b160 0x55a85f51d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.715+0000 7fb50bfedb80 4 client.26122 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.715+0000 7fb50bfedb80 1 -- 172.21.15.57:0/216888500 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a85f44ce10 con 0x55a85f51b160 2023-10-08T13:54:09.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.715+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a85f51b160 0x55a85f51d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.716+0000 7fb4e17fa700 10 client.26122 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:09.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.716+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a85f51b160 0x55a85f51d580 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54909bf071cf97a server_cookie=dd46248f5aff7371 in_seq=0 out_seq=0 2023-10-08T13:54:09.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb4e17fa700 1 -- 172.21.15.57:0/216888500 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a85f44ce10 con 0x55a85f51b160 2023-10-08T13:54:09.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb4e17fa700 10 client.26122 handle_command_reply: tid=0 2023-10-08T13:54:09.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 client.26122 shutdown 2023-10-08T13:54:09.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 -- 172.21.15.57:0/216888500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f5007a0 msgr2=0x55a85f5083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:09.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f5007a0 0x55a85f5083a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8003960 tx=0x7fb4e8026630 comp rx=0 tx=0).stop 2023-10-08T13:54:09.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 -- 172.21.15.57:0/216888500 shutdown_connections 2023-10-08T13:54:09.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a85f51b160 0x55a85f51d580 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f508a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb4e17fa700 0 client.26122 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f4ffe80 0x55a85f50b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/216888500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f5007a0 0x55a85f5083a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 -- 172.21.15.57:0/216888500 >> 172.21.15.57:0/216888500 conn(0x55a85f475260 msgr2=0x55a85f4fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 -- 172.21.15.57:0/216888500 shutdown_connections 2023-10-08T13:54:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 1 -- 172.21.15.57:0/216888500 wait complete. 2023-10-08T13:54:09.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.719+0000 7fb50bfedb80 20 client.26122 trim_cache size 0 max 16384 2023-10-08T13:54:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb50bfedb80 1 Processor -- start 2023-10-08T13:54:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb50bfedb80 1 -- start start 2023-10-08T13:54:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f508360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 0x55a85f5121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f50ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f2d4920 con 0x55a85f50b650 2023-10-08T13:54:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f4f8ae0 con 0x55a85f5007a0 2023-10-08T13:54:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f509810 con 0x55a85f4ffe80 2023-10-08T13:54:09.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fe3a1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f50ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fdba0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f508360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fd39f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 0x55a85f5121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fdba0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f508360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58800/0 (socket says 172.21.15.57:58800) 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fe3a1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f50ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57992/0 (socket says 172.21.15.57:57992) 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fd39f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 0x55a85f5121a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59416/0 (socket says 172.21.15.57:59416) 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fdba0700 1 -- 172.21.15.57:0/835941367 learned_addr learned my addr 172.21.15.57:0/835941367 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fdba0700 1 -- 172.21.15.57:0/835941367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 msgr2=0x55a85f5121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/835941367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 0x55a85f5121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fdba0700 1 -- 172.21.15.57:0/835941367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 msgr2=0x55a85f50ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/835941367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f50ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fdba0700 1 -- 172.21.15.57:0/835941367 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e8017070 con 0x55a85f4ffe80 2023-10-08T13:54:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.737+0000 7fb4fe3a1700 1 --2- 172.21.15.57:0/835941367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f50ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.738+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/835941367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f508360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400cfd0 tx=0x7fb4f4009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acabef37e51f8c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.738+0000 7fb4ecff9700 1 -- 172.21.15.57:0/835941367 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f409a840 con 0x55a85f4ffe80 2023-10-08T13:54:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.738+0000 7fb4ecff9700 1 -- 172.21.15.57:0/835941367 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f409a140 con 0x55a85f4ffe80 2023-10-08T13:54:09.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.738+0000 7fb4ecff9700 1 -- 172.21.15.57:0/835941367 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4096850 con 0x55a85f4ffe80 2023-10-08T13:54:09.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 -- 172.21.15.57:0/835941367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 msgr2=0x55a85f508360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:09.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/835941367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f508360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400cfd0 tx=0x7fb4f4009550 comp rx=0 tx=0).stop 2023-10-08T13:54:09.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 -- 172.21.15.57:0/835941367 shutdown_connections 2023-10-08T13:54:09.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/835941367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f50b650 0x55a85f50ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/835941367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f4ffe80 0x55a85f508360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/835941367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 0x55a85f5121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 -- 172.21.15.57:0/835941367 >> 172.21.15.57:0/835941367 conn(0x55a85f475260 msgr2=0x55a85f507c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 -- 172.21.15.57:0/835941367 shutdown_connections 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 -- 172.21.15.57:0/835941367 wait complete. 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 -- start start 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.739+0000 7fb50bfedb80 1 Processor -- start 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f4ffe80 0x55a85f511fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 0x55a85f510590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb50bfedb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f50b650 0x55a85f510ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f511010 con 0x55a85f4ffe80 2023-10-08T13:54:09.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f511180 con 0x55a85f5007a0 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb50bfedb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85f511340 con 0x55a85f50b650 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb50bfedb80 10 client.0 _ll_register_callbacks cb 0x7fb4e8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fdba0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f4ffe80 0x55a85f511fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fd39f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 0x55a85f510590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fdba0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f4ffe80 0x55a85f511fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57994/0 (socket says 172.21.15.57:57994) 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fdba0700 1 -- 172.21.15.57:0/153128679 learned_addr learned my addr 172.21.15.57:0/153128679 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fe3a1700 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f50b650 0x55a85f510ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fe3a1700 1 -- 172.21.15.57:0/153128679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 msgr2=0x55a85f510590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fe3a1700 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 0x55a85f510590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fe3a1700 1 -- 172.21.15.57:0/153128679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f4ffe80 msgr2=0x55a85f511fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:09.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fe3a1700 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f4ffe80 0x55a85f511fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fe3a1700 1 -- 172.21.15.57:0/153128679 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f0003b30 con 0x55a85f50b650 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f4ffe80 0x55a85f511fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fd39f700 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 0x55a85f510590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4fe3a1700 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f50b650 0x55a85f510ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009550 tx=0x7fb4f0007790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=151860aa92838813 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4e17fa700 1 -- 172.21.15.57:0/153128679 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000e040 con 0x55a85f50b650 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb50bfedb80 1 -- 172.21.15.57:0/153128679 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4e8003e70 con 0x55a85f50b650 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.740+0000 7fb4e17fa700 1 -- 172.21.15.57:0/153128679 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb4f0005450 con 0x55a85f50b650 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb4e17fa700 1 -- 172.21.15.57:0/153128679 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f0005610 con 0x55a85f50b650 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb50bfedb80 10 client.26128 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb50bfedb80 1 -- 172.21.15.57:0/153128679 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fb4f81a54b0 con 0x55a85f50b650 2023-10-08T13:54:09.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb4e17fa700 1 -- 172.21.15.57:0/153128679 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4e8003e70 con 0x55a85f50b650 2023-10-08T13:54:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb4e17fa700 1 -- 172.21.15.57:0/153128679 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb4f000b050 con 0x55a85f50b650 2023-10-08T13:54:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb50bfedb80 10 client.26128 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb50bfedb80 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb50bfedb80 20 client.26128 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a85f5704f0 0x55a85f572910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb50bfedb80 4 client.26128 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:54:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb50bfedb80 1 -- 172.21.15.57:0/153128679 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a85f44cb30 con 0x55a85f5704f0 2023-10-08T13:54:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.741+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a85f5704f0 0x55a85f572910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.742+0000 7fb4e17fa700 10 client.26128 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:09.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.742+0000 7fb4fdba0700 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a85f5704f0 0x55a85f572910 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e142da605315f8f2 server_cookie=ee62685fc3ded0c3 in_seq=0 out_seq=0 2023-10-08T13:54:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb4e17fa700 1 -- 172.21.15.57:0/153128679 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a85f44cb30 con 0x55a85f5704f0 2023-10-08T13:54:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb4e17fa700 10 client.26128 handle_command_reply: tid=0 2023-10-08T13:54:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb50bfedb80 1 client.26128 shutdown 2023-10-08T13:54:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb50bfedb80 1 -- 172.21.15.57:0/153128679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f50b650 msgr2=0x55a85f510ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f50b650 0x55a85f510ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009550 tx=0x7fb4f0007790 comp rx=0 tx=0).stop 2023-10-08T13:54:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb50bfedb80 1 -- 172.21.15.57:0/153128679 shutdown_connections 2023-10-08T13:54:09.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a85f5704f0 0x55a85f572910 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a85f50b650 0x55a85f510ad0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb4e17fa700 0 client.26128 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a85f4ffe80 0x55a85f511fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb50bfedb80 1 --2- 172.21.15.57:0/153128679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a85f5007a0 0x55a85f510590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.749+0000 7fb50bfedb80 1 -- 172.21.15.57:0/153128679 >> 172.21.15.57:0/153128679 conn(0x55a85f475260 msgr2=0x55a85f4fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb50bfedb80 1 -- 172.21.15.57:0/153128679 shutdown_connections 2023-10-08T13:54:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb50bfedb80 1 -- 172.21.15.57:0/153128679 wait complete. 2023-10-08T13:54:09.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb50bfedb80 20 client.26128 trim_cache size 0 max 16384 2023-10-08T13:54:09.752 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:54:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-08T13:54:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:54:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "6db03f06-e888-4427-a580-5fa3643c8e0d": { 2023-10-08T13:54:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:54:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "6db03f06-e888-4427-a580-5fa3643c8e0d", 2023-10-08T13:54:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:54:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:54:09.753 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:54:09.753 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:54:09.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 -- 172.21.15.57:0/2169857140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb4e40a15f0 msgr2=0x7fb4e40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:09.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb4e40a15f0 0x7fb4e40a3ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80741d0 tx=0x7fb4e8009260 comp rx=0 tx=0).stop 2023-10-08T13:54:09.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 -- 172.21.15.57:0/2169857140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f8100720 msgr2=0x7fb4f8072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:09.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f8100720 0x7fb4f8072af0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400e9c0 tx=0x7fb4f400ecd0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 -- 172.21.15.57:0/2169857140 shutdown_connections 2023-10-08T13:54:09.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb4e40a15f0 0x7fb4e40a3ab0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f8106730 0x7fb4f8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb4f8100720 0x7fb4f8072af0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 --2- 172.21.15.57:0/2169857140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f81010e0 0x7fb4f8073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:09.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 -- 172.21.15.57:0/2169857140 >> 172.21.15.57:0/2169857140 conn(0x7fb4f80fc230 msgr2=0x7fb4f80fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:09.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.750+0000 7fb4ffe04700 1 -- 172.21.15.57:0/2169857140 shutdown_connections 2023-10-08T13:54:09.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:09.751+0000 7fb4ffe04700 1 -- 172.21.15.57:0/2169857140 wait complete. 2023-10-08T13:54:09.765 INFO:tasks.cephfs.filesystem:scrub status for tag:6db03f06-e888-4427-a580-5fa3643c8e0d - {'path': '/', 'tag': '6db03f06-e888-4427-a580-5fa3643c8e0d', 'options': 'recursive,force'} 2023-10-08T13:54:10.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:10 smithi099 ceph-mon[103849]: pgmap v377: 129 pgs: 129 active+clean; 371 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.4 MiB/s wr, 340 op/s 2023-10-08T13:54:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:10 smithi115 ceph-mon[104408]: pgmap v377: 129 pgs: 129 active+clean; 371 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.4 MiB/s wr, 340 op/s 2023-10-08T13:54:10.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:10 smithi057 ceph-mon[99476]: pgmap v377: 129 pgs: 129 active+clean; 371 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.4 MiB/s wr, 340 op/s 2023-10-08T13:54:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:12 smithi099 ceph-mon[103849]: pgmap v378: 129 pgs: 129 active+clean; 378 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 345 op/s 2023-10-08T13:54:12.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:12 smithi115 ceph-mon[104408]: pgmap v378: 129 pgs: 129 active+clean; 378 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 345 op/s 2023-10-08T13:54:12.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:12 smithi057 ceph-mon[99476]: pgmap v378: 129 pgs: 129 active+clean; 378 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 345 op/s 2023-10-08T13:54:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:14 smithi099 ceph-mon[103849]: pgmap v379: 129 pgs: 129 active+clean; 384 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 350 op/s 2023-10-08T13:54:14.365 INFO:tasks.workunit.client.0.smithi057.stdout: 168 253226 1937 179826 2460 27677 49 2023-10-08T13:54:14.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:14 smithi115 ceph-mon[104408]: pgmap v379: 129 pgs: 129 active+clean; 384 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 350 op/s 2023-10-08T13:54:14.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:14 smithi057 ceph-mon[99476]: pgmap v379: 129 pgs: 129 active+clean; 384 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 350 op/s 2023-10-08T13:54:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.690+0000 7f8cee64f700 1 -- 172.21.15.57:0/4221609041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ce810d000 msgr2=0x7f8ce810d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.690+0000 7f8cee64f700 1 --2- 172.21.15.57:0/4221609041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ce810d000 0x7f8ce810d9e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000b0a0 tx=0x7f8ce000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.707+0000 7f8cee64f700 1 -- 172.21.15.57:0/4221609041 shutdown_connections 2023-10-08T13:54:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.707+0000 7f8cee64f700 1 --2- 172.21.15.57:0/4221609041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ce810d000 0x7f8ce810d9e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.707+0000 7f8cee64f700 1 --2- 172.21.15.57:0/4221609041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce8107a60 0x7f8ce8107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.707+0000 7f8cee64f700 1 --2- 172.21.15.57:0/4221609041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ce8108410 0x7f8ce810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.707+0000 7f8cee64f700 1 -- 172.21.15.57:0/4221609041 >> 172.21.15.57:0/4221609041 conn(0x7f8ce806b600 msgr2=0x7f8ce806ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.711+0000 7f8cee64f700 1 -- 172.21.15.57:0/4221609041 shutdown_connections 2023-10-08T13:54:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.712+0000 7f8cee64f700 1 -- 172.21.15.57:0/4221609041 wait complete. 2023-10-08T13:54:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.713+0000 7f8cee64f700 1 Processor -- start 2023-10-08T13:54:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.714+0000 7f8cee64f700 1 -- start start 2023-10-08T13:54:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.714+0000 7f8cee64f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ce8107a60 0x7f8ce81ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.714+0000 7f8cee64f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce810d000 0x7f8ce81abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.714+0000 7f8cee64f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ce81a57d0 0x7f8ce81a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.714+0000 7f8cee64f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce81a6260 con 0x7f8ce8107a60 2023-10-08T13:54:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.714+0000 7f8cee64f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce81a63d0 con 0x7f8ce810d000 2023-10-08T13:54:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.714+0000 7f8cee64f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce81a6590 con 0x7f8ce81a57d0 2023-10-08T13:54:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.717+0000 7f8ced64d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ce8107a60 0x7f8ce81ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.717+0000 7f8ced64d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ce8107a60 0x7f8ce81ab630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58010/0 (socket says 172.21.15.57:58010) 2023-10-08T13:54:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.717+0000 7f8ced64d700 1 -- 172.21.15.57:0/680823149 learned_addr learned my addr 172.21.15.57:0/680823149 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.725+0000 7f8cece4c700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce810d000 0x7f8ce81abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.725+0000 7f8cede4e700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ce81a57d0 0x7f8ce81a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.729+0000 7f8cece4c700 1 -- 172.21.15.57:0/680823149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ce81a57d0 msgr2=0x7f8ce81a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.729+0000 7f8cece4c700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ce81a57d0 0x7f8ce81a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.729+0000 7f8cece4c700 1 -- 172.21.15.57:0/680823149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ce8107a60 msgr2=0x7f8ce81ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.729+0000 7f8cece4c700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ce8107a60 0x7f8ce81ab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.729+0000 7f8cece4c700 1 -- 172.21.15.57:0/680823149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce0009d00 con 0x7f8ce810d000 2023-10-08T13:54:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.743+0000 7f8cece4c700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce810d000 0x7f8ce81abb70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ce806c3f0 tx=0x7f8cd800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:15.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.754+0000 7f8cde7fc700 1 -- 172.21.15.57:0/680823149 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800cbd0 con 0x7f8ce810d000 2023-10-08T13:54:15.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.754+0000 7f8cde7fc700 1 -- 172.21.15.57:0/680823149 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cd800ee80 con 0x7f8ce810d000 2023-10-08T13:54:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.754+0000 7f8cde7fc700 1 -- 172.21.15.57:0/680823149 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8018680 con 0x7f8ce810d000 2023-10-08T13:54:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.754+0000 7f8cee64f700 1 -- 172.21.15.57:0/680823149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce810ef80 con 0x7f8ce810d000 2023-10-08T13:54:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.754+0000 7f8cee64f700 1 -- 172.21.15.57:0/680823149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce810f440 con 0x7f8ce810d000 2023-10-08T13:54:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.765+0000 7f8cfba9ab80 1 Processor -- start 2023-10-08T13:54:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.765+0000 7f8cfba9ab80 1 -- start start 2023-10-08T13:54:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.765+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 0x55f75a121f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.765+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a11bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.765+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a11c300 0x55f75a11c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.765+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a084400 con 0x55f75a121ba0 2023-10-08T13:54:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.765+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a075bc0 con 0x55f75a11c300 2023-10-08T13:54:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.765+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a0734e0 con 0x55f75a1224c0 2023-10-08T13:54:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.766+0000 7f8cede4e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 0x55f75a121f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.766+0000 7f8cede4e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 0x55f75a121f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58016/0 (socket says 172.21.15.57:58016) 2023-10-08T13:54:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.766+0000 7f8cede4e700 1 -- 172.21.15.57:0/703290088 learned_addr learned my addr 172.21.15.57:0/703290088 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.766+0000 7f8ced64d700 1 --2- 172.21.15.57:0/703290088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a11bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.767+0000 7f8ced64d700 1 -- 172.21.15.57:0/703290088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a11c300 msgr2=0x55f75a11c740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:54:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.767+0000 7f8ced64d700 1 --2- 172.21.15.57:0/703290088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a11c300 0x55f75a11c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.767+0000 7f8ced64d700 1 -- 172.21.15.57:0/703290088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 msgr2=0x55f75a121f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.767+0000 7f8ced64d700 1 --2- 172.21.15.57:0/703290088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 0x55f75a121f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.767+0000 7f8ced64d700 1 -- 172.21.15.57:0/703290088 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f759e99d80 con 0x55f75a1224c0 2023-10-08T13:54:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.770+0000 7f8cde7fc700 1 -- 172.21.15.57:0/680823149 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8cd800cd30 con 0x7f8ce810d000 2023-10-08T13:54:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.771+0000 7f8cde7fc700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8cd40a1800 0x7f8cd40a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.772+0000 7f8cde7fc700 1 -- 172.21.15.57:0/680823149 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8cd8014070 con 0x7f8ce810d000 2023-10-08T13:54:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.776+0000 7f8cede4e700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8cd40a1800 0x7f8cd40a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.784+0000 7f8ced64d700 1 --2- 172.21.15.57:0/703290088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a11bdc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4003aa0 tx=0x7f8ce4003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf5299d092cd07e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.811+0000 7f8cede4e700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8cd40a1800 0x7f8cd40a3cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0007c10 tx=0x7f8ce000b740 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.811+0000 7f8cd37fe700 1 -- 172.21.15.57:0/703290088 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400fb10 con 0x55f75a1224c0 2023-10-08T13:54:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.811+0000 7f8cd37fe700 1 -- 172.21.15.57:0/703290088 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ce400fc70 con 0x55f75a1224c0 2023-10-08T13:54:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.811+0000 7f8cd37fe700 1 -- 172.21.15.57:0/703290088 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4019660 con 0x55f75a1224c0 2023-10-08T13:54:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.812+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/703290088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 msgr2=0x55f75a11bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.812+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/703290088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a11bdc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4003aa0 tx=0x7f8ce4003db0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.813+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/703290088 shutdown_connections 2023-10-08T13:54:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.813+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/703290088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a11c300 0x55f75a11c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.813+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/703290088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 0x55f75a121f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.813+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/703290088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a11bdc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.813+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/703290088 >> 172.21.15.57:0/703290088 conn(0x55f75a090200 msgr2=0x55f75a116c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.813+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/703290088 shutdown_connections 2023-10-08T13:54:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.813+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/703290088 wait complete. 2023-10-08T13:54:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.814+0000 7f8cfba9ab80 1 -- start start 2023-10-08T13:54:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.814+0000 7f8cfba9ab80 1 Processor -- start 2023-10-08T13:54:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.815+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a11c300 0x55f75a1283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.815+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 0x55f75a1256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.815+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a125e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.815+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a06edd0 con 0x55f75a11c300 2023-10-08T13:54:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.815+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a00b6b0 con 0x55f75a121ba0 2023-10-08T13:54:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.815+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a00b350 con 0x55f75a1224c0 2023-10-08T13:54:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.815+0000 7f8cfba9ab80 10 client.0 _ll_register_callbacks cb 0x55f759e687a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.815+0000 7f8cece4c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 0x55f75a1256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.816+0000 7f8cece4c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 0x55f75a1256c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59448/0 (socket says 172.21.15.57:59448) 2023-10-08T13:54:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.816+0000 7f8cece4c700 1 -- 172.21.15.57:0/1587775047 learned_addr learned my addr 172.21.15.57:0/1587775047 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.822+0000 7f8cece4c700 1 -- 172.21.15.57:0/1587775047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 msgr2=0x55f75a125e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:54:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.822+0000 7f8cece4c700 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a125e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.822+0000 7f8cece4c700 1 -- 172.21.15.57:0/1587775047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a11c300 msgr2=0x55f75a1283d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.822+0000 7f8cece4c700 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a11c300 0x55f75a1283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.822+0000 7f8cece4c700 1 -- 172.21.15.57:0/1587775047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce4003700 con 0x55f75a121ba0 2023-10-08T13:54:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.829+0000 7f8cece4c700 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 0x55f75a1256c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8099e70 tx=0x7f8cd809a220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5a677f9918611ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.833+0000 7f8cd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:54:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.833+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/1587775047 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd809a5d0 con 0x55f75a121ba0 2023-10-08T13:54:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.833+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/1587775047 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8cd8099660 con 0x55f75a121ba0 2023-10-08T13:54:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.833+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/1587775047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f75a036e50 con 0x55f75a121ba0 2023-10-08T13:54:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.836+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/1587775047 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cd8099820 con 0x55f75a121ba0 2023-10-08T13:54:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.836+0000 7f8cfba9ab80 10 client.26081 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.836+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/1587775047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f759e82c00 con 0x55f75a121ba0 2023-10-08T13:54:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.849+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/1587775047 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f75a036e50 con 0x55f75a121ba0 2023-10-08T13:54:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.849+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/1587775047 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f8cd80a2190 con 0x55f75a121ba0 2023-10-08T13:54:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.850+0000 7f8cfba9ab80 10 client.26081 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.850+0000 7f8cfba9ab80 10 client.26081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.850+0000 7f8cfba9ab80 20 client.26081 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.850+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f75a135f60 0x55f75a138380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.850+0000 7f8cfba9ab80 4 client.26081 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:54:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.850+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/1587775047 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f75a067db0 con 0x55f75a135f60 2023-10-08T13:54:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.852+0000 7f8ced64d700 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f75a135f60 0x55f75a138380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.873+0000 7f8ced64d700 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f75a135f60 0x55f75a138380 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3caa30dea06b5c8 server_cookie=e3e13a7ca00c595f in_seq=0 out_seq=0 2023-10-08T13:54:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.873+0000 7f8cd0ff9700 10 client.26081 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.884+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/1587775047 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f75a067db0 con 0x55f75a135f60 2023-10-08T13:54:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.884+0000 7f8cd0ff9700 10 client.26081 handle_command_reply: tid=0 2023-10-08T13:54:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.885+0000 7f8cfba9ab80 1 client.26081 shutdown 2023-10-08T13:54:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.888+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/1587775047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 msgr2=0x55f75a1256c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.888+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 0x55f75a1256c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8099e70 tx=0x7f8cd809a220 comp rx=0 tx=0).stop 2023-10-08T13:54:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.896+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/1587775047 shutdown_connections 2023-10-08T13:54:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.896+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f75a135f60 0x55f75a138380 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.896+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a125e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.896+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a11c300 0x55f75a1283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.896+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/1587775047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 0x55f75a1256c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.896+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/1587775047 >> 172.21.15.57:0/1587775047 conn(0x55f75a090200 msgr2=0x55f75a116260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.896+0000 7f8cd0ff9700 0 client.26081 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.903+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/1587775047 shutdown_connections 2023-10-08T13:54:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.903+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/1587775047 wait complete. 2023-10-08T13:54:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.904+0000 7f8cfba9ab80 20 client.26081 trim_cache size 0 max 16384 2023-10-08T13:54:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.969+0000 7f8cfba9ab80 1 Processor -- start 2023-10-08T13:54:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.971+0000 7f8cfba9ab80 1 -- start start 2023-10-08T13:54:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.971+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a11c300 0x55f75a120e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.971+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 0x55f75a1169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.971+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a127f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.971+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f759eef8c0 con 0x55f75a121ba0 2023-10-08T13:54:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.971+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a120150 con 0x55f75a11c300 2023-10-08T13:54:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.972+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a117b10 con 0x55f75a1224c0 2023-10-08T13:54:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.972+0000 7f8ced64d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a11c300 0x55f75a120e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.972+0000 7f8ced64d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a11c300 0x55f75a120e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59456/0 (socket says 172.21.15.57:59456) 2023-10-08T13:54:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.972+0000 7f8ced64d700 1 -- 172.21.15.57:0/765862835 learned_addr learned my addr 172.21.15.57:0/765862835 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.972+0000 7f8cede4e700 1 --2- 172.21.15.57:0/765862835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a127f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.973+0000 7f8cece4c700 1 --2- 172.21.15.57:0/765862835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 0x55f75a1169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.973+0000 7f8cede4e700 1 -- 172.21.15.57:0/765862835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a11c300 msgr2=0x55f75a120e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.973+0000 7f8cede4e700 1 --2- 172.21.15.57:0/765862835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a11c300 0x55f75a120e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.973+0000 7f8cede4e700 1 -- 172.21.15.57:0/765862835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 msgr2=0x55f75a1169e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.973+0000 7f8cede4e700 1 --2- 172.21.15.57:0/765862835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 0x55f75a1169e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.973+0000 7f8cede4e700 1 -- 172.21.15.57:0/765862835 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f759ec01f0 con 0x55f75a1224c0 2023-10-08T13:54:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.974+0000 7f8cede4e700 1 --2- 172.21.15.57:0/765862835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a127f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0007cf0 tx=0x7f8ce003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bda468c5f3400fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.989+0000 7f8cd3fff700 1 -- 172.21.15.57:0/765862835 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000e040 con 0x55f75a1224c0 2023-10-08T13:54:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.989+0000 7f8cd3fff700 1 -- 172.21.15.57:0/765862835 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ce0003d10 con 0x55f75a1224c0 2023-10-08T13:54:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.989+0000 7f8cd3fff700 1 -- 172.21.15.57:0/765862835 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0005970 con 0x55f75a1224c0 2023-10-08T13:54:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.990+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/765862835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 msgr2=0x55f75a127f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.990+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/765862835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a127f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0007cf0 tx=0x7f8ce003f000 comp rx=0 tx=0).stop 2023-10-08T13:54:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.994+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/765862835 shutdown_connections 2023-10-08T13:54:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.994+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/765862835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a1224c0 0x55f75a127f00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.994+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/765862835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a11c300 0x55f75a120e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.994+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/765862835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a121ba0 0x55f75a1169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:15.994+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/765862835 >> 172.21.15.57:0/765862835 conn(0x55f75a090200 msgr2=0x55f75a120780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.008+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/765862835 shutdown_connections 2023-10-08T13:54:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.008+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/765862835 wait complete. 2023-10-08T13:54:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.009+0000 7f8cfba9ab80 1 -- start start 2023-10-08T13:54:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.009+0000 7f8cfba9ab80 1 Processor -- start 2023-10-08T13:54:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.009+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a11c300 0x55f75a12c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.009+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 0x55f75a12cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.010+0000 7f8cfba9ab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a1224c0 0x55f75a12ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.010+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a12b3e0 con 0x55f75a1224c0 2023-10-08T13:54:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.010+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a12b550 con 0x55f75a121ba0 2023-10-08T13:54:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.010+0000 7f8cfba9ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a12b710 con 0x55f75a11c300 2023-10-08T13:54:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.010+0000 7f8cfba9ab80 10 client.0 _ll_register_callbacks cb 0x7f8cd8099bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.012+0000 7f8cede4e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a1224c0 0x55f75a12ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.013+0000 7f8cede4e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a1224c0 0x55f75a12ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58040/0 (socket says 172.21.15.57:58040) 2023-10-08T13:54:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.013+0000 7f8cede4e700 1 -- 172.21.15.57:0/2951732131 learned_addr learned my addr 172.21.15.57:0/2951732131 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.016+0000 7f8cece4c700 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 0x55f75a12cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.016+0000 7f8ced64d700 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a11c300 0x55f75a12c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.017+0000 7f8ced64d700 1 -- 172.21.15.57:0/2951732131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 msgr2=0x55f75a12cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.017+0000 7f8ced64d700 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 0x55f75a12cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.017+0000 7f8ced64d700 1 -- 172.21.15.57:0/2951732131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a1224c0 msgr2=0x55f75a12ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.017+0000 7f8ced64d700 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a1224c0 0x55f75a12ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.017+0000 7f8ced64d700 1 -- 172.21.15.57:0/2951732131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce810e2a0 con 0x55f75a11c300 2023-10-08T13:54:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.017+0000 7f8cede4e700 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a1224c0 0x55f75a12ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.018+0000 7f8ced64d700 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a11c300 0x55f75a12c760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4011d50 tx=0x7f8ce4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ee0b16ce7bf786d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.018+0000 7f8cd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:54:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.018+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/2951732131 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4027e80 con 0x55f75a11c300 2023-10-08T13:54:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.018+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/2951732131 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8ce40270b0 con 0x55f75a11c300 2023-10-08T13:54:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.018+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/2951732131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ce400fc70 con 0x55f75a11c300 2023-10-08T13:54:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.018+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/2951732131 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ce400fad0 con 0x55f75a11c300 2023-10-08T13:54:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.018+0000 7f8cfba9ab80 10 client.26143 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.018+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/2951732131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f759e98250 con 0x55f75a11c300 2023-10-08T13:54:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.019+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/2951732131 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ce400fc70 con 0x55f75a11c300 2023-10-08T13:54:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.019+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/2951732131 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f8ce4017020 con 0x55f75a11c300 2023-10-08T13:54:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.019+0000 7f8cfba9ab80 10 client.26143 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.019+0000 7f8cfba9ab80 10 client.26143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.019+0000 7f8cfba9ab80 20 client.26143 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.019+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f75a18afa0 0x55f75a18d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.019+0000 7f8cfba9ab80 4 client.26143 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:54:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.019+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/2951732131 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f75a067ad0 con 0x55f75a18afa0 2023-10-08T13:54:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.019+0000 7f8cece4c700 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f75a18afa0 0x55f75a18d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.019+0000 7f8cd0ff9700 10 client.26143 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:16.020+0000 7f8cece4c700 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f75a18afa0 0x55f75a18d3c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a71e63aedbd1ad5 server_cookie=ed7ccdfe83808cce in_seq=0 out_seq=0 2023-10-08T13:54:16.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:16 smithi115 ceph-mon[104408]: pgmap v380: 129 pgs: 129 active+clean; 443 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.2 MiB/s wr, 510 op/s 2023-10-08T13:54:16.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:16 smithi057 ceph-mon[99476]: pgmap v380: 129 pgs: 129 active+clean; 443 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.2 MiB/s wr, 510 op/s 2023-10-08T13:54:16.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:16 smithi099 ceph-mon[103849]: pgmap v380: 129 pgs: 129 active+clean; 443 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.2 MiB/s wr, 510 op/s 2023-10-08T13:54:17.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:17 smithi115 ceph-mon[104408]: pgmap v381: 129 pgs: 129 active+clean; 484 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 10 MiB/s wr, 486 op/s 2023-10-08T13:54:17.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:17 smithi057 ceph-mon[99476]: pgmap v381: 129 pgs: 129 active+clean; 484 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 10 MiB/s wr, 486 op/s 2023-10-08T13:54:17.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:17 smithi099 ceph-mon[103849]: pgmap v381: 129 pgs: 129 active+clean; 484 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 10 MiB/s wr, 486 op/s 2023-10-08T13:54:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.938+0000 7f8cd0ff9700 1 -- 172.21.15.57:0/2951732131 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f75a067ad0 con 0x55f75a18afa0 2023-10-08T13:54:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.938+0000 7f8cd0ff9700 10 client.26143 handle_command_reply: tid=0 2023-10-08T13:54:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.938+0000 7f8cfba9ab80 1 client.26143 shutdown 2023-10-08T13:54:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.948+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/2951732131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a11c300 msgr2=0x55f75a12c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.948+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a11c300 0x55f75a12c760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4011d50 tx=0x7f8ce4000bc0 comp rx=0 tx=0).stop 2023-10-08T13:54:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.957+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/2951732131 shutdown_connections 2023-10-08T13:54:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.957+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f75a18afa0 0x55f75a18d3c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.957+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f75a1224c0 0x55f75a12ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.957+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f75a11c300 0x55f75a12c760 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.957+0000 7f8cfba9ab80 1 --2- 172.21.15.57:0/2951732131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f75a121ba0 0x55f75a12cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.957+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/2951732131 >> 172.21.15.57:0/2951732131 conn(0x55f75a090200 msgr2=0x55f75a121010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.957+0000 7f8cd0ff9700 0 client.26143 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:18.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.959+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/2951732131 shutdown_connections 2023-10-08T13:54:18.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.959+0000 7f8cfba9ab80 1 -- 172.21.15.57:0/2951732131 wait complete. 2023-10-08T13:54:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.960+0000 7f8cfba9ab80 20 client.26143 trim_cache size 0 max 16384 2023-10-08T13:54:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:54:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:54:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:54:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:54:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.983+0000 7f8cee64f700 1 -- 172.21.15.57:0/680823149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8cd40a1800 msgr2=0x7f8cd40a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.983+0000 7f8cee64f700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8cd40a1800 0x7f8cd40a3cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0007c10 tx=0x7f8ce000b740 comp rx=0 tx=0).stop 2023-10-08T13:54:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.983+0000 7f8cee64f700 1 -- 172.21.15.57:0/680823149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce810d000 msgr2=0x7f8ce81abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.983+0000 7f8cee64f700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce810d000 0x7f8ce81abb70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ce806c3f0 tx=0x7f8cd800eca0 comp rx=0 tx=0).stop 2023-10-08T13:54:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.991+0000 7f8cee64f700 1 -- 172.21.15.57:0/680823149 shutdown_connections 2023-10-08T13:54:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.991+0000 7f8cee64f700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ce81a57d0 0x7f8ce81a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.991+0000 7f8cee64f700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ce8107a60 0x7f8ce81ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.991+0000 7f8cee64f700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8cd40a1800 0x7f8cd40a3cc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.991+0000 7f8cee64f700 1 --2- 172.21.15.57:0/680823149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce810d000 0x7f8ce81abb70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.991+0000 7f8cee64f700 1 -- 172.21.15.57:0/680823149 >> 172.21.15.57:0/680823149 conn(0x7f8ce806b600 msgr2=0x7f8ce8117730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.993+0000 7f8cee64f700 1 -- 172.21.15.57:0/680823149 shutdown_connections 2023-10-08T13:54:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:18.993+0000 7f8cee64f700 1 -- 172.21.15.57:0/680823149 wait complete. 2023-10-08T13:54:20.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:19 smithi115 ceph-mon[104408]: pgmap v382: 129 pgs: 129 active+clean; 499 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 463 op/s 2023-10-08T13:54:20.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:19 smithi057 ceph-mon[99476]: pgmap v382: 129 pgs: 129 active+clean; 499 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 463 op/s 2023-10-08T13:54:20.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:19 smithi099 ceph-mon[103849]: pgmap v382: 129 pgs: 129 active+clean; 499 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 463 op/s 2023-10-08T13:54:22.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:21 smithi115 ceph-mon[104408]: pgmap v383: 129 pgs: 129 active+clean; 569 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 17 MiB/s wr, 814 op/s 2023-10-08T13:54:22.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:21 smithi057 ceph-mon[99476]: pgmap v383: 129 pgs: 129 active+clean; 569 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 17 MiB/s wr, 814 op/s 2023-10-08T13:54:22.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:21 smithi099 ceph-mon[103849]: pgmap v383: 129 pgs: 129 active+clean; 569 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 17 MiB/s wr, 814 op/s 2023-10-08T13:54:24.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:23 smithi057 ceph-mon[99476]: pgmap v384: 129 pgs: 129 active+clean; 594 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 19 MiB/s wr, 934 op/s 2023-10-08T13:54:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:23 smithi099 ceph-mon[103849]: pgmap v384: 129 pgs: 129 active+clean; 594 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 19 MiB/s wr, 934 op/s 2023-10-08T13:54:24.365 INFO:tasks.workunit.client.0.smithi057.stdout: 185 1846853 948 1329455 936 208662 446 2023-10-08T13:54:24.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:23 smithi115 ceph-mon[104408]: pgmap v384: 129 pgs: 129 active+clean; 594 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 19 MiB/s wr, 934 op/s 2023-10-08T13:54:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:26 smithi099 ceph-mon[103849]: pgmap v385: 129 pgs: 129 active+clean; 611 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 20 MiB/s wr, 1.01k op/s 2023-10-08T13:54:26.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:26 smithi115 ceph-mon[104408]: pgmap v385: 129 pgs: 129 active+clean; 611 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 20 MiB/s wr, 1.01k op/s 2023-10-08T13:54:26.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:26 smithi057 ceph-mon[99476]: pgmap v385: 129 pgs: 129 active+clean; 611 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 20 MiB/s wr, 1.01k op/s 2023-10-08T13:54:28.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:28 smithi115 ceph-mon[104408]: pgmap v386: 129 pgs: 129 active+clean; 617 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 16 MiB/s wr, 849 op/s 2023-10-08T13:54:28.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:28 smithi057 ceph-mon[99476]: pgmap v386: 129 pgs: 129 active+clean; 617 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 16 MiB/s wr, 849 op/s 2023-10-08T13:54:28.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:28 smithi099 ceph-mon[103849]: pgmap v386: 129 pgs: 129 active+clean; 617 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 16 MiB/s wr, 849 op/s 2023-10-08T13:54:30.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:30 smithi115 ceph-mon[104408]: pgmap v387: 129 pgs: 129 active+clean; 618 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 726 op/s 2023-10-08T13:54:30.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:30 smithi057 ceph-mon[99476]: pgmap v387: 129 pgs: 129 active+clean; 618 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 726 op/s 2023-10-08T13:54:30.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:30 smithi099 ceph-mon[103849]: pgmap v387: 129 pgs: 129 active+clean; 618 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 726 op/s 2023-10-08T13:54:31.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:31 smithi115 ceph-mon[104408]: pgmap v388: 129 pgs: 129 active+clean; 622 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 693 op/s 2023-10-08T13:54:31.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:31 smithi057 ceph-mon[99476]: pgmap v388: 129 pgs: 129 active+clean; 622 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 693 op/s 2023-10-08T13:54:31.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:31 smithi099 ceph-mon[103849]: pgmap v388: 129 pgs: 129 active+clean; 622 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 693 op/s 2023-10-08T13:54:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.372+0000 7f11e8236700 1 -- 172.21.15.57:0/2142425547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e00ffbe0 msgr2=0x7f11e0103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.372+0000 7f11e8236700 1 --2- 172.21.15.57:0/2142425547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e00ffbe0 0x7f11e0103990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009990 tx=0x7f11dc009ca0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.372+0000 7f11e8236700 1 -- 172.21.15.57:0/2142425547 shutdown_connections 2023-10-08T13:54:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.372+0000 7f11e8236700 1 --2- 172.21.15.57:0/2142425547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e01040c0 0x7f11e010bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.372+0000 7f11e8236700 1 --2- 172.21.15.57:0/2142425547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e00ff230 0x7f11e00ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.372+0000 7f11e8236700 1 --2- 172.21.15.57:0/2142425547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e00ffbe0 0x7f11e0103990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.373+0000 7f11e8236700 1 -- 172.21.15.57:0/2142425547 >> 172.21.15.57:0/2142425547 conn(0x7f11e0075bc0 msgr2=0x7f11e0075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.373+0000 7f11e8236700 1 -- 172.21.15.57:0/2142425547 shutdown_connections 2023-10-08T13:54:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.373+0000 7f11e8236700 1 -- 172.21.15.57:0/2142425547 wait complete. 2023-10-08T13:54:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e8236700 1 Processor -- start 2023-10-08T13:54:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e8236700 1 -- start start 2023-10-08T13:54:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e8236700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e00ff230 0x7f11e010d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e8236700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e01040c0 0x7f11e010d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e8236700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e010de20 0x7f11e01a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e8236700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01a5400 con 0x7f11e00ff230 2023-10-08T13:54:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e8236700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01a5570 con 0x7f11e010de20 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e8236700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01a5730 con 0x7f11e01040c0 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e5fd2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e00ff230 0x7f11e010d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e5fd2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e00ff230 0x7f11e010d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58052/0 (socket says 172.21.15.57:58052) 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e57d1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e01040c0 0x7f11e010d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e5fd2700 1 -- 172.21.15.57:0/335607345 learned_addr learned my addr 172.21.15.57:0/335607345 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.374+0000 7f11e67d3700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e010de20 0x7f11e01a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.375+0000 7f11e57d1700 1 -- 172.21.15.57:0/335607345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e010de20 msgr2=0x7f11e01a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.375+0000 7f11e57d1700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e010de20 0x7f11e01a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.375+0000 7f11e57d1700 1 -- 172.21.15.57:0/335607345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e00ff230 msgr2=0x7f11e010d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.375+0000 7f11e57d1700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e00ff230 0x7f11e010d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.375+0000 7f11e57d1700 1 -- 172.21.15.57:0/335607345 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc0095f0 con 0x7f11e01040c0 2023-10-08T13:54:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.375+0000 7f11e5fd2700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e00ff230 0x7f11e010d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.379+0000 7f11e57d1700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e01040c0 0x7f11e010d8e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f11dc006010 tx=0x7f11dc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.379+0000 7f11d6ffd700 1 -- 172.21.15.57:0/335607345 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc01d070 con 0x7f11e01040c0 2023-10-08T13:54:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.379+0000 7f11d6ffd700 1 -- 172.21.15.57:0/335607345 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11dc021410 con 0x7f11e01040c0 2023-10-08T13:54:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.379+0000 7f11e8236700 1 -- 172.21.15.57:0/335607345 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e01969b0 con 0x7f11e01040c0 2023-10-08T13:54:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.379+0000 7f11d6ffd700 1 -- 172.21.15.57:0/335607345 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc021ea0 con 0x7f11e01040c0 2023-10-08T13:54:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.379+0000 7f11e8236700 1 -- 172.21.15.57:0/335607345 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e0196e30 con 0x7f11e01040c0 2023-10-08T13:54:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.383+0000 7f11d6ffd700 1 -- 172.21.15.57:0/335607345 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f11dc017650 con 0x7f11e01040c0 2023-10-08T13:54:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.383+0000 7f11f441fb80 1 Processor -- start 2023-10-08T13:54:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.384+0000 7f11f441fb80 1 -- start start 2023-10-08T13:54:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11d6ffd700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a15f0 0x7f11cc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf8174bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174c1a0 0x55bf81745b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf81746510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11f441fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf816ae400 con 0x55bf817460b0 2023-10-08T13:54:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11f441fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf8169fbc0 con 0x55bf8174c1a0 2023-10-08T13:54:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11f441fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf8169d4e0 con 0x55bf8174b880 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11d6ffd700 1 -- 172.21.15.57:0/335607345 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f11dc0ce570 con 0x7f11e01040c0 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11e57d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174c1a0 0x55bf81745b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11e67d3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf81746510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11e5fd2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf8174bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11e5fd2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf8174bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58872/0 (socket says 172.21.15.57:58872) 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11e67d3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf81746510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58062/0 (socket says 172.21.15.57:58062) 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11e67d3700 1 -- 172.21.15.57:0/1378845147 learned_addr learned my addr 172.21.15.57:0/1378845147 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11e57d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174c1a0 0x55bf81745b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59490/0 (socket says 172.21.15.57:59490) 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.385+0000 7f11e67d3700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a15f0 0x7f11cc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11e57d1700 1 -- 172.21.15.57:0/1378845147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 msgr2=0x55bf8174bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11e57d1700 1 --2- 172.21.15.57:0/1378845147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf8174bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11e57d1700 1 -- 172.21.15.57:0/1378845147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 msgr2=0x55bf81746510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11e57d1700 1 --2- 172.21.15.57:0/1378845147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf81746510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11e57d1700 1 -- 172.21.15.57:0/1378845147 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf814c3d80 con 0x55bf8174c1a0 2023-10-08T13:54:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11e5fd2700 1 --2- 172.21.15.57:0/1378845147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf8174bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11e67d3700 1 --2- 172.21.15.57:0/1378845147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf81746510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11e57d1700 1 --2- 172.21.15.57:0/1378845147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174c1a0 0x55bf81745b70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0cb9c0 tx=0x7f11dc09aea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8494cc3a77af4d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11cbfff700 1 -- 172.21.15.57:0/1378845147 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc09b2e0 con 0x55bf8174c1a0 2023-10-08T13:54:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11e67d3700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a15f0 0x7f11cc0a3ab0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009fd0 tx=0x7f11d800b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11cbfff700 1 -- 172.21.15.57:0/1378845147 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11dc09f430 con 0x55bf8174c1a0 2023-10-08T13:54:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.386+0000 7f11cbfff700 1 -- 172.21.15.57:0/1378845147 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc09b540 con 0x55bf8174c1a0 2023-10-08T13:54:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.387+0000 7f11f441fb80 1 -- 172.21.15.57:0/1378845147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174c1a0 msgr2=0x55bf81745b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.387+0000 7f11f441fb80 1 --2- 172.21.15.57:0/1378845147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174c1a0 0x55bf81745b70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0cb9c0 tx=0x7f11dc09aea0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.387+0000 7f11f441fb80 1 -- 172.21.15.57:0/1378845147 shutdown_connections 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.387+0000 7f11f441fb80 1 --2- 172.21.15.57:0/1378845147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf81746510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.387+0000 7f11f441fb80 1 --2- 172.21.15.57:0/1378845147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf8174bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.387+0000 7f11f441fb80 1 --2- 172.21.15.57:0/1378845147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174c1a0 0x55bf81745b70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.387+0000 7f11f441fb80 1 -- 172.21.15.57:0/1378845147 >> 172.21.15.57:0/1378845147 conn(0x55bf816ba200 msgr2=0x55bf81740b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.387+0000 7f11f441fb80 1 -- 172.21.15.57:0/1378845147 shutdown_connections 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.387+0000 7f11f441fb80 1 -- 172.21.15.57:0/1378845147 wait complete. 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.388+0000 7f11f441fb80 1 -- start start 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.388+0000 7f11f441fb80 1 Processor -- start 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.388+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf817521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.388+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174b880 0x55bf8174f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.388+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174c1a0 0x55bf8174fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.388+0000 7f11f441fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf81698dd0 con 0x55bf817460b0 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.388+0000 7f11f441fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf816356b0 con 0x55bf8174b880 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.388+0000 7f11f441fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf81635350 con 0x55bf8174c1a0 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11f441fb80 10 client.0 _ll_register_callbacks cb 0x55bf814927a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e57d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174b880 0x55bf8174f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e57d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174b880 0x55bf8174f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59494/0 (socket says 172.21.15.57:59494) 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e57d1700 1 -- 172.21.15.57:0/2768486507 learned_addr learned my addr 172.21.15.57:0/2768486507 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e5fd2700 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf817521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e67d3700 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174c1a0 0x55bf8174fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e57d1700 1 -- 172.21.15.57:0/2768486507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174c1a0 msgr2=0x55bf8174fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e57d1700 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174c1a0 0x55bf8174fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e57d1700 1 -- 172.21.15.57:0/2768486507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 msgr2=0x55bf817521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e57d1700 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf817521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e57d1700 1 -- 172.21.15.57:0/2768486507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d8012820 con 0x55bf8174b880 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e5fd2700 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf817521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11e57d1700 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174b880 0x55bf8174f500 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009990 tx=0x7f11dc009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82beab615cfb44d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11c97fa700 1 -- 172.21.15.57:0/2768486507 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0a35e0 con 0x55bf8174b880 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11c97fa700 1 -- 172.21.15.57:0/2768486507 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f11dc09f130 con 0x55bf8174b880 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.389+0000 7f11f441fb80 1 -- 172.21.15.57:0/2768486507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf81660e50 con 0x55bf8174b880 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11c97fa700 1 -- 172.21.15.57:0/2768486507 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11dc0b92c0 con 0x55bf8174b880 2023-10-08T13:54:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11f441fb80 10 client.26099 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11f441fb80 1 -- 172.21.15.57:0/2768486507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f11d001e040 con 0x55bf8174b880 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11c97fa700 1 -- 172.21.15.57:0/2768486507 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11dc081080 con 0x55bf8174b880 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11c97fa700 1 -- 172.21.15.57:0/2768486507 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f11dc09b540 con 0x55bf8174b880 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11f441fb80 10 client.26099 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11f441fb80 10 client.26099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11f441fb80 20 client.26099 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11f441fb80 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bf8175f510 0x55bf81761930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11f441fb80 4 client.26099 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11f441fb80 1 -- 172.21.15.57:0/2768486507 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf81691db0 con 0x55bf8175f510 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.390+0000 7f11e5fd2700 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bf8175f510 0x55bf81761930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.391+0000 7f11c97fa700 10 client.26099 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.391+0000 7f11e5fd2700 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bf8175f510 0x55bf81761930 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f948d26152b88686 server_cookie=e183b50215b61209 in_seq=0 out_seq=0 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.391+0000 7f11c97fa700 1 -- 172.21.15.57:0/2768486507 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bf81691db0 con 0x55bf8175f510 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.391+0000 7f11c97fa700 10 client.26099 handle_command_reply: tid=0 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.391+0000 7f11f441fb80 1 client.26099 shutdown 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.391+0000 7f11f441fb80 1 -- 172.21.15.57:0/2768486507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174b880 msgr2=0x55bf8174f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.391+0000 7f11f441fb80 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174b880 0x55bf8174f500 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009990 tx=0x7f11dc009290 comp rx=0 tx=0).stop 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.392+0000 7f11f441fb80 1 -- 172.21.15.57:0/2768486507 shutdown_connections 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.392+0000 7f11f441fb80 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bf8175f510 0x55bf81761930 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.392+0000 7f11f441fb80 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174c1a0 0x55bf8174fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.392+0000 7f11c97fa700 0 client.26099 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.392+0000 7f11f441fb80 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf817460b0 0x55bf817521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.392+0000 7f11f441fb80 1 --2- 172.21.15.57:0/2768486507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8174b880 0x55bf8174f500 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.392+0000 7f11f441fb80 1 -- 172.21.15.57:0/2768486507 >> 172.21.15.57:0/2768486507 conn(0x55bf816ba200 msgr2=0x55bf81740010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.392+0000 7f11f441fb80 1 -- 172.21.15.57:0/2768486507 shutdown_connections 2023-10-08T13:54:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.392+0000 7f11f441fb80 1 -- 172.21.15.57:0/2768486507 wait complete. 2023-10-08T13:54:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.392+0000 7f11f441fb80 20 client.26099 trim_cache size 0 max 16384 2023-10-08T13:54:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.410+0000 7f11f441fb80 1 Processor -- start 2023-10-08T13:54:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.410+0000 7f11f441fb80 1 -- start start 2023-10-08T13:54:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81745b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf81756900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf8174c1a0 0x55bf81740790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11f441fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf815198c0 con 0x55bf8174c1a0 2023-10-08T13:54:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11f441fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf81741610 con 0x55bf817460b0 2023-10-08T13:54:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11f441fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf8175c3f0 con 0x55bf8174b880 2023-10-08T13:54:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e5fd2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81745b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e57d1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf81756900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e57d1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf81756900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58888/0 (socket says 172.21.15.57:58888) 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e67d3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf8174c1a0 0x55bf81740790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e5fd2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81745b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59500/0 (socket says 172.21.15.57:59500) 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e57d1700 1 -- 172.21.15.57:0/3871222756 learned_addr learned my addr 172.21.15.57:0/3871222756 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e57d1700 1 -- 172.21.15.57:0/3871222756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 msgr2=0x55bf81745b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e57d1700 1 --2- 172.21.15.57:0/3871222756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81745b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e57d1700 1 -- 172.21.15.57:0/3871222756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf8174c1a0 msgr2=0x55bf81740790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e57d1700 1 --2- 172.21.15.57:0/3871222756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf8174c1a0 0x55bf81740790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e57d1700 1 -- 172.21.15.57:0/3871222756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e0196e30 con 0x55bf8174b880 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e5fd2700 1 --2- 172.21.15.57:0/3871222756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81745b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.411+0000 7f11e57d1700 1 --2- 172.21.15.57:0/3871222756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf81756900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11dc09afe0 tx=0x7f11dc099890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7738d4140af54c51 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.412+0000 7f11d4ff9700 1 -- 172.21.15.57:0/3871222756 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0a35e0 con 0x55bf8174b880 2023-10-08T13:54:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.412+0000 7f11d4ff9700 1 -- 172.21.15.57:0/3871222756 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bf81660e50 con 0x55bf8174b880 2023-10-08T13:54:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.412+0000 7f11d4ff9700 1 -- 172.21.15.57:0/3871222756 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0b92c0 con 0x55bf8174b880 2023-10-08T13:54:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.412+0000 7f11f441fb80 1 -- 172.21.15.57:0/3871222756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 msgr2=0x55bf81756900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.412+0000 7f11f441fb80 1 --2- 172.21.15.57:0/3871222756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf81756900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11dc09afe0 tx=0x7f11dc099890 comp rx=0 tx=0).stop 2023-10-08T13:54:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.413+0000 7f11f441fb80 1 -- 172.21.15.57:0/3871222756 shutdown_connections 2023-10-08T13:54:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.413+0000 7f11f441fb80 1 --2- 172.21.15.57:0/3871222756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf8174c1a0 0x55bf81740790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.413+0000 7f11f441fb80 1 --2- 172.21.15.57:0/3871222756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81745b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.413+0000 7f11f441fb80 1 --2- 172.21.15.57:0/3871222756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174b880 0x55bf81756900 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.413+0000 7f11f441fb80 1 -- 172.21.15.57:0/3871222756 >> 172.21.15.57:0/3871222756 conn(0x55bf816ba200 msgr2=0x55bf81749700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.413+0000 7f11f441fb80 1 -- 172.21.15.57:0/3871222756 shutdown_connections 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.413+0000 7f11f441fb80 1 -- 172.21.15.57:0/3871222756 wait complete. 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.413+0000 7f11f441fb80 1 -- start start 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.413+0000 7f11f441fb80 1 Processor -- start 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81754ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf8174b880 0x55bf81755010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11f441fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174c1a0 0x55bf817b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11f441fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf817b2840 con 0x55bf8174b880 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11f441fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf817b2980 con 0x55bf817460b0 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11f441fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf817b2ac0 con 0x55bf8174c1a0 2023-10-08T13:54:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11f441fb80 10 client.0 _ll_register_callbacks cb 0x7f11d8009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11e5fd2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81754ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11e5fd2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81754ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59506/0 (socket says 172.21.15.57:59506) 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11e5fd2700 1 -- 172.21.15.57:0/3273765720 learned_addr learned my addr 172.21.15.57:0/3273765720 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11e67d3700 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174c1a0 0x55bf817b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11e57d1700 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf8174b880 0x55bf81755010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11e5fd2700 1 -- 172.21.15.57:0/3273765720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174c1a0 msgr2=0x55bf817b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11e5fd2700 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174c1a0 0x55bf817b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11e5fd2700 1 -- 172.21.15.57:0/3273765720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf8174b880 msgr2=0x55bf81755010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11e5fd2700 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf8174b880 0x55bf81755010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.414+0000 7f11e5fd2700 1 -- 172.21.15.57:0/3273765720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d00254c0 con 0x55bf817460b0 2023-10-08T13:54:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11e5fd2700 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81754ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55bf817621a0 tx=0x7f11d0009210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97cb8399852dfe1e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11c97fa700 1 -- 172.21.15.57:0/3273765720 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0025920 con 0x55bf817460b0 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11c97fa700 1 -- 172.21.15.57:0/3273765720 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f11d0025a80 con 0x55bf817460b0 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11f441fb80 1 -- 172.21.15.57:0/3273765720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11dc09f430 con 0x55bf817460b0 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11c97fa700 1 -- 172.21.15.57:0/3273765720 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11d0025c40 con 0x55bf817460b0 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11f441fb80 10 client.26105 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11f441fb80 1 -- 172.21.15.57:0/3273765720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f11d001e040 con 0x55bf817460b0 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11c97fa700 1 -- 172.21.15.57:0/3273765720 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11dc09f430 con 0x55bf817460b0 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11c97fa700 1 -- 172.21.15.57:0/3273765720 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f11d00263d0 con 0x55bf817460b0 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11f441fb80 10 client.26105 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11f441fb80 10 client.26105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11f441fb80 20 client.26105 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11f441fb80 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bf817b43e0 0x55bf817b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.415+0000 7f11f441fb80 4 client.26105 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:54:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.416+0000 7f11f441fb80 1 -- 172.21.15.57:0/3273765720 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bf81691ad0 con 0x55bf817b43e0 2023-10-08T13:54:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.416+0000 7f11e57d1700 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bf817b43e0 0x55bf817b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.416+0000 7f11c97fa700 10 client.26105 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:32.416+0000 7f11e57d1700 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bf817b43e0 0x55bf817b6800 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e73c49a0b2b66a server_cookie=49085ff3e586ea0b in_seq=0 out_seq=0 2023-10-08T13:54:34.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:33 smithi115 ceph-mon[104408]: pgmap v389: 129 pgs: 129 active+clean; 622 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 355 op/s 2023-10-08T13:54:34.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:33 smithi057 ceph-mon[99476]: pgmap v389: 129 pgs: 129 active+clean; 622 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 355 op/s 2023-10-08T13:54:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:33 smithi099 ceph-mon[103849]: pgmap v389: 129 pgs: 129 active+clean; 622 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 355 op/s 2023-10-08T13:54:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11c97fa700 1 -- 172.21.15.57:0/3273765720 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bf81691ad0 con 0x55bf817b43e0 2023-10-08T13:54:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11c97fa700 10 client.26105 handle_command_reply: tid=0 2023-10-08T13:54:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 client.26105 shutdown 2023-10-08T13:54:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 -- 172.21.15.57:0/3273765720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 msgr2=0x55bf81754ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81754ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55bf817621a0 tx=0x7f11d0009210 comp rx=0 tx=0).stop 2023-10-08T13:54:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 -- 172.21.15.57:0/3273765720 shutdown_connections 2023-10-08T13:54:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bf817b43e0 0x55bf817b6800 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bf8174c1a0 0x55bf817b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf817460b0 0x55bf81754ad0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11c97fa700 0 client.26105 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 --2- 172.21.15.57:0/3273765720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bf8174b880 0x55bf81755010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 -- 172.21.15.57:0/3273765720 >> 172.21.15.57:0/3273765720 conn(0x55bf816ba200 msgr2=0x55bf8174eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 -- 172.21.15.57:0/3273765720 shutdown_connections 2023-10-08T13:54:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 1 -- 172.21.15.57:0/3273765720 wait complete. 2023-10-08T13:54:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.310+0000 7f11f441fb80 20 client.26105 trim_cache size 0 max 16384 2023-10-08T13:54:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:54:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:54:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:54:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:54:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 -- 172.21.15.57:0/335607345 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a15f0 msgr2=0x7f11cc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a15f0 0x7f11cc0a3ab0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009fd0 tx=0x7f11d800b040 comp rx=0 tx=0).stop 2023-10-08T13:54:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 -- 172.21.15.57:0/335607345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e01040c0 msgr2=0x7f11e010d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e01040c0 0x7f11e010d8e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f11dc006010 tx=0x7f11dc005740 comp rx=0 tx=0).stop 2023-10-08T13:54:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 -- 172.21.15.57:0/335607345 shutdown_connections 2023-10-08T13:54:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f11cc0a15f0 0x7f11cc0a3ab0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e010de20 0x7f11e01a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e00ff230 0x7f11e010d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 --2- 172.21.15.57:0/335607345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11e01040c0 0x7f11e010d8e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 -- 172.21.15.57:0/335607345 >> 172.21.15.57:0/335607345 conn(0x7f11e0075bc0 msgr2=0x7f11e00fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 -- 172.21.15.57:0/335607345 shutdown_connections 2023-10-08T13:54:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:34.311+0000 7f11e8236700 1 -- 172.21.15.57:0/335607345 wait complete. 2023-10-08T13:54:34.365 INFO:tasks.workunit.client.0.smithi057.stdout: 201 2105839 886 1525598 948 257905 432 2023-10-08T13:54:36.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:35 smithi115 ceph-mon[104408]: pgmap v390: 129 pgs: 129 active+clean; 625 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 423 op/s 2023-10-08T13:54:36.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:35 smithi057 ceph-mon[99476]: pgmap v390: 129 pgs: 129 active+clean; 625 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 423 op/s 2023-10-08T13:54:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:35 smithi099 ceph-mon[103849]: pgmap v390: 129 pgs: 129 active+clean; 625 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 423 op/s 2023-10-08T13:54:38.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:37 smithi115 ceph-mon[104408]: pgmap v391: 129 pgs: 129 active+clean; 629 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 542 op/s 2023-10-08T13:54:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:37 smithi057 ceph-mon[99476]: pgmap v391: 129 pgs: 129 active+clean; 629 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 542 op/s 2023-10-08T13:54:38.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:37 smithi099 ceph-mon[103849]: pgmap v391: 129 pgs: 129 active+clean; 629 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 542 op/s 2023-10-08T13:54:39.767 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:54:39.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.902+0000 7fd67cb43700 1 -- 172.21.15.57:0/3587171163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd67810c5f0 msgr2=0x7fd67810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:39.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.902+0000 7fd67cb43700 1 --2- 172.21.15.57:0/3587171163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd67810c5f0 0x7fd67810c9d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd66800aad0 tx=0x7fd66800ade0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.902+0000 7fd67cb43700 1 -- 172.21.15.57:0/3587171163 shutdown_connections 2023-10-08T13:54:39.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.902+0000 7fd67cb43700 1 --2- 172.21.15.57:0/3587171163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd678108a40 0x7fd678114480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.902+0000 7fd67cb43700 1 --2- 172.21.15.57:0/3587171163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd67810c5f0 0x7fd67810c9d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.902+0000 7fd67cb43700 1 --2- 172.21.15.57:0/3587171163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd678108080 0x7fd678108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.902+0000 7fd67cb43700 1 -- 172.21.15.57:0/3587171163 >> 172.21.15.57:0/3587171163 conn(0x7fd67806b830 msgr2=0x7fd67806bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:39.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.902+0000 7fd67cb43700 1 -- 172.21.15.57:0/3587171163 shutdown_connections 2023-10-08T13:54:39.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.902+0000 7fd67cb43700 1 -- 172.21.15.57:0/3587171163 wait complete. 2023-10-08T13:54:39.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.903+0000 7fd67cb43700 1 Processor -- start 2023-10-08T13:54:39.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.903+0000 7fd67cb43700 1 -- start start 2023-10-08T13:54:39.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.903+0000 7fd67cb43700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd678108080 0x7fd67806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.903+0000 7fd67cb43700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd678108a40 0x7fd678070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd67cb43700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd67810c5f0 0x7fd6780708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd67cb43700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781ad780 con 0x7fd67810c5f0 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd67cb43700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781ad8f0 con 0x7fd678108080 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd67cb43700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781adab0 con 0x7fd678108a40 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd676d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd67810c5f0 0x7fd6780708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd67659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd678108080 0x7fd67806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd678108a40 0x7fd678070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd678108a40 0x7fd678070390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58908/0 (socket says 172.21.15.57:58908) 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd676d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd67810c5f0 0x7fd6780708d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58098/0 (socket says 172.21.15.57:58098) 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd67659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd678108080 0x7fd67806fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59520/0 (socket says 172.21.15.57:59520) 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd675d9b700 1 -- 172.21.15.57:0/3884524219 learned_addr learned my addr 172.21.15.57:0/3884524219 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd675d9b700 1 -- 172.21.15.57:0/3884524219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd678108080 msgr2=0x7fd67806fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:39.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd675d9b700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd678108080 0x7fd67806fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd675d9b700 1 -- 172.21.15.57:0/3884524219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd67810c5f0 msgr2=0x7fd6780708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:39.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd675d9b700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd67810c5f0 0x7fd6780708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd675d9b700 1 -- 172.21.15.57:0/3884524219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66800a700 con 0x7fd678108a40 2023-10-08T13:54:39.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd675d9b700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd678108a40 0x7fd678070390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00e9a0 tx=0x7fd66c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:39.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd6677fe700 1 -- 172.21.15.57:0/3884524219 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c009790 con 0x7fd678108a40 2023-10-08T13:54:39.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd67cb43700 1 -- 172.21.15.57:0/3884524219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6781add90 con 0x7fd678108a40 2023-10-08T13:54:39.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd6677fe700 1 -- 172.21.15.57:0/3884524219 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd66c010430 con 0x7fd678108a40 2023-10-08T13:54:39.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd6677fe700 1 -- 172.21.15.57:0/3884524219 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c018620 con 0x7fd678108a40 2023-10-08T13:54:39.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.904+0000 7fd67cb43700 1 -- 172.21.15.57:0/3884524219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6781ae210 con 0x7fd678108a40 2023-10-08T13:54:39.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.907+0000 7fd6677fe700 1 -- 172.21.15.57:0/3884524219 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd66c0105a0 con 0x7fd678108a40 2023-10-08T13:54:39.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.908+0000 7fd6677fe700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd6600a15f0 0x7fd6600a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.908+0000 7fd6677fe700 1 -- 172.21.15.57:0/3884524219 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fd66c014070 con 0x7fd678108a40 2023-10-08T13:54:39.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.908+0000 7fd688d2cb80 1 Processor -- start 2023-10-08T13:54:39.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.908+0000 7fd67659c700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd6600a15f0 0x7fd6600a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.908+0000 7fd688d2cb80 1 -- start start 2023-10-08T13:54:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4047e80 0x55f3a4048260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a40487a0 0x55f3a4053110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a4055a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a3fb1460 con 0x55f3a4053650 2023-10-08T13:54:39.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a3fa2c20 con 0x55f3a40487a0 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a3fa0540 con 0x55f3a4047e80 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd67659c700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd6600a15f0 0x7fd6600a3ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd66800afd0 tx=0x7fd668005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd676d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a4055a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a40487a0 0x55f3a4053110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd676d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a4055a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58106/0 (socket says 172.21.15.57:58106) 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd676d9d700 1 -- 172.21.15.57:0/490300056 learned_addr learned my addr 172.21.15.57:0/490300056 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd67659c700 1 --2- 172.21.15.57:0/490300056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4047e80 0x55f3a4048260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd675d9b700 1 -- 172.21.15.57:0/490300056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4047e80 msgr2=0x55f3a4048260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd675d9b700 1 --2- 172.21.15.57:0/490300056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4047e80 0x55f3a4048260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd675d9b700 1 -- 172.21.15.57:0/490300056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 msgr2=0x55f3a4055a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd675d9b700 1 --2- 172.21.15.57:0/490300056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a4055a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd675d9b700 1 -- 172.21.15.57:0/490300056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3a3dc6de0 con 0x55f3a40487a0 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd67659c700 1 --2- 172.21.15.57:0/490300056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4047e80 0x55f3a4048260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.909+0000 7fd675d9b700 1 --2- 172.21.15.57:0/490300056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a40487a0 0x55f3a4053110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0ca990 tx=0x7fd66c099ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86a95bb9971a92d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.910+0000 7fd664ff9700 1 -- 172.21.15.57:0/490300056 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c09e3a0 con 0x55f3a40487a0 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.910+0000 7fd664ff9700 1 -- 172.21.15.57:0/490300056 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd66c0a25e0 con 0x55f3a40487a0 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.910+0000 7fd664ff9700 1 -- 172.21.15.57:0/490300056 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c09a6f0 con 0x55f3a40487a0 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.910+0000 7fd688d2cb80 1 -- 172.21.15.57:0/490300056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a40487a0 msgr2=0x55f3a4053110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.910+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/490300056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a40487a0 0x55f3a4053110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0ca990 tx=0x7fd66c099ec0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.910+0000 7fd688d2cb80 1 -- 172.21.15.57:0/490300056 shutdown_connections 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.910+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/490300056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a4055a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.910+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/490300056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4047e80 0x55f3a4048260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.910+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/490300056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a40487a0 0x55f3a4053110 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.910+0000 7fd688d2cb80 1 -- 172.21.15.57:0/490300056 >> 172.21.15.57:0/490300056 conn(0x55f3a3fbd260 msgr2=0x55f3a40442d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 1 -- 172.21.15.57:0/490300056 shutdown_connections 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 1 -- 172.21.15.57:0/490300056 wait complete. 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 1 -- start start 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 1 Processor -- start 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 0x55f3a40530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a40487a0 0x55f3a4050430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4053650 0x55f3a4050b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a3f9be30 con 0x55f3a40487a0 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a3f38710 con 0x55f3a4047e80 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a3f383b0 con 0x55f3a4053650 2023-10-08T13:54:39.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd688d2cb80 10 client.0 _ll_register_callbacks cb 0x55f3a3d95800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a40487a0 0x55f3a4050430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd67659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 0x55f3a40530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a40487a0 0x55f3a4050430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58110/0 (socket says 172.21.15.57:58110) 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.911+0000 7fd675d9b700 1 -- 172.21.15.57:0/542220087 learned_addr learned my addr 172.21.15.57:0/542220087 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd676d9d700 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4053650 0x55f3a4050b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd675d9b700 1 -- 172.21.15.57:0/542220087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4053650 msgr2=0x55f3a4050b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd675d9b700 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4053650 0x55f3a4050b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd675d9b700 1 -- 172.21.15.57:0/542220087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 msgr2=0x55f3a40530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd675d9b700 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 0x55f3a40530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd675d9b700 1 -- 172.21.15.57:0/542220087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6781ae210 con 0x55f3a40487a0 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd676d9d700 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4053650 0x55f3a4050b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:54:39.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd67659c700 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 0x55f3a40530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd675d9b700 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a40487a0 0x55f3a4050430 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0ca9f0 tx=0x7fd66c096610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec21b5ab01b8c8c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd65dffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd65dffb700 1 -- 172.21.15.57:0/542220087 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c099940 con 0x55f3a40487a0 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd65dffb700 1 -- 172.21.15.57:0/542220087 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fd66c09a4b0 con 0x55f3a40487a0 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd688d2cb80 1 -- 172.21.15.57:0/542220087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3a3f63eb0 con 0x55f3a40487a0 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd65dffb700 1 -- 172.21.15.57:0/542220087 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd66c09a670 con 0x55f3a40487a0 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd688d2cb80 10 client.16584 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.912+0000 7fd688d2cb80 1 -- 172.21.15.57:0/542220087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f3a3dafc60 con 0x55f3a40487a0 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.913+0000 7fd65dffb700 1 -- 172.21.15.57:0/542220087 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3a3f63eb0 con 0x55f3a40487a0 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.913+0000 7fd65dffb700 1 -- 172.21.15.57:0/542220087 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd66c0b4d20 con 0x55f3a40487a0 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.913+0000 7fd688d2cb80 10 client.16584 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:39.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.913+0000 7fd688d2cb80 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:39.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.913+0000 7fd688d2cb80 20 client.16584 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:39.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.913+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f3a4062e60 0x55f3a4065280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.913+0000 7fd688d2cb80 4 client.16584 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:54:39.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.913+0000 7fd688d2cb80 1 -- 172.21.15.57:0/542220087 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3a3f94e10 con 0x55f3a4062e60 2023-10-08T13:54:39.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.913+0000 7fd67659c700 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f3a4062e60 0x55f3a4065280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.913+0000 7fd65dffb700 10 client.16584 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:39.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.914+0000 7fd67659c700 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f3a4062e60 0x55f3a4065280 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233bbcbbc0d92823 server_cookie=189bb4164a29ece6 in_seq=0 out_seq=0 2023-10-08T13:54:39.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd65dffb700 1 -- 172.21.15.57:0/542220087 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f3a3f94e10 con 0x55f3a4062e60 2023-10-08T13:54:39.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd65dffb700 10 client.16584 handle_command_reply: tid=0 2023-10-08T13:54:39.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 client.16584 shutdown 2023-10-08T13:54:39.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 -- 172.21.15.57:0/542220087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a40487a0 msgr2=0x55f3a4050430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a40487a0 0x55f3a4050430 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0ca9f0 tx=0x7fd66c096610 comp rx=0 tx=0).stop 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 -- 172.21.15.57:0/542220087 shutdown_connections 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f3a4062e60 0x55f3a4065280 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a4053650 0x55f3a4050b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd65dffb700 0 client.16584 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 0x55f3a40530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/542220087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a40487a0 0x55f3a4050430 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 -- 172.21.15.57:0/542220087 >> 172.21.15.57:0/542220087 conn(0x55f3a3fbd260 msgr2=0x55f3a4043ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 -- 172.21.15.57:0/542220087 shutdown_connections 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 1 -- 172.21.15.57:0/542220087 wait complete. 2023-10-08T13:54:39.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.916+0000 7fd688d2cb80 20 client.16584 trim_cache size 0 max 16384 2023-10-08T13:54:39.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd688d2cb80 1 Processor -- start 2023-10-08T13:54:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd688d2cb80 1 -- start start 2023-10-08T13:54:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 0x55f3a405a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a404b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a404bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a3e1c920 con 0x55f3a4053650 2023-10-08T13:54:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a40490d0 con 0x55f3a4047e80 2023-10-08T13:54:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a403bb10 con 0x55f3a40487a0 2023-10-08T13:54:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd676d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a404bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a404b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd67659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 0x55f3a405a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd676d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a404bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58120/0 (socket says 172.21.15.57:58120) 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd676d9d700 1 -- 172.21.15.57:0/2157409361 learned_addr learned my addr 172.21.15.57:0/2157409361 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a404b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58930/0 (socket says 172.21.15.57:58930) 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd676d9d700 1 -- 172.21.15.57:0/2157409361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 msgr2=0x55f3a404b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd676d9d700 1 --2- 172.21.15.57:0/2157409361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a404b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd676d9d700 1 -- 172.21.15.57:0/2157409361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 msgr2=0x55f3a405a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd676d9d700 1 --2- 172.21.15.57:0/2157409361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 0x55f3a405a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd676d9d700 1 -- 172.21.15.57:0/2157409361 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3a3ded250 con 0x55f3a4053650 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.934+0000 7fd675d9b700 1 --2- 172.21.15.57:0/2157409361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a404b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.935+0000 7fd676d9d700 1 --2- 172.21.15.57:0/2157409361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a404bcf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd670003a50 tx=0x7fd670009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9d99d097483c9f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:39.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.935+0000 7fd6657fa700 1 -- 172.21.15.57:0/2157409361 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67000e040 con 0x55f3a4053650 2023-10-08T13:54:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.935+0000 7fd6657fa700 1 -- 172.21.15.57:0/2157409361 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd67000f040 con 0x55f3a4053650 2023-10-08T13:54:39.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.935+0000 7fd6657fa700 1 -- 172.21.15.57:0/2157409361 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67001f670 con 0x55f3a4053650 2023-10-08T13:54:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.935+0000 7fd688d2cb80 1 -- 172.21.15.57:0/2157409361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 msgr2=0x55f3a404bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.935+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/2157409361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a404bcf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd670003a50 tx=0x7fd670009920 comp rx=0 tx=0).stop 2023-10-08T13:54:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 -- 172.21.15.57:0/2157409361 shutdown_connections 2023-10-08T13:54:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/2157409361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4053650 0x55f3a404bcf0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/2157409361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4047e80 0x55f3a405a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/2157409361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a404b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 -- 172.21.15.57:0/2157409361 >> 172.21.15.57:0/2157409361 conn(0x55f3a3fbd260 msgr2=0x55f3a40434d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 -- 172.21.15.57:0/2157409361 shutdown_connections 2023-10-08T13:54:39.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 -- 172.21.15.57:0/2157409361 wait complete. 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 -- start start 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 Processor -- start 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4047e80 0x55f3a40582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a4058830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4053650 0x55f3a40afc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a40b01a0 con 0x55f3a4047e80 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a40b0310 con 0x55f3a4053650 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a40b04d0 con 0x55f3a40487a0 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd688d2cb80 10 client.0 _ll_register_callbacks cb 0x7fd670009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd67659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4047e80 0x55f3a40582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.936+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a4058830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd67659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4047e80 0x55f3a40582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58122/0 (socket says 172.21.15.57:58122) 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a4058830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58936/0 (socket says 172.21.15.57:58936) 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd67659c700 1 -- 172.21.15.57:0/3750604334 learned_addr learned my addr 172.21.15.57:0/3750604334 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd676d9d700 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4053650 0x55f3a40afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd675d9b700 1 -- 172.21.15.57:0/3750604334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4053650 msgr2=0x55f3a40afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd675d9b700 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4053650 0x55f3a40afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd675d9b700 1 -- 172.21.15.57:0/3750604334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4047e80 msgr2=0x55f3a40582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd675d9b700 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4047e80 0x55f3a40582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd675d9b700 1 -- 172.21.15.57:0/3750604334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3a3dafc60 con 0x55f3a40487a0 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd67659c700 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4047e80 0x55f3a40582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd675d9b700 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a4058830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00c2d0 tx=0x7fd66c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b81e43a0355d77f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:39.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd65dffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd65dffb700 1 -- 172.21.15.57:0/3750604334 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c09a950 con 0x55f3a40487a0 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd688d2cb80 1 -- 172.21.15.57:0/3750604334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd66c0a25e0 con 0x55f3a40487a0 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd65dffb700 1 -- 172.21.15.57:0/3750604334 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fd66c01abe0 con 0x55f3a40487a0 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd65dffb700 1 -- 172.21.15.57:0/3750604334 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd66c0994d0 con 0x55f3a40487a0 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd688d2cb80 10 client.26176 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.937+0000 7fd688d2cb80 1 -- 172.21.15.57:0/3750604334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fd6700092a0 con 0x55f3a40487a0 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd65dffb700 1 -- 172.21.15.57:0/3750604334 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd66c0a25e0 con 0x55f3a40487a0 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd65dffb700 1 -- 172.21.15.57:0/3750604334 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd66c09f0e0 con 0x55f3a40487a0 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd688d2cb80 10 client.26176 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd688d2cb80 10 client.26176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:39.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd688d2cb80 20 client.26176 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f3a40b7a30 0x55f3a40b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd688d2cb80 4 client.26176 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd688d2cb80 1 -- 172.21.15.57:0/3750604334 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f3a3f94b30 con 0x55f3a40b7a30 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd67659c700 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f3a40b7a30 0x55f3a40b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd65dffb700 10 client.26176 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.938+0000 7fd67659c700 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f3a40b7a30 0x55f3a40b9e50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4225940cd9a07b8f server_cookie=e804325fb7012e3b in_seq=0 out_seq=0 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.939+0000 7fd65dffb700 1 -- 172.21.15.57:0/3750604334 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f3a3f94b30 con 0x55f3a40b7a30 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.939+0000 7fd65dffb700 10 client.26176 handle_command_reply: tid=0 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.939+0000 7fd688d2cb80 1 client.26176 shutdown 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.939+0000 7fd688d2cb80 1 -- 172.21.15.57:0/3750604334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 msgr2=0x55f3a4058830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:39.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.939+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a4058830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00c2d0 tx=0x7fd66c000bc0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd688d2cb80 1 -- 172.21.15.57:0/3750604334 shutdown_connections 2023-10-08T13:54:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f3a40b7a30 0x55f3a40b9e50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a4053650 0x55f3a40afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd65dffb700 0 client.26176 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f3a4047e80 0x55f3a40582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd688d2cb80 1 --2- 172.21.15.57:0/3750604334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f3a40487a0 0x55f3a4058830 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd688d2cb80 1 -- 172.21.15.57:0/3750604334 >> 172.21.15.57:0/3750604334 conn(0x55f3a3fbd260 msgr2=0x55f3a404b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd688d2cb80 1 -- 172.21.15.57:0/3750604334 shutdown_connections 2023-10-08T13:54:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd688d2cb80 1 -- 172.21.15.57:0/3750604334 wait complete. 2023-10-08T13:54:39.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd688d2cb80 20 client.26176 trim_cache size 0 max 16384 2023-10-08T13:54:39.948 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:54:39.948 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-08T13:54:39.948 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:54:39.948 INFO:teuthology.orchestra.run.smithi057.stdout: "6db03f06-e888-4427-a580-5fa3643c8e0d": { 2023-10-08T13:54:39.948 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:54:39.948 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "6db03f06-e888-4427-a580-5fa3643c8e0d", 2023-10-08T13:54:39.948 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:54:39.948 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:54:39.948 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:54:39.948 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:54:39.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 -- 172.21.15.57:0/3884524219 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd6600a15f0 msgr2=0x7fd6600a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:39.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd6600a15f0 0x7fd6600a3ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd66800afd0 tx=0x7fd668005fb0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 -- 172.21.15.57:0/3884524219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd678108a40 msgr2=0x7fd678070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd678108a40 0x7fd678070390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00e9a0 tx=0x7fd66c00ecb0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 -- 172.21.15.57:0/3884524219 shutdown_connections 2023-10-08T13:54:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd67810c5f0 0x7fd6780708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd6600a15f0 0x7fd6600a3ab0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd678108080 0x7fd67806fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 --2- 172.21.15.57:0/3884524219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd678108a40 0x7fd678070390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 -- 172.21.15.57:0/3884524219 >> 172.21.15.57:0/3884524219 conn(0x7fd67806b830 msgr2=0x7fd67810ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 -- 172.21.15.57:0/3884524219 shutdown_connections 2023-10-08T13:54:39.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:54:39.940+0000 7fd67cb43700 1 -- 172.21.15.57:0/3884524219 wait complete. 2023-10-08T13:54:39.953 INFO:tasks.cephfs.filesystem:scrub status for tag:6db03f06-e888-4427-a580-5fa3643c8e0d - {'path': '/', 'tag': '6db03f06-e888-4427-a580-5fa3643c8e0d', 'options': 'recursive,force'} 2023-10-08T13:54:40.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:39 smithi115 ceph-mon[104408]: pgmap v392: 129 pgs: 129 active+clean; 631 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 643 op/s 2023-10-08T13:54:40.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:39 smithi057 ceph-mon[99476]: pgmap v392: 129 pgs: 129 active+clean; 631 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 643 op/s 2023-10-08T13:54:40.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:39 smithi099 ceph-mon[103849]: pgmap v392: 129 pgs: 129 active+clean; 631 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 643 op/s 2023-10-08T13:54:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:41 smithi115 ceph-mon[104408]: pgmap v393: 129 pgs: 129 active+clean; 630 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 677 op/s 2023-10-08T13:54:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:41 smithi057 ceph-mon[99476]: pgmap v393: 129 pgs: 129 active+clean; 630 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 677 op/s 2023-10-08T13:54:42.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:41 smithi099 ceph-mon[103849]: pgmap v393: 129 pgs: 129 active+clean; 630 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 677 op/s 2023-10-08T13:54:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:43 smithi115 ceph-mon[104408]: pgmap v394: 129 pgs: 129 active+clean; 630 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 616 op/s 2023-10-08T13:54:44.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:43 smithi057 ceph-mon[99476]: pgmap v394: 129 pgs: 129 active+clean; 630 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 616 op/s 2023-10-08T13:54:44.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:43 smithi099 ceph-mon[103849]: pgmap v394: 129 pgs: 129 active+clean; 630 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 616 op/s 2023-10-08T13:54:44.365 INFO:tasks.workunit.client.0.smithi057.stdout: 216 703617 1068 507730 724 80837 269 2023-10-08T13:54:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:45 smithi115 ceph-mon[104408]: pgmap v395: 129 pgs: 129 active+clean; 635 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 628 op/s 2023-10-08T13:54:46.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:45 smithi057 ceph-mon[99476]: pgmap v395: 129 pgs: 129 active+clean; 635 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 628 op/s 2023-10-08T13:54:46.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:45 smithi099 ceph-mon[103849]: pgmap v395: 129 pgs: 129 active+clean; 635 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 628 op/s 2023-10-08T13:54:47.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:46 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:54:47.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:46 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:54:47.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:46 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:54:47.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:46 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:54:47.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:46 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:54:47.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:46 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:54:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.424+0000 7f0ed642c700 1 -- 172.21.15.57:0/2256650898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ed01066f0 msgr2=0x7f0ed0106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.424+0000 7f0ed642c700 1 --2- 172.21.15.57:0/2256650898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ed01066f0 0x7f0ed0106ad0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009990 tx=0x7f0ec0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.425+0000 7f0ed642c700 1 -- 172.21.15.57:0/2256650898 shutdown_connections 2023-10-08T13:54:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.425+0000 7f0ed642c700 1 --2- 172.21.15.57:0/2256650898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ed01010a0 0x7f0ed010d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.425+0000 7f0ed642c700 1 --2- 172.21.15.57:0/2256650898 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ed01066f0 0x7f0ed0106ad0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.425+0000 7f0ed642c700 1 --2- 172.21.15.57:0/2256650898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed01006e0 0x7f0ed0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.425+0000 7f0ed642c700 1 -- 172.21.15.57:0/2256650898 >> 172.21.15.57:0/2256650898 conn(0x7f0ed00fc1f0 msgr2=0x7f0ed00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.425+0000 7f0ed642c700 1 -- 172.21.15.57:0/2256650898 shutdown_connections 2023-10-08T13:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.425+0000 7f0ed642c700 1 -- 172.21.15.57:0/2256650898 wait complete. 2023-10-08T13:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.425+0000 7f0ed642c700 1 Processor -- start 2023-10-08T13:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.425+0000 7f0ed642c700 1 -- start start 2023-10-08T13:54:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ed642c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ed01006e0 0x7f0ed019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ed642c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed01010a0 0x7f0ed019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ed642c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ed01066f0 0x7f0ed01969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ed642c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0197010 con 0x7f0ed01006e0 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ed642c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0197180 con 0x7f0ed01010a0 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ed642c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0197340 con 0x7f0ed01066f0 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ed01006e0 0x7f0ed019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ed01006e0 0x7f0ed019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58136/0 (socket says 172.21.15.57:58136) 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecffff700 1 -- 172.21.15.57:0/3335777668 learned_addr learned my addr 172.21.15.57:0/3335777668 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed01010a0 0x7f0ed019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ed49c9700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ed01066f0 0x7f0ed01969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecffff700 1 -- 172.21.15.57:0/3335777668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ed01066f0 msgr2=0x7f0ed01969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecffff700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ed01066f0 0x7f0ed01969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecffff700 1 -- 172.21.15.57:0/3335777668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed01010a0 msgr2=0x7f0ed019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecffff700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed01010a0 0x7f0ed019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecffff700 1 -- 172.21.15.57:0/3335777668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec00095f0 con 0x7f0ed01006e0 2023-10-08T13:54:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecffff700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ed01006e0 0x7f0ed019c880 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0005910 tx=0x7f0ec0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecd7fa700 1 -- 172.21.15.57:0/3335777668 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec001d070 con 0x7f0ed01006e0 2023-10-08T13:54:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecd7fa700 1 -- 172.21.15.57:0/3335777668 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ec0022470 con 0x7f0ed01006e0 2023-10-08T13:54:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ed642c700 1 -- 172.21.15.57:0/3335777668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed01975c0 con 0x7f0ed01006e0 2023-10-08T13:54:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.426+0000 7f0ecd7fa700 1 -- 172.21.15.57:0/3335777668 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0017650 con 0x7f0ed01006e0 2023-10-08T13:54:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.427+0000 7f0ed642c700 1 -- 172.21.15.57:0/3335777668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed01a3870 con 0x7f0ed01006e0 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.429+0000 7f0ecd7fa700 1 -- 172.21.15.57:0/3335777668 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0ec00225e0 con 0x7f0ed01006e0 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.430+0000 7f0ecd7fa700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0eb80a15f0 0x7f0eb80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.430+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0eb80a15f0 0x7f0eb80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.430+0000 7f0ecd7fa700 1 -- 172.21.15.57:0/3335777668 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0ec00ce620 con 0x7f0ed01006e0 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.430+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0eb80a15f0 0x7f0eb80a3ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0197b70 tx=0x7f0ec4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.430+0000 7f0ee2615b80 1 Processor -- start 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ee2615b80 1 -- start start 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3f880 0x55cd6de3fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de39b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de3a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6dda2400 con 0x55cd6de401a0 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6dd93bc0 con 0x55cd6de3a0b0 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6dd914e0 con 0x55cd6de3f880 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de39b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3f880 0x55cd6de3fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ed49c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de3a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de39b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58146/0 (socket says 172.21.15.57:58146) 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3f880 0x55cd6de3fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58956/0 (socket says 172.21.15.57:58956) 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecf7fe700 1 -- 172.21.15.57:0/1941966159 learned_addr learned my addr 172.21.15.57:0/1941966159 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecffff700 1 -- 172.21.15.57:0/1941966159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 msgr2=0x55cd6de3a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecffff700 1 --2- 172.21.15.57:0/1941966159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de3a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecffff700 1 -- 172.21.15.57:0/1941966159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 msgr2=0x55cd6de39b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecffff700 1 --2- 172.21.15.57:0/1941966159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de39b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecffff700 1 -- 172.21.15.57:0/1941966159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd6dbb7d80 con 0x55cd6de3f880 2023-10-08T13:54:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ed49c9700 1 --2- 172.21.15.57:0/1941966159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de3a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/1941966159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de39b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:54:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.431+0000 7f0ecffff700 1 --2- 172.21.15.57:0/1941966159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3f880 0x55cd6de3fc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00cba70 tx=0x7f0ec0022f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f71a6ba1f55bfa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.432+0000 7f0eb67fc700 1 -- 172.21.15.57:0/1941966159 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec009ae10 con 0x55cd6de3f880 2023-10-08T13:54:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.432+0000 7f0eb67fc700 1 -- 172.21.15.57:0/1941966159 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ec009bbb0 con 0x55cd6de3f880 2023-10-08T13:54:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.432+0000 7f0eb67fc700 1 -- 172.21.15.57:0/1941966159 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00ce2d0 con 0x55cd6de3f880 2023-10-08T13:54:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.432+0000 7f0ee2615b80 1 -- 172.21.15.57:0/1941966159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3f880 msgr2=0x55cd6de3fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.432+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/1941966159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3f880 0x55cd6de3fc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00cba70 tx=0x7f0ec0022f90 comp rx=0 tx=0).stop 2023-10-08T13:54:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.432+0000 7f0ee2615b80 1 -- 172.21.15.57:0/1941966159 shutdown_connections 2023-10-08T13:54:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.432+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/1941966159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de3a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.432+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/1941966159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3f880 0x55cd6de3fc60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.432+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/1941966159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de39b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.432+0000 7f0ee2615b80 1 -- 172.21.15.57:0/1941966159 >> 172.21.15.57:0/1941966159 conn(0x55cd6ddae200 msgr2=0x55cd6de34b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.433+0000 7f0ee2615b80 1 -- 172.21.15.57:0/1941966159 shutdown_connections 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.433+0000 7f0ee2615b80 1 -- 172.21.15.57:0/1941966159 wait complete. 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.433+0000 7f0ee2615b80 1 -- start start 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.433+0000 7f0ee2615b80 1 Processor -- start 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.434+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3a0b0 0x55cd6de435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.434+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de401a0 0x55cd6de43b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.434+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de44050 0x55cd6de4f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.434+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6dd8cdd0 con 0x55cd6de44050 2023-10-08T13:54:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.434+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6dd296b0 con 0x55cd6de401a0 2023-10-08T13:54:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.434+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6dd29350 con 0x55cd6de3a0b0 2023-10-08T13:54:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.434+0000 7f0ee2615b80 10 client.0 _ll_register_callbacks cb 0x55cd6db867a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.436+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3a0b0 0x55cd6de435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.436+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3a0b0 0x55cd6de435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58964/0 (socket says 172.21.15.57:58964) 2023-10-08T13:54:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.436+0000 7f0ecffff700 1 -- 172.21.15.57:0/2043089275 learned_addr learned my addr 172.21.15.57:0/2043089275 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.436+0000 7f0ecffff700 1 -- 172.21.15.57:0/2043089275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de401a0 msgr2=0x55cd6de43b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:54:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.436+0000 7f0ecffff700 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de401a0 0x55cd6de43b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.436+0000 7f0ecffff700 1 -- 172.21.15.57:0/2043089275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de44050 msgr2=0x55cd6de4f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.436+0000 7f0ecffff700 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de44050 0x55cd6de4f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.437+0000 7f0ecffff700 1 -- 172.21.15.57:0/2043089275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec8011230 con 0x55cd6de3a0b0 2023-10-08T13:54:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.437+0000 7f0ecffff700 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3a0b0 0x55cd6de435d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00cc070 tx=0x7f0ec0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c24b7d6a4b02d66d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.499+0000 7f0eaf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:54:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.499+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2043089275 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00b7070 con 0x55cd6de3a0b0 2023-10-08T13:54:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.499+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2043089275 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0ec009f600 con 0x55cd6de3a0b0 2023-10-08T13:54:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.501+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2043089275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd6dd54e50 con 0x55cd6de3a0b0 2023-10-08T13:54:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.502+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2043089275 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec009b530 con 0x55cd6de3a0b0 2023-10-08T13:54:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.503+0000 7f0ee2615b80 10 client.26188 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.503+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2043089275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f0ec4017070 con 0x55cd6de3a0b0 2023-10-08T13:54:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.506+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2043089275 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd6dd54e50 con 0x55cd6de3a0b0 2023-10-08T13:54:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.506+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2043089275 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f0ec0082070 con 0x55cd6de3a0b0 2023-10-08T13:54:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.508+0000 7f0ee2615b80 10 client.26188 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.508+0000 7f0ee2615b80 10 client.26188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.508+0000 7f0ee2615b80 20 client.26188 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.508+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd6de57e60 0x55cd6de5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.508+0000 7f0ee2615b80 4 client.26188 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:54:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.508+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2043089275 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd6dd85db0 con 0x55cd6de57e60 2023-10-08T13:54:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.510+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd6de57e60 0x55cd6de5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.511+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd6de57e60 0x55cd6de5a300 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41cdcf6fd6a54ed3 server_cookie=d2eda8b2960983f2 in_seq=0 out_seq=0 2023-10-08T13:54:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.511+0000 7f0eaf7fe700 10 client.26188 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.511+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2043089275 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cd6dd85db0 con 0x55cd6de57e60 2023-10-08T13:54:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.511+0000 7f0eaf7fe700 10 client.26188 handle_command_reply: tid=0 2023-10-08T13:54:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.511+0000 7f0ee2615b80 1 client.26188 shutdown 2023-10-08T13:54:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.512+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2043089275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3a0b0 msgr2=0x55cd6de435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.512+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3a0b0 0x55cd6de435d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00cc070 tx=0x7f0ec0009310 comp rx=0 tx=0).stop 2023-10-08T13:54:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.513+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2043089275 shutdown_connections 2023-10-08T13:54:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.513+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd6de57e60 0x55cd6de5a300 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.513+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de44050 0x55cd6de4f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.513+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de3a0b0 0x55cd6de435d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.513+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2043089275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de401a0 0x55cd6de43b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.513+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2043089275 >> 172.21.15.57:0/2043089275 conn(0x55cd6ddae200 msgr2=0x55cd6de34010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.513+0000 7f0eaf7fe700 0 client.26188 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.513+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2043089275 shutdown_connections 2023-10-08T13:54:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.513+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2043089275 wait complete. 2023-10-08T13:54:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.514+0000 7f0ee2615b80 20 client.26188 trim_cache size 0 max 16384 2023-10-08T13:54:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.553+0000 7f0ee2615b80 1 Processor -- start 2023-10-08T13:54:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.553+0000 7f0ee2615b80 1 -- start start 2023-10-08T13:54:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.554+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de4a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.554+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de3edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.554+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de44050 0x55cd6de34790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.554+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6dc0d8c0 con 0x55cd6de401a0 2023-10-08T13:54:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.554+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6de36350 con 0x55cd6de3a0b0 2023-10-08T13:54:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.554+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6de44590 con 0x55cd6de44050 2023-10-08T13:54:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.554+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de4a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.554+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de4a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59582/0 (socket says 172.21.15.57:59582) 2023-10-08T13:54:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.554+0000 7f0ecffff700 1 -- 172.21.15.57:0/1380350700 learned_addr learned my addr 172.21.15.57:0/1380350700 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.554+0000 7f0ed49c9700 1 --2- 172.21.15.57:0/1380350700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de44050 0x55cd6de34790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.556+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/1380350700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de3edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.557+0000 7f0ecf7fe700 1 -- 172.21.15.57:0/1380350700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de44050 msgr2=0x55cd6de34790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.557+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/1380350700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de44050 0x55cd6de34790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.557+0000 7f0ecf7fe700 1 -- 172.21.15.57:0/1380350700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 msgr2=0x55cd6de4a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.557+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/1380350700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de4a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.557+0000 7f0ecf7fe700 1 -- 172.21.15.57:0/1380350700 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed01a3870 con 0x55cd6de401a0 2023-10-08T13:54:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.558+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/1380350700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de3edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x55cd6de5abe0 tx=0x7f0ec4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82432bce06ef3f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.564+0000 7f0eb6ffd700 1 -- 172.21.15.57:0/1380350700 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4022040 con 0x55cd6de401a0 2023-10-08T13:54:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.564+0000 7f0eb6ffd700 1 -- 172.21.15.57:0/1380350700 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ec4026d50 con 0x55cd6de401a0 2023-10-08T13:54:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.564+0000 7f0eb6ffd700 1 -- 172.21.15.57:0/1380350700 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4026730 con 0x55cd6de401a0 2023-10-08T13:54:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.566+0000 7f0ee2615b80 1 -- 172.21.15.57:0/1380350700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 msgr2=0x55cd6de3edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.566+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/1380350700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de3edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x55cd6de5abe0 tx=0x7f0ec4014000 comp rx=0 tx=0).stop 2023-10-08T13:54:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.566+0000 7f0ee2615b80 1 -- 172.21.15.57:0/1380350700 shutdown_connections 2023-10-08T13:54:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.566+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/1380350700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de44050 0x55cd6de34790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.566+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/1380350700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de4a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.566+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/1380350700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de401a0 0x55cd6de3edb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.566+0000 7f0ee2615b80 1 -- 172.21.15.57:0/1380350700 >> 172.21.15.57:0/1380350700 conn(0x55cd6ddae200 msgr2=0x55cd6de3e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.567+0000 7f0ee2615b80 1 -- 172.21.15.57:0/1380350700 shutdown_connections 2023-10-08T13:54:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.567+0000 7f0ee2615b80 1 -- 172.21.15.57:0/1380350700 wait complete. 2023-10-08T13:54:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.567+0000 7f0ee2615b80 1 -- start start 2023-10-08T13:54:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.568+0000 7f0ee2615b80 1 Processor -- start 2023-10-08T13:54:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.568+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.568+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de401a0 0x55cd6de53c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.568+0000 7f0ee2615b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de44050 0x55cd6de49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.568+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6de49ea0 con 0x55cd6de44050 2023-10-08T13:54:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.568+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6de4a010 con 0x55cd6de3a0b0 2023-10-08T13:54:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.568+0000 7f0ee2615b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd6de4a1d0 con 0x55cd6de401a0 2023-10-08T13:54:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.568+0000 7f0ee2615b80 10 client.0 _ll_register_callbacks cb 0x7f0ec4011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:54:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.569+0000 7f0ed49c9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de44050 0x55cd6de49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.569+0000 7f0ed49c9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de44050 0x55cd6de49960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58162/0 (socket says 172.21.15.57:58162) 2023-10-08T13:54:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.569+0000 7f0ed49c9700 1 -- 172.21.15.57:0/2816032389 learned_addr learned my addr 172.21.15.57:0/2816032389 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:54:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.577+0000 7f0ecffff700 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.577+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de401a0 0x55cd6de53c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.577+0000 7f0ecffff700 1 -- 172.21.15.57:0/2816032389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de401a0 msgr2=0x55cd6de53c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.577+0000 7f0ecffff700 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de401a0 0x55cd6de53c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.578+0000 7f0ecffff700 1 -- 172.21.15.57:0/2816032389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de44050 msgr2=0x55cd6de49960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:54:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.578+0000 7f0ecffff700 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de44050 0x55cd6de49960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.578+0000 7f0ecffff700 1 -- 172.21.15.57:0/2816032389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec4011af0 con 0x55cd6de3a0b0 2023-10-08T13:54:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.587+0000 7f0ecffff700 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de536f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0006010 tx=0x7f0ec000f830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d422105b403af709 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:54:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.605+0000 7f0eaf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:54:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.605+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2816032389 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00977f0 con 0x55cd6de3a0b0 2023-10-08T13:54:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.606+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2816032389 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f0ec009f600 con 0x55cd6de3a0b0 2023-10-08T13:54:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.606+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2816032389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ec009bbb0 con 0x55cd6de3a0b0 2023-10-08T13:54:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.629+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2816032389 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec00b7030 con 0x55cd6de3a0b0 2023-10-08T13:54:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.630+0000 7f0ee2615b80 10 client.26138 fetch_fsmap learned FSMap version 32 2023-10-08T13:54:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.630+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2816032389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cd6dba0c00 con 0x55cd6de3a0b0 2023-10-08T13:54:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.641+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2816032389 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ec009bbb0 con 0x55cd6de3a0b0 2023-10-08T13:54:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.641+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2816032389 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f0ec0082040 con 0x55cd6de3a0b0 2023-10-08T13:54:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.642+0000 7f0ee2615b80 10 client.26138 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:54:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.642+0000 7f0ee2615b80 10 client.26138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:54:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.642+0000 7f0ee2615b80 20 client.26138 populate_metadata read hostname 'smithi057' 2023-10-08T13:54:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.642+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd6dea8010 0x55cd6debb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:54:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.642+0000 7f0ee2615b80 4 client.26138 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:54:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.642+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2816032389 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd6dd85ad0 con 0x55cd6dea8010 2023-10-08T13:54:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.643+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd6dea8010 0x55cd6debb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:54:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.669+0000 7f0ecf7fe700 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd6dea8010 0x55cd6debb4d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b862ed936c2df176 server_cookie=35cb8b26b50be8d in_seq=0 out_seq=0 2023-10-08T13:54:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:47.670+0000 7f0eaf7fe700 10 client.26138 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:48.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:47 smithi057 ceph-mon[99476]: pgmap v396: 129 pgs: 129 active+clean; 637 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 461 op/s 2023-10-08T13:54:48.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:47 smithi099 ceph-mon[103849]: pgmap v396: 129 pgs: 129 active+clean; 637 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 461 op/s 2023-10-08T13:54:48.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:47 smithi115 ceph-mon[104408]: pgmap v396: 129 pgs: 129 active+clean; 637 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 461 op/s 2023-10-08T13:54:50.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:49 smithi057 ceph-mon[99476]: pgmap v397: 129 pgs: 129 active+clean; 644 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-10-08T13:54:50.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:49 smithi099 ceph-mon[103849]: pgmap v397: 129 pgs: 129 active+clean; 644 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-10-08T13:54:50.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:49 smithi115 ceph-mon[104408]: pgmap v397: 129 pgs: 129 active+clean; 644 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-10-08T13:54:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.395+0000 7f0eaf7fe700 1 -- 172.21.15.57:0/2816032389 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd6dd85ad0 con 0x55cd6dea8010 2023-10-08T13:54:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.395+0000 7f0eaf7fe700 10 client.26138 handle_command_reply: tid=0 2023-10-08T13:54:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.396+0000 7f0ee2615b80 1 client.26138 shutdown 2023-10-08T13:54:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.396+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2816032389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 msgr2=0x55cd6de536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.396+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de536f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0006010 tx=0x7f0ec000f830 comp rx=0 tx=0).stop 2023-10-08T13:54:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.397+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2816032389 shutdown_connections 2023-10-08T13:54:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.397+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd6dea8010 0x55cd6debb4d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.397+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd6de44050 0x55cd6de49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.397+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd6de3a0b0 0x55cd6de536f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.397+0000 7f0ee2615b80 1 --2- 172.21.15.57:0/2816032389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd6de401a0 0x55cd6de53c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.397+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2816032389 >> 172.21.15.57:0/2816032389 conn(0x55cd6ddae200 msgr2=0x55cd6de43920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.397+0000 7f0eaf7fe700 0 client.26138 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:54:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.400+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2816032389 shutdown_connections 2023-10-08T13:54:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.400+0000 7f0ee2615b80 1 -- 172.21.15.57:0/2816032389 wait complete. 2023-10-08T13:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:54:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:54:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:54:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.407+0000 7f0ee2615b80 20 client.26138 trim_cache size 0 max 16384 2023-10-08T13:54:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:52 smithi115 ceph-mon[104408]: pgmap v398: 129 pgs: 129 active+clean; 694 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 360 op/s 2023-10-08T13:54:52.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:52 smithi057 ceph-mon[99476]: pgmap v398: 129 pgs: 129 active+clean; 694 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 360 op/s 2023-10-08T13:54:52.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:52 smithi099 ceph-mon[103849]: pgmap v398: 129 pgs: 129 active+clean; 694 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.9 MiB/s wr, 360 op/s 2023-10-08T13:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.601+0000 7f0ed642c700 1 -- 172.21.15.57:0/3335777668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0eb80a15f0 msgr2=0x7f0eb80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.601+0000 7f0ed642c700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0eb80a15f0 0x7f0eb80a3ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0197b70 tx=0x7f0ec4009260 comp rx=0 tx=0).stop 2023-10-08T13:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.601+0000 7f0ed642c700 1 -- 172.21.15.57:0/3335777668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ed01006e0 msgr2=0x7f0ed019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.601+0000 7f0ed642c700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ed01006e0 0x7f0ed019c880 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0005910 tx=0x7f0ec0004730 comp rx=0 tx=0).stop 2023-10-08T13:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.602+0000 7f0ed642c700 1 -- 172.21.15.57:0/3335777668 shutdown_connections 2023-10-08T13:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.602+0000 7f0ed642c700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ed01066f0 0x7f0ed01969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.602+0000 7f0ed642c700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ed01006e0 0x7f0ed019c880 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.602+0000 7f0ed642c700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0eb80a15f0 0x7f0eb80a3ab0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.602+0000 7f0ed642c700 1 --2- 172.21.15.57:0/3335777668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed01010a0 0x7f0ed019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:54:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.602+0000 7f0ed642c700 1 -- 172.21.15.57:0/3335777668 >> 172.21.15.57:0/3335777668 conn(0x7f0ed00fc1f0 msgr2=0x7f0ed00fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:54:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.607+0000 7f0ed642c700 1 -- 172.21.15.57:0/3335777668 shutdown_connections 2023-10-08T13:54:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:54:52.608+0000 7f0ed642c700 1 -- 172.21.15.57:0/3335777668 wait complete. 2023-10-08T13:54:54.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:53 smithi115 ceph-mon[104408]: pgmap v399: 129 pgs: 129 active+clean; 728 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.5 MiB/s wr, 456 op/s 2023-10-08T13:54:54.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:53 smithi057 ceph-mon[99476]: pgmap v399: 129 pgs: 129 active+clean; 728 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.5 MiB/s wr, 456 op/s 2023-10-08T13:54:54.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:53 smithi099 ceph-mon[103849]: pgmap v399: 129 pgs: 129 active+clean; 728 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.5 MiB/s wr, 456 op/s 2023-10-08T13:54:54.365 INFO:tasks.workunit.client.0.smithi057.stdout: 228 1312707 739 941326 727 156394 300 2023-10-08T13:54:56.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:55 smithi115 ceph-mon[104408]: pgmap v400: 129 pgs: 129 active+clean; 798 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 15 MiB/s wr, 770 op/s 2023-10-08T13:54:56.250 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:55 smithi057 ceph-mon[99476]: pgmap v400: 129 pgs: 129 active+clean; 798 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 15 MiB/s wr, 770 op/s 2023-10-08T13:54:56.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:55 smithi099 ceph-mon[103849]: pgmap v400: 129 pgs: 129 active+clean; 798 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 15 MiB/s wr, 770 op/s 2023-10-08T13:54:58.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:57 smithi115 ceph-mon[104408]: pgmap v401: 129 pgs: 129 active+clean; 825 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 852 op/s 2023-10-08T13:54:58.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:57 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:54:58.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:57 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:54:58.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:57 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:54:58.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:57 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:54:58.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:57 smithi057 ceph-mon[99476]: pgmap v401: 129 pgs: 129 active+clean; 825 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 852 op/s 2023-10-08T13:54:58.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:57 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:54:58.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:57 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:54:58.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:57 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:54:58.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:57 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:54:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:57 smithi099 ceph-mon[103849]: pgmap v401: 129 pgs: 129 active+clean; 825 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 852 op/s 2023-10-08T13:54:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:57 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:54:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:57 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:54:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:57 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:54:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:57 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:55:00.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:54:59 smithi115 ceph-mon[104408]: pgmap v402: 129 pgs: 129 active+clean; 835 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 863 op/s 2023-10-08T13:55:00.248 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:54:59 smithi057 ceph-mon[99476]: pgmap v402: 129 pgs: 129 active+clean; 835 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 863 op/s 2023-10-08T13:55:00.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:54:59 smithi099 ceph-mon[103849]: pgmap v402: 129 pgs: 129 active+clean; 835 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 863 op/s 2023-10-08T13:55:02.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:01 smithi057 ceph-mon[99476]: pgmap v403: 129 pgs: 129 active+clean; 840 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 859 op/s 2023-10-08T13:55:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:01 smithi099 ceph-mon[103849]: pgmap v403: 129 pgs: 129 active+clean; 840 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 859 op/s 2023-10-08T13:55:02.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:01 smithi115 ceph-mon[104408]: pgmap v403: 129 pgs: 129 active+clean; 840 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 859 op/s 2023-10-08T13:55:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:04 smithi099 ceph-mon[103849]: pgmap v404: 129 pgs: 129 active+clean; 840 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 667 op/s 2023-10-08T13:55:04.366 INFO:tasks.workunit.client.0.smithi057.stdout: 237 2086088 503 1498370 542 263512 671 2023-10-08T13:55:04.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:04 smithi115 ceph-mon[104408]: pgmap v404: 129 pgs: 129 active+clean; 840 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 667 op/s 2023-10-08T13:55:04.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:04 smithi057 ceph-mon[99476]: pgmap v404: 129 pgs: 129 active+clean; 840 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 667 op/s 2023-10-08T13:55:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.955+0000 7f67a6f54700 1 -- 172.21.15.57:0/313201058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a01040c0 msgr2=0x7f67a010bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.955+0000 7f67a6f54700 1 --2- 172.21.15.57:0/313201058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a01040c0 0x7f67a010bc80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f679800b0a0 tx=0x7f679800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.955+0000 7f67a6f54700 1 -- 172.21.15.57:0/313201058 shutdown_connections 2023-10-08T13:55:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.955+0000 7f67a6f54700 1 --2- 172.21.15.57:0/313201058 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a01040c0 0x7f67a010bc80 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.955+0000 7f67a6f54700 1 --2- 172.21.15.57:0/313201058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a00ff230 0x7f67a00ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.955+0000 7f67a6f54700 1 --2- 172.21.15.57:0/313201058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a00ffbe0 0x7f67a0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.955+0000 7f67a6f54700 1 -- 172.21.15.57:0/313201058 >> 172.21.15.57:0/313201058 conn(0x7f67a0075bc0 msgr2=0x7f67a0075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.955+0000 7f67a6f54700 1 -- 172.21.15.57:0/313201058 shutdown_connections 2023-10-08T13:55:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.955+0000 7f67a6f54700 1 -- 172.21.15.57:0/313201058 wait complete. 2023-10-08T13:55:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.956+0000 7f67a6f54700 1 Processor -- start 2023-10-08T13:55:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.956+0000 7f67a6f54700 1 -- start start 2023-10-08T13:55:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a6f54700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a00ff230 0x7f67a019b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a6f54700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a01040c0 0x7f67a019be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a6f54700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a0195a40 0x7f67a0195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a6f54700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a0196540 con 0x7f67a01040c0 2023-10-08T13:55:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a6f54700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01966b0 con 0x7f67a0195a40 2023-10-08T13:55:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a6f54700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a0196870 con 0x7f67a00ff230 2023-10-08T13:55:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a4cf0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a00ff230 0x7f67a019b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a4cf0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a00ff230 0x7f67a019b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:58992/0 (socket says 172.21.15.57:58992) 2023-10-08T13:55:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a4cf0700 1 -- 172.21.15.57:0/3883258951 learned_addr learned my addr 172.21.15.57:0/3883258951 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f679ffff700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a01040c0 0x7f67a019be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a54f1700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a0195a40 0x7f67a0195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a4cf0700 1 -- 172.21.15.57:0/3883258951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a0195a40 msgr2=0x7f67a0195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a4cf0700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a0195a40 0x7f67a0195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a4cf0700 1 -- 172.21.15.57:0/3883258951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a01040c0 msgr2=0x7f67a019be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a4cf0700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a01040c0 0x7f67a019be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a4cf0700 1 -- 172.21.15.57:0/3883258951 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798009d00 con 0x7f67a00ff230 2023-10-08T13:55:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a54f1700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a0195a40 0x7f67a0195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:55:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.957+0000 7f67a4cf0700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a00ff230 0x7f67a019b950 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f679400eab0 tx=0x7f679400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.958+0000 7f679dffb700 1 -- 172.21.15.57:0/3883258951 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794013070 con 0x7f67a00ff230 2023-10-08T13:55:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.958+0000 7f679dffb700 1 -- 172.21.15.57:0/3883258951 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f679400be80 con 0x7f67a00ff230 2023-10-08T13:55:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.958+0000 7f679dffb700 1 -- 172.21.15.57:0/3883258951 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67940163b0 con 0x7f67a00ff230 2023-10-08T13:55:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.958+0000 7f67a6f54700 1 -- 172.21.15.57:0/3883258951 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a010f6f0 con 0x7f67a00ff230 2023-10-08T13:55:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.958+0000 7f67a6f54700 1 -- 172.21.15.57:0/3883258951 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a010fbb0 con 0x7f67a00ff230 2023-10-08T13:55:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.961+0000 7f679dffb700 1 -- 172.21.15.57:0/3883258951 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6794017580 con 0x7f67a00ff230 2023-10-08T13:55:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f679dffb700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f67900a15f0 0x7f67900a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f67b313db80 1 Processor -- start 2023-10-08T13:55:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f679dffb700 1 -- 172.21.15.57:0/3883258951 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f67940ceb40 con 0x7f67a00ff230 2023-10-08T13:55:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f67b313db80 1 -- start start 2023-10-08T13:55:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f67b313db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 0x556ed13c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f679ffff700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f67900a15f0 0x7f67900a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f67b313db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 0x556ed13cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f67b313db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f67b313db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed132b400 con 0x556ed13c3680 2023-10-08T13:55:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f67b313db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed131cbc0 con 0x556ed13c2d60 2023-10-08T13:55:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.963+0000 7f67b313db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed131a4e0 con 0x556ed13d02b0 2023-10-08T13:55:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.964+0000 7f67a4cf0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 0x556ed13c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.964+0000 7f67a4cf0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 0x556ed13c3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59614/0 (socket says 172.21.15.57:59614) 2023-10-08T13:55:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.964+0000 7f67a4cf0700 1 -- 172.21.15.57:0/880083026 learned_addr learned my addr 172.21.15.57:0/880083026 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.964+0000 7f679ffff700 1 --2- 172.21.15.57:0/880083026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 0x556ed13cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.964+0000 7f679ffff700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f67900a15f0 0x7f67900a3ab0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f67a0197160 tx=0x7f678c009880 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.964+0000 7f679ffff700 1 -- 172.21.15.57:0/880083026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 msgr2=0x556ed13d0710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:55:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.964+0000 7f679ffff700 1 --2- 172.21.15.57:0/880083026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d0710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.964+0000 7f679ffff700 1 -- 172.21.15.57:0/880083026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 msgr2=0x556ed13c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.964+0000 7f679ffff700 1 --2- 172.21.15.57:0/880083026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 0x556ed13c3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.964+0000 7f679ffff700 1 -- 172.21.15.57:0/880083026 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ed1140d80 con 0x556ed13c3680 2023-10-08T13:55:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.965+0000 7f679ffff700 1 --2- 172.21.15.57:0/880083026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 0x556ed13cfd70 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f67a0072d30 tx=0x7f678c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0dae60d032cac4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.965+0000 7f678affd700 1 -- 172.21.15.57:0/880083026 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c01d960 con 0x556ed13c3680 2023-10-08T13:55:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.965+0000 7f678affd700 1 -- 172.21.15.57:0/880083026 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f678c004030 con 0x556ed13c3680 2023-10-08T13:55:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.965+0000 7f678affd700 1 -- 172.21.15.57:0/880083026 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c003ae0 con 0x556ed13c3680 2023-10-08T13:55:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.966+0000 7f67b313db80 1 -- 172.21.15.57:0/880083026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 msgr2=0x556ed13cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.966+0000 7f67b313db80 1 --2- 172.21.15.57:0/880083026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 0x556ed13cfd70 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f67a0072d30 tx=0x7f678c01c000 comp rx=0 tx=0).stop 2023-10-08T13:55:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.969+0000 7f67b313db80 1 -- 172.21.15.57:0/880083026 shutdown_connections 2023-10-08T13:55:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.969+0000 7f67b313db80 1 --2- 172.21.15.57:0/880083026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.969+0000 7f67b313db80 1 --2- 172.21.15.57:0/880083026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 0x556ed13c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.969+0000 7f67b313db80 1 --2- 172.21.15.57:0/880083026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 0x556ed13cfd70 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.969+0000 7f67b313db80 1 -- 172.21.15.57:0/880083026 >> 172.21.15.57:0/880083026 conn(0x556ed1337200 msgr2=0x556ed13bdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.969+0000 7f67b313db80 1 -- 172.21.15.57:0/880083026 shutdown_connections 2023-10-08T13:55:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.969+0000 7f67b313db80 1 -- 172.21.15.57:0/880083026 wait complete. 2023-10-08T13:55:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.969+0000 7f67b313db80 1 -- start start 2023-10-08T13:55:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67b313db80 1 Processor -- start 2023-10-08T13:55:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67b313db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 0x556ed13d79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67b313db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c3680 0x556ed13d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67b313db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67b313db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed1315dd0 con 0x556ed13c2d60 2023-10-08T13:55:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67b313db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed12b26b0 con 0x556ed13c3680 2023-10-08T13:55:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67b313db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed12b2350 con 0x556ed13d02b0 2023-10-08T13:55:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67b313db80 10 client.0 _ll_register_callbacks cb 0x556ed110f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67a4cf0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 0x556ed13d79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67a4cf0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 0x556ed13d79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58194/0 (socket says 172.21.15.57:58194) 2023-10-08T13:55:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.970+0000 7f67a4cf0700 1 -- 172.21.15.57:0/2474532177 learned_addr learned my addr 172.21.15.57:0/2474532177 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f67a54f1700 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f67a4cf0700 1 -- 172.21.15.57:0/2474532177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 msgr2=0x556ed13d53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f67a4cf0700 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f67a4cf0700 1 -- 172.21.15.57:0/2474532177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c3680 msgr2=0x556ed13d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f67a4cf0700 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c3680 0x556ed13d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f67a4cf0700 1 -- 172.21.15.57:0/2474532177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c019050 con 0x556ed13c2d60 2023-10-08T13:55:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f67a4cf0700 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 0x556ed13d79f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f67940cc590 tx=0x7f679409b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5360fc36bb44fab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f6783fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:55:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f6783fff700 1 -- 172.21.15.57:0/2474532177 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679409f970 con 0x556ed13c2d60 2023-10-08T13:55:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f6783fff700 1 -- 172.21.15.57:0/2474532177 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f67940a3bb0 con 0x556ed13c2d60 2023-10-08T13:55:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.971+0000 7f67b313db80 1 -- 172.21.15.57:0/2474532177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ed12dde50 con 0x556ed13c2d60 2023-10-08T13:55:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f6783fff700 1 -- 172.21.15.57:0/2474532177 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67940860c0 con 0x556ed13c2d60 2023-10-08T13:55:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f67b313db80 10 client.16617 fetch_fsmap learned FSMap version 32 2023-10-08T13:55:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f67b313db80 1 -- 172.21.15.57:0/2474532177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556ed1129c00 con 0x556ed13c2d60 2023-10-08T13:55:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f6783fff700 1 -- 172.21.15.57:0/2474532177 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ed12dde50 con 0x556ed13c2d60 2023-10-08T13:55:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f6783fff700 1 -- 172.21.15.57:0/2474532177 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f67940b9d20 con 0x556ed13c2d60 2023-10-08T13:55:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f67b313db80 10 client.16617 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:55:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f67b313db80 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f67b313db80 20 client.16617 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f67b313db80 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ed13f6580 0x556ed13f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f67b313db80 4 client.16617 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:55:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.972+0000 7f67b313db80 1 -- 172.21.15.57:0/2474532177 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ed130edb0 con 0x556ed13f6580 2023-10-08T13:55:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.973+0000 7f679ffff700 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ed13f6580 0x556ed13f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.973+0000 7f679ffff700 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ed13f6580 0x556ed13f89a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887bb4fe4162a2f6 server_cookie=c312748e54fdb3a1 in_seq=0 out_seq=0 2023-10-08T13:55:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.973+0000 7f6783fff700 10 client.16617 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f6783fff700 1 -- 172.21.15.57:0/2474532177 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556ed130edb0 con 0x556ed13f6580 2023-10-08T13:55:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f6783fff700 10 client.16617 handle_command_reply: tid=0 2023-10-08T13:55:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 client.16617 shutdown 2023-10-08T13:55:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 -- 172.21.15.57:0/2474532177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 msgr2=0x556ed13d79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 0x556ed13d79f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f67940cc590 tx=0x7f679409b200 comp rx=0 tx=0).stop 2023-10-08T13:55:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 -- 172.21.15.57:0/2474532177 shutdown_connections 2023-10-08T13:55:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ed13f6580 0x556ed13f89a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 0x556ed13d79f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f6783fff700 0 client.16617 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 --2- 172.21.15.57:0/2474532177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c3680 0x556ed13d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 -- 172.21.15.57:0/2474532177 >> 172.21.15.57:0/2474532177 conn(0x556ed1337200 msgr2=0x556ed13d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 -- 172.21.15.57:0/2474532177 shutdown_connections 2023-10-08T13:55:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 1 -- 172.21.15.57:0/2474532177 wait complete. 2023-10-08T13:55:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.975+0000 7f67b313db80 20 client.16617 trim_cache size 0 max 16384 2023-10-08T13:55:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.997+0000 7f67b313db80 1 Processor -- start 2023-10-08T13:55:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.997+0000 7f67b313db80 1 -- start start 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.997+0000 7f67b313db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 0x556ed13bc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.997+0000 7f67b313db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 0x556ed13d3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.997+0000 7f67b313db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.997+0000 7f67b313db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed11968c0 con 0x556ed13c3680 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.997+0000 7f67b313db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed13f3ab0 con 0x556ed13c2d60 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.997+0000 7f67b313db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed13c3d60 con 0x556ed13d02b0 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.997+0000 7f67a4cf0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 0x556ed13bc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f679ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 0x556ed13d3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f67a4cf0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 0x556ed13bc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59628/0 (socket says 172.21.15.57:59628) 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f67a4cf0700 1 -- 172.21.15.57:0/2640350979 learned_addr learned my addr 172.21.15.57:0/2640350979 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f67a54f1700 1 --2- 172.21.15.57:0/2640350979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f679ffff700 1 -- 172.21.15.57:0/2640350979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 msgr2=0x556ed13d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f679ffff700 1 --2- 172.21.15.57:0/2640350979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f679ffff700 1 -- 172.21.15.57:0/2640350979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 msgr2=0x556ed13bc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f679ffff700 1 --2- 172.21.15.57:0/2640350979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 0x556ed13bc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f679ffff700 1 -- 172.21.15.57:0/2640350979 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ed11671f0 con 0x556ed13c3680 2023-10-08T13:55:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f67a54f1700 1 --2- 172.21.15.57:0/2640350979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:55:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f679ffff700 1 --2- 172.21.15.57:0/2640350979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 0x556ed13d3600 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f678c009320 tx=0x7f678c009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c197a8f8917373b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f678b7fe700 1 -- 172.21.15.57:0/2640350979 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c0044e0 con 0x556ed13c3680 2023-10-08T13:55:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f678b7fe700 1 -- 172.21.15.57:0/2640350979 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f678c02b1f0 con 0x556ed13c3680 2023-10-08T13:55:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:05.998+0000 7f678b7fe700 1 -- 172.21.15.57:0/2640350979 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c005030 con 0x556ed13c3680 2023-10-08T13:55:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 -- 172.21.15.57:0/2640350979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 msgr2=0x556ed13d3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 --2- 172.21.15.57:0/2640350979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 0x556ed13d3600 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f678c009320 tx=0x7f678c009fa0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 -- 172.21.15.57:0/2640350979 shutdown_connections 2023-10-08T13:55:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 --2- 172.21.15.57:0/2640350979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 --2- 172.21.15.57:0/2640350979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed13c2d60 0x556ed13bc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 --2- 172.21.15.57:0/2640350979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c3680 0x556ed13d3600 secure :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f678c009320 tx=0x7f678c009fa0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 -- 172.21.15.57:0/2640350979 >> 172.21.15.57:0/2640350979 conn(0x556ed1337200 msgr2=0x556ed13fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 -- 172.21.15.57:0/2640350979 shutdown_connections 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 -- 172.21.15.57:0/2640350979 wait complete. 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 -- start start 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.000+0000 7f67b313db80 1 Processor -- start 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f67b313db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 0x556ed13eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f67b313db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f67b313db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed14473e0 0x556ed14497d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f67b313db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed1449d10 con 0x556ed13c2d60 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f67b313db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed1449e80 con 0x556ed14473e0 2023-10-08T13:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f67b313db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ed144a040 con 0x556ed13d02b0 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f67b313db80 10 client.0 _ll_register_callbacks cb 0x7f679400e350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f67a4cf0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 0x556ed13eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f679ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f679ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13ebb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59022/0 (socket says 172.21.15.57:59022) 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.001+0000 7f679ffff700 1 -- 172.21.15.57:0/1468998573 learned_addr learned my addr 172.21.15.57:0/1468998573 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.002+0000 7f67a54f1700 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed14473e0 0x556ed14497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.002+0000 7f679ffff700 1 -- 172.21.15.57:0/1468998573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed14473e0 msgr2=0x556ed14497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.002+0000 7f679ffff700 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed14473e0 0x556ed14497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.002+0000 7f679ffff700 1 -- 172.21.15.57:0/1468998573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 msgr2=0x556ed13eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.002+0000 7f679ffff700 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 0x556ed13eb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.002+0000 7f679ffff700 1 -- 172.21.15.57:0/1468998573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a010c4f0 con 0x556ed13d02b0 2023-10-08T13:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.006+0000 7f679ffff700 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13ebb30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f678c014000 tx=0x7f678c005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f86804490846690a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.006+0000 7f6783fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.006+0000 7f6783fff700 1 -- 172.21.15.57:0/1468998573 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c038070 con 0x556ed13d02b0 2023-10-08T13:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.006+0000 7f6783fff700 1 -- 172.21.15.57:0/1468998573 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f678c0043e0 con 0x556ed13d02b0 2023-10-08T13:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.006+0000 7f67b313db80 1 -- 172.21.15.57:0/1468998573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f678c004030 con 0x556ed13d02b0 2023-10-08T13:55:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f6783fff700 1 -- 172.21.15.57:0/1468998573 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f678c02b360 con 0x556ed13d02b0 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f67b313db80 10 client.26206 fetch_fsmap learned FSMap version 32 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f67b313db80 1 -- 172.21.15.57:0/1468998573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556ed1140d80 con 0x556ed13d02b0 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f6783fff700 1 -- 172.21.15.57:0/1468998573 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f678c004030 con 0x556ed13d02b0 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f6783fff700 1 -- 172.21.15.57:0/1468998573 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f678c03b020 con 0x556ed13d02b0 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f67b313db80 10 client.26206 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f67b313db80 10 client.26206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f67b313db80 20 client.26206 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f67b313db80 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ed144b4f0 0x556ed144d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f67b313db80 4 client.26206 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.015+0000 7f67b313db80 1 -- 172.21.15.57:0/1468998573 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ed130ead0 con 0x556ed144b4f0 2023-10-08T13:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.016+0000 7f67a4cf0700 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ed144b4f0 0x556ed144d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.016+0000 7f6783fff700 10 client.26206 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.016+0000 7f67a4cf0700 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ed144b4f0 0x556ed144d910 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a475ef11e5cd3a server_cookie=497257d63e0137cb in_seq=0 out_seq=0 2023-10-08T13:55:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:06 smithi099 ceph-mon[103849]: pgmap v405: 129 pgs: 129 active+clean; 843 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 544 op/s 2023-10-08T13:55:06.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:06 smithi115 ceph-mon[104408]: pgmap v405: 129 pgs: 129 active+clean; 843 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 544 op/s 2023-10-08T13:55:06.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:06 smithi057 ceph-mon[99476]: pgmap v405: 129 pgs: 129 active+clean; 843 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 544 op/s 2023-10-08T13:55:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f6783fff700 1 -- 172.21.15.57:0/1468998573 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ed130ead0 con 0x556ed144b4f0 2023-10-08T13:55:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f6783fff700 10 client.26206 handle_command_reply: tid=0 2023-10-08T13:55:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 client.26206 shutdown 2023-10-08T13:55:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 -- 172.21.15.57:0/1468998573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 msgr2=0x556ed13ebb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13ebb30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f678c014000 tx=0x7f678c005d80 comp rx=0 tx=0).stop 2023-10-08T13:55:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 -- 172.21.15.57:0/1468998573 shutdown_connections 2023-10-08T13:55:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ed144b4f0 0x556ed144d910 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed14473e0 0x556ed14497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ed13c2d60 0x556ed13eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f6783fff700 0 client.26206 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 --2- 172.21.15.57:0/1468998573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ed13d02b0 0x556ed13ebb30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 -- 172.21.15.57:0/1468998573 >> 172.21.15.57:0/1468998573 conn(0x556ed1337200 msgr2=0x556ed13ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 -- 172.21.15.57:0/1468998573 shutdown_connections 2023-10-08T13:55:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.798+0000 7f67b313db80 1 -- 172.21.15.57:0/1468998573 wait complete. 2023-10-08T13:55:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67b313db80 20 client.26206 trim_cache size 0 max 16384 2023-10-08T13:55:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:55:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:55:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:55:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:55:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 -- 172.21.15.57:0/3883258951 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f67900a15f0 msgr2=0x7f67900a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f67900a15f0 0x7f67900a3ab0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f67a0197160 tx=0x7f678c009880 comp rx=0 tx=0).stop 2023-10-08T13:55:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 -- 172.21.15.57:0/3883258951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a00ff230 msgr2=0x7f67a019b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a00ff230 0x7f67a019b950 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f679400eab0 tx=0x7f679400edc0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 -- 172.21.15.57:0/3883258951 shutdown_connections 2023-10-08T13:55:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f67900a15f0 0x7f67900a3ab0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67a0195a40 0x7f67a0195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a00ff230 0x7f67a019b950 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 --2- 172.21.15.57:0/3883258951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67a01040c0 0x7f67a019be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 -- 172.21.15.57:0/3883258951 >> 172.21.15.57:0/3883258951 conn(0x7f67a0075bc0 msgr2=0x7f67a00fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 -- 172.21.15.57:0/3883258951 shutdown_connections 2023-10-08T13:55:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:06.799+0000 7f67a6f54700 1 -- 172.21.15.57:0/3883258951 wait complete. 2023-10-08T13:55:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:08 smithi115 ceph-mon[104408]: pgmap v406: 129 pgs: 129 active+clean; 844 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 241 op/s 2023-10-08T13:55:08.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:08 smithi057 ceph-mon[99476]: pgmap v406: 129 pgs: 129 active+clean; 844 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 241 op/s 2023-10-08T13:55:08.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:08 smithi099 ceph-mon[103849]: pgmap v406: 129 pgs: 129 active+clean; 844 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 241 op/s 2023-10-08T13:55:09.954 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:55:10.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.096+0000 7f5c09ff8700 1 -- 172.21.15.57:0/2819253340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c041104e0 msgr2=0x7f5c04118060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:10.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.096+0000 7f5c09ff8700 1 --2- 172.21.15.57:0/2819253340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c041104e0 0x7f5c04118060 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b0a0 tx=0x7f5bfc00b3b0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.096+0000 7f5c09ff8700 1 -- 172.21.15.57:0/2819253340 shutdown_connections 2023-10-08T13:55:10.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.096+0000 7f5c09ff8700 1 --2- 172.21.15.57:0/2819253340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c041104e0 0x7f5c04118060 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.096+0000 7f5c09ff8700 1 --2- 172.21.15.57:0/2819253340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c0410b3d0 0x7f5c0410b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.096+0000 7f5c09ff8700 1 --2- 172.21.15.57:0/2819253340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c0410bd80 0x7f5c0410fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.096+0000 7f5c09ff8700 1 -- 172.21.15.57:0/2819253340 >> 172.21.15.57:0/2819253340 conn(0x7f5c0406b720 msgr2=0x7f5c04109800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:10.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.096+0000 7f5c09ff8700 1 -- 172.21.15.57:0/2819253340 shutdown_connections 2023-10-08T13:55:10.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.096+0000 7f5c09ff8700 1 -- 172.21.15.57:0/2819253340 wait complete. 2023-10-08T13:55:10.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.097+0000 7f5c09ff8700 1 Processor -- start 2023-10-08T13:55:10.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.097+0000 7f5c09ff8700 1 -- start start 2023-10-08T13:55:10.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.097+0000 7f5c09ff8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c0410b3d0 0x7f5c041a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.097+0000 7f5c09ff8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c041104e0 0x7f5c041a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.097+0000 7f5c09ff8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c041a4f90 0x7f5c041a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.097+0000 7f5c09ff8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c041a5980 con 0x7f5c0410b3d0 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.097+0000 7f5c09ff8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c041a5af0 con 0x7f5c041a4f90 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.097+0000 7f5c09ff8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c041a5cb0 con 0x7f5c041104e0 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c037fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c0410b3d0 0x7f5c041a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c037fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c0410b3d0 0x7f5c041a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58222/0 (socket says 172.21.15.57:58222) 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c037fe700 1 -- 172.21.15.57:0/1737127267 learned_addr learned my addr 172.21.15.57:0/1737127267 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c041104e0 0x7f5c041a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c03fff700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c041a4f90 0x7f5c041a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c037fe700 1 -- 172.21.15.57:0/1737127267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c041104e0 msgr2=0x7f5c041a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:10.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c037fe700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c041104e0 0x7f5c041a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c037fe700 1 -- 172.21.15.57:0/1737127267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c041a4f90 msgr2=0x7f5c041a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c037fe700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c041a4f90 0x7f5c041a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c037fe700 1 -- 172.21.15.57:0/1737127267 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc009d00 con 0x7f5c0410b3d0 2023-10-08T13:55:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c037fe700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c0410b3d0 0x7f5c041a17b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00e9a0 tx=0x7f5bec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c00ff9700 1 -- 172.21.15.57:0/1737127267 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec009790 con 0x7f5c0410b3d0 2023-10-08T13:55:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c00ff9700 1 -- 172.21.15.57:0/1737127267 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bec010430 con 0x7f5c0410b3d0 2023-10-08T13:55:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c09ff8700 1 -- 172.21.15.57:0/1737127267 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c041ae320 con 0x7f5c0410b3d0 2023-10-08T13:55:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c00ff9700 1 -- 172.21.15.57:0/1737127267 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec018680 con 0x7f5c0410b3d0 2023-10-08T13:55:10.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.098+0000 7f5c09ff8700 1 -- 172.21.15.57:0/1737127267 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c041ae880 con 0x7f5c0410b3d0 2023-10-08T13:55:10.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.100+0000 7f5c00ff9700 1 -- 172.21.15.57:0/1737127267 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5bec0105a0 con 0x7f5c0410b3d0 2023-10-08T13:55:10.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.101+0000 7f5c00ff9700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bf00a16a0 0x7f5bf00a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.101+0000 7f5c00ff9700 1 -- 172.21.15.57:0/1737127267 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5bec014070 con 0x7f5c0410b3d0 2023-10-08T13:55:10.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.101+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bf00a16a0 0x7f5bf00a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.102+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bf00a16a0 0x7f5bf00a3b60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5c041a2cb0 tx=0x7f5bf4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:10.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.102+0000 7f5c161e1b80 1 Processor -- start 2023-10-08T13:55:10.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.102+0000 7f5c161e1b80 1 -- start start 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.102+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5118e80 0x55dea5119260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.102+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 0x55dea5124110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.102+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5124650 0x55dea5126a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.102+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea5082460 con 0x55dea5118e80 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.102+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea5073c20 con 0x55dea51197a0 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.102+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea5071540 con 0x55dea5124650 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c037fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5118e80 0x55dea5119260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c037fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5118e80 0x55dea5119260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58230/0 (socket says 172.21.15.57:58230) 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c037fe700 1 -- 172.21.15.57:0/2774643290 learned_addr learned my addr 172.21.15.57:0/2774643290 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/2774643290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 0x55dea5124110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c03fff700 1 --2- 172.21.15.57:0/2774643290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5124650 0x55dea5126a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c037fe700 1 -- 172.21.15.57:0/2774643290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5124650 msgr2=0x55dea5126a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c037fe700 1 --2- 172.21.15.57:0/2774643290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5124650 0x55dea5126a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c037fe700 1 -- 172.21.15.57:0/2774643290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 msgr2=0x55dea5124110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c037fe700 1 --2- 172.21.15.57:0/2774643290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 0x55dea5124110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c037fe700 1 -- 172.21.15.57:0/2774643290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dea4e97de0 con 0x55dea5118e80 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5c037fe700 1 --2- 172.21.15.57:0/2774643290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5118e80 0x55dea5119260 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5bec096770 tx=0x7f5bec09a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a274bb6d31a9a856 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5bf9ffb700 1 -- 172.21.15.57:0/2774643290 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0cc040 con 0x55dea5118e80 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5bf9ffb700 1 -- 172.21.15.57:0/2774643290 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bec0b1070 con 0x55dea5118e80 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.103+0000 7f5bf9ffb700 1 -- 172.21.15.57:0/2774643290 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec09a4d0 con 0x55dea5118e80 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.104+0000 7f5c161e1b80 1 -- 172.21.15.57:0/2774643290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5118e80 msgr2=0x55dea5119260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.104+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/2774643290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5118e80 0x55dea5119260 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5bec096770 tx=0x7f5bec09a970 comp rx=0 tx=0).stop 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.104+0000 7f5c161e1b80 1 -- 172.21.15.57:0/2774643290 shutdown_connections 2023-10-08T13:55:10.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.104+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/2774643290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5124650 0x55dea5126a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.104+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/2774643290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5118e80 0x55dea5119260 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.104+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/2774643290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 0x55dea5124110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.104+0000 7f5c161e1b80 1 -- 172.21.15.57:0/2774643290 >> 172.21.15.57:0/2774643290 conn(0x55dea508e260 msgr2=0x55dea51152d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:10.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.104+0000 7f5c161e1b80 1 -- 172.21.15.57:0/2774643290 shutdown_connections 2023-10-08T13:55:10.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.104+0000 7f5c161e1b80 1 -- 172.21.15.57:0/2774643290 wait complete. 2023-10-08T13:55:10.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.104+0000 7f5c161e1b80 1 -- start start 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c161e1b80 1 Processor -- start 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5118e80 0x55dea5124060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea5121350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5124650 0x55dea51219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea506ce30 con 0x55dea51197a0 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea5009710 con 0x55dea5118e80 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea50093b0 con 0x55dea5124650 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c161e1b80 10 client.0 _ll_register_callbacks cb 0x55dea4e66800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c02ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea5121350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c037fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5118e80 0x55dea5124060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c02ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea5121350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58238/0 (socket says 172.21.15.57:58238) 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c02ffd700 1 -- 172.21.15.57:0/1300357986 learned_addr learned my addr 172.21.15.57:0/1300357986 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c03fff700 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5124650 0x55dea51219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c02ffd700 1 -- 172.21.15.57:0/1300357986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5124650 msgr2=0x55dea51219f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5124650 0x55dea51219f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c02ffd700 1 -- 172.21.15.57:0/1300357986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5118e80 msgr2=0x55dea5124060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5118e80 0x55dea5124060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c02ffd700 1 -- 172.21.15.57:0/1300357986 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c041ae880 con 0x55dea51197a0 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.105+0000 7f5c037fe700 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5118e80 0x55dea5124060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea5121350 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5bf401f8e0 tx=0x7f5bf40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e879d4d98e9d59 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5be2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5be2ffd700 1 -- 172.21.15.57:0/1300357986 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4004260 con 0x55dea51197a0 2023-10-08T13:55:10.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5be2ffd700 1 -- 172.21.15.57:0/1300357986 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5bf40043c0 con 0x55dea51197a0 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1300357986 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dea5034eb0 con 0x55dea51197a0 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5be2ffd700 1 -- 172.21.15.57:0/1300357986 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bf4004580 con 0x55dea51197a0 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5c161e1b80 10 client.16635 fetch_fsmap learned FSMap version 32 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1300357986 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dea4e80c60 con 0x55dea51197a0 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5be2ffd700 1 -- 172.21.15.57:0/1300357986 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bf4031020 con 0x55dea51197a0 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5be2ffd700 1 -- 172.21.15.57:0/1300357986 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5bf40288e0 con 0x55dea51197a0 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5c161e1b80 10 client.16635 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5c161e1b80 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.106+0000 7f5c161e1b80 20 client.16635 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.107+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55dea5134270 0x55dea5136690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.107+0000 7f5c161e1b80 4 client.16635 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:55:10.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.107+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1300357986 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dea5065e10 con 0x55dea5134270 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.107+0000 7f5c037fe700 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55dea5134270 0x55dea5136690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.107+0000 7f5c037fe700 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55dea5134270 0x55dea5136690 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac7128d9c0d77f3 server_cookie=17d8bca14d2d854c in_seq=0 out_seq=0 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.107+0000 7f5be2ffd700 10 client.16635 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5be2ffd700 1 -- 172.21.15.57:0/1300357986 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dea5065e10 con 0x55dea5134270 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5be2ffd700 10 client.16635 handle_command_reply: tid=0 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 client.16635 shutdown 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1300357986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 msgr2=0x55dea5121350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea5121350 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5bf401f8e0 tx=0x7f5bf40047c0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1300357986 shutdown_connections 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55dea5134270 0x55dea5136690 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5124650 0x55dea51219f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5118e80 0x55dea5124060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/1300357986 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea5121350 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5be2ffd700 0 client.16635 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1300357986 >> 172.21.15.57:0/1300357986 conn(0x55dea508e260 msgr2=0x55dea5114ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1300357986 shutdown_connections 2023-10-08T13:55:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1300357986 wait complete. 2023-10-08T13:55:10.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.108+0000 7f5c161e1b80 20 client.16635 trim_cache size 0 max 16384 2023-10-08T13:55:10.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.126+0000 7f5c161e1b80 1 Processor -- start 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.126+0000 7f5c161e1b80 1 -- start start 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 0x55dea512b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea511fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5124650 0x55dea5120160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea4eed920 con 0x55dea51197a0 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea5130ce0 con 0x55dea5124650 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea511a0d0 con 0x55dea5118e80 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c02ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea511fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c02ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea511fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58244/0 (socket says 172.21.15.57:58244) 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c037fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 0x55dea512b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c02ffd700 1 -- 172.21.15.57:0/1484291854 learned_addr learned my addr 172.21.15.57:0/1484291854 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:10.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c03fff700 1 --2- 172.21.15.57:0/1484291854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5124650 0x55dea5120160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c02ffd700 1 -- 172.21.15.57:0/1484291854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 msgr2=0x55dea512b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/1484291854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 0x55dea512b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c02ffd700 1 -- 172.21.15.57:0/1484291854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5124650 msgr2=0x55dea5120160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/1484291854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5124650 0x55dea5120160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c02ffd700 1 -- 172.21.15.57:0/1484291854 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dea4ebe250 con 0x55dea51197a0 2023-10-08T13:55:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/1484291854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea511fc20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4016040 tx=0x7f5bf40092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3ce0e9c4b3e408 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5bfa7fc700 1 -- 172.21.15.57:0/1484291854 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4004460 con 0x55dea51197a0 2023-10-08T13:55:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5bfa7fc700 1 -- 172.21.15.57:0/1484291854 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bf4031030 con 0x55dea51197a0 2023-10-08T13:55:10.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.127+0000 7f5bfa7fc700 1 -- 172.21.15.57:0/1484291854 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4003be0 con 0x55dea51197a0 2023-10-08T13:55:10.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.128+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1484291854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 msgr2=0x55dea511fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:10.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.128+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/1484291854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea511fc20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4016040 tx=0x7f5bf40092d0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.128+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1484291854 shutdown_connections 2023-10-08T13:55:10.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.128+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/1484291854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea5124650 0x55dea5120160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.128+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/1484291854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 0x55dea512b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.128+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/1484291854 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea51197a0 0x55dea511fc20 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.128+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1484291854 >> 172.21.15.57:0/1484291854 conn(0x55dea508e260 msgr2=0x55dea511c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.128+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1484291854 shutdown_connections 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.128+0000 7f5c161e1b80 1 -- 172.21.15.57:0/1484291854 wait complete. 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.128+0000 7f5c161e1b80 1 -- start start 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c161e1b80 1 Processor -- start 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 0x55dea512b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 0x55dea51295e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c161e1b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5124650 0x55dea5129bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea512a2d0 con 0x55dea5124650 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea512a440 con 0x55dea51197a0 2023-10-08T13:55:10.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c161e1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea51880e0 con 0x55dea5118e80 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c161e1b80 10 client.0 _ll_register_callbacks cb 0x55dea4e66800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c03fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5124650 0x55dea5129bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c037fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 0x55dea512b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c02ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 0x55dea51295e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c03fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5124650 0x55dea5129bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58254/0 (socket says 172.21.15.57:58254) 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c03fff700 1 -- 172.21.15.57:0/518175878 learned_addr learned my addr 172.21.15.57:0/518175878 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c02ffd700 1 -- 172.21.15.57:0/518175878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 msgr2=0x55dea512b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 0x55dea512b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c02ffd700 1 -- 172.21.15.57:0/518175878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5124650 msgr2=0x55dea5129bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5124650 0x55dea5129bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c02ffd700 1 -- 172.21.15.57:0/518175878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dea4e80c60 con 0x55dea51197a0 2023-10-08T13:55:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c037fe700 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 0x55dea512b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.129+0000 7f5c03fff700 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5124650 0x55dea5129bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5c02ffd700 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 0x55dea51295e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55dea5134020 tx=0x7f5bf4028670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64a9ff4516c5c872 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5be2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5be2ffd700 1 -- 172.21.15.57:0/518175878 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4003a60 con 0x55dea51197a0 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5be2ffd700 1 -- 172.21.15.57:0/518175878 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5bf4032080 con 0x55dea51197a0 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5c161e1b80 1 -- 172.21.15.57:0/518175878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bec0b1070 con 0x55dea51197a0 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5be2ffd700 1 -- 172.21.15.57:0/518175878 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bf4028a70 con 0x55dea51197a0 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5c161e1b80 10 client.26171 fetch_fsmap learned FSMap version 32 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5c161e1b80 1 -- 172.21.15.57:0/518175878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dea4e962b0 con 0x55dea51197a0 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5be2ffd700 1 -- 172.21.15.57:0/518175878 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bec0b1070 con 0x55dea51197a0 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5be2ffd700 1 -- 172.21.15.57:0/518175878 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5bf4040a20 con 0x55dea51197a0 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5c161e1b80 10 client.26171 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:55:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5c161e1b80 10 client.26171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5c161e1b80 20 client.26171 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55dea5189600 0x55dea518ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5c161e1b80 4 client.26171 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.130+0000 7f5c161e1b80 1 -- 172.21.15.57:0/518175878 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dea5065b30 con 0x55dea5189600 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c037fe700 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55dea5189600 0x55dea518ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c037fe700 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55dea5189600 0x55dea518ba20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c323652388ab16 server_cookie=bf4782d762a2ea30 in_seq=0 out_seq=0 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5be2ffd700 10 client.26171 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5be2ffd700 1 -- 172.21.15.57:0/518175878 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dea5065b30 con 0x55dea5189600 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5be2ffd700 10 client.26171 handle_command_reply: tid=0 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c161e1b80 1 client.26171 shutdown 2023-10-08T13:55:10.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c161e1b80 1 -- 172.21.15.57:0/518175878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 msgr2=0x55dea51295e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 0x55dea51295e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55dea5134020 tx=0x7f5bf4028670 comp rx=0 tx=0).stop 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c161e1b80 1 -- 172.21.15.57:0/518175878 shutdown_connections 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55dea5189600 0x55dea518ba20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55dea5124650 0x55dea5129bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dea5118e80 0x55dea512b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c161e1b80 1 --2- 172.21.15.57:0/518175878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dea51197a0 0x55dea51295e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5be2ffd700 0 client.26171 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c161e1b80 1 -- 172.21.15.57:0/518175878 >> 172.21.15.57:0/518175878 conn(0x55dea508e260 msgr2=0x55dea511c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.131+0000 7f5c161e1b80 1 -- 172.21.15.57:0/518175878 shutdown_connections 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c161e1b80 1 -- 172.21.15.57:0/518175878 wait complete. 2023-10-08T13:55:10.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c161e1b80 20 client.26171 trim_cache size 0 max 16384 2023-10-08T13:55:10.140 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:55:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T13:55:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:55:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "6db03f06-e888-4427-a580-5fa3643c8e0d": { 2023-10-08T13:55:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:55:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "6db03f06-e888-4427-a580-5fa3643c8e0d", 2023-10-08T13:55:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:55:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:55:10.140 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:55:10.140 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:55:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 -- 172.21.15.57:0/1737127267 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bf00a16a0 msgr2=0x7f5bf00a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bf00a16a0 0x7f5bf00a3b60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5c041a2cb0 tx=0x7f5bf4009260 comp rx=0 tx=0).stop 2023-10-08T13:55:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 -- 172.21.15.57:0/1737127267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c0410b3d0 msgr2=0x7f5c041a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c0410b3d0 0x7f5c041a17b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00e9a0 tx=0x7f5bec00ecb0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 -- 172.21.15.57:0/1737127267 shutdown_connections 2023-10-08T13:55:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f5bf00a16a0 0x7f5bf00a3b60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c041a4f90 0x7f5c041a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c0410b3d0 0x7f5c041a17b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 --2- 172.21.15.57:0/1737127267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c041104e0 0x7f5c041a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 -- 172.21.15.57:0/1737127267 >> 172.21.15.57:0/1737127267 conn(0x7f5c0406b720 msgr2=0x7f5c0410e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 -- 172.21.15.57:0/1737127267 shutdown_connections 2023-10-08T13:55:10.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:10.132+0000 7f5c09ff8700 1 -- 172.21.15.57:0/1737127267 wait complete. 2023-10-08T13:55:10.146 INFO:tasks.cephfs.filesystem:scrub status for tag:6db03f06-e888-4427-a580-5fa3643c8e0d - {'path': '/', 'tag': '6db03f06-e888-4427-a580-5fa3643c8e0d', 'options': 'recursive,force'} 2023-10-08T13:55:10.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:10 smithi057 ceph-mon[99476]: pgmap v407: 129 pgs: 129 active+clean; 841 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2023-10-08T13:55:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:10 smithi115 ceph-mon[104408]: pgmap v407: 129 pgs: 129 active+clean; 841 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2023-10-08T13:55:10.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:10 smithi099 ceph-mon[103849]: pgmap v407: 129 pgs: 129 active+clean; 841 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2023-10-08T13:55:12.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:12 smithi115 ceph-mon[104408]: pgmap v408: 129 pgs: 129 active+clean; 852 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2023-10-08T13:55:12.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:12 smithi057 ceph-mon[99476]: pgmap v408: 129 pgs: 129 active+clean; 852 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2023-10-08T13:55:12.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:12 smithi099 ceph-mon[103849]: pgmap v408: 129 pgs: 129 active+clean; 852 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2023-10-08T13:55:14.366 INFO:tasks.workunit.client.0.smithi057.stdout: 271 1037498 1594 744116 1594 156015 1155 2023-10-08T13:55:14.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:14 smithi115 ceph-mon[104408]: pgmap v409: 129 pgs: 129 active+clean; 856 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 221 op/s 2023-10-08T13:55:14.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:14 smithi057 ceph-mon[99476]: pgmap v409: 129 pgs: 129 active+clean; 856 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 221 op/s 2023-10-08T13:55:14.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:14 smithi099 ceph-mon[103849]: pgmap v409: 129 pgs: 129 active+clean; 856 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 221 op/s 2023-10-08T13:55:16.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:16 smithi115 ceph-mon[104408]: pgmap v410: 129 pgs: 129 active+clean; 862 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 228 op/s 2023-10-08T13:55:16.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:16 smithi057 ceph-mon[99476]: pgmap v410: 129 pgs: 129 active+clean; 862 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 228 op/s 2023-10-08T13:55:16.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:16 smithi099 ceph-mon[103849]: pgmap v410: 129 pgs: 129 active+clean; 862 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 228 op/s 2023-10-08T13:55:18.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:18 smithi115 ceph-mon[104408]: pgmap v411: 129 pgs: 129 active+clean; 864 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-10-08T13:55:18.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:18 smithi057 ceph-mon[99476]: pgmap v411: 129 pgs: 129 active+clean; 864 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-10-08T13:55:18.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:18 smithi099 ceph-mon[103849]: pgmap v411: 129 pgs: 129 active+clean; 864 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-10-08T13:55:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.947+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/1621846643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0b010cf10 msgr2=0x7fb0b010f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.947+0000 7fb0b6e9d700 1 --2- 172.21.15.57:0/1621846643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0b010cf10 0x7fb0b010f360 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800b0a0 tx=0x7fb0a800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.947+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/1621846643 shutdown_connections 2023-10-08T13:55:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.947+0000 7fb0b6e9d700 1 --2- 172.21.15.57:0/1621846643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0b010cf10 0x7fb0b010f360 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.947+0000 7fb0b6e9d700 1 --2- 172.21.15.57:0/1621846643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b0072c00 0x7fb0b0072fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.947+0000 7fb0b6e9d700 1 --2- 172.21.15.57:0/1621846643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b0073520 0x7fb0b010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.947+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/1621846643 >> 172.21.15.57:0/1621846643 conn(0x7fb0b00fc1c0 msgr2=0x7fb0b00fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.947+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/1621846643 shutdown_connections 2023-10-08T13:55:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.947+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/1621846643 wait complete. 2023-10-08T13:55:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.948+0000 7fb0b6e9d700 1 Processor -- start 2023-10-08T13:55:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.948+0000 7fb0b6e9d700 1 -- start start 2023-10-08T13:55:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.948+0000 7fb0b6e9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b0072c00 0x7fb0b010c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.948+0000 7fb0b6e9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b0073520 0x7fb0b0109a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.948+0000 7fb0b6e9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0b010cf10 0x7fb0b0109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.948+0000 7fb0b6e9d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b010a780 con 0x7fb0b010cf10 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.948+0000 7fb0b6e9d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b01a96c0 con 0x7fb0b0073520 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.948+0000 7fb0b6e9d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b01a9800 con 0x7fb0b0072c00 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b543a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0b010cf10 0x7fb0b0109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b543a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0b010cf10 0x7fb0b0109fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58266/0 (socket says 172.21.15.57:58266) 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b4c39700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b0072c00 0x7fb0b010c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b543a700 1 -- 172.21.15.57:0/884932590 learned_addr learned my addr 172.21.15.57:0/884932590 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0affff700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b0073520 0x7fb0b0109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b543a700 1 -- 172.21.15.57:0/884932590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b0072c00 msgr2=0x7fb0b010c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b543a700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b0072c00 0x7fb0b010c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b543a700 1 -- 172.21.15.57:0/884932590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b0073520 msgr2=0x7fb0b0109a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b543a700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b0073520 0x7fb0b0109a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b543a700 1 -- 172.21.15.57:0/884932590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a8009d00 con 0x7fb0b010cf10 2023-10-08T13:55:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b543a700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0b010cf10 0x7fb0b0109fb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800b070 tx=0x7fb0a80078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0adffb700 1 -- 172.21.15.57:0/884932590 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800e040 con 0x7fb0b010cf10 2023-10-08T13:55:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0adffb700 1 -- 172.21.15.57:0/884932590 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0a801d470 con 0x7fb0b010cf10 2023-10-08T13:55:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0adffb700 1 -- 172.21.15.57:0/884932590 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a80052a0 con 0x7fb0b010cf10 2023-10-08T13:55:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.949+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/884932590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b01a9a00 con 0x7fb0b010cf10 2023-10-08T13:55:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.950+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/884932590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b01a9ee0 con 0x7fb0b010cf10 2023-10-08T13:55:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.953+0000 7fb0adffb700 1 -- 172.21.15.57:0/884932590 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb0a801d5e0 con 0x7fb0b010cf10 2023-10-08T13:55:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.953+0000 7fb0adffb700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0a00a1670 0x7fb0a00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.954+0000 7fb0adffb700 1 -- 172.21.15.57:0/884932590 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb0a80ce670 con 0x7fb0b010cf10 2023-10-08T13:55:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.954+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0a00a1670 0x7fb0a00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.954+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0a00a1670 0x7fb0a00a3b30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb0b010b180 tx=0x7fb0a4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.954+0000 7fb0c3086b80 1 Processor -- start 2023-10-08T13:55:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.954+0000 7fb0c3086b80 1 -- start start 2023-10-08T13:55:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 0x55da82e10c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e0ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e0b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82d73400 con 0x55da82e111a0 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82d64bc0 con 0x55da82e0b0b0 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82d624e0 con 0x55da82e10880 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0affff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e0ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0affff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e0ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58276/0 (socket says 172.21.15.57:58276) 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0affff700 1 -- 172.21.15.57:0/2917515490 learned_addr learned my addr 172.21.15.57:0/2917515490 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0b543a700 1 --2- 172.21.15.57:0/2917515490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e0b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/2917515490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 0x55da82e10c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0affff700 1 -- 172.21.15.57:0/2917515490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 msgr2=0x55da82e10c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0affff700 1 --2- 172.21.15.57:0/2917515490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 0x55da82e10c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0affff700 1 -- 172.21.15.57:0/2917515490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 msgr2=0x55da82e0b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0affff700 1 --2- 172.21.15.57:0/2917515490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e0b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.955+0000 7fb0affff700 1 -- 172.21.15.57:0/2917515490 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da82b88d80 con 0x55da82e111a0 2023-10-08T13:55:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.956+0000 7fb0affff700 1 --2- 172.21.15.57:0/2917515490 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e0ab70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009ca0 tx=0x7fb09c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1478b0232d0529c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.956+0000 7fb09affd700 1 -- 172.21.15.57:0/2917515490 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c011d80 con 0x55da82e111a0 2023-10-08T13:55:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.956+0000 7fb09affd700 1 -- 172.21.15.57:0/2917515490 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb09c004cf0 con 0x55da82e111a0 2023-10-08T13:55:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.956+0000 7fb09affd700 1 -- 172.21.15.57:0/2917515490 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c019630 con 0x55da82e111a0 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2917515490 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 msgr2=0x55da82e0ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2917515490 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e0ab70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009ca0 tx=0x7fb09c011440 comp rx=0 tx=0).stop 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2917515490 shutdown_connections 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2917515490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e0b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2917515490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 0x55da82e10c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2917515490 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e0ab70 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2917515490 >> 172.21.15.57:0/2917515490 conn(0x55da82d7f200 msgr2=0x55da82e05b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2917515490 shutdown_connections 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2917515490 wait complete. 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 -- start start 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.957+0000 7fb0c3086b80 1 Processor -- start 2023-10-08T13:55:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e10880 0x55da82e14a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 0x55da82e14f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82d5ddd0 con 0x55da82e10880 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82cfa6b0 con 0x55da82e0b0b0 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82cfa350 con 0x55da82e111a0 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0c3086b80 10 client.0 _ll_register_callbacks cb 0x55da82b577a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0b4c39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0b543a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 0x55da82e14f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0b4c39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e144f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59706/0 (socket says 172.21.15.57:59706) 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0b543a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 0x55da82e14f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59090/0 (socket says 172.21.15.57:59090) 2023-10-08T13:55:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0b4c39700 1 -- 172.21.15.57:0/2556011108 learned_addr learned my addr 172.21.15.57:0/2556011108 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0b4c39700 1 -- 172.21.15.57:0/2556011108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 msgr2=0x55da82e14f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 0x55da82e14f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.958+0000 7fb0b4c39700 1 -- 172.21.15.57:0/2556011108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e10880 msgr2=0x55da82e14a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e10880 0x55da82e14a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb0b4c39700 1 -- 172.21.15.57:0/2556011108 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09c009900 con 0x55da82e0b0b0 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e144f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40045b0 tx=0x7fb0a40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe0df5cc150aa53 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb08ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb08ffff700 1 -- 172.21.15.57:0/2556011108 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4022070 con 0x55da82e0b0b0 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2556011108 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da82d25e50 con 0x55da82e0b0b0 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb08ffff700 1 -- 172.21.15.57:0/2556011108 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb0a4003e20 con 0x55da82e0b0b0 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb08ffff700 1 -- 172.21.15.57:0/2556011108 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0a4004830 con 0x55da82e0b0b0 2023-10-08T13:55:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb0c3086b80 10 client.26183 fetch_fsmap learned FSMap version 32 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2556011108 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55da82b71c00 con 0x55da82e0b0b0 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.959+0000 7fb08ffff700 1 -- 172.21.15.57:0/2556011108 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da82d25e50 con 0x55da82e0b0b0 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.960+0000 7fb08ffff700 1 -- 172.21.15.57:0/2556011108 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb0a401d620 con 0x55da82e0b0b0 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.960+0000 7fb0c3086b80 10 client.26183 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.960+0000 7fb0c3086b80 10 client.26183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.960+0000 7fb0c3086b80 20 client.26183 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.960+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da82e24ee0 0x55da82e27300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.960+0000 7fb0c3086b80 4 client.26183 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.960+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2556011108 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da82d56db0 con 0x55da82e24ee0 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.960+0000 7fb0affff700 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da82e24ee0 0x55da82e27300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.960+0000 7fb08ffff700 10 client.26183 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.960+0000 7fb0affff700 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da82e24ee0 0x55da82e27300 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797d8c5f5d2136f8 server_cookie=40c72514a18b7e78 in_seq=0 out_seq=0 2023-10-08T13:55:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb08ffff700 1 -- 172.21.15.57:0/2556011108 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55da82d56db0 con 0x55da82e24ee0 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb08ffff700 10 client.26183 handle_command_reply: tid=0 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb0c3086b80 1 client.26183 shutdown 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2556011108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 msgr2=0x55da82e144f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e144f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40045b0 tx=0x7fb0a40047c0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2556011108 shutdown_connections 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da82e24ee0 0x55da82e27300 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 0x55da82e14f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e144f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2556011108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e10880 0x55da82e14a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2556011108 >> 172.21.15.57:0/2556011108 conn(0x55da82d7f200 msgr2=0x55da82e05010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb08ffff700 0 client.26183 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.961+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2556011108 shutdown_connections 2023-10-08T13:55:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.962+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2556011108 wait complete. 2023-10-08T13:55:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.962+0000 7fb0c3086b80 20 client.26183 trim_cache size 0 max 16384 2023-10-08T13:55:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.977+0000 7fb0c3086b80 1 Processor -- start 2023-10-08T13:55:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.977+0000 7fb0c3086b80 1 -- start start 2023-10-08T13:55:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e0b0b0 0x55da82e0ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e10880 0x55da82e1bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 0x55da82e05790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82bde8c0 con 0x55da82e0b0b0 2023-10-08T13:55:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82e06610 con 0x55da82e10880 2023-10-08T13:55:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82e15b60 con 0x55da82e111a0 2023-10-08T13:55:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0b4c39700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e0b0b0 0x55da82e0ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0b4c39700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e0b0b0 0x55da82e0ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58286/0 (socket says 172.21.15.57:58286) 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0b4c39700 1 -- 172.21.15.57:0/2888873127 learned_addr learned my addr 172.21.15.57:0/2888873127 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0affff700 1 --2- 172.21.15.57:0/2888873127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e10880 0x55da82e1bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0b543a700 1 --2- 172.21.15.57:0/2888873127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 0x55da82e05790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0b4c39700 1 -- 172.21.15.57:0/2888873127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 msgr2=0x55da82e05790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/2888873127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 0x55da82e05790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0b4c39700 1 -- 172.21.15.57:0/2888873127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e10880 msgr2=0x55da82e1bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/2888873127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e10880 0x55da82e1bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0b4c39700 1 -- 172.21.15.57:0/2888873127 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da82baf1f0 con 0x55da82e0b0b0 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/2888873127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e0b0b0 0x55da82e0ab70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4000c00 tx=0x7fb0a4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb62082e5488f657 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb09b7fe700 1 -- 172.21.15.57:0/2888873127 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4022070 con 0x55da82e0b0b0 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb09b7fe700 1 -- 172.21.15.57:0/2888873127 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0a401dd20 con 0x55da82e0b0b0 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.978+0000 7fb09b7fe700 1 -- 172.21.15.57:0/2888873127 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4027b90 con 0x55da82e0b0b0 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.979+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2888873127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e0b0b0 msgr2=0x55da82e0ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.979+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2888873127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e0b0b0 0x55da82e0ab70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4000c00 tx=0x7fb0a4011ac0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.979+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2888873127 shutdown_connections 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.979+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2888873127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e111a0 0x55da82e05790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.979+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2888873127 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e0b0b0 0x55da82e0ab70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.979+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/2888873127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e10880 0x55da82e1bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.979+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2888873127 >> 172.21.15.57:0/2888873127 conn(0x55da82d7f200 msgr2=0x55da82e0e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.979+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2888873127 shutdown_connections 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.979+0000 7fb0c3086b80 1 -- 172.21.15.57:0/2888873127 wait complete. 2023-10-08T13:55:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.979+0000 7fb0c3086b80 1 -- start start 2023-10-08T13:55:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0c3086b80 1 Processor -- start 2023-10-08T13:55:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e6ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 0x55da82e6d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0c3086b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e77d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82e6dab0 con 0x55da82e111a0 2023-10-08T13:55:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82e78280 con 0x55da82e0b0b0 2023-10-08T13:55:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0c3086b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da82e783c0 con 0x55da82e10880 2023-10-08T13:55:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0c3086b80 10 client.0 _ll_register_callbacks cb 0x7fb0a40043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0affff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 0x55da82e6d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0b543a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e77d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0b4c39700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e6ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0affff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 0x55da82e6d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59106/0 (socket says 172.21.15.57:59106) 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.980+0000 7fb0affff700 1 -- 172.21.15.57:0/882451692 learned_addr learned my addr 172.21.15.57:0/882451692 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0affff700 1 -- 172.21.15.57:0/882451692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 msgr2=0x55da82e6ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0affff700 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e6ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0affff700 1 -- 172.21.15.57:0/882451692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 msgr2=0x55da82e77d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0affff700 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e77d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0affff700 1 -- 172.21.15.57:0/882451692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da82b71c00 con 0x55da82e10880 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e6ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0b543a700 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e77d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0affff700 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 0x55da82e6d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb09c000c00 tx=0x7fb09c009720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=182ee72bfddab12d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb08ffff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb08ffff700 1 -- 172.21.15.57:0/882451692 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0056a0 con 0x55da82e10880 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0c3086b80 1 -- 172.21.15.57:0/882451692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb09c004cf0 con 0x55da82e10880 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb08ffff700 1 -- 172.21.15.57:0/882451692 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fb09c0050f0 con 0x55da82e10880 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb08ffff700 1 -- 172.21.15.57:0/882451692 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb09c011040 con 0x55da82e10880 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0c3086b80 10 client.26242 fetch_fsmap learned FSMap version 32 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb0c3086b80 1 -- 172.21.15.57:0/882451692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55da82b87250 con 0x55da82e10880 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.981+0000 7fb08ffff700 1 -- 172.21.15.57:0/882451692 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb09c004cf0 con 0x55da82e10880 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.982+0000 7fb08ffff700 1 -- 172.21.15.57:0/882451692 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb09c009cd0 con 0x55da82e10880 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.982+0000 7fb0c3086b80 10 client.26242 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.982+0000 7fb0c3086b80 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.982+0000 7fb0c3086b80 20 client.26242 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.982+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da82e79b90 0x55da82e7bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.982+0000 7fb0c3086b80 4 client.26242 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:55:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.982+0000 7fb0c3086b80 1 -- 172.21.15.57:0/882451692 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55da82d56ad0 con 0x55da82e79b90 2023-10-08T13:55:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.982+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da82e79b90 0x55da82e7bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.982+0000 7fb08ffff700 10 client.26242 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:19.982+0000 7fb0b4c39700 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da82e79b90 0x55da82e7bfb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ef921bed074a92 server_cookie=19c15e1365098835 in_seq=0 out_seq=0 2023-10-08T13:55:20.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:20 smithi115 ceph-mon[104408]: pgmap v412: 129 pgs: 129 active+clean; 865 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-10-08T13:55:20.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:20 smithi057 ceph-mon[99476]: pgmap v412: 129 pgs: 129 active+clean; 865 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-10-08T13:55:20.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:20 smithi099 ceph-mon[103849]: pgmap v412: 129 pgs: 129 active+clean; 865 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2023-10-08T13:55:22.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:22 smithi115 ceph-mon[104408]: pgmap v413: 129 pgs: 129 active+clean; 877 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2023-10-08T13:55:22.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:22 smithi057 ceph-mon[99476]: pgmap v413: 129 pgs: 129 active+clean; 877 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2023-10-08T13:55:22.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:22 smithi099 ceph-mon[103849]: pgmap v413: 129 pgs: 129 active+clean; 877 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2023-10-08T13:55:24.373 INFO:tasks.workunit.client.0.smithi057.stdout: 273 685384 226 492007 211 117345 108 2023-10-08T13:55:24.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:24 smithi115 ceph-mon[104408]: pgmap v414: 129 pgs: 129 active+clean; 896 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 286 op/s 2023-10-08T13:55:24.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:24 smithi057 ceph-mon[99476]: pgmap v414: 129 pgs: 129 active+clean; 896 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 286 op/s 2023-10-08T13:55:24.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:24 smithi099 ceph-mon[103849]: pgmap v414: 129 pgs: 129 active+clean; 896 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 286 op/s 2023-10-08T13:55:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb08ffff700 1 -- 172.21.15.57:0/882451692 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55da82d56ad0 con 0x55da82e79b90 2023-10-08T13:55:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb08ffff700 10 client.26242 handle_command_reply: tid=0 2023-10-08T13:55:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 client.26242 shutdown 2023-10-08T13:55:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 -- 172.21.15.57:0/882451692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 msgr2=0x55da82e6d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 0x55da82e6d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb09c000c00 tx=0x7fb09c009720 comp rx=0 tx=0).stop 2023-10-08T13:55:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 -- 172.21.15.57:0/882451692 shutdown_connections 2023-10-08T13:55:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55da82e79b90 0x55da82e7bfb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55da82e111a0 0x55da82e77d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da82e0b0b0 0x55da82e6ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 --2- 172.21.15.57:0/882451692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da82e10880 0x55da82e6d3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 -- 172.21.15.57:0/882451692 >> 172.21.15.57:0/882451692 conn(0x55da82d7f200 msgr2=0x55da82e11630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb08ffff700 0 client.26242 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 -- 172.21.15.57:0/882451692 shutdown_connections 2023-10-08T13:55:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.057+0000 7fb0c3086b80 1 -- 172.21.15.57:0/882451692 wait complete. 2023-10-08T13:55:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:55:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:55:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:55:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:55:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.107+0000 7fb0c3086b80 20 client.26242 trim_cache size 0 max 16384 2023-10-08T13:55:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.197+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/884932590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0a00a1670 msgr2=0x7fb0a00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.198+0000 7fb0b6e9d700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0a00a1670 0x7fb0a00a3b30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb0b010b180 tx=0x7fb0a4009260 comp rx=0 tx=0).stop 2023-10-08T13:55:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.198+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/884932590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0b010cf10 msgr2=0x7fb0b0109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.198+0000 7fb0b6e9d700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0b010cf10 0x7fb0b0109fb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800b070 tx=0x7fb0a80078e0 comp rx=0 tx=0).stop 2023-10-08T13:55:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.204+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/884932590 shutdown_connections 2023-10-08T13:55:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.204+0000 7fb0b6e9d700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0b010cf10 0x7fb0b0109fb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.204+0000 7fb0b6e9d700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b0072c00 0x7fb0b010c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.204+0000 7fb0b6e9d700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb0a00a1670 0x7fb0a00a3b30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.204+0000 7fb0b6e9d700 1 --2- 172.21.15.57:0/884932590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b0073520 0x7fb0b0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.204+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/884932590 >> 172.21.15.57:0/884932590 conn(0x7fb0b00fc1c0 msgr2=0x7fb0b00fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.214+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/884932590 shutdown_connections 2023-10-08T13:55:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:25.228+0000 7fb0b6e9d700 1 -- 172.21.15.57:0/884932590 wait complete. 2023-10-08T13:55:25.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:25 smithi099 ceph-mon[103849]: pgmap v415: 129 pgs: 129 active+clean; 933 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.9 MiB/s wr, 517 op/s 2023-10-08T13:55:25.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:25 smithi115 ceph-mon[104408]: pgmap v415: 129 pgs: 129 active+clean; 933 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.9 MiB/s wr, 517 op/s 2023-10-08T13:55:25.747 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:25 smithi057 ceph-mon[99476]: pgmap v415: 129 pgs: 129 active+clean; 933 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.9 MiB/s wr, 517 op/s 2023-10-08T13:55:27.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:27 smithi115 ceph-mon[104408]: pgmap v416: 129 pgs: 129 active+clean; 955 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.1 MiB/s wr, 658 op/s 2023-10-08T13:55:27.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:27 smithi057 ceph-mon[99476]: pgmap v416: 129 pgs: 129 active+clean; 955 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.1 MiB/s wr, 658 op/s 2023-10-08T13:55:27.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:27 smithi099 ceph-mon[103849]: pgmap v416: 129 pgs: 129 active+clean; 955 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.1 MiB/s wr, 658 op/s 2023-10-08T13:55:30.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:29 smithi057 ceph-mon[99476]: pgmap v417: 129 pgs: 129 active+clean; 972 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.8 MiB/s wr, 731 op/s 2023-10-08T13:55:30.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:29 smithi099 ceph-mon[103849]: pgmap v417: 129 pgs: 129 active+clean; 972 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.8 MiB/s wr, 731 op/s 2023-10-08T13:55:30.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:29 smithi115 ceph-mon[104408]: pgmap v417: 129 pgs: 129 active+clean; 972 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.8 MiB/s wr, 731 op/s 2023-10-08T13:55:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:31 smithi057 ceph-mon[99476]: pgmap v418: 129 pgs: 129 active+clean; 1015 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 895 op/s 2023-10-08T13:55:32.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:31 smithi099 ceph-mon[103849]: pgmap v418: 129 pgs: 129 active+clean; 1015 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 895 op/s 2023-10-08T13:55:32.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:31 smithi115 ceph-mon[104408]: pgmap v418: 129 pgs: 129 active+clean; 1015 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 895 op/s 2023-10-08T13:55:34.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:33 smithi057 ceph-mon[99476]: pgmap v419: 129 pgs: 129 active+clean; 1022 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 873 op/s 2023-10-08T13:55:34.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:33 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:33 smithi099 ceph-mon[103849]: pgmap v419: 129 pgs: 129 active+clean; 1022 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 873 op/s 2023-10-08T13:55:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:33 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:34.372 INFO:tasks.workunit.client.0.smithi057.stdout: 293 1380332 935 989905 959 228331 160 2023-10-08T13:55:34.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:33 smithi115 ceph-mon[104408]: pgmap v419: 129 pgs: 129 active+clean; 1022 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 873 op/s 2023-10-08T13:55:34.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:33 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:36.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:35 smithi057 ceph-mon[99476]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 766 op/s 2023-10-08T13:55:36.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:35 smithi099 ceph-mon[103849]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 766 op/s 2023-10-08T13:55:36.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:35 smithi115 ceph-mon[104408]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 766 op/s 2023-10-08T13:55:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:37 smithi057 ceph-mon[99476]: pgmap v421: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.4 MiB/s wr, 531 op/s 2023-10-08T13:55:38.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:37 smithi099 ceph-mon[103849]: pgmap v421: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.4 MiB/s wr, 531 op/s 2023-10-08T13:55:38.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:37 smithi115 ceph-mon[104408]: pgmap v421: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.4 MiB/s wr, 531 op/s 2023-10-08T13:55:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.081+0000 7f95da40b700 1 -- 172.21.15.57:0/2425124184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d4119fd0 msgr2=0x7f95d411c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.081+0000 7f95da40b700 1 --2- 172.21.15.57:0/2425124184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d4119fd0 0x7f95d411c3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b0d0 tx=0x7f95cc00b3e0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.082+0000 7f95da40b700 1 -- 172.21.15.57:0/2425124184 shutdown_connections 2023-10-08T13:55:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.082+0000 7f95da40b700 1 --2- 172.21.15.57:0/2425124184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d4119fd0 0x7f95d411c3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.082+0000 7f95da40b700 1 --2- 172.21.15.57:0/2425124184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95d410ef10 0x7f95d41077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.082+0000 7f95da40b700 1 --2- 172.21.15.57:0/2425124184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d410f380 0x7f95d4107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.082+0000 7f95da40b700 1 -- 172.21.15.57:0/2425124184 >> 172.21.15.57:0/2425124184 conn(0x7f95d406b7c0 msgr2=0x7f95d406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.082+0000 7f95da40b700 1 -- 172.21.15.57:0/2425124184 shutdown_connections 2023-10-08T13:55:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.082+0000 7f95da40b700 1 -- 172.21.15.57:0/2425124184 wait complete. 2023-10-08T13:55:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.083+0000 7f95da40b700 1 Processor -- start 2023-10-08T13:55:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.084+0000 7f95da40b700 1 -- start start 2023-10-08T13:55:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.084+0000 7f95da40b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95d410ef10 0x7f95d41184a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.084+0000 7f95da40b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d410f380 0x7f95d4118a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.084+0000 7f95da40b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4119fd0 0x7f95d41117a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.084+0000 7f95da40b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d4111f70 con 0x7f95d410ef10 2023-10-08T13:55:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.084+0000 7f95da40b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41120e0 con 0x7f95d4119fd0 2023-10-08T13:55:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.084+0000 7f95da40b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41122a0 con 0x7f95d410f380 2023-10-08T13:55:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.085+0000 7f95d89a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4119fd0 0x7f95d41117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.085+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95d410ef10 0x7f95d41184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.085+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95d410ef10 0x7f95d41184a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58308/0 (socket says 172.21.15.57:58308) 2023-10-08T13:55:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.085+0000 7f95d3fff700 1 -- 172.21.15.57:0/65888114 learned_addr learned my addr 172.21.15.57:0/65888114 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.085+0000 7f95d37fe700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d410f380 0x7f95d4118a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.085+0000 7f95d89a8700 1 -- 172.21.15.57:0/65888114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d410f380 msgr2=0x7f95d4118a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.085+0000 7f95d89a8700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d410f380 0x7f95d4118a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.085+0000 7f95d89a8700 1 -- 172.21.15.57:0/65888114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95d410ef10 msgr2=0x7f95d41184a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.085+0000 7f95d89a8700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95d410ef10 0x7f95d41184a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.085+0000 7f95d89a8700 1 -- 172.21.15.57:0/65888114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8009520 con 0x7f95d4119fd0 2023-10-08T13:55:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.086+0000 7f95d89a8700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4119fd0 0x7f95d41117a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b770 tx=0x7f95cc00bd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.086+0000 7f95d17fa700 1 -- 172.21.15.57:0/65888114 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc00e040 con 0x7f95d4119fd0 2023-10-08T13:55:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.086+0000 7f95d17fa700 1 -- 172.21.15.57:0/65888114 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95cc005030 con 0x7f95d4119fd0 2023-10-08T13:55:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.086+0000 7f95d17fa700 1 -- 172.21.15.57:0/65888114 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc01b6a0 con 0x7f95d4119fd0 2023-10-08T13:55:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.086+0000 7f95da40b700 1 -- 172.21.15.57:0/65888114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc009d00 con 0x7f95d4119fd0 2023-10-08T13:55:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.086+0000 7f95da40b700 1 -- 172.21.15.57:0/65888114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d4116210 con 0x7f95d4119fd0 2023-10-08T13:55:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.089+0000 7f95d17fa700 1 -- 172.21.15.57:0/65888114 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f95cc004a50 con 0x7f95d4119fd0 2023-10-08T13:55:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.090+0000 7f95d17fa700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f95bc0a16b0 0x7f95bc0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.090+0000 7f95d3fff700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f95bc0a16b0 0x7f95bc0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.090+0000 7f95d17fa700 1 -- 172.21.15.57:0/65888114 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f95cc0cddb0 con 0x7f95d4119fd0 2023-10-08T13:55:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.091+0000 7f95e65f4b80 1 Processor -- start 2023-10-08T13:55:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.091+0000 7f95e65f4b80 1 -- start start 2023-10-08T13:55:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.091+0000 7f95d3fff700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f95bc0a16b0 0x7f95bc0a3b70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009ad0 tx=0x7f95c4009290 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.091+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f4408c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.091+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f4402b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.091+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 0x5560f4403510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.091+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f436b400 con 0x5560f4408880 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.091+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f435cbc0 con 0x5560f44091a0 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.091+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f435a4e0 con 0x5560f44030b0 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.092+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f4408c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.092+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f4408c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58316/0 (socket says 172.21.15.57:58316) 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.092+0000 7f95d3fff700 1 -- 172.21.15.57:0/3578427535 learned_addr learned my addr 172.21.15.57:0/3578427535 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.092+0000 7f95d89a8700 1 --2- 172.21.15.57:0/3578427535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 0x5560f4403510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.092+0000 7f95d3fff700 1 -- 172.21.15.57:0/3578427535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 msgr2=0x5560f4403510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.092+0000 7f95d3fff700 1 --2- 172.21.15.57:0/3578427535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 0x5560f4403510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.092+0000 7f95d3fff700 1 -- 172.21.15.57:0/3578427535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 msgr2=0x5560f4402b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:55:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.092+0000 7f95d3fff700 1 --2- 172.21.15.57:0/3578427535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f4402b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.092+0000 7f95d3fff700 1 -- 172.21.15.57:0/3578427535 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560f4180d80 con 0x5560f4408880 2023-10-08T13:55:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.093+0000 7f95d3fff700 1 --2- 172.21.15.57:0/3578427535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f4408c60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f95c400eac0 tx=0x7f95c4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50632db7fd4d36b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.093+0000 7f95ba7fc700 1 -- 172.21.15.57:0/3578427535 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4015480 con 0x5560f4408880 2023-10-08T13:55:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.093+0000 7f95ba7fc700 1 -- 172.21.15.57:0/3578427535 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c4015aa0 con 0x5560f4408880 2023-10-08T13:55:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.093+0000 7f95ba7fc700 1 -- 172.21.15.57:0/3578427535 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4013b10 con 0x5560f4408880 2023-10-08T13:55:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.094+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3578427535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 msgr2=0x5560f4408c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.094+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/3578427535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f4408c60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f95c400eac0 tx=0x7f95c4011000 comp rx=0 tx=0).stop 2023-10-08T13:55:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.095+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3578427535 shutdown_connections 2023-10-08T13:55:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.095+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/3578427535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 0x5560f4403510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.095+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/3578427535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f4408c60 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.095+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/3578427535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f4402b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.095+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3578427535 >> 172.21.15.57:0/3578427535 conn(0x5560f4377200 msgr2=0x5560f43fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.095+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3578427535 shutdown_connections 2023-10-08T13:55:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.095+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3578427535 wait complete. 2023-10-08T13:55:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.095+0000 7f95e65f4b80 1 -- start start 2023-10-08T13:55:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.096+0000 7f95e65f4b80 1 Processor -- start 2023-10-08T13:55:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.096+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 0x5560f440c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.096+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f440ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.096+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f440cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.096+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f4355dd0 con 0x5560f4408880 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.096+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f42f26b0 con 0x5560f44091a0 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.096+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f42f2350 con 0x5560f44030b0 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.096+0000 7f95e65f4b80 10 client.0 _ll_register_callbacks cb 0x5560f414f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.096+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 0x5560f440c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 0x5560f440c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59132/0 (socket says 172.21.15.57:59132) 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d3fff700 1 -- 172.21.15.57:0/2627650261 learned_addr learned my addr 172.21.15.57:0/2627650261 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d89a8700 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f440cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d37fe700 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f440ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d3fff700 1 -- 172.21.15.57:0/2627650261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 msgr2=0x5560f440cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d3fff700 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f440cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d3fff700 1 -- 172.21.15.57:0/2627650261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 msgr2=0x5560f440ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d3fff700 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f440ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d3fff700 1 -- 172.21.15.57:0/2627650261 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c400e7a0 con 0x5560f44030b0 2023-10-08T13:55:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d89a8700 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f440cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:55:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d37fe700 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f440ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:55:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95d3fff700 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 0x5560f440c510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f95c40095d0 tx=0x7f95c4015730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b717d3fc1e6003d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:55:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.097+0000 7f95b37fe700 1 -- 172.21.15.57:0/2627650261 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4013b20 con 0x5560f44030b0 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95b37fe700 1 -- 172.21.15.57:0/2627650261 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f95c4013c80 con 0x5560f44030b0 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2627650261 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560f431de50 con 0x5560f44030b0 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95b37fe700 1 -- 172.21.15.57:0/2627650261 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95c4013e40 con 0x5560f44030b0 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95e65f4b80 10 client.26251 fetch_fsmap learned FSMap version 34 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2627650261 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5560f4169c00 con 0x5560f44030b0 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95b37fe700 1 -- 172.21.15.57:0/2627650261 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5560f431de50 con 0x5560f44030b0 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95b37fe700 1 -- 172.21.15.57:0/2627650261 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f95c4028420 con 0x5560f44030b0 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95e65f4b80 10 client.26251 fetch_fsmap finished waiting for FSMap version 34 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95e65f4b80 10 client.26251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95e65f4b80 20 client.26251 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5560f441cd40 0x5560f441f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95e65f4b80 4 client.26251 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.098+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2627650261 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560f434edb0 con 0x5560f441cd40 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.099+0000 7f95d37fe700 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5560f441cd40 0x5560f441f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.099+0000 7f95b37fe700 10 client.26251 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.099+0000 7f95d37fe700 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5560f441cd40 0x5560f441f160 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99058cef92bbb43c server_cookie=adfecd102e99364a in_seq=0 out_seq=0 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.100+0000 7f95b37fe700 1 -- 172.21.15.57:0/2627650261 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5560f434edb0 con 0x5560f441cd40 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95b37fe700 10 client.26251 handle_command_reply: tid=0 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 client.26251 shutdown 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2627650261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 msgr2=0x5560f440c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 0x5560f440c510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f95c40095d0 tx=0x7f95c4015730 comp rx=0 tx=0).stop 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2627650261 shutdown_connections 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5560f441cd40 0x5560f441f160 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f440cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44030b0 0x5560f440c510 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95b37fe700 0 client.26251 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/2627650261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f4408880 0x5560f440ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2627650261 >> 172.21.15.57:0/2627650261 conn(0x5560f4377200 msgr2=0x5560f43fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2627650261 shutdown_connections 2023-10-08T13:55:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2627650261 wait complete. 2023-10-08T13:55:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.101+0000 7f95e65f4b80 20 client.26251 trim_cache size 0 max 16384 2023-10-08T13:55:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.134+0000 7f95e65f4b80 1 Processor -- start 2023-10-08T13:55:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.134+0000 7f95e65f4b80 1 -- start start 2023-10-08T13:55:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.134+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 0x5560f4413be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.134+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f4408880 0x5560f440cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.134+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44091a0 0x5560f43fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.134+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f41d68c0 con 0x5560f44030b0 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.134+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f43fa830 con 0x5560f4408880 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.134+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f440db60 con 0x5560f44091a0 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 0x5560f4413be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f4408880 0x5560f440cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f4408880 0x5560f440cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59756/0 (socket says 172.21.15.57:59756) 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 0x5560f4413be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58328/0 (socket says 172.21.15.57:58328) 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d37fe700 1 -- 172.21.15.57:0/2124039564 learned_addr learned my addr 172.21.15.57:0/2124039564 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d89a8700 1 --2- 172.21.15.57:0/2124039564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44091a0 0x5560f43fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d37fe700 1 -- 172.21.15.57:0/2124039564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44091a0 msgr2=0x5560f43fd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d37fe700 1 --2- 172.21.15.57:0/2124039564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44091a0 0x5560f43fd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d37fe700 1 -- 172.21.15.57:0/2124039564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 msgr2=0x5560f4413be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d37fe700 1 --2- 172.21.15.57:0/2124039564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 0x5560f4413be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d37fe700 1 -- 172.21.15.57:0/2124039564 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560f41a71f0 con 0x5560f4408880 2023-10-08T13:55:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95d37fe700 1 --2- 172.21.15.57:0/2124039564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f4408880 0x5560f440cdb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f95d406c720 tx=0x7f95c8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990747c2f5d6b4e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95baffd700 1 -- 172.21.15.57:0/2124039564 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8017040 con 0x5560f4408880 2023-10-08T13:55:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95baffd700 1 -- 172.21.15.57:0/2124039564 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c8019570 con 0x5560f4408880 2023-10-08T13:55:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.135+0000 7f95baffd700 1 -- 172.21.15.57:0/2124039564 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8012440 con 0x5560f4408880 2023-10-08T13:55:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.136+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2124039564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f4408880 msgr2=0x5560f440cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.136+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/2124039564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f4408880 0x5560f440cdb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f95d406c720 tx=0x7f95c8009260 comp rx=0 tx=0).stop 2023-10-08T13:55:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.136+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2124039564 shutdown_connections 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.136+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/2124039564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f44091a0 0x5560f43fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.136+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/2124039564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 0x5560f4413be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.136+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/2124039564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f4408880 0x5560f440cdb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.136+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2124039564 >> 172.21.15.57:0/2124039564 conn(0x5560f4377200 msgr2=0x5560f4423a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.136+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2124039564 shutdown_connections 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.136+0000 7f95e65f4b80 1 -- 172.21.15.57:0/2124039564 wait complete. 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.137+0000 7f95e65f4b80 1 -- start start 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.137+0000 7f95e65f4b80 1 Processor -- start 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 0x5560f4404400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f4408880 0x5560f4404940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95e65f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f4404e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f44053c0 con 0x5560f44030b0 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f4405530 con 0x5560f44091a0 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95e65f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560f44056f0 con 0x5560f4408880 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95e65f4b80 10 client.0 _ll_register_callbacks cb 0x7f95c400e9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f4408880 0x5560f4404940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 0x5560f4404400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d89a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f4404e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f4408880 0x5560f4404940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59148/0 (socket says 172.21.15.57:59148) 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d89a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f4404e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59764/0 (socket says 172.21.15.57:59764) 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d37fe700 1 -- 172.21.15.57:0/3437893540 learned_addr learned my addr 172.21.15.57:0/3437893540 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d3fff700 1 -- 172.21.15.57:0/3437893540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f4408880 msgr2=0x5560f4404940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d3fff700 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f4408880 0x5560f4404940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d3fff700 1 -- 172.21.15.57:0/3437893540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 msgr2=0x5560f4404e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d3fff700 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f4404e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d3fff700 1 -- 172.21.15.57:0/3437893540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c8009db0 con 0x5560f44030b0 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d37fe700 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f4408880 0x5560f4404940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d89a8700 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f4404e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95d3fff700 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 0x5560f4404400 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x5560f441a630 tx=0x7f95c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3d517ff6717f59 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.138+0000 7f95b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95b37fe700 1 -- 172.21.15.57:0/3437893540 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4028b50 con 0x5560f44030b0 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95b37fe700 1 -- 172.21.15.57:0/3437893540 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f95c4013920 con 0x5560f44030b0 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3437893540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95c4015aa0 con 0x5560f44030b0 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95b37fe700 1 -- 172.21.15.57:0/3437893540 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95c4028620 con 0x5560f44030b0 2023-10-08T13:55:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95e65f4b80 10 client.16671 fetch_fsmap learned FSMap version 34 2023-10-08T13:55:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3437893540 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5560f417f250 con 0x5560f44030b0 2023-10-08T13:55:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95b37fe700 1 -- 172.21.15.57:0/3437893540 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c4015aa0 con 0x5560f44030b0 2023-10-08T13:55:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95b37fe700 1 -- 172.21.15.57:0/3437893540 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 34) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f95c4032b40 con 0x5560f44030b0 2023-10-08T13:55:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95e65f4b80 10 client.16671 fetch_fsmap finished waiting for FSMap version 34 2023-10-08T13:55:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95e65f4b80 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95e65f4b80 20 client.16671 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5560f4471350 0x5560f4473770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95e65f4b80 4 client.16671 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:55:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.139+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3437893540 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5560f434ead0 con 0x5560f4471350 2023-10-08T13:55:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.140+0000 7f95d37fe700 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5560f4471350 0x5560f4473770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.140+0000 7f95b37fe700 10 client.16671 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:39.140+0000 7f95d37fe700 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5560f4471350 0x5560f4473770 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171022dcec818431 server_cookie=4b8a0cfffa901615 in_seq=0 out_seq=0 2023-10-08T13:55:39.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:39 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:39.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:39 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:39.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:38 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:40.146 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:55:40.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:40 smithi115 ceph-mon[104408]: pgmap v422: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 386 op/s 2023-10-08T13:55:40.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:40 smithi099 ceph-mon[103849]: pgmap v422: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 386 op/s 2023-10-08T13:55:40.545 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:40 smithi057 ceph-mon[99476]: pgmap v422: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 386 op/s 2023-10-08T13:55:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.892+0000 7f95b37fe700 1 -- 172.21.15.57:0/3437893540 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5560f434ead0 con 0x5560f4471350 2023-10-08T13:55:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.892+0000 7f95b37fe700 10 client.16671 handle_command_reply: tid=0 2023-10-08T13:55:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.892+0000 7f95e65f4b80 1 client.16671 shutdown 2023-10-08T13:55:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.892+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3437893540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 msgr2=0x5560f4404400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.892+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 0x5560f4404400 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x5560f441a630 tx=0x7f95c400e4b0 comp rx=0 tx=0).stop 2023-10-08T13:55:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.893+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3437893540 shutdown_connections 2023-10-08T13:55:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.893+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5560f4471350 0x5560f4473770 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.893+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560f44091a0 0x5560f4404e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.893+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5560f44030b0 0x5560f4404400 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.893+0000 7f95e65f4b80 1 --2- 172.21.15.57:0/3437893540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5560f4408880 0x5560f4404940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.893+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3437893540 >> 172.21.15.57:0/3437893540 conn(0x5560f4377200 msgr2=0x5560f440ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.893+0000 7f95b37fe700 0 client.16671 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.893+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3437893540 shutdown_connections 2023-10-08T13:55:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.893+0000 7f95e65f4b80 1 -- 172.21.15.57:0/3437893540 wait complete. 2023-10-08T13:55:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.893+0000 7f95e65f4b80 20 client.16671 trim_cache size 0 max 16384 2023-10-08T13:55:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:55:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:55:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:55:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:55:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.894+0000 7f95da40b700 1 -- 172.21.15.57:0/65888114 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f95bc0a16b0 msgr2=0x7f95bc0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.894+0000 7f95da40b700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f95bc0a16b0 0x7f95bc0a3b70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009ad0 tx=0x7f95c4009290 comp rx=0 tx=0).stop 2023-10-08T13:55:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.894+0000 7f95da40b700 1 -- 172.21.15.57:0/65888114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4119fd0 msgr2=0x7f95d41117a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.894+0000 7f95da40b700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4119fd0 0x7f95d41117a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b770 tx=0x7f95cc00bd40 comp rx=0 tx=0).stop 2023-10-08T13:55:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.895+0000 7f95da40b700 1 -- 172.21.15.57:0/65888114 shutdown_connections 2023-10-08T13:55:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.895+0000 7f95da40b700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f95bc0a16b0 0x7f95bc0a3b70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.895+0000 7f95da40b700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4119fd0 0x7f95d41117a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.895+0000 7f95da40b700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f95d410ef10 0x7f95d41184a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.895+0000 7f95da40b700 1 --2- 172.21.15.57:0/65888114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d410f380 0x7f95d4118a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.895+0000 7f95da40b700 1 -- 172.21.15.57:0/65888114 >> 172.21.15.57:0/65888114 conn(0x7f95d406b7c0 msgr2=0x7f95d406f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.895+0000 7f95da40b700 1 -- 172.21.15.57:0/65888114 shutdown_connections 2023-10-08T13:55:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:40.895+0000 7f95da40b700 1 -- 172.21.15.57:0/65888114 wait complete. 2023-10-08T13:55:41.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.104+0000 7f7975072700 1 -- 172.21.15.57:0/644411675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7970112740 msgr2=0x7f7970114b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:41.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.104+0000 7f7975072700 1 --2- 172.21.15.57:0/644411675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7970112740 0x7f7970114b30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f796000faf0 tx=0x7f796000fe00 comp rx=0 tx=0).stop 2023-10-08T13:55:41.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.104+0000 7f7975072700 1 -- 172.21.15.57:0/644411675 shutdown_connections 2023-10-08T13:55:41.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.104+0000 7f7975072700 1 --2- 172.21.15.57:0/644411675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7970112740 0x7f7970114b30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.104+0000 7f7975072700 1 --2- 172.21.15.57:0/644411675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7970109190 0x7f7970109570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.104+0000 7f7975072700 1 --2- 172.21.15.57:0/644411675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7970109ab0 0x7f7970112200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.104+0000 7f7975072700 1 -- 172.21.15.57:0/644411675 >> 172.21.15.57:0/644411675 conn(0x7f797006b830 msgr2=0x7f797006bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:41.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.105+0000 7f7975072700 1 -- 172.21.15.57:0/644411675 shutdown_connections 2023-10-08T13:55:41.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.105+0000 7f7975072700 1 -- 172.21.15.57:0/644411675 wait complete. 2023-10-08T13:55:41.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.106+0000 7f7975072700 1 Processor -- start 2023-10-08T13:55:41.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.106+0000 7f7975072700 1 -- start start 2023-10-08T13:55:41.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.106+0000 7f7975072700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7970109190 0x7f797019d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.106+0000 7f7975072700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7970109ab0 0x7f797019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.106+0000 7f7975072700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7970112740 0x7f79701a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.106+0000 7f7975072700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79701a2140 con 0x7f7970109ab0 2023-10-08T13:55:41.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.106+0000 7f7975072700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79701a22b0 con 0x7f7970109190 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.106+0000 7f7975072700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79701a2470 con 0x7f7970112740 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f796ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7970109190 0x7f797019d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f796ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7970109190 0x7f797019d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59774/0 (socket says 172.21.15.57:59774) 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f796ed9d700 1 -- 172.21.15.57:0/1112882426 learned_addr learned my addr 172.21.15.57:0/1112882426 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f796f59e700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7970112740 0x7f79701a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f796ed9d700 1 -- 172.21.15.57:0/1112882426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7970112740 msgr2=0x7f79701a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f796ed9d700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7970112740 0x7f79701a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f796ed9d700 1 -- 172.21.15.57:0/1112882426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7970109ab0 msgr2=0x7f797019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f796ed9d700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7970109ab0 0x7f797019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f796ed9d700 1 -- 172.21.15.57:0/1112882426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796800bc60 con 0x7f7970109190 2023-10-08T13:55:41.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f796ed9d700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7970109190 0x7f797019d270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f796800f5a0 tx=0x7f796800f8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:41.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.107+0000 7f7957fff700 1 -- 172.21.15.57:0/1112882426 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968019070 con 0x7f7970109190 2023-10-08T13:55:41.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.108+0000 7f7975072700 1 -- 172.21.15.57:0/1112882426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796000f750 con 0x7f7970109190 2023-10-08T13:55:41.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.108+0000 7f7975072700 1 -- 172.21.15.57:0/1112882426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79701badc0 con 0x7f7970109190 2023-10-08T13:55:41.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.108+0000 7f7957fff700 1 -- 172.21.15.57:0/1112882426 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f796800b2e0 con 0x7f7970109190 2023-10-08T13:55:41.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.108+0000 7f7957fff700 1 -- 172.21.15.57:0/1112882426 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968010040 con 0x7f7970109190 2023-10-08T13:55:41.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.110+0000 7f7957fff700 1 -- 172.21.15.57:0/1112882426 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7968004020 con 0x7f7970109190 2023-10-08T13:55:41.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.110+0000 7f7957fff700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79580a1670 0x7f79580a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.111+0000 7f7957fff700 1 -- 172.21.15.57:0/1112882426 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f79680ce2d0 con 0x7f7970109190 2023-10-08T13:55:41.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.113+0000 7f798125bb80 1 Processor -- start 2023-10-08T13:55:41.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.113+0000 7f798125bb80 1 -- start start 2023-10-08T13:55:41.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f798125bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad932fe80 0x558ad9330260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f798125bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad93307a0 0x558ad933b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f798125bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad933b650 0x558ad933da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f798125bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad9299460 con 0x558ad932fe80 2023-10-08T13:55:41.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f798125bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad928ac20 con 0x558ad93307a0 2023-10-08T13:55:41.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f798125bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad9288540 con 0x558ad933b650 2023-10-08T13:55:41.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796e59c700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79580a1670 0x7f79580a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad933b650 0x558ad933da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad933b650 0x558ad933da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59168/0 (socket says 172.21.15.57:59168) 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796f59e700 1 -- 172.21.15.57:0/2277237678 learned_addr learned my addr 172.21.15.57:0/2277237678 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796e59c700 1 --2- 172.21.15.57:0/2277237678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad93307a0 0x558ad933b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796f59e700 1 -- 172.21.15.57:0/2277237678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad93307a0 msgr2=0x558ad933b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796f59e700 1 --2- 172.21.15.57:0/2277237678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad93307a0 0x558ad933b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796f59e700 1 -- 172.21.15.57:0/2277237678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad932fe80 msgr2=0x558ad9330260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796f59e700 1 --2- 172.21.15.57:0/2277237678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad932fe80 0x558ad9330260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796f59e700 1 -- 172.21.15.57:0/2277237678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ad90aede0 con 0x558ad933b650 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.114+0000 7f796e59c700 1 --2- 172.21.15.57:0/2277237678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad93307a0 0x558ad933b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.115+0000 7f796f59e700 1 --2- 172.21.15.57:0/2277237678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad933b650 0x558ad933da40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7960014000 tx=0x7f796000ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12c97bf388abc3cf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.115+0000 7f79557fa700 1 -- 172.21.15.57:0/2277237678 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796001e040 con 0x558ad933b650 2023-10-08T13:55:41.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.115+0000 7f79557fa700 1 -- 172.21.15.57:0/2277237678 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7960023480 con 0x558ad933b650 2023-10-08T13:55:41.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.115+0000 7f79557fa700 1 -- 172.21.15.57:0/2277237678 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960018650 con 0x558ad933b650 2023-10-08T13:55:41.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.116+0000 7f798125bb80 1 -- 172.21.15.57:0/2277237678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad933b650 msgr2=0x558ad933da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:41.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.116+0000 7f798125bb80 1 --2- 172.21.15.57:0/2277237678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad933b650 0x558ad933da40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7960014000 tx=0x7f796000ff90 comp rx=0 tx=0).stop 2023-10-08T13:55:41.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.116+0000 7f798125bb80 1 -- 172.21.15.57:0/2277237678 shutdown_connections 2023-10-08T13:55:41.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.116+0000 7f798125bb80 1 --2- 172.21.15.57:0/2277237678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad933b650 0x558ad933da40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.116+0000 7f798125bb80 1 --2- 172.21.15.57:0/2277237678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad932fe80 0x558ad9330260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.116+0000 7f798125bb80 1 --2- 172.21.15.57:0/2277237678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad93307a0 0x558ad933b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.116+0000 7f798125bb80 1 -- 172.21.15.57:0/2277237678 >> 172.21.15.57:0/2277237678 conn(0x558ad92a5260 msgr2=0x558ad932c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:41.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.117+0000 7f798125bb80 1 -- 172.21.15.57:0/2277237678 shutdown_connections 2023-10-08T13:55:41.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.118+0000 7f798125bb80 1 -- 172.21.15.57:0/2277237678 wait complete. 2023-10-08T13:55:41.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.118+0000 7f796e59c700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79580a1670 0x7f79580a3b30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7964005e90 tx=0x7f796400a3a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:41.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.118+0000 7f798125bb80 1 -- start start 2023-10-08T13:55:41.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.119+0000 7f798125bb80 1 Processor -- start 2023-10-08T13:55:41.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.119+0000 7f798125bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad932fe80 0x558ad93385b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.119+0000 7f798125bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad93307a0 0x558ad9338af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.119+0000 7f798125bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad933b650 0x558ad9339030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.119+0000 7f798125bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad9283e30 con 0x558ad933b650 2023-10-08T13:55:41.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.119+0000 7f798125bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad9220710 con 0x558ad93307a0 2023-10-08T13:55:41.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.119+0000 7f798125bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad92203b0 con 0x558ad932fe80 2023-10-08T13:55:41.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.119+0000 7f798125bb80 10 client.0 _ll_register_callbacks cb 0x558ad907d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:41.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.120+0000 7f796f59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad933b650 0x558ad9339030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.120+0000 7f796f59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad933b650 0x558ad9339030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58366/0 (socket says 172.21.15.57:58366) 2023-10-08T13:55:41.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.120+0000 7f796f59e700 1 -- 172.21.15.57:0/1763910476 learned_addr learned my addr 172.21.15.57:0/1763910476 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:41.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.122+0000 7f796ed9d700 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad932fe80 0x558ad93385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.123+0000 7f796ed9d700 1 -- 172.21.15.57:0/1763910476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad93307a0 msgr2=0x558ad9338af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:55:41.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.123+0000 7f796ed9d700 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad93307a0 0x558ad9338af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.123+0000 7f796ed9d700 1 -- 172.21.15.57:0/1763910476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad933b650 msgr2=0x558ad9339030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:41.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.123+0000 7f796ed9d700 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad933b650 0x558ad9339030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.123+0000 7f796ed9d700 1 -- 172.21.15.57:0/1763910476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79701155c0 con 0x558ad932fe80 2023-10-08T13:55:41.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.123+0000 7f796ed9d700 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad932fe80 0x558ad93385b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79680cb720 tx=0x7f796809aa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d418c2e9fb00941 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:41.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.123+0000 7f7952ff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:55:41.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.123+0000 7f7952ff5700 1 -- 172.21.15.57:0/1763910476 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79680ce170 con 0x558ad932fe80 2023-10-08T13:55:41.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.123+0000 7f7952ff5700 1 -- 172.21.15.57:0/1763910476 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f796809f190 con 0x558ad932fe80 2023-10-08T13:55:41.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.123+0000 7f798125bb80 1 -- 172.21.15.57:0/1763910476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ad924beb0 con 0x558ad932fe80 2023-10-08T13:55:41.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.140+0000 7f7952ff5700 1 -- 172.21.15.57:0/1763910476 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7968009830 con 0x558ad932fe80 2023-10-08T13:55:41.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.140+0000 7f798125bb80 10 client.26269 fetch_fsmap learned FSMap version 34 2023-10-08T13:55:41.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.140+0000 7f798125bb80 1 -- 172.21.15.57:0/1763910476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558ad9097c60 con 0x558ad932fe80 2023-10-08T13:55:41.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.151+0000 7f7952ff5700 1 -- 172.21.15.57:0/1763910476 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79680b7020 con 0x558ad932fe80 2023-10-08T13:55:41.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.151+0000 7f7952ff5700 1 -- 172.21.15.57:0/1763910476 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f796809a530 con 0x558ad932fe80 2023-10-08T13:55:41.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.151+0000 7f798125bb80 10 client.26269 fetch_fsmap finished waiting for FSMap version 34 2023-10-08T13:55:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.151+0000 7f798125bb80 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.151+0000 7f798125bb80 20 client.26269 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.151+0000 7f798125bb80 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558ad934b2c0 0x558ad934d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.151+0000 7f798125bb80 4 client.26269 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:55:41.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.151+0000 7f798125bb80 1 -- 172.21.15.57:0/1763910476 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ad927ce10 con 0x558ad934b2c0 2023-10-08T13:55:41.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.171+0000 7f796e59c700 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558ad934b2c0 0x558ad934d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.186+0000 7f796e59c700 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558ad934b2c0 0x558ad934d6e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da71cf0c61dcafac server_cookie=ce444dcd939209f8 in_seq=0 out_seq=0 2023-10-08T13:55:41.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.186+0000 7f7952ff5700 10 client.26269 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.189+0000 7f7952ff5700 1 -- 172.21.15.57:0/1763910476 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558ad927ce10 con 0x558ad934b2c0 2023-10-08T13:55:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.189+0000 7f7952ff5700 10 client.26269 handle_command_reply: tid=0 2023-10-08T13:55:41.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.189+0000 7f798125bb80 1 client.26269 shutdown 2023-10-08T13:55:41.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.542+0000 7f798125bb80 1 -- 172.21.15.57:0/1763910476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad932fe80 msgr2=0x558ad93385b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:41.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.542+0000 7f798125bb80 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad932fe80 0x558ad93385b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79680cb720 tx=0x7f796809aa50 comp rx=0 tx=0).stop 2023-10-08T13:55:41.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.543+0000 7f798125bb80 1 -- 172.21.15.57:0/1763910476 shutdown_connections 2023-10-08T13:55:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.543+0000 7f798125bb80 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558ad934b2c0 0x558ad934d6e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.543+0000 7f798125bb80 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad933b650 0x558ad9339030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.543+0000 7f798125bb80 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad932fe80 0x558ad93385b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.543+0000 7f798125bb80 1 --2- 172.21.15.57:0/1763910476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad93307a0 0x558ad9338af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.543+0000 7f798125bb80 1 -- 172.21.15.57:0/1763910476 >> 172.21.15.57:0/1763910476 conn(0x558ad92a5260 msgr2=0x558ad932bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.543+0000 7f7952ff5700 0 client.26269 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:41.544 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.544+0000 7f798125bb80 1 -- 172.21.15.57:0/1763910476 shutdown_connections 2023-10-08T13:55:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.544+0000 7f798125bb80 1 -- 172.21.15.57:0/1763910476 wait complete. 2023-10-08T13:55:41.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.545+0000 7f798125bb80 20 client.26269 trim_cache size 0 max 16384 2023-10-08T13:55:41.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.751+0000 7f798125bb80 1 Processor -- start 2023-10-08T13:55:41.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.752+0000 7f798125bb80 1 -- start start 2023-10-08T13:55:41.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.752+0000 7f798125bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad92a5260 0x558ad9342230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.753+0000 7f798125bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad932bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.753+0000 7f798125bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad93307a0 0x558ad93338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.753+0000 7f798125bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad9104920 con 0x558ad92a5260 2023-10-08T13:55:41.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.753+0000 7f798125bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad9328ae0 con 0x558ad932fe80 2023-10-08T13:55:41.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.753+0000 7f798125bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad9347ee0 con 0x558ad93307a0 2023-10-08T13:55:41.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.753+0000 7f796e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad932bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.753+0000 7f796e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad932bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59798/0 (socket says 172.21.15.57:59798) 2023-10-08T13:55:41.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.753+0000 7f796e59c700 1 -- 172.21.15.57:0/28415611 learned_addr learned my addr 172.21.15.57:0/28415611 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:41.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.754+0000 7f796f59e700 1 --2- 172.21.15.57:0/28415611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad93307a0 0x558ad93338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.756+0000 7f796ed9d700 1 --2- 172.21.15.57:0/28415611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad92a5260 0x558ad9342230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.758+0000 7f796ed9d700 1 -- 172.21.15.57:0/28415611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad93307a0 msgr2=0x558ad93338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:41.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.758+0000 7f796ed9d700 1 --2- 172.21.15.57:0/28415611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad93307a0 0x558ad93338f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.758+0000 7f796ed9d700 1 -- 172.21.15.57:0/28415611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 msgr2=0x558ad932bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:41.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.758+0000 7f796ed9d700 1 --2- 172.21.15.57:0/28415611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad932bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.758+0000 7f796ed9d700 1 -- 172.21.15.57:0/28415611 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ad90d5250 con 0x558ad92a5260 2023-10-08T13:55:41.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.760+0000 7f796ed9d700 1 --2- 172.21.15.57:0/28415611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad92a5260 0x558ad9342230 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7968097ab0 tx=0x7f7968009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a970bcbfaecdcd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:41.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.767+0000 7f7955ffb700 1 -- 172.21.15.57:0/28415611 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79680ce170 con 0x558ad92a5260 2023-10-08T13:55:41.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.767+0000 7f7955ffb700 1 -- 172.21.15.57:0/28415611 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79680b7030 con 0x558ad92a5260 2023-10-08T13:55:41.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.767+0000 7f7955ffb700 1 -- 172.21.15.57:0/28415611 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79680a3340 con 0x558ad92a5260 2023-10-08T13:55:41.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.768+0000 7f798125bb80 1 -- 172.21.15.57:0/28415611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad92a5260 msgr2=0x558ad9342230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:41.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.768+0000 7f798125bb80 1 --2- 172.21.15.57:0/28415611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad92a5260 0x558ad9342230 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7968097ab0 tx=0x7f7968009f90 comp rx=0 tx=0).stop 2023-10-08T13:55:41.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.779+0000 7f798125bb80 1 -- 172.21.15.57:0/28415611 shutdown_connections 2023-10-08T13:55:41.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.779+0000 7f798125bb80 1 --2- 172.21.15.57:0/28415611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad93307a0 0x558ad93338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.779+0000 7f798125bb80 1 --2- 172.21.15.57:0/28415611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad92a5260 0x558ad9342230 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.779+0000 7f798125bb80 1 --2- 172.21.15.57:0/28415611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad932bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.779+0000 7f798125bb80 1 -- 172.21.15.57:0/28415611 >> 172.21.15.57:0/28415611 conn(0x558ad93282a0 msgr2=0x558ad934d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:41.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.780+0000 7f798125bb80 1 -- 172.21.15.57:0/28415611 shutdown_connections 2023-10-08T13:55:41.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.780+0000 7f798125bb80 1 -- 172.21.15.57:0/28415611 wait complete. 2023-10-08T13:55:41.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.780+0000 7f798125bb80 1 -- start start 2023-10-08T13:55:41.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.780+0000 7f798125bb80 1 Processor -- start 2023-10-08T13:55:41.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f798125bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad92a5260 0x558ad9340400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f798125bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad9340940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f798125bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad93307a0 0x558ad9398080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f798125bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad9398630 con 0x558ad92a5260 2023-10-08T13:55:41.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f798125bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad93987a0 con 0x558ad932fe80 2023-10-08T13:55:41.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f798125bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad9398960 con 0x558ad93307a0 2023-10-08T13:55:41.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f798125bb80 10 client.0 _ll_register_callbacks cb 0x7f796809ae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:41.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f796e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad9340940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f796e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad9340940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59802/0 (socket says 172.21.15.57:59802) 2023-10-08T13:55:41.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f796e59c700 1 -- 172.21.15.57:0/2323909189 learned_addr learned my addr 172.21.15.57:0/2323909189 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:41.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.781+0000 7f796f59e700 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad93307a0 0x558ad9398080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.782+0000 7f796e59c700 1 -- 172.21.15.57:0/2323909189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad93307a0 msgr2=0x558ad9398080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:41.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.782+0000 7f796e59c700 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad93307a0 0x558ad9398080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.782+0000 7f796e59c700 1 -- 172.21.15.57:0/2323909189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad92a5260 msgr2=0x558ad9340400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:41.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.782+0000 7f796e59c700 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad92a5260 0x558ad9340400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.782+0000 7f796e59c700 1 -- 172.21.15.57:0/2323909189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ad9097c60 con 0x558ad932fe80 2023-10-08T13:55:41.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.782+0000 7f796e59c700 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad9340940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7964005fd0 tx=0x7f7964005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f4c779fed533f12 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:41.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.803+0000 7f7952ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:55:41.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.803+0000 7f7952ff5700 1 -- 172.21.15.57:0/2323909189 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79640212c0 con 0x558ad932fe80 2023-10-08T13:55:41.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.803+0000 7f7952ff5700 1 -- 172.21.15.57:0/2323909189 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f7964016040 con 0x558ad932fe80 2023-10-08T13:55:41.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.804+0000 7f798125bb80 1 -- 172.21.15.57:0/2323909189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7960023480 con 0x558ad932fe80 2023-10-08T13:55:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.814+0000 7f7952ff5700 1 -- 172.21.15.57:0/2323909189 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7964021420 con 0x558ad932fe80 2023-10-08T13:55:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.814+0000 7f798125bb80 10 client.26216 fetch_fsmap learned FSMap version 34 2023-10-08T13:55:41.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.814+0000 7f798125bb80 1 -- 172.21.15.57:0/2323909189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558ad90ad2b0 con 0x558ad932fe80 2023-10-08T13:55:41.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.824+0000 7f7952ff5700 1 -- 172.21.15.57:0/2323909189 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7960023480 con 0x558ad932fe80 2023-10-08T13:55:41.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.824+0000 7f7952ff5700 1 -- 172.21.15.57:0/2323909189 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f79640053e0 con 0x558ad932fe80 2023-10-08T13:55:41.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.825+0000 7f798125bb80 10 client.26216 fetch_fsmap finished waiting for FSMap version 34 2023-10-08T13:55:41.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.825+0000 7f798125bb80 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:41.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.825+0000 7f798125bb80 20 client.26216 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:41.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.825+0000 7f798125bb80 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558ad933b650 0x558ad93a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:41.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.825+0000 7f798125bb80 4 client.26216 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:55:41.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.825+0000 7f798125bb80 1 -- 172.21.15.57:0/2323909189 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558ad927cb30 con 0x558ad933b650 2023-10-08T13:55:41.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.826+0000 7f796ed9d700 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558ad933b650 0x558ad93a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:41.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.840+0000 7f796ed9d700 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558ad933b650 0x558ad93a2390 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b39f4cc670588a51 server_cookie=8f7fe595ed767e19 in_seq=0 out_seq=0 2023-10-08T13:55:41.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.840+0000 7f7952ff5700 10 client.26216 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:41.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.845+0000 7f7952ff5700 1 -- 172.21.15.57:0/2323909189 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558ad927cb30 con 0x558ad933b650 2023-10-08T13:55:41.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.845+0000 7f7952ff5700 10 client.26216 handle_command_reply: tid=0 2023-10-08T13:55:41.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.846+0000 7f798125bb80 1 client.26216 shutdown 2023-10-08T13:55:41.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.856+0000 7f798125bb80 1 -- 172.21.15.57:0/2323909189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 msgr2=0x558ad9340940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:41.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.856+0000 7f798125bb80 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad9340940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7964005fd0 tx=0x7f7964005f60 comp rx=0 tx=0).stop 2023-10-08T13:55:41.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.867+0000 7f798125bb80 1 -- 172.21.15.57:0/2323909189 shutdown_connections 2023-10-08T13:55:41.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.867+0000 7f798125bb80 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558ad933b650 0x558ad93a2390 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.867+0000 7f798125bb80 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ad93307a0 0x558ad9398080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.867+0000 7f798125bb80 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558ad92a5260 0x558ad9340400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.867+0000 7f798125bb80 1 --2- 172.21.15.57:0/2323909189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ad932fe80 0x558ad9340940 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.867+0000 7f798125bb80 1 -- 172.21.15.57:0/2323909189 >> 172.21.15.57:0/2323909189 conn(0x558ad93282a0 msgr2=0x558ad932bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:41.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.868+0000 7f7952ff5700 0 client.26216 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:41.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.868+0000 7f798125bb80 1 -- 172.21.15.57:0/2323909189 shutdown_connections 2023-10-08T13:55:41.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.868+0000 7f798125bb80 1 -- 172.21.15.57:0/2323909189 wait complete. 2023-10-08T13:55:41.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.868+0000 7f798125bb80 20 client.26216 trim_cache size 0 max 16384 2023-10-08T13:55:41.869 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:55:41.870 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T13:55:41.870 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:55:41.870 INFO:teuthology.orchestra.run.smithi057.stdout: "6db03f06-e888-4427-a580-5fa3643c8e0d": { 2023-10-08T13:55:41.871 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:55:41.871 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "6db03f06-e888-4427-a580-5fa3643c8e0d", 2023-10-08T13:55:41.871 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:55:41.871 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:55:41.871 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:55:41.871 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:55:41.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.894+0000 7f7975072700 1 -- 172.21.15.57:0/1112882426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79580a1670 msgr2=0x7f79580a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:41.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.894+0000 7f7975072700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79580a1670 0x7f79580a3b30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7964005e90 tx=0x7f796400a3a0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.894+0000 7f7975072700 1 -- 172.21.15.57:0/1112882426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7970109190 msgr2=0x7f797019d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:41.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.894+0000 7f7975072700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7970109190 0x7f797019d270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f796800f5a0 tx=0x7f796800f8b0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.932+0000 7f7975072700 1 -- 172.21.15.57:0/1112882426 shutdown_connections 2023-10-08T13:55:41.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.932+0000 7f7975072700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7970112740 0x7f79701a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.932+0000 7f7975072700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f79580a1670 0x7f79580a3b30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.932+0000 7f7975072700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7970109190 0x7f797019d270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.932+0000 7f7975072700 1 --2- 172.21.15.57:0/1112882426 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7970109ab0 0x7f797019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:41.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.932+0000 7f7975072700 1 -- 172.21.15.57:0/1112882426 >> 172.21.15.57:0/1112882426 conn(0x7f797006b830 msgr2=0x7f797006f680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:41.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.932+0000 7f7975072700 1 -- 172.21.15.57:0/1112882426 shutdown_connections 2023-10-08T13:55:41.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:55:41.932+0000 7f7975072700 1 -- 172.21.15.57:0/1112882426 wait complete. 2023-10-08T13:55:42.247 INFO:tasks.cephfs.filesystem:scrub status for tag:6db03f06-e888-4427-a580-5fa3643c8e0d - {'path': '/', 'tag': '6db03f06-e888-4427-a580-5fa3643c8e0d', 'options': 'recursive,force'} 2023-10-08T13:55:42.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:42 smithi115 ceph-mon[104408]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 402 op/s 2023-10-08T13:55:42.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:42 smithi057 ceph-mon[99476]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 402 op/s 2023-10-08T13:55:42.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:42 smithi099 ceph-mon[103849]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 402 op/s 2023-10-08T13:55:44.379 INFO:tasks.workunit.client.0.smithi057.stdout: 307 1932130 675 1383012 677 299226 224 2023-10-08T13:55:44.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:44 smithi115 ceph-mon[104408]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 315 op/s 2023-10-08T13:55:44.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:44 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:44.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:44 smithi057 ceph-mon[99476]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 315 op/s 2023-10-08T13:55:44.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:44 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:44.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:44 smithi099 ceph-mon[103849]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 315 op/s 2023-10-08T13:55:44.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:44 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:46.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:46 smithi115 ceph-mon[104408]: pgmap v425: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 506 op/s 2023-10-08T13:55:46.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:46 smithi057 ceph-mon[99476]: pgmap v425: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 506 op/s 2023-10-08T13:55:46.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:46 smithi099 ceph-mon[103849]: pgmap v425: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 506 op/s 2023-10-08T13:55:48.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:48 smithi115 ceph-mon[104408]: pgmap v426: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 503 op/s 2023-10-08T13:55:48.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:48 smithi057 ceph-mon[99476]: pgmap v426: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 503 op/s 2023-10-08T13:55:48.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:48 smithi099 ceph-mon[103849]: pgmap v426: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 503 op/s 2023-10-08T13:55:50.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:50 smithi057 ceph-mon[99476]: pgmap v427: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.1 MiB/s wr, 486 op/s 2023-10-08T13:55:50.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:50 smithi099 ceph-mon[103849]: pgmap v427: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.1 MiB/s wr, 486 op/s 2023-10-08T13:55:50.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:50 smithi115 ceph-mon[104408]: pgmap v427: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.1 MiB/s wr, 486 op/s 2023-10-08T13:55:52.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:52 smithi057 ceph-mon[99476]: pgmap v428: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 485 op/s 2023-10-08T13:55:52.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:52 smithi099 ceph-mon[103849]: pgmap v428: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 485 op/s 2023-10-08T13:55:52.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:52 smithi115 ceph-mon[104408]: pgmap v428: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 485 op/s 2023-10-08T13:55:54.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:53 smithi057 ceph-mon[99476]: pgmap v429: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.0 MiB/s wr, 398 op/s 2023-10-08T13:55:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:53 smithi099 ceph-mon[103849]: pgmap v429: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.0 MiB/s wr, 398 op/s 2023-10-08T13:55:54.391 INFO:tasks.workunit.client.0.smithi057.stdout: 314 2553539 551 1826419 385 397669 507 2023-10-08T13:55:54.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:53 smithi115 ceph-mon[104408]: pgmap v429: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.0 MiB/s wr, 398 op/s 2023-10-08T13:55:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:56 smithi099 ceph-mon[103849]: pgmap v430: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 869 KiB/s wr, 318 op/s 2023-10-08T13:55:56.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:56 smithi115 ceph-mon[104408]: pgmap v430: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 869 KiB/s wr, 318 op/s 2023-10-08T13:55:56.510 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:56 smithi057 ceph-mon[99476]: pgmap v430: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 869 KiB/s wr, 318 op/s 2023-10-08T13:55:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.509+0000 7f624c774700 1 -- 172.21.15.57:0/99345050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6244119fd0 msgr2=0x7f624411c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.509+0000 7f624c774700 1 --2- 172.21.15.57:0/99345050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6244119fd0 0x7f624411c3c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f623c00b0d0 tx=0x7f623c00b3e0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.509+0000 7f624c774700 1 -- 172.21.15.57:0/99345050 shutdown_connections 2023-10-08T13:55:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.509+0000 7f624c774700 1 --2- 172.21.15.57:0/99345050 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6244119fd0 0x7f624411c3c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.509+0000 7f624c774700 1 --2- 172.21.15.57:0/99345050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f624410ef10 0x7f62441077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.509+0000 7f624c774700 1 --2- 172.21.15.57:0/99345050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f624410f380 0x7f6244107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.509+0000 7f624c774700 1 -- 172.21.15.57:0/99345050 >> 172.21.15.57:0/99345050 conn(0x7f624406b7c0 msgr2=0x7f624406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.509+0000 7f624c774700 1 -- 172.21.15.57:0/99345050 shutdown_connections 2023-10-08T13:55:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.509+0000 7f624c774700 1 -- 172.21.15.57:0/99345050 wait complete. 2023-10-08T13:55:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.510+0000 7f624c774700 1 Processor -- start 2023-10-08T13:55:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.510+0000 7f624c774700 1 -- start start 2023-10-08T13:55:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.510+0000 7f624c774700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f624410ef10 0x7f62441195d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.510+0000 7f624c774700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f624410f380 0x7f6244113620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.510+0000 7f624c774700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6244119fd0 0x7f6244113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.510+0000 7f624c774700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62441140a0 con 0x7f6244119fd0 2023-10-08T13:55:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.510+0000 7f624c774700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244114210 con 0x7f624410ef10 2023-10-08T13:55:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.510+0000 7f624c774700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244114380 con 0x7f624410f380 2023-10-08T13:55:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.526+0000 7f6249d0f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f624410f380 0x7f6244113620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.526+0000 7f6249d0f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f624410f380 0x7f6244113620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59206/0 (socket says 172.21.15.57:59206) 2023-10-08T13:55:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.526+0000 7f6249d0f700 1 -- 172.21.15.57:0/3679248925 learned_addr learned my addr 172.21.15.57:0/3679248925 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.526+0000 7f624a510700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f624410ef10 0x7f62441195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.535+0000 7f624ad11700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6244119fd0 0x7f6244113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.544+0000 7f624a510700 1 -- 172.21.15.57:0/3679248925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f624410f380 msgr2=0x7f6244113620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.544+0000 7f624a510700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f624410f380 0x7f6244113620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.544+0000 7f624a510700 1 -- 172.21.15.57:0/3679248925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6244119fd0 msgr2=0x7f6244113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.544+0000 7f624a510700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6244119fd0 0x7f6244113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.544+0000 7f624a510700 1 -- 172.21.15.57:0/3679248925 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c009d00 con 0x7f624410ef10 2023-10-08T13:55:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.555+0000 7f624a510700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f624410ef10 0x7f62441195d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f624000d950 tx=0x7f624000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.709+0000 7f623b7fe700 1 -- 172.21.15.57:0/3679248925 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240009930 con 0x7f624410ef10 2023-10-08T13:55:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.709+0000 7f623b7fe700 1 -- 172.21.15.57:0/3679248925 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6240010460 con 0x7f624410ef10 2023-10-08T13:55:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.709+0000 7f623b7fe700 1 -- 172.21.15.57:0/3679248925 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624000b5d0 con 0x7f624410ef10 2023-10-08T13:55:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.709+0000 7f624c774700 1 -- 172.21.15.57:0/3679248925 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62441bab90 con 0x7f624410ef10 2023-10-08T13:55:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.709+0000 7f624c774700 1 -- 172.21.15.57:0/3679248925 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62441bb020 con 0x7f624410ef10 2023-10-08T13:55:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.716+0000 7f625895db80 1 Processor -- start 2023-10-08T13:55:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.721+0000 7f623b7fe700 1 -- 172.21.15.57:0/3679248925 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f624000f620 con 0x7f624410ef10 2023-10-08T13:55:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.722+0000 7f623b7fe700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f62300a1800 0x7f62300a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.722+0000 7f623b7fe700 1 -- 172.21.15.57:0/3679248925 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f62400cdbc0 con 0x7f624410ef10 2023-10-08T13:55:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.740+0000 7f625895db80 1 -- start start 2023-10-08T13:55:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.741+0000 7f625895db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 0x55e1a37ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.741+0000 7f625895db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.741+0000 7f625895db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a37fa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.741+0000 7f625895db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a3762400 con 0x55e1a37ff880 2023-10-08T13:55:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.741+0000 7f625895db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a3753bc0 con 0x55e1a37fa0b0 2023-10-08T13:55:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.741+0000 7f625895db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a37514e0 con 0x55e1a38001a0 2023-10-08T13:55:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.741+0000 7f6249d0f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.742+0000 7f6249d0f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59214/0 (socket says 172.21.15.57:59214) 2023-10-08T13:55:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.742+0000 7f6249d0f700 1 -- 172.21.15.57:0/2686536882 learned_addr learned my addr 172.21.15.57:0/2686536882 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.763+0000 7f6249d0f700 1 -- 172.21.15.57:0/2686536882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 msgr2=0x55e1a37fa510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:55:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.763+0000 7f6249d0f700 1 --2- 172.21.15.57:0/2686536882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a37fa510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.763+0000 7f6249d0f700 1 -- 172.21.15.57:0/2686536882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 msgr2=0x55e1a37ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.763+0000 7f6249d0f700 1 --2- 172.21.15.57:0/2686536882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 0x55e1a37ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.763+0000 7f6249d0f700 1 -- 172.21.15.57:0/2686536882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1a3577d80 con 0x55e1a38001a0 2023-10-08T13:55:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.771+0000 7f624ad11700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f62300a1800 0x7f62300a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.775+0000 7f6249d0f700 1 --2- 172.21.15.57:0/2686536882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f9b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f623400bcc0 tx=0x7f62340103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=599d4c8be5a526fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.784+0000 7f6238ff9700 1 -- 172.21.15.57:0/2686536882 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234010cd0 con 0x55e1a38001a0 2023-10-08T13:55:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.784+0000 7f6238ff9700 1 -- 172.21.15.57:0/2686536882 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6234015070 con 0x55e1a38001a0 2023-10-08T13:55:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.784+0000 7f6238ff9700 1 -- 172.21.15.57:0/2686536882 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623401a5e0 con 0x55e1a38001a0 2023-10-08T13:55:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.790+0000 7f625895db80 1 -- 172.21.15.57:0/2686536882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 msgr2=0x55e1a37f9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.790+0000 7f625895db80 1 --2- 172.21.15.57:0/2686536882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f9b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f623400bcc0 tx=0x7f62340103d0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.853+0000 7f624ad11700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f62300a1800 0x7f62300a3cc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f623c00d010 tx=0x7f623c00b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.938+0000 7f625895db80 1 -- 172.21.15.57:0/2686536882 shutdown_connections 2023-10-08T13:55:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.938+0000 7f625895db80 1 --2- 172.21.15.57:0/2686536882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a37fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.938+0000 7f625895db80 1 --2- 172.21.15.57:0/2686536882 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 0x55e1a37ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.938+0000 7f625895db80 1 --2- 172.21.15.57:0/2686536882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f9b70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.938+0000 7f625895db80 1 -- 172.21.15.57:0/2686536882 >> 172.21.15.57:0/2686536882 conn(0x55e1a376e200 msgr2=0x55e1a37f4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.938+0000 7f625895db80 1 -- 172.21.15.57:0/2686536882 shutdown_connections 2023-10-08T13:55:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.938+0000 7f625895db80 1 -- 172.21.15.57:0/2686536882 wait complete. 2023-10-08T13:55:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.939+0000 7f625895db80 1 -- start start 2023-10-08T13:55:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:56.939+0000 7f625895db80 1 Processor -- start 2023-10-08T13:55:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.087+0000 7f625895db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a3803600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.087+0000 7f625895db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 0x55e1a3803b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.087+0000 7f625895db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a3804080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.087+0000 7f625895db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a374cdd0 con 0x55e1a37ff880 2023-10-08T13:55:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.087+0000 7f625895db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a36e96b0 con 0x55e1a37fa0b0 2023-10-08T13:55:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.088+0000 7f625895db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a36e9350 con 0x55e1a38001a0 2023-10-08T13:55:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.088+0000 7f625895db80 10 client.0 _ll_register_callbacks cb 0x55e1a35467a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.095+0000 7f624a510700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a3803600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.095+0000 7f624a510700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a3803600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59830/0 (socket says 172.21.15.57:59830) 2023-10-08T13:55:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.095+0000 7f624a510700 1 -- 172.21.15.57:0/2101033868 learned_addr learned my addr 172.21.15.57:0/2101033868 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.095+0000 7f624ad11700 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a3804080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.096+0000 7f624a510700 1 -- 172.21.15.57:0/2101033868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 msgr2=0x55e1a3804080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.096+0000 7f624a510700 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a3804080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.096+0000 7f624a510700 1 -- 172.21.15.57:0/2101033868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 msgr2=0x55e1a3803b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.096+0000 7f624a510700 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 0x55e1a3803b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.096+0000 7f624a510700 1 -- 172.21.15.57:0/2101033868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623400b8f0 con 0x55e1a37fa0b0 2023-10-08T13:55:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.118+0000 7f624a510700 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a3803600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f624009a540 tx=0x7f624009b030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1088845a2330a2a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.390+0000 7f622dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:55:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.390+0000 7f622dffb700 1 -- 172.21.15.57:0/2101033868 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624009aca0 con 0x55e1a37fa0b0 2023-10-08T13:55:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.390+0000 7f622dffb700 1 -- 172.21.15.57:0/2101033868 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f62400a2bd0 con 0x55e1a37fa0b0 2023-10-08T13:55:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.408+0000 7f625895db80 1 -- 172.21.15.57:0/2101033868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1a3714e50 con 0x55e1a37fa0b0 2023-10-08T13:55:57.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:57 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.442+0000 7f622dffb700 1 -- 172.21.15.57:0/2101033868 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6240099d30 con 0x55e1a37fa0b0 2023-10-08T13:55:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.491+0000 7f625895db80 10 client.26225 fetch_fsmap learned FSMap version 36 2023-10-08T13:55:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.491+0000 7f625895db80 1 -- 172.21.15.57:0/2101033868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e1a3560c00 con 0x55e1a37fa0b0 2023-10-08T13:55:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.491+0000 7f622dffb700 1 -- 172.21.15.57:0/2101033868 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1a3714e50 con 0x55e1a37fa0b0 2023-10-08T13:55:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.491+0000 7f622dffb700 1 -- 172.21.15.57:0/2101033868 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 36) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f624009bb70 con 0x55e1a37fa0b0 2023-10-08T13:55:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.491+0000 7f625895db80 10 client.26225 fetch_fsmap finished waiting for FSMap version 36 2023-10-08T13:55:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.491+0000 7f625895db80 10 client.26225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.491+0000 7f625895db80 20 client.26225 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.492+0000 7f625895db80 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e1a3813eb0 0x55e1a38162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.492+0000 7f625895db80 4 client.26225 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:55:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.492+0000 7f625895db80 1 -- 172.21.15.57:0/2101033868 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1a3745db0 con 0x55e1a3813eb0 2023-10-08T13:55:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.492+0000 7f6249d0f700 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e1a3813eb0 0x55e1a38162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:57.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:57 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.492+0000 7f622dffb700 10 client.26225 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.516+0000 7f6249d0f700 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e1a3813eb0 0x55e1a38162d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2fddda885d3072e server_cookie=eed517eb94da33b1 in_seq=0 out_seq=0 2023-10-08T13:55:57.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:57 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:55:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.523+0000 7f622dffb700 1 -- 172.21.15.57:0/2101033868 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e1a3745db0 con 0x55e1a3813eb0 2023-10-08T13:55:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.523+0000 7f622dffb700 10 client.26225 handle_command_reply: tid=0 2023-10-08T13:55:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.527+0000 7f625895db80 1 client.26225 shutdown 2023-10-08T13:55:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.527+0000 7f625895db80 1 -- 172.21.15.57:0/2101033868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 msgr2=0x55e1a3803600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.527+0000 7f625895db80 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a3803600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f624009a540 tx=0x7f624009b030 comp rx=0 tx=0).stop 2023-10-08T13:55:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.527+0000 7f625895db80 1 -- 172.21.15.57:0/2101033868 shutdown_connections 2023-10-08T13:55:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.527+0000 7f625895db80 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e1a3813eb0 0x55e1a38162d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.527+0000 7f625895db80 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a3804080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.527+0000 7f625895db80 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a3803600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.527+0000 7f625895db80 1 --2- 172.21.15.57:0/2101033868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 0x55e1a3803b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.527+0000 7f625895db80 1 -- 172.21.15.57:0/2101033868 >> 172.21.15.57:0/2101033868 conn(0x55e1a376e200 msgr2=0x55e1a37f4010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.527+0000 7f622dffb700 0 client.26225 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.559+0000 7f625895db80 1 -- 172.21.15.57:0/2101033868 shutdown_connections 2023-10-08T13:55:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.559+0000 7f625895db80 1 -- 172.21.15.57:0/2101033868 wait complete. 2023-10-08T13:55:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.560+0000 7f625895db80 20 client.26225 trim_cache size 0 max 16384 2023-10-08T13:55:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.769+0000 7f625895db80 1 Processor -- start 2023-10-08T13:55:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.770+0000 7f625895db80 1 -- start start 2023-10-08T13:55:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.770+0000 7f625895db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a37f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.770+0000 7f625895db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 0x55e1a380ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.770+0000 7f625895db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.771+0000 7f625895db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a35cd8c0 con 0x55e1a37ff880 2023-10-08T13:55:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.771+0000 7f625895db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a37f5940 con 0x55e1a37fa0b0 2023-10-08T13:55:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.771+0000 7f625895db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a37ef2b0 con 0x55e1a38001a0 2023-10-08T13:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.784+0000 7f624ad11700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.784+0000 7f624a510700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a37f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.784+0000 7f624ad11700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59228/0 (socket says 172.21.15.57:59228) 2023-10-08T13:55:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.784+0000 7f624ad11700 1 -- 172.21.15.57:0/3089664363 learned_addr learned my addr 172.21.15.57:0/3089664363 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.784+0000 7f6249d0f700 1 --2- 172.21.15.57:0/3089664363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 0x55e1a380ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.795+0000 7f624a510700 1 -- 172.21.15.57:0/3089664363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 msgr2=0x55e1a37f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.795+0000 7f624a510700 1 --2- 172.21.15.57:0/3089664363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.795+0000 7f624a510700 1 -- 172.21.15.57:0/3089664363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 msgr2=0x55e1a380ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.795+0000 7f624a510700 1 --2- 172.21.15.57:0/3089664363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 0x55e1a380ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.795+0000 7f624a510700 1 -- 172.21.15.57:0/3089664363 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1a381af00 con 0x55e1a37fa0b0 2023-10-08T13:55:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.795+0000 7f624a510700 1 --2- 172.21.15.57:0/3089664363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a37f9b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6240009520 tx=0x7f624009eb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af1a374b07e21d7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.875+0000 7f62397fa700 1 -- 172.21.15.57:0/3089664363 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62400a2bd0 con 0x55e1a37fa0b0 2023-10-08T13:55:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.875+0000 7f62397fa700 1 -- 172.21.15.57:0/3089664363 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f624009bb70 con 0x55e1a37fa0b0 2023-10-08T13:55:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.875+0000 7f62397fa700 1 -- 172.21.15.57:0/3089664363 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62400a2740 con 0x55e1a37fa0b0 2023-10-08T13:55:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.876+0000 7f625895db80 1 -- 172.21.15.57:0/3089664363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 msgr2=0x55e1a37f9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:55:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.876+0000 7f625895db80 1 --2- 172.21.15.57:0/3089664363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a37f9b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6240009520 tx=0x7f624009eb10 comp rx=0 tx=0).stop 2023-10-08T13:55:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.900+0000 7f625895db80 1 -- 172.21.15.57:0/3089664363 shutdown_connections 2023-10-08T13:55:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.900+0000 7f625895db80 1 --2- 172.21.15.57:0/3089664363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a38001a0 0x55e1a37f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.900+0000 7f625895db80 1 --2- 172.21.15.57:0/3089664363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37fa0b0 0x55e1a37f9b70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.900+0000 7f625895db80 1 --2- 172.21.15.57:0/3089664363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a37ff880 0x55e1a380ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.900+0000 7f625895db80 1 -- 172.21.15.57:0/3089664363 >> 172.21.15.57:0/3089664363 conn(0x55e1a376e200 msgr2=0x55e1a3803ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:55:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.900+0000 7f625895db80 1 -- 172.21.15.57:0/3089664363 shutdown_connections 2023-10-08T13:55:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.900+0000 7f625895db80 1 -- 172.21.15.57:0/3089664363 wait complete. 2023-10-08T13:55:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.901+0000 7f625895db80 1 -- start start 2023-10-08T13:55:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.901+0000 7f625895db80 1 Processor -- start 2023-10-08T13:55:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.901+0000 7f625895db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a37fa0b0 0x55e1a385f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.901+0000 7f625895db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37ff880 0x55e1a385fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.901+0000 7f625895db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a38001a0 0x55e1a380a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.901+0000 7f625895db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a380a800 con 0x55e1a38001a0 2023-10-08T13:55:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.901+0000 7f625895db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a380a940 con 0x55e1a37ff880 2023-10-08T13:55:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.901+0000 7f625895db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a380ab00 con 0x55e1a37fa0b0 2023-10-08T13:55:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.901+0000 7f625895db80 10 client.0 _ll_register_callbacks cb 0x7f624009a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:55:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.902+0000 7f6249d0f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37ff880 0x55e1a385fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.902+0000 7f6249d0f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37ff880 0x55e1a385fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59846/0 (socket says 172.21.15.57:59846) 2023-10-08T13:55:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.902+0000 7f6249d0f700 1 -- 172.21.15.57:0/4081620361 learned_addr learned my addr 172.21.15.57:0/4081620361 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:55:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.902+0000 7f624a510700 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a37fa0b0 0x55e1a385f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.902+0000 7f6249d0f700 1 -- 172.21.15.57:0/4081620361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a37fa0b0 msgr2=0x55e1a385f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.902+0000 7f6249d0f700 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a37fa0b0 0x55e1a385f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.902+0000 7f6249d0f700 1 -- 172.21.15.57:0/4081620361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a38001a0 msgr2=0x55e1a380a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:55:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.902+0000 7f6249d0f700 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a38001a0 0x55e1a380a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:55:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.902+0000 7f6249d0f700 1 -- 172.21.15.57:0/4081620361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1a3560c00 con 0x55e1a37ff880 2023-10-08T13:55:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.923+0000 7f6249d0f700 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37ff880 0x55e1a385fa60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f623400b480 tx=0x7f6234010f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0e30c4a50ee6fab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:55:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.992+0000 7f622dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:55:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.992+0000 7f622dffb700 1 -- 172.21.15.57:0/4081620361 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234020040 con 0x55e1a37ff880 2023-10-08T13:55:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.992+0000 7f622dffb700 1 -- 172.21.15.57:0/4081620361 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f6234017070 con 0x55e1a37ff880 2023-10-08T13:55:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:57.992+0000 7f625895db80 1 -- 172.21.15.57:0/4081620361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6234015070 con 0x55e1a37ff880 2023-10-08T13:55:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.064+0000 7f622dffb700 1 -- 172.21.15.57:0/4081620361 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6234026b70 con 0x55e1a37ff880 2023-10-08T13:55:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.064+0000 7f625895db80 10 client.26231 fetch_fsmap learned FSMap version 36 2023-10-08T13:55:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.064+0000 7f625895db80 1 -- 172.21.15.57:0/4081620361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e1a359e1f0 con 0x55e1a37ff880 2023-10-08T13:55:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.075+0000 7f622dffb700 1 -- 172.21.15.57:0/4081620361 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6234015070 con 0x55e1a37ff880 2023-10-08T13:55:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.075+0000 7f622dffb700 1 -- 172.21.15.57:0/4081620361 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 36) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6234011510 con 0x55e1a37ff880 2023-10-08T13:55:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.075+0000 7f625895db80 10 client.26231 fetch_fsmap finished waiting for FSMap version 36 2023-10-08T13:55:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.075+0000 7f625895db80 10 client.26231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:55:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.075+0000 7f625895db80 20 client.26231 populate_metadata read hostname 'smithi057' 2023-10-08T13:55:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.075+0000 7f625895db80 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e1a38685b0 0x55e1a386a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:55:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.075+0000 7f625895db80 4 client.26231 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:55:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.075+0000 7f625895db80 1 -- 172.21.15.57:0/4081620361 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e1a3745ad0 con 0x55e1a38685b0 2023-10-08T13:55:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.122+0000 7f624a510700 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e1a38685b0 0x55e1a386a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:55:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.194+0000 7f624a510700 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e1a38685b0 0x55e1a386a9d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c598532812420a server_cookie=5074c06800362f30 in_seq=0 out_seq=0 2023-10-08T13:55:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:55:58.195+0000 7f622dffb700 10 client.26231 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:55:58.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:58 smithi057 ceph-mon[99476]: pgmap v431: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 980 KiB/s wr, 94 op/s 2023-10-08T13:55:58.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:55:58 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:55:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:58 smithi099 ceph-mon[103849]: pgmap v431: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 980 KiB/s wr, 94 op/s 2023-10-08T13:55:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:55:58 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:55:58.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:58 smithi115 ceph-mon[104408]: pgmap v431: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 980 KiB/s wr, 94 op/s 2023-10-08T13:55:58.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:55:58 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:56:00.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:00 smithi115 ceph-mon[104408]: pgmap v432: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2023-10-08T13:56:00.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:00 smithi057 ceph-mon[99476]: pgmap v432: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2023-10-08T13:56:00.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:00 smithi099 ceph-mon[103849]: pgmap v432: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2023-10-08T13:56:01.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:01 smithi115 ceph-mon[104408]: pgmap v433: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.6 MiB/s wr, 363 op/s 2023-10-08T13:56:01.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:01 smithi057 ceph-mon[99476]: pgmap v433: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.6 MiB/s wr, 363 op/s 2023-10-08T13:56:01.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:01 smithi099 ceph-mon[103849]: pgmap v433: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.6 MiB/s wr, 363 op/s 2023-10-08T13:56:02.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:02 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:02.749 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:02 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:02.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:02 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:03.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:03 smithi115 ceph-mon[104408]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.4 MiB/s wr, 461 op/s 2023-10-08T13:56:03.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:03 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:56:03.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:03 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:56:03.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:03 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:56:03.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:03 smithi057 ceph-mon[99476]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.4 MiB/s wr, 461 op/s 2023-10-08T13:56:03.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:03 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:56:03.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:03 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:56:03.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:03 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:56:03.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:03 smithi099 ceph-mon[103849]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.4 MiB/s wr, 461 op/s 2023-10-08T13:56:03.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:03 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:56:03.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:03 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:56:03.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:03 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:56:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.923+0000 7f622dffb700 1 -- 172.21.15.57:0/4081620361 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e1a3745ad0 con 0x55e1a38685b0 2023-10-08T13:56:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.923+0000 7f622dffb700 10 client.26231 handle_command_reply: tid=0 2023-10-08T13:56:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.923+0000 7f625895db80 1 client.26231 shutdown 2023-10-08T13:56:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.925+0000 7f625895db80 1 -- 172.21.15.57:0/4081620361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37ff880 msgr2=0x55e1a385fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.925+0000 7f625895db80 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37ff880 0x55e1a385fa60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f623400b480 tx=0x7f6234010f00 comp rx=0 tx=0).stop 2023-10-08T13:56:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.943+0000 7f625895db80 1 -- 172.21.15.57:0/4081620361 shutdown_connections 2023-10-08T13:56:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.943+0000 7f625895db80 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e1a38685b0 0x55e1a386a9d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.943+0000 7f625895db80 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1a38001a0 0x55e1a380a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.943+0000 7f625895db80 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1a37fa0b0 0x55e1a385f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.943+0000 7f625895db80 1 --2- 172.21.15.57:0/4081620361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1a37ff880 0x55e1a385fa60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.943+0000 7f625895db80 1 -- 172.21.15.57:0/4081620361 >> 172.21.15.57:0/4081620361 conn(0x55e1a376e200 msgr2=0x55e1a3800630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.943+0000 7f622dffb700 0 client.26231 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.945+0000 7f625895db80 1 -- 172.21.15.57:0/4081620361 shutdown_connections 2023-10-08T13:56:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.945+0000 7f625895db80 1 -- 172.21.15.57:0/4081620361 wait complete. 2023-10-08T13:56:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.955+0000 7f625895db80 20 client.26231 trim_cache size 0 max 16384 2023-10-08T13:56:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:56:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:56:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:56:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:56:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.983+0000 7f624c774700 1 -- 172.21.15.57:0/3679248925 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f62300a1800 msgr2=0x7f62300a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.983+0000 7f624c774700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f62300a1800 0x7f62300a3cc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f623c00d010 tx=0x7f623c00b040 comp rx=0 tx=0).stop 2023-10-08T13:56:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.983+0000 7f624c774700 1 -- 172.21.15.57:0/3679248925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f624410ef10 msgr2=0x7f62441195d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.983+0000 7f624c774700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f624410ef10 0x7f62441195d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f624000d950 tx=0x7f624000dc60 comp rx=0 tx=0).stop 2023-10-08T13:56:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.992+0000 7f624c774700 1 -- 172.21.15.57:0/3679248925 shutdown_connections 2023-10-08T13:56:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.992+0000 7f624c774700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6244119fd0 0x7f6244113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.992+0000 7f624c774700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f62300a1800 0x7f62300a3cc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.992+0000 7f624c774700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f624410ef10 0x7f62441195d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.992+0000 7f624c774700 1 --2- 172.21.15.57:0/3679248925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f624410f380 0x7f6244113620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:03.992+0000 7f624c774700 1 -- 172.21.15.57:0/3679248925 >> 172.21.15.57:0/3679248925 conn(0x7f624406b7c0 msgr2=0x7f624411b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:04.013+0000 7f624c774700 1 -- 172.21.15.57:0/3679248925 shutdown_connections 2023-10-08T13:56:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:04.013+0000 7f624c774700 1 -- 172.21.15.57:0/3679248925 wait complete. 2023-10-08T13:56:04.379 INFO:tasks.workunit.client.0.smithi057.stdout: 319 1864239 254 1327522 318 309409 446 2023-10-08T13:56:05.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:05 smithi115 ceph-mon[104408]: pgmap v435: 129 pgs: 129 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 668 op/s 2023-10-08T13:56:05.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:05 smithi057 ceph-mon[99476]: pgmap v435: 129 pgs: 129 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 668 op/s 2023-10-08T13:56:05.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:05 smithi099 ceph-mon[103849]: pgmap v435: 129 pgs: 129 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 668 op/s 2023-10-08T13:56:06.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:06 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:06.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:06 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:06.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:06 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:07.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:07 smithi115 ceph-mon[104408]: pgmap v436: 129 pgs: 129 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 673 op/s 2023-10-08T13:56:07.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:07 smithi057 ceph-mon[99476]: pgmap v436: 129 pgs: 129 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 673 op/s 2023-10-08T13:56:07.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:07 smithi099 ceph-mon[103849]: pgmap v436: 129 pgs: 129 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 673 op/s 2023-10-08T13:56:10.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:09 smithi057 ceph-mon[99476]: pgmap v437: 129 pgs: 129 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 677 op/s 2023-10-08T13:56:10.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:09 smithi099 ceph-mon[103849]: pgmap v437: 129 pgs: 129 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 677 op/s 2023-10-08T13:56:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:09 smithi115 ceph-mon[104408]: pgmap v437: 129 pgs: 129 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 677 op/s 2023-10-08T13:56:12.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:11 smithi057 ceph-mon[99476]: pgmap v438: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 13 MiB/s wr, 680 op/s 2023-10-08T13:56:12.249 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:56:12.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:11 smithi099 ceph-mon[103849]: pgmap v438: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 13 MiB/s wr, 680 op/s 2023-10-08T13:56:12.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.381+0000 7f43e14ac700 1 -- 172.21.15.57:0/1197695684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43dc100b80 msgr2=0x7f43dc10c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:12.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.381+0000 7f43e14ac700 1 --2- 172.21.15.57:0/1197695684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43dc100b80 0x7f43dc10c510 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f43d400b0a0 tx=0x7f43d400b3b0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.381+0000 7f43e14ac700 1 -- 172.21.15.57:0/1197695684 shutdown_connections 2023-10-08T13:56:12.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.381+0000 7f43e14ac700 1 --2- 172.21.15.57:0/1197695684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43dc100b80 0x7f43dc10c510 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.381+0000 7f43e14ac700 1 --2- 172.21.15.57:0/1197695684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43dc1046a0 0x7f43dc104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.381+0000 7f43e14ac700 1 --2- 172.21.15.57:0/1197695684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43dc1001c0 0x7f43dc100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.381+0000 7f43e14ac700 1 -- 172.21.15.57:0/1197695684 >> 172.21.15.57:0/1197695684 conn(0x7f43dc0749a0 msgr2=0x7f43dc074db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:12.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.381+0000 7f43e14ac700 1 -- 172.21.15.57:0/1197695684 shutdown_connections 2023-10-08T13:56:12.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.381+0000 7f43e14ac700 1 -- 172.21.15.57:0/1197695684 wait complete. 2023-10-08T13:56:12.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43e14ac700 1 Processor -- start 2023-10-08T13:56:12.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43e14ac700 1 -- start start 2023-10-08T13:56:12.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43e14ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43dc1001c0 0x7f43dc19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43e14ac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43dc100b80 0x7f43dc19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43e14ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43dc1046a0 0x7f43dc1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43e14ac700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc196050 con 0x7f43dc100b80 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43e14ac700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc1961c0 con 0x7f43dc1001c0 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43e14ac700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc196380 con 0x7f43dc1046a0 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43dc100b80 0x7f43dc19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43dc1001c0 0x7f43dc19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43dc1001c0 0x7f43dc19b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59860/0 (socket says 172.21.15.57:59860) 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43daffd700 1 -- 172.21.15.57:0/1335619283 learned_addr learned my addr 172.21.15.57:0/1335619283 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43dc100b80 0x7f43dc19be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58436/0 (socket says 172.21.15.57:58436) 2023-10-08T13:56:12.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.382+0000 7f43db7fe700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43dc1046a0 0x7f43dc1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43da7fc700 1 -- 172.21.15.57:0/1335619283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43dc1046a0 msgr2=0x7f43dc1959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43da7fc700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43dc1046a0 0x7f43dc1959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43da7fc700 1 -- 172.21.15.57:0/1335619283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43dc1001c0 msgr2=0x7f43dc19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43da7fc700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43dc1001c0 0x7f43dc19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43da7fc700 1 -- 172.21.15.57:0/1335619283 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d4009d00 con 0x7f43dc100b80 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43daffd700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43dc1001c0 0x7f43dc19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43da7fc700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43dc100b80 0x7f43dc19be00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f43d000e9f0 tx=0x7f43d000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43c3fff700 1 -- 172.21.15.57:0/1335619283 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d000c9c0 con 0x7f43dc100b80 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43c3fff700 1 -- 172.21.15.57:0/1335619283 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43d0010430 con 0x7f43dc100b80 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43e14ac700 1 -- 172.21.15.57:0/1335619283 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc196660 con 0x7f43dc100b80 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43c3fff700 1 -- 172.21.15.57:0/1335619283 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d0018680 con 0x7f43dc100b80 2023-10-08T13:56:12.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.383+0000 7f43e14ac700 1 -- 172.21.15.57:0/1335619283 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc1a28b0 con 0x7f43dc100b80 2023-10-08T13:56:12.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.385+0000 7f43c3fff700 1 -- 172.21.15.57:0/1335619283 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f43d0010860 con 0x7f43dc100b80 2023-10-08T13:56:12.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.386+0000 7f43c3fff700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43c40a15f0 0x7f43c40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.386+0000 7f43daffd700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43c40a15f0 0x7f43c40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.386+0000 7f43c3fff700 1 -- 172.21.15.57:0/1335619283 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f43d0014070 con 0x7f43dc100b80 2023-10-08T13:56:12.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43daffd700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43c40a15f0 0x7f43c40a3ab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f43cc005950 tx=0x7f43cc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:12.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43ed695b80 1 Processor -- start 2023-10-08T13:56:12.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43ed695b80 1 -- start start 2023-10-08T13:56:12.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c474260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c4747a0 0x55de9c481320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c481860 0x55de9c483c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43ed695b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c3dd460 con 0x55de9c473e80 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43ed695b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c3cec20 con 0x55de9c481860 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43ed695b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c3cc540 con 0x55de9c4747a0 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43daffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c474260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43daffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c474260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58442/0 (socket says 172.21.15.57:58442) 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43daffd700 1 -- 172.21.15.57:0/3848944974 learned_addr learned my addr 172.21.15.57:0/3848944974 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.387+0000 7f43da7fc700 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c4747a0 0x55de9c481320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43db7fe700 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c481860 0x55de9c483c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43daffd700 1 -- 172.21.15.57:0/3848944974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c4747a0 msgr2=0x55de9c481320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:12.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43daffd700 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c4747a0 0x55de9c481320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43daffd700 1 -- 172.21.15.57:0/3848944974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c481860 msgr2=0x55de9c483c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43daffd700 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c481860 0x55de9c483c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43daffd700 1 -- 172.21.15.57:0/3848944974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de9c1f2de0 con 0x55de9c473e80 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43db7fe700 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c481860 0x55de9c483c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43da7fc700 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c4747a0 0x55de9c481320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43daffd700 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c474260 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009880 tx=0x7f43cc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cc5209a90b4899 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43c17fa700 1 -- 172.21.15.57:0/3848944974 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc004a30 con 0x55de9c473e80 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43c17fa700 1 -- 172.21.15.57:0/3848944974 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43cc021070 con 0x55de9c473e80 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.388+0000 7f43c17fa700 1 -- 172.21.15.57:0/3848944974 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc01c670 con 0x55de9c473e80 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 -- 172.21.15.57:0/3848944974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 msgr2=0x55de9c474260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c474260 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009880 tx=0x7f43cc014000 comp rx=0 tx=0).stop 2023-10-08T13:56:12.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 -- 172.21.15.57:0/3848944974 shutdown_connections 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c481860 0x55de9c483c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c474260 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 --2- 172.21.15.57:0/3848944974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c4747a0 0x55de9c481320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 -- 172.21.15.57:0/3848944974 >> 172.21.15.57:0/3848944974 conn(0x55de9c3e9260 msgr2=0x55de9c4702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 -- 172.21.15.57:0/3848944974 shutdown_connections 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 -- 172.21.15.57:0/3848944974 wait complete. 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 -- start start 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.389+0000 7f43ed695b80 1 Processor -- start 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c4812b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c4747a0 0x55de9c47e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c47ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43ed695b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c3c7e30 con 0x55de9c473e80 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43ed695b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c364710 con 0x55de9c4747a0 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43ed695b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c3643b0 con 0x55de9c481860 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43ed695b80 10 client.0 _ll_register_callbacks cb 0x55de9c1c1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43daffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c4812b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43daffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c4812b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58448/0 (socket says 172.21.15.57:58448) 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43daffd700 1 -- 172.21.15.57:0/215905075 learned_addr learned my addr 172.21.15.57:0/215905075 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43da7fc700 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c4747a0 0x55de9c47e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43db7fe700 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c47ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43daffd700 1 -- 172.21.15.57:0/215905075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 msgr2=0x55de9c47ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43daffd700 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c47ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43daffd700 1 -- 172.21.15.57:0/215905075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c4747a0 msgr2=0x55de9c47e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:12.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43daffd700 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c4747a0 0x55de9c47e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43daffd700 1 -- 172.21.15.57:0/215905075 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc1a28b0 con 0x55de9c473e80 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43da7fc700 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c4747a0 0x55de9c47e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43daffd700 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c4812b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009850 tx=0x7f43cc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a520dd5f0cca5108 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43beff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43beff5700 1 -- 172.21.15.57:0/215905075 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc02c070 con 0x55de9c473e80 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43ed695b80 1 -- 172.21.15.57:0/215905075 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de9c38feb0 con 0x55de9c473e80 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.390+0000 7f43beff5700 1 -- 172.21.15.57:0/215905075 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f43cc004920 con 0x55de9c473e80 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43beff5700 1 -- 172.21.15.57:0/215905075 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43cc004ae0 con 0x55de9c473e80 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43ed695b80 10 client.16701 fetch_fsmap learned FSMap version 38 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43ed695b80 1 -- 172.21.15.57:0/215905075 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f43dc10cda0 con 0x55de9c473e80 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43beff5700 1 -- 172.21.15.57:0/215905075 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de9c38feb0 con 0x55de9c473e80 2023-10-08T13:56:12.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43beff5700 1 -- 172.21.15.57:0/215905075 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 38) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f43cc026930 con 0x55de9c473e80 2023-10-08T13:56:12.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43ed695b80 10 client.16701 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T13:56:12.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43ed695b80 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:56:12.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43ed695b80 20 client.16701 populate_metadata read hostname 'smithi057' 2023-10-08T13:56:12.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43ed695b80 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55de9c491250 0x55de9c493670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43ed695b80 4 client.16701 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:56:12.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43ed695b80 1 -- 172.21.15.57:0/215905075 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de9c3c0e10 con 0x55de9c491250 2023-10-08T13:56:12.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.391+0000 7f43da7fc700 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55de9c491250 0x55de9c493670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.392+0000 7f43beff5700 10 client.16701 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:12.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.392+0000 7f43da7fc700 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55de9c491250 0x55de9c493670 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a89768a036f3f1 server_cookie=b7fcc2118661d779 in_seq=0 out_seq=0 2023-10-08T13:56:12.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43beff5700 1 -- 172.21.15.57:0/215905075 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55de9c3c0e10 con 0x55de9c491250 2023-10-08T13:56:12.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43beff5700 10 client.16701 handle_command_reply: tid=0 2023-10-08T13:56:12.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 client.16701 shutdown 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 -- 172.21.15.57:0/215905075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 msgr2=0x55de9c4812b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c4812b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009850 tx=0x7f43cc00ee40 comp rx=0 tx=0).stop 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 -- 172.21.15.57:0/215905075 shutdown_connections 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55de9c491250 0x55de9c493670 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c47ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43beff5700 0 client.16701 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c473e80 0x55de9c4812b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 --2- 172.21.15.57:0/215905075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c4747a0 0x55de9c47e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 -- 172.21.15.57:0/215905075 >> 172.21.15.57:0/215905075 conn(0x55de9c3e9260 msgr2=0x55de9c46fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 -- 172.21.15.57:0/215905075 shutdown_connections 2023-10-08T13:56:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 1 -- 172.21.15.57:0/215905075 wait complete. 2023-10-08T13:56:12.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.394+0000 7f43ed695b80 20 client.16701 trim_cache size 0 max 16384 2023-10-08T13:56:12.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.412+0000 7f43ed695b80 1 Processor -- start 2023-10-08T13:56:12.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.412+0000 7f43ed695b80 1 -- start start 2023-10-08T13:56:12.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.412+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 0x55de9c47ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.412+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.412+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c479960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.412+0000 7f43ed695b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c248920 con 0x55de9c4747a0 2023-10-08T13:56:12.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.412+0000 7f43ed695b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c474b80 con 0x55de9c473e80 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.412+0000 7f43ed695b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c46cae0 con 0x55de9c481860 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.412+0000 7f43da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.412+0000 7f43da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4883b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58458/0 (socket says 172.21.15.57:58458) 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 0x55de9c47ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43da7fc700 1 -- 172.21.15.57:0/483797753 learned_addr learned my addr 172.21.15.57:0/483797753 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43db7fe700 1 --2- 172.21.15.57:0/483797753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c479960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43da7fc700 1 -- 172.21.15.57:0/483797753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 msgr2=0x55de9c479960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43da7fc700 1 --2- 172.21.15.57:0/483797753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c479960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43da7fc700 1 -- 172.21.15.57:0/483797753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 msgr2=0x55de9c47ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43da7fc700 1 --2- 172.21.15.57:0/483797753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 0x55de9c47ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43da7fc700 1 -- 172.21.15.57:0/483797753 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc017070 con 0x55de9c4747a0 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43da7fc700 1 --2- 172.21.15.57:0/483797753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4883b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f43d009a1a0 tx=0x7f43d001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfeab4f3d48fcf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43c1ffb700 1 -- 172.21.15.57:0/483797753 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00b1070 con 0x55de9c4747a0 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43c1ffb700 1 -- 172.21.15.57:0/483797753 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43d00cd040 con 0x55de9c4747a0 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.413+0000 7f43c1ffb700 1 -- 172.21.15.57:0/483797753 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00985b0 con 0x55de9c4747a0 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 -- 172.21.15.57:0/483797753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 msgr2=0x55de9c4883b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 --2- 172.21.15.57:0/483797753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4883b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f43d009a1a0 tx=0x7f43d001a040 comp rx=0 tx=0).stop 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 -- 172.21.15.57:0/483797753 shutdown_connections 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 --2- 172.21.15.57:0/483797753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c479960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 --2- 172.21.15.57:0/483797753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 0x55de9c47ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 --2- 172.21.15.57:0/483797753 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4883b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 -- 172.21.15.57:0/483797753 >> 172.21.15.57:0/483797753 conn(0x55de9c3e9260 msgr2=0x55de9c46f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 -- 172.21.15.57:0/483797753 shutdown_connections 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 -- 172.21.15.57:0/483797753 wait complete. 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 -- start start 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 Processor -- start 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 0x55de9c487cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4881f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c486390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c486a70 con 0x55de9c4747a0 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c486be0 con 0x55de9c473e80 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.414+0000 7f43ed695b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de9c486da0 con 0x55de9c481860 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43ed695b80 10 client.0 _ll_register_callbacks cb 0x7f43d0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4881f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 0x55de9c487cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4881f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58464/0 (socket says 172.21.15.57:58464) 2023-10-08T13:56:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43da7fc700 1 -- 172.21.15.57:0/2646451912 learned_addr learned my addr 172.21.15.57:0/2646451912 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43db7fe700 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c486390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43da7fc700 1 -- 172.21.15.57:0/2646451912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 msgr2=0x55de9c486390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43da7fc700 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c486390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43da7fc700 1 -- 172.21.15.57:0/2646451912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 msgr2=0x55de9c487cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43da7fc700 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 0x55de9c487cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43da7fc700 1 -- 172.21.15.57:0/2646451912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc196660 con 0x55de9c4747a0 2023-10-08T13:56:12.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43daffd700 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 0x55de9c487cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43da7fc700 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4881f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f43d0099f10 tx=0x7f43d0005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98966be794b50f7a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43beff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43beff5700 1 -- 172.21.15.57:0/2646451912 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00b1070 con 0x55de9c4747a0 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43beff5700 1 -- 172.21.15.57:0/2646451912 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f43d009a210 con 0x55de9c4747a0 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43ed695b80 1 -- 172.21.15.57:0/2646451912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43cc021070 con 0x55de9c4747a0 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43beff5700 1 -- 172.21.15.57:0/2646451912 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43d009e150 con 0x55de9c4747a0 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43ed695b80 10 client.16707 fetch_fsmap learned FSMap version 38 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.415+0000 7f43ed695b80 1 -- 172.21.15.57:0/2646451912 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55de9c1dbc60 con 0x55de9c4747a0 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43beff5700 1 -- 172.21.15.57:0/2646451912 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43cc021070 con 0x55de9c4747a0 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43beff5700 1 -- 172.21.15.57:0/2646451912 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 38) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f43d009a470 con 0x55de9c4747a0 2023-10-08T13:56:12.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43ed695b80 10 client.16707 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T13:56:12.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43ed695b80 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:56:12.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43ed695b80 20 client.16707 populate_metadata read hostname 'smithi057' 2023-10-08T13:56:12.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43ed695b80 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55de9c4e60e0 0x55de9c4e8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:12.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43ed695b80 4 client.16707 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:56:12.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43ed695b80 1 -- 172.21.15.57:0/2646451912 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de9c3c0b30 con 0x55de9c4e60e0 2023-10-08T13:56:12.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43daffd700 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55de9c4e60e0 0x55de9c4e8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:12.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43daffd700 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55de9c4e60e0 0x55de9c4e8500 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b8514470dfec51 server_cookie=e00b0b94ea220e5f in_seq=0 out_seq=0 2023-10-08T13:56:12.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.416+0000 7f43beff5700 10 client.16707 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:12.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43beff5700 1 -- 172.21.15.57:0/2646451912 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55de9c3c0b30 con 0x55de9c4e60e0 2023-10-08T13:56:12.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43beff5700 10 client.16707 handle_command_reply: tid=0 2023-10-08T13:56:12.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 client.16707 shutdown 2023-10-08T13:56:12.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 -- 172.21.15.57:0/2646451912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 msgr2=0x55de9c4881f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4881f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f43d0099f10 tx=0x7f43d0005ce0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 -- 172.21.15.57:0/2646451912 shutdown_connections 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55de9c4e60e0 0x55de9c4e8500 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55de9c481860 0x55de9c486390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de9c473e80 0x55de9c487cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43beff5700 0 client.16707 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 --2- 172.21.15.57:0/2646451912 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55de9c4747a0 0x55de9c4881f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 -- 172.21.15.57:0/2646451912 >> 172.21.15.57:0/2646451912 conn(0x55de9c3e9260 msgr2=0x55de9c479960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 -- 172.21.15.57:0/2646451912 shutdown_connections 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 1 -- 172.21.15.57:0/2646451912 wait complete. 2023-10-08T13:56:12.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43ed695b80 20 client.16707 trim_cache size 0 max 16384 2023-10-08T13:56:12.426 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:56:12.426 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T13:56:12.426 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:56:12.426 INFO:teuthology.orchestra.run.smithi057.stdout: "6db03f06-e888-4427-a580-5fa3643c8e0d": { 2023-10-08T13:56:12.426 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:56:12.426 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "6db03f06-e888-4427-a580-5fa3643c8e0d", 2023-10-08T13:56:12.426 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:56:12.426 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:56:12.426 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:56:12.426 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:56:12.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43e14ac700 1 -- 172.21.15.57:0/1335619283 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43c40a15f0 msgr2=0x7f43c40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:12.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43e14ac700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43c40a15f0 0x7f43c40a3ab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f43cc005950 tx=0x7f43cc009260 comp rx=0 tx=0).stop 2023-10-08T13:56:12.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43e14ac700 1 -- 172.21.15.57:0/1335619283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43dc100b80 msgr2=0x7f43dc19be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:12.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.420+0000 7f43e14ac700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43dc100b80 0x7f43dc19be00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f43d000e9f0 tx=0x7f43d000ed00 comp rx=0 tx=0).stop 2023-10-08T13:56:12.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.421+0000 7f43e14ac700 1 -- 172.21.15.57:0/1335619283 shutdown_connections 2023-10-08T13:56:12.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.421+0000 7f43e14ac700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43dc1046a0 0x7f43dc1959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.421+0000 7f43e14ac700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f43c40a15f0 0x7f43c40a3ab0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.421+0000 7f43e14ac700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43dc1001c0 0x7f43dc19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.421+0000 7f43e14ac700 1 --2- 172.21.15.57:0/1335619283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43dc100b80 0x7f43dc19be00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:12.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.421+0000 7f43e14ac700 1 -- 172.21.15.57:0/1335619283 >> 172.21.15.57:0/1335619283 conn(0x7f43dc0749a0 msgr2=0x7f43dc0fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:12.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.421+0000 7f43e14ac700 1 -- 172.21.15.57:0/1335619283 shutdown_connections 2023-10-08T13:56:12.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:12.421+0000 7f43e14ac700 1 -- 172.21.15.57:0/1335619283 wait complete. 2023-10-08T13:56:12.434 INFO:tasks.cephfs.filesystem:scrub status for tag:6db03f06-e888-4427-a580-5fa3643c8e0d - {'path': '/', 'tag': '6db03f06-e888-4427-a580-5fa3643c8e0d', 'options': 'recursive,force'} 2023-10-08T13:56:12.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:11 smithi115 ceph-mon[104408]: pgmap v438: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 13 MiB/s wr, 680 op/s 2023-10-08T13:56:14.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:13 smithi057 ceph-mon[99476]: pgmap v439: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 429 op/s 2023-10-08T13:56:14.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:13 smithi099 ceph-mon[103849]: pgmap v439: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 429 op/s 2023-10-08T13:56:14.379 INFO:tasks.workunit.client.0.smithi057.stdout: 379 515635 2835 371331 3080 88049 250 2023-10-08T13:56:14.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:13 smithi115 ceph-mon[104408]: pgmap v439: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 429 op/s 2023-10-08T13:56:16.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:15 smithi057 ceph-mon[99476]: pgmap v440: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 332 op/s 2023-10-08T13:56:16.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:15 smithi099 ceph-mon[103849]: pgmap v440: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 332 op/s 2023-10-08T13:56:16.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:15 smithi115 ceph-mon[104408]: pgmap v440: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 332 op/s 2023-10-08T13:56:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.209+0000 7f93a11a4700 1 -- 172.21.15.57:0/3396314611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f939c0ff5d0 msgr2=0x7f939c0ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.209+0000 7f93a11a4700 1 --2- 172.21.15.57:0/3396314611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f939c0ff5d0 0x7f939c0ffa50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9390009990 tx=0x7f9390009ca0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.209+0000 7f93a11a4700 1 -- 172.21.15.57:0/3396314611 shutdown_connections 2023-10-08T13:56:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.209+0000 7f93a11a4700 1 --2- 172.21.15.57:0/3396314611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f939c0fff90 0x7f939c10c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.209+0000 7f93a11a4700 1 --2- 172.21.15.57:0/3396314611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939c1045e0 0x7f939c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.209+0000 7f93a11a4700 1 --2- 172.21.15.57:0/3396314611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f939c0ff5d0 0x7f939c0ffa50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.209+0000 7f93a11a4700 1 -- 172.21.15.57:0/3396314611 >> 172.21.15.57:0/3396314611 conn(0x7f939c076790 msgr2=0x7f939c076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.209+0000 7f93a11a4700 1 -- 172.21.15.57:0/3396314611 shutdown_connections 2023-10-08T13:56:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.209+0000 7f93a11a4700 1 -- 172.21.15.57:0/3396314611 wait complete. 2023-10-08T13:56:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.210+0000 7f93a11a4700 1 Processor -- start 2023-10-08T13:56:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.210+0000 7f93a11a4700 1 -- start start 2023-10-08T13:56:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f93a11a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f939c0ff5d0 0x7f939c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f93a11a4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939c0fff90 0x7f939c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f93a11a4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f939c1045e0 0x7f939c1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f93a11a4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c197020 con 0x7f939c1045e0 2023-10-08T13:56:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f93a11a4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c197190 con 0x7f939c0ff5d0 2023-10-08T13:56:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f93a11a4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c197350 con 0x7f939c0fff90 2023-10-08T13:56:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939c0fff90 0x7f939c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939c0fff90 0x7f939c19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59288/0 (socket says 172.21.15.57:59288) 2023-10-08T13:56:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939a59c700 1 -- 172.21.15.57:0/2338254139 learned_addr learned my addr 172.21.15.57:0/2338254139 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939b59e700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f939c1045e0 0x7f939c1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939ad9d700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f939c0ff5d0 0x7f939c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939a59c700 1 -- 172.21.15.57:0/2338254139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f939c0ff5d0 msgr2=0x7f939c19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939a59c700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f939c0ff5d0 0x7f939c19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939a59c700 1 -- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f939c1045e0 msgr2=0x7f939c1969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939a59c700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f939c1045e0 0x7f939c1969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939a59c700 1 -- 172.21.15.57:0/2338254139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93900095f0 con 0x7f939c0fff90 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939b59e700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f939c1045e0 0x7f939c1969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.211+0000 7f939a59c700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939c0fff90 0x7f939c19cdd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9390000c00 tx=0x7f9390004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.212+0000 7f9383fff700 1 -- 172.21.15.57:0/2338254139 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939001d070 con 0x7f939c0fff90 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.212+0000 7f9383fff700 1 -- 172.21.15.57:0/2338254139 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9390022470 con 0x7f939c0fff90 2023-10-08T13:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.212+0000 7f93a11a4700 1 -- 172.21.15.57:0/2338254139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c1975d0 con 0x7f939c0fff90 2023-10-08T13:56:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.212+0000 7f9383fff700 1 -- 172.21.15.57:0/2338254139 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390017650 con 0x7f939c0fff90 2023-10-08T13:56:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.212+0000 7f93a11a4700 1 -- 172.21.15.57:0/2338254139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c1a3880 con 0x7f939c0fff90 2023-10-08T13:56:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.215+0000 7f9383fff700 1 -- 172.21.15.57:0/2338254139 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f93900225e0 con 0x7f939c0fff90 2023-10-08T13:56:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.216+0000 7f9383fff700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93840a1670 0x7f93840a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.216+0000 7f9383fff700 1 -- 172.21.15.57:0/2338254139 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f93900ce520 con 0x7f939c0fff90 2023-10-08T13:56:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.216+0000 7f939ad9d700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93840a1670 0x7f93840a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.216+0000 7f93ad38db80 1 Processor -- start 2023-10-08T13:56:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.216+0000 7f93ad38db80 1 -- start start 2023-10-08T13:56:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.216+0000 7f939ad9d700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93840a1670 0x7f93840a3b30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f939c069cf0 tx=0x7f938c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 0x55c5018beb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018bf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f93ad38db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c501827400 con 0x55c5018bf0b0 2023-10-08T13:56:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f93ad38db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c501818bc0 con 0x55c5018c4880 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f93ad38db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5018164e0 con 0x55c5018c51a0 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018bf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 0x55c5018beb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 0x55c5018beb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59298/0 (socket says 172.21.15.57:59298) 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018bf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58488/0 (socket says 172.21.15.57:58488) 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939a59c700 1 -- 172.21.15.57:0/240407118 learned_addr learned my addr 172.21.15.57:0/240407118 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939ad9d700 1 -- 172.21.15.57:0/240407118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 msgr2=0x55c5018beb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939ad9d700 1 --2- 172.21.15.57:0/240407118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 0x55c5018beb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939ad9d700 1 -- 172.21.15.57:0/240407118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 msgr2=0x55c5018bf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939ad9d700 1 --2- 172.21.15.57:0/240407118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018bf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939ad9d700 1 -- 172.21.15.57:0/240407118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c50163cd80 con 0x55c5018c4880 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939a59c700 1 --2- 172.21.15.57:0/240407118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 0x55c5018beb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.217+0000 7f939ad9d700 1 --2- 172.21.15.57:0/240407118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018c4c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f938c009fd0 tx=0x7f938c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea82141dd9e8ed5b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.218+0000 7f93817fa700 1 -- 172.21.15.57:0/240407118 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c004020 con 0x55c5018c4880 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.218+0000 7f93817fa700 1 -- 172.21.15.57:0/240407118 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f938c021070 con 0x55c5018c4880 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.218+0000 7f93817fa700 1 -- 172.21.15.57:0/240407118 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c003900 con 0x55c5018c4880 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.218+0000 7f93ad38db80 1 -- 172.21.15.57:0/240407118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 msgr2=0x55c5018c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.218+0000 7f93ad38db80 1 --2- 172.21.15.57:0/240407118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018c4c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f938c009fd0 tx=0x7f938c014000 comp rx=0 tx=0).stop 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.219+0000 7f93ad38db80 1 -- 172.21.15.57:0/240407118 shutdown_connections 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.219+0000 7f93ad38db80 1 --2- 172.21.15.57:0/240407118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018bf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.219+0000 7f93ad38db80 1 --2- 172.21.15.57:0/240407118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018c4c60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.219+0000 7f93ad38db80 1 --2- 172.21.15.57:0/240407118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 0x55c5018beb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.219+0000 7f93ad38db80 1 -- 172.21.15.57:0/240407118 >> 172.21.15.57:0/240407118 conn(0x55c501833200 msgr2=0x55c5018b9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.219+0000 7f93ad38db80 1 -- 172.21.15.57:0/240407118 shutdown_connections 2023-10-08T13:56:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.219+0000 7f93ad38db80 1 -- 172.21.15.57:0/240407118 wait complete. 2023-10-08T13:56:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.219+0000 7f93ad38db80 1 -- start start 2023-10-08T13:56:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.219+0000 7f93ad38db80 1 Processor -- start 2023-10-08T13:56:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c4880 0x55c5018c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c51a0 0x55c5018c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f93ad38db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c501811dd0 con 0x55c5018bf0b0 2023-10-08T13:56:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f93ad38db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5017ae6b0 con 0x55c5018c51a0 2023-10-08T13:56:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f93ad38db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5017ae350 con 0x55c5018c4880 2023-10-08T13:56:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f93ad38db80 10 client.0 _ll_register_callbacks cb 0x55c50160b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:56:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c4880 0x55c5018c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939b59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c51a0 0x55c5018c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c4880 0x55c5018c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59304/0 (socket says 172.21.15.57:59304) 2023-10-08T13:56:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939a59c700 1 -- 172.21.15.57:0/640629061 learned_addr learned my addr 172.21.15.57:0/640629061 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939ad9d700 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58490/0 (socket says 172.21.15.57:58490) 2023-10-08T13:56:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939b59e700 1 -- 172.21.15.57:0/640629061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c4880 msgr2=0x55c5018c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939b59e700 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c4880 0x55c5018c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939b59e700 1 -- 172.21.15.57:0/640629061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 msgr2=0x55c5018c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939b59e700 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939b59e700 1 -- 172.21.15.57:0/640629061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c1a3880 con 0x55c5018c51a0 2023-10-08T13:56:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939a59c700 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c4880 0x55c5018c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939ad9d700 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f939b59e700 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c51a0 0x55c5018c8fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93940046a0 tx=0x7f93940049b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cef162b5576630f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f937eff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:56:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.220+0000 7f937eff5700 1 -- 172.21.15.57:0/640629061 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394004db0 con 0x55c5018c51a0 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f937eff5700 1 -- 172.21.15.57:0/640629061 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f93940093a0 con 0x55c5018c51a0 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f93ad38db80 1 -- 172.21.15.57:0/640629061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5017d9e50 con 0x55c5018c51a0 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f937eff5700 1 -- 172.21.15.57:0/640629061 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f939400ea10 con 0x55c5018c51a0 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f93ad38db80 10 client.26261 fetch_fsmap learned FSMap version 38 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f93ad38db80 1 -- 172.21.15.57:0/640629061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f938c017070 con 0x55c5018c51a0 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f937eff5700 1 -- 172.21.15.57:0/640629061 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5017d9e50 con 0x55c5018c51a0 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f937eff5700 1 -- 172.21.15.57:0/640629061 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f939401c030 con 0x55c5018c51a0 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f93ad38db80 10 client.26261 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f93ad38db80 10 client.26261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f93ad38db80 20 client.26261 populate_metadata read hostname 'smithi057' 2023-10-08T13:56:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f93ad38db80 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55c5018d8e70 0x55c5018db290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f93ad38db80 4 client.26261 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f93ad38db80 1 -- 172.21.15.57:0/640629061 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c50180adb0 con 0x55c5018d8e70 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.221+0000 7f939ad9d700 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55c5018d8e70 0x55c5018db290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.222+0000 7f937eff5700 10 client.26261 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.222+0000 7f939ad9d700 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55c5018d8e70 0x55c5018db290 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f3fa0e6f7d051f server_cookie=8988148f63ddb899 in_seq=0 out_seq=0 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.222+0000 7f937eff5700 1 -- 172.21.15.57:0/640629061 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c50180adb0 con 0x55c5018d8e70 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.222+0000 7f937eff5700 10 client.26261 handle_command_reply: tid=0 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.222+0000 7f93ad38db80 1 client.26261 shutdown 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 1 -- 172.21.15.57:0/640629061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c51a0 msgr2=0x55c5018c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c51a0 0x55c5018c8fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93940046a0 tx=0x7f93940049b0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 1 -- 172.21.15.57:0/640629061 shutdown_connections 2023-10-08T13:56:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55c5018d8e70 0x55c5018db290 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c51a0 0x55c5018c8fa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018bf0b0 0x55c5018c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 1 --2- 172.21.15.57:0/640629061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c4880 0x55c5018c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 1 -- 172.21.15.57:0/640629061 >> 172.21.15.57:0/640629061 conn(0x55c501833200 msgr2=0x55c5018b9010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f937eff5700 0 client.26261 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 1 -- 172.21.15.57:0/640629061 shutdown_connections 2023-10-08T13:56:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 1 -- 172.21.15.57:0/640629061 wait complete. 2023-10-08T13:56:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.223+0000 7f93ad38db80 20 client.26261 trim_cache size 0 max 16384 2023-10-08T13:56:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.239+0000 7f93ad38db80 1 Processor -- start 2023-10-08T13:56:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f93ad38db80 1 -- start start 2023-10-08T13:56:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018bf0b0 0x55c5018beb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c4880 0x55c5018c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 0x55c5018c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f93ad38db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5016928c0 con 0x55c5018c4880 2023-10-08T13:56:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f93ad38db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5018c9bd0 con 0x55c5018bf0b0 2023-10-08T13:56:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f93ad38db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5018d6320 con 0x55c5018c51a0 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c4880 0x55c5018c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018bf0b0 0x55c5018beb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c4880 0x55c5018c3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58500/0 (socket says 172.21.15.57:58500) 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939a59c700 1 -- 172.21.15.57:0/2803650232 learned_addr learned my addr 172.21.15.57:0/2803650232 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939ad9d700 1 --2- 172.21.15.57:0/2803650232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018bf0b0 0x55c5018beb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59924/0 (socket says 172.21.15.57:59924) 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939b59e700 1 --2- 172.21.15.57:0/2803650232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 0x55c5018c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939a59c700 1 -- 172.21.15.57:0/2803650232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 msgr2=0x55c5018c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939a59c700 1 --2- 172.21.15.57:0/2803650232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 0x55c5018c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939a59c700 1 -- 172.21.15.57:0/2803650232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018bf0b0 msgr2=0x55c5018beb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939a59c700 1 --2- 172.21.15.57:0/2803650232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018bf0b0 0x55c5018beb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939a59c700 1 -- 172.21.15.57:0/2803650232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c501625c00 con 0x55c5018c4880 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.240+0000 7f939a59c700 1 --2- 172.21.15.57:0/2803650232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c4880 0x55c5018c3d20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9390022890 tx=0x7f939001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9942e40a8809b349 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.241+0000 7f9381ffb700 1 -- 172.21.15.57:0/2803650232 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390095ea0 con 0x55c5018c4880 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.241+0000 7f9381ffb700 1 -- 172.21.15.57:0/2803650232 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f939009b990 con 0x55c5018c4880 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.241+0000 7f9381ffb700 1 -- 172.21.15.57:0/2803650232 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900ce3a0 con 0x55c5018c4880 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.241+0000 7f93ad38db80 1 -- 172.21.15.57:0/2803650232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c4880 msgr2=0x55c5018c3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.241+0000 7f93ad38db80 1 --2- 172.21.15.57:0/2803650232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c4880 0x55c5018c3d20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9390022890 tx=0x7f939001a040 comp rx=0 tx=0).stop 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.241+0000 7f93ad38db80 1 -- 172.21.15.57:0/2803650232 shutdown_connections 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 --2- 172.21.15.57:0/2803650232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018c51a0 0x55c5018c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 --2- 172.21.15.57:0/2803650232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018bf0b0 0x55c5018beb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 --2- 172.21.15.57:0/2803650232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c4880 0x55c5018c3d20 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 -- 172.21.15.57:0/2803650232 >> 172.21.15.57:0/2803650232 conn(0x55c501833200 msgr2=0x55c5018c3610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 -- 172.21.15.57:0/2803650232 shutdown_connections 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 -- 172.21.15.57:0/2803650232 wait complete. 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 -- start start 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 Processor -- start 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018bf0b0 0x55c5018cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c51a0 0x55c5018cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5018cd8d0 con 0x55c5018c51a0 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5018cda40 con 0x55c5018c4880 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5018cdc00 con 0x55c5018bf0b0 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f93ad38db80 10 client.0 _ll_register_callbacks cb 0x7f9394004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:56:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f939ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018bf0b0 0x55c5018cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f939a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f939ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018bf0b0 0x55c5018cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59316/0 (socket says 172.21.15.57:59316) 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f939ad9d700 1 -- 172.21.15.57:0/2923431794 learned_addr learned my addr 172.21.15.57:0/2923431794 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f939a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59932/0 (socket says 172.21.15.57:59932) 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.242+0000 7f939b59e700 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c51a0 0x55c5018cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f939ad9d700 1 -- 172.21.15.57:0/2923431794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 msgr2=0x55c5018cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f939ad9d700 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f939ad9d700 1 -- 172.21.15.57:0/2923431794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c51a0 msgr2=0x55c5018cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f939ad9d700 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c51a0 0x55c5018cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f939ad9d700 1 -- 172.21.15.57:0/2923431794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c1975d0 con 0x55c5018bf0b0 2023-10-08T13:56:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f939a59c700 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f939ad9d700 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018bf0b0 0x55c5018cfa80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f938c011c20 tx=0x7f938c005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcfb7febd32b8d42 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f937eff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:56:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f937eff5700 1 -- 172.21.15.57:0/2923431794 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c02c040 con 0x55c5018bf0b0 2023-10-08T13:56:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f937eff5700 1 -- 172.21.15.57:0/2923431794 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f938c031460 con 0x55c5018bf0b0 2023-10-08T13:56:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f93ad38db80 1 -- 172.21.15.57:0/2923431794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f938c021070 con 0x55c5018bf0b0 2023-10-08T13:56:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f937eff5700 1 -- 172.21.15.57:0/2923431794 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938c00e810 con 0x55c5018bf0b0 2023-10-08T13:56:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f93ad38db80 10 client.26326 fetch_fsmap learned FSMap version 38 2023-10-08T13:56:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.243+0000 7f93ad38db80 1 -- 172.21.15.57:0/2923431794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c5016631f0 con 0x55c5018bf0b0 2023-10-08T13:56:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f937eff5700 1 -- 172.21.15.57:0/2923431794 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f938c021070 con 0x55c5018bf0b0 2023-10-08T13:56:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f937eff5700 1 -- 172.21.15.57:0/2923431794 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f938c031a20 con 0x55c5018bf0b0 2023-10-08T13:56:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f93ad38db80 10 client.26326 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T13:56:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f93ad38db80 10 client.26326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:56:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f93ad38db80 20 client.26326 populate_metadata read hostname 'smithi057' 2023-10-08T13:56:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f93ad38db80 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55c50192d2f0 0x55c50192f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f93ad38db80 4 client.26326 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:56:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f93ad38db80 1 -- 172.21.15.57:0/2923431794 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c50180aad0 con 0x55c50192d2f0 2023-10-08T13:56:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f939a59c700 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55c50192d2f0 0x55c50192f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f937eff5700 10 client.26326 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:17.244+0000 7f939a59c700 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55c50192d2f0 0x55c50192f710 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7458ede823743ca6 server_cookie=3f2c9e4d97e870a6 in_seq=0 out_seq=0 2023-10-08T13:56:18.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:17 smithi099 ceph-mon[103849]: pgmap v441: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 121 op/s 2023-10-08T13:56:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:17 smithi115 ceph-mon[104408]: pgmap v441: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 121 op/s 2023-10-08T13:56:18.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:17 smithi057 ceph-mon[99476]: pgmap v441: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 121 op/s 2023-10-08T13:56:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f937eff5700 1 -- 172.21.15.57:0/2923431794 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c50180aad0 con 0x55c50192d2f0 2023-10-08T13:56:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f937eff5700 10 client.26326 handle_command_reply: tid=0 2023-10-08T13:56:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 client.26326 shutdown 2023-10-08T13:56:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 -- 172.21.15.57:0/2923431794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018bf0b0 msgr2=0x55c5018cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018bf0b0 0x55c5018cfa80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f938c011c20 tx=0x7f938c005700 comp rx=0 tx=0).stop 2023-10-08T13:56:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 -- 172.21.15.57:0/2923431794 shutdown_connections 2023-10-08T13:56:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55c50192d2f0 0x55c50192f710 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c5018c51a0 0x55c5018cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5018bf0b0 0x55c5018cfa80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 --2- 172.21.15.57:0/2923431794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5018c4880 0x55c5018cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f937eff5700 0 client.26326 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 -- 172.21.15.57:0/2923431794 >> 172.21.15.57:0/2923431794 conn(0x55c501833200 msgr2=0x55c5018c3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 -- 172.21.15.57:0/2923431794 shutdown_connections 2023-10-08T13:56:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.923+0000 7f93ad38db80 1 -- 172.21.15.57:0/2923431794 wait complete. 2023-10-08T13:56:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93ad38db80 20 client.26326 trim_cache size 0 max 16384 2023-10-08T13:56:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:56:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:56:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:56:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:56:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 -- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93840a1670 msgr2=0x7f93840a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93840a1670 0x7f93840a3b30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f939c069cf0 tx=0x7f938c009260 comp rx=0 tx=0).stop 2023-10-08T13:56:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 -- 172.21.15.57:0/2338254139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939c0fff90 msgr2=0x7f939c19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939c0fff90 0x7f939c19cdd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9390000c00 tx=0x7f9390004730 comp rx=0 tx=0).stop 2023-10-08T13:56:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 -- 172.21.15.57:0/2338254139 shutdown_connections 2023-10-08T13:56:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f939c1045e0 0x7f939c1969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f93840a1670 0x7f93840a3b30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f939c0ff5d0 0x7f939c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 --2- 172.21.15.57:0/2338254139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939c0fff90 0x7f939c19cdd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 -- 172.21.15.57:0/2338254139 >> 172.21.15.57:0/2338254139 conn(0x7f939c076790 msgr2=0x7f939c067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 -- 172.21.15.57:0/2338254139 shutdown_connections 2023-10-08T13:56:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:19.924+0000 7f93a11a4700 1 -- 172.21.15.57:0/2338254139 wait complete. 2023-10-08T13:56:20.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:19 smithi099 ceph-mon[103849]: pgmap v442: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 151 op/s 2023-10-08T13:56:20.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:19 smithi115 ceph-mon[104408]: pgmap v442: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 151 op/s 2023-10-08T13:56:20.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:19 smithi057 ceph-mon[99476]: pgmap v442: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 151 op/s 2023-10-08T13:56:22.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:22 smithi099 ceph-mon[103849]: pgmap v443: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 706 op/s 2023-10-08T13:56:22.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:22 smithi115 ceph-mon[104408]: pgmap v443: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 706 op/s 2023-10-08T13:56:22.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:22 smithi057 ceph-mon[99476]: pgmap v443: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 706 op/s 2023-10-08T13:56:24.379 INFO:tasks.workunit.client.0.smithi057.stdout: 425 222024 2189 158487 1877 38675 198 2023-10-08T13:56:24.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:24 smithi115 ceph-mon[104408]: pgmap v444: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 828 op/s 2023-10-08T13:56:24.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:24 smithi057 ceph-mon[99476]: pgmap v444: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 828 op/s 2023-10-08T13:56:24.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:24 smithi099 ceph-mon[103849]: pgmap v444: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 828 op/s 2023-10-08T13:56:26.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:26 smithi115 ceph-mon[104408]: pgmap v445: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 1.17k op/s 2023-10-08T13:56:26.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:26 smithi057 ceph-mon[99476]: pgmap v445: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 1.17k op/s 2023-10-08T13:56:26.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:26 smithi099 ceph-mon[103849]: pgmap v445: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 1.17k op/s 2023-10-08T13:56:28.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:28 smithi115 ceph-mon[104408]: pgmap v446: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 15 MiB/s wr, 1.33k op/s 2023-10-08T13:56:28.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:28 smithi099 ceph-mon[103849]: pgmap v446: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 15 MiB/s wr, 1.33k op/s 2023-10-08T13:56:28.526 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:28 smithi057 ceph-mon[99476]: pgmap v446: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 15 MiB/s wr, 1.33k op/s 2023-10-08T13:56:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:30 smithi099 ceph-mon[103849]: pgmap v447: 129 pgs: 129 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-10-08T13:56:30.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:30 smithi057 ceph-mon[99476]: pgmap v447: 129 pgs: 129 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-10-08T13:56:30.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:30 smithi115 ceph-mon[104408]: pgmap v447: 129 pgs: 129 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-10-08T13:56:32.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:32 smithi115 ceph-mon[104408]: pgmap v448: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 24 MiB/s wr, 1.61k op/s 2023-10-08T13:56:32.751 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:32 smithi057 ceph-mon[99476]: pgmap v448: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 24 MiB/s wr, 1.61k op/s 2023-10-08T13:56:32.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:32 smithi099 ceph-mon[103849]: pgmap v448: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 24 MiB/s wr, 1.61k op/s 2023-10-08T13:56:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:33 smithi099 ceph-mon[103849]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 1.14k op/s 2023-10-08T13:56:34.386 INFO:tasks.workunit.client.0.smithi057.stdout: 439 1657457 754 1163160 551 221990 143 2023-10-08T13:56:34.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:33 smithi115 ceph-mon[104408]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 1.14k op/s 2023-10-08T13:56:34.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:33 smithi057 ceph-mon[99476]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 1.14k op/s 2023-10-08T13:56:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.086+0000 7f281546d700 1 -- 172.21.15.57:0/2278069630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281010f380 msgr2=0x7f2810107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.086+0000 7f281546d700 1 --2- 172.21.15.57:0/2278069630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281010f380 0x7f2810107d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f280800b0a0 tx=0x7f280800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.091+0000 7f281546d700 1 -- 172.21.15.57:0/2278069630 shutdown_connections 2023-10-08T13:56:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.091+0000 7f281546d700 1 --2- 172.21.15.57:0/2278069630 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2810119fd0 0x7f281011c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.091+0000 7f281546d700 1 --2- 172.21.15.57:0/2278069630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f281010ef10 0x7f28101077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.091+0000 7f281546d700 1 --2- 172.21.15.57:0/2278069630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281010f380 0x7f2810107d30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.091+0000 7f281546d700 1 -- 172.21.15.57:0/2278069630 >> 172.21.15.57:0/2278069630 conn(0x7f281006b7c0 msgr2=0x7f281006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.091+0000 7f281546d700 1 -- 172.21.15.57:0/2278069630 shutdown_connections 2023-10-08T13:56:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.091+0000 7f281546d700 1 -- 172.21.15.57:0/2278069630 wait complete. 2023-10-08T13:56:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.092+0000 7f281546d700 1 Processor -- start 2023-10-08T13:56:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.093+0000 7f281546d700 1 -- start start 2023-10-08T13:56:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.093+0000 7f281546d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281010ef10 0x7f2810118440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.093+0000 7f281546d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f281010f380 0x7f2810118980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.093+0000 7f281546d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2810119fd0 0x7f2810111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.093+0000 7f281546d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2810111d60 con 0x7f281010f380 2023-10-08T13:56:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.093+0000 7f281546d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2810111ed0 con 0x7f281010ef10 2023-10-08T13:56:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.093+0000 7f281546d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2810112090 con 0x7f2810119fd0 2023-10-08T13:56:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.094+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f281010f380 0x7f2810118980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.094+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f281010f380 0x7f2810118980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58516/0 (socket says 172.21.15.57:58516) 2023-10-08T13:56:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.095+0000 7f280e7fc700 1 -- 172.21.15.57:0/531844708 learned_addr learned my addr 172.21.15.57:0/531844708 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.095+0000 7f280effd700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281010ef10 0x7f2810118440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.095+0000 7f280f7fe700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2810119fd0 0x7f2810111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.102+0000 7f280f7fe700 1 -- 172.21.15.57:0/531844708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281010ef10 msgr2=0x7f2810118440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.102+0000 7f280f7fe700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281010ef10 0x7f2810118440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.102+0000 7f280f7fe700 1 -- 172.21.15.57:0/531844708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f281010f380 msgr2=0x7f2810118980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.102+0000 7f280f7fe700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f281010f380 0x7f2810118980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.102+0000 7f280f7fe700 1 -- 172.21.15.57:0/531844708 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2808009d00 con 0x7f2810119fd0 2023-10-08T13:56:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.105+0000 7f280f7fe700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2810119fd0 0x7f2810111590 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f281006c810 tx=0x7f280000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.114+0000 7f27fffff700 1 -- 172.21.15.57:0/531844708 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800011620 con 0x7f2810119fd0 2023-10-08T13:56:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.114+0000 7f27fffff700 1 -- 172.21.15.57:0/531844708 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2800011c40 con 0x7f2810119fd0 2023-10-08T13:56:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.114+0000 7f27fffff700 1 -- 172.21.15.57:0/531844708 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800010c90 con 0x7f2810119fd0 2023-10-08T13:56:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.114+0000 7f281546d700 1 -- 172.21.15.57:0/531844708 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2810112370 con 0x7f2810119fd0 2023-10-08T13:56:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.114+0000 7f281546d700 1 -- 172.21.15.57:0/531844708 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2810116260 con 0x7f2810119fd0 2023-10-08T13:56:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.121+0000 7f2821656b80 1 Processor -- start 2023-10-08T13:56:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.121+0000 7f2821656b80 1 -- start start 2023-10-08T13:56:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.122+0000 7f2821656b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9da2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.122+0000 7f2821656b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9da31a0 0x55f8f9d9cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.122+0000 7f2821656b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.122+0000 7f2821656b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9d05400 con 0x55f8f9da31a0 2023-10-08T13:56:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.122+0000 7f2821656b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9cf6bc0 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.122+0000 7f2821656b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9cf44e0 con 0x55f8f9da2880 2023-10-08T13:56:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.123+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.123+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9da2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.123+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9da2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59334/0 (socket says 172.21.15.57:59334) 2023-10-08T13:56:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.123+0000 7f280e7fc700 1 -- 172.21.15.57:0/1634730214 learned_addr learned my addr 172.21.15.57:0/1634730214 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.124+0000 7f280effd700 1 --2- 172.21.15.57:0/1634730214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9da31a0 0x55f8f9d9cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.124+0000 7f280e7fc700 1 -- 172.21.15.57:0/1634730214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 msgr2=0x55f8f9da2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.124+0000 7f280e7fc700 1 --2- 172.21.15.57:0/1634730214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9da2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.124+0000 7f280e7fc700 1 -- 172.21.15.57:0/1634730214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9da31a0 msgr2=0x55f8f9d9cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.124+0000 7f280e7fc700 1 --2- 172.21.15.57:0/1634730214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9da31a0 0x55f8f9d9cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.124+0000 7f280e7fc700 1 -- 172.21.15.57:0/1634730214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8f9b1ad80 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.125+0000 7f280e7fc700 1 --2- 172.21.15.57:0/1634730214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9d510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f280800b070 tx=0x7f28080099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e8b240c07b1dd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.128+0000 7f27fffff700 1 -- 172.21.15.57:0/531844708 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2800011780 con 0x7f2810119fd0 2023-10-08T13:56:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.129+0000 7f27fffff700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27f80a1800 0x7f27f80a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.130+0000 7f27fffff700 1 -- 172.21.15.57:0/531844708 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f28000cca70 con 0x7f2810119fd0 2023-10-08T13:56:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.136+0000 7f280effd700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27f80a1800 0x7f27f80a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.157+0000 7f27fd7fa700 1 -- 172.21.15.57:0/1634730214 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280800e070 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.157+0000 7f27fd7fa700 1 -- 172.21.15.57:0/1634730214 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28080079a0 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.157+0000 7f27fd7fa700 1 -- 172.21.15.57:0/1634730214 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28080057f0 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.158+0000 7f2821656b80 1 -- 172.21.15.57:0/1634730214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 msgr2=0x55f8f9d9d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.158+0000 7f2821656b80 1 --2- 172.21.15.57:0/1634730214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9d510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f280800b070 tx=0x7f28080099e0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.159+0000 7f280effd700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27f80a1800 0x7f27f80a3cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55f8f9da3600 tx=0x7f2804008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.159+0000 7f2821656b80 1 -- 172.21.15.57:0/1634730214 shutdown_connections 2023-10-08T13:56:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.159+0000 7f2821656b80 1 --2- 172.21.15.57:0/1634730214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9d510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.159+0000 7f2821656b80 1 --2- 172.21.15.57:0/1634730214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9da2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.159+0000 7f2821656b80 1 --2- 172.21.15.57:0/1634730214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9da31a0 0x55f8f9d9cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.159+0000 7f2821656b80 1 -- 172.21.15.57:0/1634730214 >> 172.21.15.57:0/1634730214 conn(0x55f8f9d11200 msgr2=0x55f8f9d97b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.168+0000 7f2821656b80 1 -- 172.21.15.57:0/1634730214 shutdown_connections 2023-10-08T13:56:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.168+0000 7f2821656b80 1 -- 172.21.15.57:0/1634730214 wait complete. 2023-10-08T13:56:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.169+0000 7f2821656b80 1 -- start start 2023-10-08T13:56:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.169+0000 7f2821656b80 1 Processor -- start 2023-10-08T13:56:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.169+0000 7f2821656b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9da9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.169+0000 7f2821656b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9da64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.169+0000 7f2821656b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da31a0 0x55f8f9da6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.169+0000 7f2821656b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9cefdd0 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.169+0000 7f2821656b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9c8c6b0 con 0x55f8f9da31a0 2023-10-08T13:56:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.169+0000 7f2821656b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9c8c350 con 0x55f8f9da2880 2023-10-08T13:56:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.170+0000 7f2821656b80 10 client.0 _ll_register_callbacks cb 0x55f8f9ae97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:56:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.178+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9da64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.178+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9da64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59344/0 (socket says 172.21.15.57:59344) 2023-10-08T13:56:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.178+0000 7f280e7fc700 1 -- 172.21.15.57:0/2734603552 learned_addr learned my addr 172.21.15.57:0/2734603552 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.178+0000 7f280f7fe700 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da31a0 0x55f8f9da6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.179+0000 7f280f7fe700 1 -- 172.21.15.57:0/2734603552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 msgr2=0x55f8f9da64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.179+0000 7f280f7fe700 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9da64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.179+0000 7f280f7fe700 1 -- 172.21.15.57:0/2734603552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9d9d0b0 msgr2=0x55f8f9da9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.179+0000 7f280f7fe700 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9da9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.179+0000 7f280f7fe700 1 -- 172.21.15.57:0/2734603552 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281011cc50 con 0x55f8f9da31a0 2023-10-08T13:56:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.179+0000 7f280f7fe700 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da31a0 0x55f8f9da6c30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28000c9f70 tx=0x7f2800099880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12bb683b5082e891 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.338+0000 7f27f67fc700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:56:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.338+0000 7f27f67fc700 1 -- 172.21.15.57:0/2734603552 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800010df0 con 0x55f8f9da31a0 2023-10-08T13:56:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.338+0000 7f27f67fc700 1 -- 172.21.15.57:0/2734603552 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f28000b90b0 con 0x55f8f9da31a0 2023-10-08T13:56:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.340+0000 7f2821656b80 1 -- 172.21.15.57:0/2734603552 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8f9cb7e50 con 0x55f8f9da31a0 2023-10-08T13:56:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.341+0000 7f27f67fc700 1 -- 172.21.15.57:0/2734603552 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28000ca0e0 con 0x55f8f9da31a0 2023-10-08T13:56:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.350+0000 7f2821656b80 10 client.26279 fetch_fsmap learned FSMap version 38 2023-10-08T13:56:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.350+0000 7f2821656b80 1 -- 172.21.15.57:0/2734603552 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55f8f9b03c00 con 0x55f8f9da31a0 2023-10-08T13:56:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.353+0000 7f27f67fc700 1 -- 172.21.15.57:0/2734603552 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8f9cb7e50 con 0x55f8f9da31a0 2023-10-08T13:56:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.353+0000 7f27f67fc700 1 -- 172.21.15.57:0/2734603552 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f2800099a60 con 0x55f8f9da31a0 2023-10-08T13:56:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.354+0000 7f2821656b80 10 client.26279 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T13:56:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.354+0000 7f2821656b80 10 client.26279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:56:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.354+0000 7f2821656b80 20 client.26279 populate_metadata read hostname 'smithi057' 2023-10-08T13:56:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.354+0000 7f2821656b80 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f8f9db6e10 0x55f8f9db9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.354+0000 7f2821656b80 4 client.26279 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:56:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.354+0000 7f2821656b80 1 -- 172.21.15.57:0/2734603552 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8f9ce8db0 con 0x55f8f9db6e10 2023-10-08T13:56:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.355+0000 7f280effd700 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f8f9db6e10 0x55f8f9db9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.355+0000 7f280effd700 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f8f9db6e10 0x55f8f9db9230 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1b7a369b7cdbed server_cookie=816462c7c592f46f in_seq=0 out_seq=0 2023-10-08T13:56:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.362+0000 7f27f67fc700 10 client.26279 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.362+0000 7f27f67fc700 1 -- 172.21.15.57:0/2734603552 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f8f9ce8db0 con 0x55f8f9db6e10 2023-10-08T13:56:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.362+0000 7f27f67fc700 10 client.26279 handle_command_reply: tid=0 2023-10-08T13:56:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.362+0000 7f2821656b80 1 client.26279 shutdown 2023-10-08T13:56:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.366+0000 7f2821656b80 1 -- 172.21.15.57:0/2734603552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da31a0 msgr2=0x55f8f9da6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.366+0000 7f2821656b80 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da31a0 0x55f8f9da6c30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28000c9f70 tx=0x7f2800099880 comp rx=0 tx=0).stop 2023-10-08T13:56:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.367+0000 7f2821656b80 1 -- 172.21.15.57:0/2734603552 shutdown_connections 2023-10-08T13:56:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.367+0000 7f2821656b80 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f8f9db6e10 0x55f8f9db9230 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.367+0000 7f2821656b80 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da31a0 0x55f8f9da6c30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.367+0000 7f2821656b80 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9da9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.367+0000 7f2821656b80 1 --2- 172.21.15.57:0/2734603552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9da64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.367+0000 7f2821656b80 1 -- 172.21.15.57:0/2734603552 >> 172.21.15.57:0/2734603552 conn(0x55f8f9d11200 msgr2=0x55f8f9d97010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.367+0000 7f27f67fc700 0 client.26279 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.373+0000 7f2821656b80 1 -- 172.21.15.57:0/2734603552 shutdown_connections 2023-10-08T13:56:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.373+0000 7f2821656b80 1 -- 172.21.15.57:0/2734603552 wait complete. 2023-10-08T13:56:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.374+0000 7f2821656b80 20 client.26279 trim_cache size 0 max 16384 2023-10-08T13:56:36.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:36 smithi115 ceph-mon[104408]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 26 MiB/s wr, 1.02k op/s 2023-10-08T13:56:36.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:36 smithi057 ceph-mon[99476]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 26 MiB/s wr, 1.02k op/s 2023-10-08T13:56:36.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:36 smithi099 ceph-mon[103849]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 26 MiB/s wr, 1.02k op/s 2023-10-08T13:56:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.538+0000 7f2821656b80 1 Processor -- start 2023-10-08T13:56:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.539+0000 7f2821656b80 1 -- start start 2023-10-08T13:56:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.539+0000 7f2821656b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.540+0000 7f2821656b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da2880 0x55f8f9d97790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.540+0000 7f2821656b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da31a0 0x55f8f9d97cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.540+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da2880 0x55f8f9d97790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.540+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da2880 0x55f8f9d97790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59966/0 (socket says 172.21.15.57:59966) 2023-10-08T13:56:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.540+0000 7f280e7fc700 1 -- 172.21.15.57:0/1049846496 learned_addr learned my addr 172.21.15.57:0/1049846496 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.541+0000 7f280f7fe700 1 --2- 172.21.15.57:0/1049846496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da31a0 0x55f8f9d97cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.541+0000 7f2821656b80 1 -- 172.21.15.57:0/1049846496 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9b708c0 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.541+0000 7f2821656b80 1 -- 172.21.15.57:0/1049846496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9d922b0 con 0x55f8f9da2880 2023-10-08T13:56:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.541+0000 7f2821656b80 1 -- 172.21.15.57:0/1049846496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9da7230 con 0x55f8f9da31a0 2023-10-08T13:56:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.542+0000 7f280effd700 1 --2- 172.21.15.57:0/1049846496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.549+0000 7f280effd700 1 -- 172.21.15.57:0/1049846496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da31a0 msgr2=0x55f8f9d97cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.549+0000 7f280effd700 1 --2- 172.21.15.57:0/1049846496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da31a0 0x55f8f9d97cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.549+0000 7f280effd700 1 -- 172.21.15.57:0/1049846496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da2880 msgr2=0x55f8f9d97790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.549+0000 7f280effd700 1 --2- 172.21.15.57:0/1049846496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da2880 0x55f8f9d97790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.549+0000 7f280effd700 1 -- 172.21.15.57:0/1049846496 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8f9b411f0 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.554+0000 7f280effd700 1 --2- 172.21.15.57:0/1049846496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9cb70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55f8f9db9ae0 tx=0x7f2804004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89cdc8c223670ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.687+0000 7f27fdffb700 1 -- 172.21.15.57:0/1049846496 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804011e70 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.687+0000 7f27fdffb700 1 -- 172.21.15.57:0/1049846496 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2804022440 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.687+0000 7f27fdffb700 1 -- 172.21.15.57:0/1049846496 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28040156a0 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.696+0000 7f2821656b80 1 -- 172.21.15.57:0/1049846496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9d9d0b0 msgr2=0x55f8f9d9cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.696+0000 7f2821656b80 1 --2- 172.21.15.57:0/1049846496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9cb70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55f8f9db9ae0 tx=0x7f2804004860 comp rx=0 tx=0).stop 2023-10-08T13:56:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.696+0000 7f2821656b80 1 -- 172.21.15.57:0/1049846496 shutdown_connections 2023-10-08T13:56:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.696+0000 7f2821656b80 1 --2- 172.21.15.57:0/1049846496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da31a0 0x55f8f9d97cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.696+0000 7f2821656b80 1 --2- 172.21.15.57:0/1049846496 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9cb70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.696+0000 7f2821656b80 1 --2- 172.21.15.57:0/1049846496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9da2880 0x55f8f9d97790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.696+0000 7f2821656b80 1 -- 172.21.15.57:0/1049846496 >> 172.21.15.57:0/1049846496 conn(0x55f8f9d11200 msgr2=0x55f8f9da0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.696+0000 7f2821656b80 1 -- 172.21.15.57:0/1049846496 shutdown_connections 2023-10-08T13:56:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.696+0000 7f2821656b80 1 -- 172.21.15.57:0/1049846496 wait complete. 2023-10-08T13:56:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.697+0000 7f2821656b80 1 -- start start 2023-10-08T13:56:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.697+0000 7f2821656b80 1 Processor -- start 2023-10-08T13:56:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.697+0000 7f2821656b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.697+0000 7f2821656b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9d9fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.697+0000 7f2821656b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9da31a0 0x55f8f9d9e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.697+0000 7f2821656b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9d9e8d0 con 0x55f8f9da31a0 2023-10-08T13:56:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.697+0000 7f2821656b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9d9ea40 con 0x55f8f9d9d0b0 2023-10-08T13:56:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.697+0000 7f2821656b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f9d9ec00 con 0x55f8f9da2880 2023-10-08T13:56:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.697+0000 7f2821656b80 10 client.0 _ll_register_callbacks cb 0x7f2800099160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:56:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.698+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9d9fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.698+0000 7f280e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9d9fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59356/0 (socket says 172.21.15.57:59356) 2023-10-08T13:56:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.698+0000 7f280e7fc700 1 -- 172.21.15.57:0/616182105 learned_addr learned my addr 172.21.15.57:0/616182105 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.698+0000 7f280effd700 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.702+0000 7f280e7fc700 1 -- 172.21.15.57:0/616182105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 msgr2=0x55f8f9d9f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.702+0000 7f280e7fc700 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.702+0000 7f280e7fc700 1 -- 172.21.15.57:0/616182105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9da31a0 msgr2=0x55f8f9d9e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.702+0000 7f280e7fc700 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9da31a0 0x55f8f9d9e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.702+0000 7f280e7fc700 1 -- 172.21.15.57:0/616182105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2804011750 con 0x55f8f9da2880 2023-10-08T13:56:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.713+0000 7f280e7fc700 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9d9fc20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f280800b830 tx=0x7f2808003c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e53c31161091aca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.780+0000 7f27f67fc700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:56:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.780+0000 7f27f67fc700 1 -- 172.21.15.57:0/616182105 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280800e040 con 0x55f8f9da2880 2023-10-08T13:56:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.780+0000 7f27f67fc700 1 -- 172.21.15.57:0/616182105 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f28080054d0 con 0x55f8f9da2880 2023-10-08T13:56:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.780+0000 7f2821656b80 1 -- 172.21.15.57:0/616182105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28080079a0 con 0x55f8f9da2880 2023-10-08T13:56:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.793+0000 7f27f67fc700 1 -- 172.21.15.57:0/616182105 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28080056b0 con 0x55f8f9da2880 2023-10-08T13:56:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.804+0000 7f2821656b80 10 client.26341 fetch_fsmap learned FSMap version 38 2023-10-08T13:56:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.804+0000 7f2821656b80 1 -- 172.21.15.57:0/616182105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55f8f9b19250 con 0x55f8f9da2880 2023-10-08T13:56:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.811+0000 7f27f67fc700 1 -- 172.21.15.57:0/616182105 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2808030030 con 0x55f8f9da2880 2023-10-08T13:56:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.811+0000 7f27f67fc700 1 -- 172.21.15.57:0/616182105 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f280800b0d0 con 0x55f8f9da2880 2023-10-08T13:56:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.812+0000 7f2821656b80 10 client.26341 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T13:56:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.812+0000 7f2821656b80 10 client.26341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:56:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.812+0000 7f2821656b80 20 client.26341 populate_metadata read hostname 'smithi057' 2023-10-08T13:56:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.812+0000 7f2821656b80 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f8f9e0b8c0 0x55f8f9e0dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.812+0000 7f2821656b80 4 client.26341 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:56:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.812+0000 7f2821656b80 1 -- 172.21.15.57:0/616182105 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f8f9ce8ad0 con 0x55f8f9e0b8c0 2023-10-08T13:56:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.815+0000 7f280effd700 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f8f9e0b8c0 0x55f8f9e0dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.822+0000 7f280effd700 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f8f9e0b8c0 0x55f8f9e0dce0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02e910441adfab6 server_cookie=815a1866fe8a1300 in_seq=0 out_seq=0 2023-10-08T13:56:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:36.831+0000 7f27f67fc700 10 client.26341 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:38.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:38 smithi115 ceph-mon[104408]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 18 MiB/s wr, 715 op/s 2023-10-08T13:56:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.483+0000 7f27f67fc700 1 -- 172.21.15.57:0/616182105 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f8f9ce8ad0 con 0x55f8f9e0b8c0 2023-10-08T13:56:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.483+0000 7f27f67fc700 10 client.26341 handle_command_reply: tid=0 2023-10-08T13:56:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.483+0000 7f2821656b80 1 client.26341 shutdown 2023-10-08T13:56:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.484+0000 7f2821656b80 1 -- 172.21.15.57:0/616182105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 msgr2=0x55f8f9d9fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.484+0000 7f2821656b80 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9d9fc20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f280800b830 tx=0x7f2808003c50 comp rx=0 tx=0).stop 2023-10-08T13:56:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.484+0000 7f2821656b80 1 -- 172.21.15.57:0/616182105 shutdown_connections 2023-10-08T13:56:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.484+0000 7f2821656b80 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f8f9e0b8c0 0x55f8f9e0dce0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.484+0000 7f2821656b80 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f8f9da31a0 0x55f8f9d9e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.484+0000 7f2821656b80 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8f9d9d0b0 0x55f8f9d9f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.484+0000 7f2821656b80 1 --2- 172.21.15.57:0/616182105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f8f9da2880 0x55f8f9d9fc20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.484+0000 7f2821656b80 1 -- 172.21.15.57:0/616182105 >> 172.21.15.57:0/616182105 conn(0x55f8f9d11200 msgr2=0x55f8f9da4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.484+0000 7f27f67fc700 0 client.26341 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.485+0000 7f2821656b80 1 -- 172.21.15.57:0/616182105 shutdown_connections 2023-10-08T13:56:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.485+0000 7f2821656b80 1 -- 172.21.15.57:0/616182105 wait complete. 2023-10-08T13:56:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.485+0000 7f2821656b80 20 client.26341 trim_cache size 0 max 16384 2023-10-08T13:56:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:56:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:56:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:56:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:56:38.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:38 smithi057 ceph-mon[99476]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 18 MiB/s wr, 715 op/s 2023-10-08T13:56:38.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:38 smithi099 ceph-mon[103849]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 18 MiB/s wr, 715 op/s 2023-10-08T13:56:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.568+0000 7f281546d700 1 -- 172.21.15.57:0/531844708 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27f80a1800 msgr2=0x7f27f80a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.568+0000 7f281546d700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27f80a1800 0x7f27f80a3cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55f8f9da3600 tx=0x7f2804008040 comp rx=0 tx=0).stop 2023-10-08T13:56:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.568+0000 7f281546d700 1 -- 172.21.15.57:0/531844708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2810119fd0 msgr2=0x7f2810111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.568+0000 7f281546d700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2810119fd0 0x7f2810111590 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f281006c810 tx=0x7f280000b670 comp rx=0 tx=0).stop 2023-10-08T13:56:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.569+0000 7f281546d700 1 -- 172.21.15.57:0/531844708 shutdown_connections 2023-10-08T13:56:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.569+0000 7f281546d700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2810119fd0 0x7f2810111590 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.569+0000 7f281546d700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f27f80a1800 0x7f27f80a3cc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.569+0000 7f281546d700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281010ef10 0x7f2810118440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.569+0000 7f281546d700 1 --2- 172.21.15.57:0/531844708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f281010f380 0x7f2810118980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.569+0000 7f281546d700 1 -- 172.21.15.57:0/531844708 >> 172.21.15.57:0/531844708 conn(0x7f281006b7c0 msgr2=0x7f281006f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.569+0000 7f281546d700 1 -- 172.21.15.57:0/531844708 shutdown_connections 2023-10-08T13:56:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:38.570+0000 7f281546d700 1 -- 172.21.15.57:0/531844708 wait complete. 2023-10-08T13:56:39.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:39 smithi057 ceph-mon[99476]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 544 op/s 2023-10-08T13:56:39.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:39 smithi099 ceph-mon[103849]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 544 op/s 2023-10-08T13:56:39.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:39 smithi115 ceph-mon[104408]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 544 op/s 2023-10-08T13:56:40.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:40 smithi057 ceph-mon[99476]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 636 op/s 2023-10-08T13:56:40.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:40 smithi099 ceph-mon[103849]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 636 op/s 2023-10-08T13:56:40.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:40 smithi115 ceph-mon[104408]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 636 op/s 2023-10-08T13:56:42.435 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:56:42.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:42 smithi099 ceph-mon[103849]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 465 op/s 2023-10-08T13:56:42.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:42 smithi115 ceph-mon[104408]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 465 op/s 2023-10-08T13:56:43.167 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:42 smithi057 ceph-mon[99476]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.2 MiB/s wr, 465 op/s 2023-10-08T13:56:44.406 INFO:tasks.workunit.client.0.smithi057.stdout: 450 2396883 599 1676313 375 342631 910 2023-10-08T13:56:44.747 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:44 smithi057 ceph-mon[99476]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 380 op/s 2023-10-08T13:56:44.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:44 smithi099 ceph-mon[103849]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 380 op/s 2023-10-08T13:56:44.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:44 smithi115 ceph-mon[104408]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 380 op/s 2023-10-08T13:56:46.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:46 smithi057 ceph-mon[99476]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 351 op/s 2023-10-08T13:56:46.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:46 smithi099 ceph-mon[103849]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 351 op/s 2023-10-08T13:56:46.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:46 smithi115 ceph-mon[104408]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 351 op/s 2023-10-08T13:56:47.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.512+0000 7f84b5816700 1 -- 172.21.15.57:0/874446440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b0119c80 msgr2=0x7f84b011c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:47.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.512+0000 7f84b5816700 1 --2- 172.21.15.57:0/874446440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b0119c80 0x7f84b011c070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b0a0 tx=0x7f84a800b3b0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.523+0000 7f84b5816700 1 -- 172.21.15.57:0/874446440 shutdown_connections 2023-10-08T13:56:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.523+0000 7f84b5816700 1 --2- 172.21.15.57:0/874446440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b0119c80 0x7f84b011c070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.523+0000 7f84b5816700 1 --2- 172.21.15.57:0/874446440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84b010c8d0 0x7f84b010ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.523+0000 7f84b5816700 1 --2- 172.21.15.57:0/874446440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b010d1f0 0x7f84b0119740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.523+0000 7f84b5816700 1 -- 172.21.15.57:0/874446440 >> 172.21.15.57:0/874446440 conn(0x7f84b006b650 msgr2=0x7f84b006ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:47.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.523+0000 7f84b5816700 1 -- 172.21.15.57:0/874446440 shutdown_connections 2023-10-08T13:56:47.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.523+0000 7f84b5816700 1 -- 172.21.15.57:0/874446440 wait complete. 2023-10-08T13:56:47.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.524+0000 7f84b5816700 1 Processor -- start 2023-10-08T13:56:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.524+0000 7f84b5816700 1 -- start start 2023-10-08T13:56:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.525+0000 7f84b5816700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84b010c8d0 0x7f84b01ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.525+0000 7f84b5816700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b010d1f0 0x7f84b01abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.525+0000 7f84b5816700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b0119c80 0x7f84b01a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.525+0000 7f84b5816700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b01a5e70 con 0x7f84b010c8d0 2023-10-08T13:56:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.525+0000 7f84b5816700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b01a5fe0 con 0x7f84b010d1f0 2023-10-08T13:56:47.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.525+0000 7f84b5816700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b01a61a0 con 0x7f84b0119c80 2023-10-08T13:56:47.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.527+0000 7f84b4814700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84b010c8d0 0x7f84b01ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.530+0000 7f84affff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b010d1f0 0x7f84b01abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.530+0000 7f84affff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b010d1f0 0x7f84b01abc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59986/0 (socket says 172.21.15.57:59986) 2023-10-08T13:56:47.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.530+0000 7f84affff700 1 -- 172.21.15.57:0/1084195472 learned_addr learned my addr 172.21.15.57:0/1084195472 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:47.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.530+0000 7f84b5015700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b0119c80 0x7f84b01a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.531+0000 7f84b5015700 1 -- 172.21.15.57:0/1084195472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b010d1f0 msgr2=0x7f84b01abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.531+0000 7f84b5015700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b010d1f0 0x7f84b01abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.531+0000 7f84b5015700 1 -- 172.21.15.57:0/1084195472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84b010c8d0 msgr2=0x7f84b01ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.531+0000 7f84b5015700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84b010c8d0 0x7f84b01ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.531+0000 7f84b5015700 1 -- 172.21.15.57:0/1084195472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a8009d00 con 0x7f84b0119c80 2023-10-08T13:56:47.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.543+0000 7f84b5015700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b0119c80 0x7f84b01a5800 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f84b0069a50 tx=0x7f84a80077f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:47.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.590+0000 7f84adffb700 1 -- 172.21.15.57:0/1084195472 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a800e040 con 0x7f84b0119c80 2023-10-08T13:56:47.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.590+0000 7f84adffb700 1 -- 172.21.15.57:0/1084195472 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a8004c60 con 0x7f84b0119c80 2023-10-08T13:56:47.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.590+0000 7f84adffb700 1 -- 172.21.15.57:0/1084195472 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a80052a0 con 0x7f84b0119c80 2023-10-08T13:56:47.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.590+0000 7f84b5816700 1 -- 172.21.15.57:0/1084195472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b01a6420 con 0x7f84b0119c80 2023-10-08T13:56:47.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.590+0000 7f84b5816700 1 -- 172.21.15.57:0/1084195472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b010f100 con 0x7f84b0119c80 2023-10-08T13:56:47.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.600+0000 7f84adffb700 1 -- 172.21.15.57:0/1084195472 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f84a8005400 con 0x7f84b0119c80 2023-10-08T13:56:47.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.601+0000 7f84adffb700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84980a1800 0x7f84980a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.602+0000 7f84adffb700 1 -- 172.21.15.57:0/1084195472 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f84a80cf850 con 0x7f84b0119c80 2023-10-08T13:56:47.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.604+0000 7f84b4814700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84980a1800 0x7f84980a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.613+0000 7f84c2c61b80 1 Processor -- start 2023-10-08T13:56:47.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.613+0000 7f84c2c61b80 1 -- start start 2023-10-08T13:56:47.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.614+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b050 0x557717a7b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.614+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.614+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a86800 0x557717a88bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.614+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577179e4460 con 0x557717a86800 2023-10-08T13:56:47.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.614+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577179d5c20 con 0x557717a7b970 2023-10-08T13:56:47.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.614+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577179d3540 con 0x557717a7b050 2023-10-08T13:56:47.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.614+0000 7f84affff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.614+0000 7f84affff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a862c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:59994/0 (socket says 172.21.15.57:59994) 2023-10-08T13:56:47.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.614+0000 7f84affff700 1 -- 172.21.15.57:0/137975115 learned_addr learned my addr 172.21.15.57:0/137975115 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:47.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.619+0000 7f84b5015700 1 --2- 172.21.15.57:0/137975115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a86800 0x557717a88bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.621+0000 7f84b4814700 1 --2- 172.21.15.57:0/137975115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b050 0x557717a7b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.621+0000 7f84affff700 1 -- 172.21.15.57:0/137975115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b050 msgr2=0x557717a7b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:47.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.621+0000 7f84affff700 1 --2- 172.21.15.57:0/137975115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b050 0x557717a7b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.621+0000 7f84affff700 1 -- 172.21.15.57:0/137975115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a86800 msgr2=0x557717a88bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:47.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.621+0000 7f84affff700 1 --2- 172.21.15.57:0/137975115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a86800 0x557717a88bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.621+0000 7f84affff700 1 -- 172.21.15.57:0/137975115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577177f9de0 con 0x557717a7b970 2023-10-08T13:56:47.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.622+0000 7f84affff700 1 --2- 172.21.15.57:0/137975115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a862c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f84a400acc0 tx=0x7f84a400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7840aa8159f35187 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:47.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.636+0000 7f84b4814700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84980a1800 0x7f84980a3cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f84a0005950 tx=0x7f84a0012670 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:47.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.859+0000 7f8496ffd700 1 -- 172.21.15.57:0/137975115 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a400fed0 con 0x557717a7b970 2023-10-08T13:56:47.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.859+0000 7f8496ffd700 1 -- 172.21.15.57:0/137975115 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a4018900 con 0x557717a7b970 2023-10-08T13:56:47.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.859+0000 7f8496ffd700 1 -- 172.21.15.57:0/137975115 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40109f0 con 0x557717a7b970 2023-10-08T13:56:47.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.870+0000 7f84c2c61b80 1 -- 172.21.15.57:0/137975115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 msgr2=0x557717a862c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:47.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.870+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/137975115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a862c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f84a400acc0 tx=0x7f84a400f3c0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.871+0000 7f84c2c61b80 1 -- 172.21.15.57:0/137975115 shutdown_connections 2023-10-08T13:56:47.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.871+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/137975115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a86800 0x557717a88bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.871+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/137975115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b050 0x557717a7b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.871+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/137975115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a862c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.871+0000 7f84c2c61b80 1 -- 172.21.15.57:0/137975115 >> 172.21.15.57:0/137975115 conn(0x5577179f0260 msgr2=0x557717a77460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:47.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.871+0000 7f84c2c61b80 1 -- 172.21.15.57:0/137975115 shutdown_connections 2023-10-08T13:56:47.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.871+0000 7f84c2c61b80 1 -- 172.21.15.57:0/137975115 wait complete. 2023-10-08T13:56:47.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.872+0000 7f84c2c61b80 1 -- start start 2023-10-08T13:56:47.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.872+0000 7f84c2c61b80 1 Processor -- start 2023-10-08T13:56:47.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.873+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 0x557717a86240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.873+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b970 0x557717a83530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.873+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a83c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.873+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577179cee30 con 0x557717a7b050 2023-10-08T13:56:47.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.873+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771796b710 con 0x557717a86800 2023-10-08T13:56:47.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.873+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771796b3b0 con 0x557717a7b970 2023-10-08T13:56:47.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.873+0000 7f84c2c61b80 10 client.0 _ll_register_callbacks cb 0x5577177c8800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:56:47.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.882+0000 7f84b5015700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a83c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.882+0000 7f84b5015700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a83c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60000/0 (socket says 172.21.15.57:60000) 2023-10-08T13:56:47.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.882+0000 7f84b5015700 1 -- 172.21.15.57:0/1483567245 learned_addr learned my addr 172.21.15.57:0/1483567245 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:47.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.882+0000 7f84affff700 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b970 0x557717a83530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.883+0000 7f84b4814700 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 0x557717a86240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.894+0000 7f84b5015700 1 -- 172.21.15.57:0/1483567245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b970 msgr2=0x557717a83530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:47.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.894+0000 7f84b5015700 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b970 0x557717a83530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.894+0000 7f84b5015700 1 -- 172.21.15.57:0/1483567245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 msgr2=0x557717a86240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:47.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.894+0000 7f84b5015700 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 0x557717a86240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:47.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.894+0000 7f84b5015700 1 -- 172.21.15.57:0/1483567245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a400a8f0 con 0x557717a86800 2023-10-08T13:56:47.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.897+0000 7f84b5015700 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a83c70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f84a8098a60 tx=0x7f84a809c0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10bc85f94f48315 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.912+0000 7f848ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:56:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.912+0000 7f848ffff700 1 -- 172.21.15.57:0/1483567245 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a809cc60 con 0x557717a86800 2023-10-08T13:56:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.912+0000 7f848ffff700 1 -- 172.21.15.57:0/1483567245 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f84a8004dd0 con 0x557717a86800 2023-10-08T13:56:47.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.912+0000 7f84c2c61b80 1 -- 172.21.15.57:0/1483567245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557717996eb0 con 0x557717a86800 2023-10-08T13:56:47.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.919+0000 7f848ffff700 1 -- 172.21.15.57:0/1483567245 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84a809c560 con 0x557717a86800 2023-10-08T13:56:47.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.926+0000 7f84c2c61b80 10 client.26297 fetch_fsmap learned FSMap version 38 2023-10-08T13:56:47.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.926+0000 7f84c2c61b80 1 -- 172.21.15.57:0/1483567245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5577177e2c60 con 0x557717a86800 2023-10-08T13:56:47.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.937+0000 7f848ffff700 1 -- 172.21.15.57:0/1483567245 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557717996eb0 con 0x557717a86800 2023-10-08T13:56:47.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.937+0000 7f848ffff700 1 -- 172.21.15.57:0/1483567245 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f84a80b1070 con 0x557717a86800 2023-10-08T13:56:47.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.938+0000 7f84c2c61b80 10 client.26297 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T13:56:47.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.938+0000 7f84c2c61b80 10 client.26297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:56:47.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.939+0000 7f84c2c61b80 20 client.26297 populate_metadata read hostname 'smithi057' 2023-10-08T13:56:47.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.939+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557717a96220 0x557717a98640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:47.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.939+0000 7f84c2c61b80 4 client.26297 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:56:47.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.939+0000 7f84c2c61b80 1 -- 172.21.15.57:0/1483567245 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577179c7e10 con 0x557717a96220 2023-10-08T13:56:47.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.943+0000 7f84b4814700 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557717a96220 0x557717a98640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:47.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.985+0000 7f84b4814700 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557717a96220 0x557717a98640 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55bbbd9bda7502d server_cookie=73a4f00ed486d034 in_seq=0 out_seq=0 2023-10-08T13:56:47.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.996+0000 7f848ffff700 10 client.26297 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:47.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.996+0000 7f848ffff700 1 -- 172.21.15.57:0/1483567245 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5577179c7e10 con 0x557717a96220 2023-10-08T13:56:47.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.996+0000 7f848ffff700 10 client.26297 handle_command_reply: tid=0 2023-10-08T13:56:47.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:47.996+0000 7f84c2c61b80 1 client.26297 shutdown 2023-10-08T13:56:48.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.007+0000 7f84c2c61b80 1 -- 172.21.15.57:0/1483567245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 msgr2=0x557717a83c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:48.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.007+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a83c70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f84a8098a60 tx=0x7f84a809c0a0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.007+0000 7f84c2c61b80 1 -- 172.21.15.57:0/1483567245 shutdown_connections 2023-10-08T13:56:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.007+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557717a96220 0x557717a98640 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.007+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a83c70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.007+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 0x557717a86240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.007+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/1483567245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b970 0x557717a83530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.007+0000 7f84c2c61b80 1 -- 172.21.15.57:0/1483567245 >> 172.21.15.57:0/1483567245 conn(0x5577179f0260 msgr2=0x557717a7d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.008+0000 7f848ffff700 0 client.26297 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.008+0000 7f84c2c61b80 1 -- 172.21.15.57:0/1483567245 shutdown_connections 2023-10-08T13:56:48.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.008+0000 7f84c2c61b80 1 -- 172.21.15.57:0/1483567245 wait complete. 2023-10-08T13:56:48.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.008+0000 7f84c2c61b80 20 client.26297 trim_cache size 0 max 16384 2023-10-08T13:56:48.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.282+0000 7f84c2c61b80 1 Processor -- start 2023-10-08T13:56:48.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.282+0000 7f84c2c61b80 1 -- start start 2023-10-08T13:56:48.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.283+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b050 0x557717a8d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:48.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.283+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b970 0x557717a7e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:48.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.283+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a7ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:48.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.284+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771784f920 con 0x557717a7b970 2023-10-08T13:56:48.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.284+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557717a92ea0 con 0x557717a86800 2023-10-08T13:56:48.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.284+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557717a73c40 con 0x557717a7b050 2023-10-08T13:56:48.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.286+0000 7f84b5015700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a7ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.286+0000 7f84b5015700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a7ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60010/0 (socket says 172.21.15.57:60010) 2023-10-08T13:56:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.286+0000 7f84b5015700 1 -- 172.21.15.57:0/657474560 learned_addr learned my addr 172.21.15.57:0/657474560 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.286+0000 7f84b4814700 1 --2- 172.21.15.57:0/657474560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b050 0x557717a8d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.286+0000 7f84b5015700 1 -- 172.21.15.57:0/657474560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b050 msgr2=0x557717a8d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.286+0000 7f84b5015700 1 --2- 172.21.15.57:0/657474560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b050 0x557717a8d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.286+0000 7f84b5015700 1 -- 172.21.15.57:0/657474560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b970 msgr2=0x557717a7e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.286+0000 7f84b5015700 1 --2- 172.21.15.57:0/657474560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b970 0x557717a7e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.286+0000 7f84b5015700 1 -- 172.21.15.57:0/657474560 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557717820250 con 0x557717a86800 2023-10-08T13:56:48.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.293+0000 7f84b5015700 1 --2- 172.21.15.57:0/657474560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a7ed40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f84a80a4330 tx=0x7f84a801bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45ba2d566d05aca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:48.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.302+0000 7f84977fe700 1 -- 172.21.15.57:0/657474560 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8004dd0 con 0x557717a86800 2023-10-08T13:56:48.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.302+0000 7f84977fe700 1 -- 172.21.15.57:0/657474560 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a809cc60 con 0x557717a86800 2023-10-08T13:56:48.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.302+0000 7f84977fe700 1 -- 172.21.15.57:0/657474560 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a80a06b0 con 0x557717a86800 2023-10-08T13:56:48.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.380+0000 7f84c2c61b80 1 -- 172.21.15.57:0/657474560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 msgr2=0x557717a7ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:48.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.380+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/657474560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a7ed40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f84a80a4330 tx=0x7f84a801bf20 comp rx=0 tx=0).stop 2023-10-08T13:56:48.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.390+0000 7f84c2c61b80 1 -- 172.21.15.57:0/657474560 shutdown_connections 2023-10-08T13:56:48.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.390+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/657474560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a86800 0x557717a7ed40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.390+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/657474560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a7b050 0x557717a8d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.390+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/657474560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b970 0x557717a7e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.390+0000 7f84c2c61b80 1 -- 172.21.15.57:0/657474560 >> 172.21.15.57:0/657474560 conn(0x5577179f0260 msgr2=0x557717a85e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:48.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.391+0000 7f84c2c61b80 1 -- 172.21.15.57:0/657474560 shutdown_connections 2023-10-08T13:56:48.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.391+0000 7f84c2c61b80 1 -- 172.21.15.57:0/657474560 wait complete. 2023-10-08T13:56:48.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.392+0000 7f84c2c61b80 1 -- start start 2023-10-08T13:56:48.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.392+0000 7f84c2c61b80 1 Processor -- start 2023-10-08T13:56:48.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.393+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 0x557717a8cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:48.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.394+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a8b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:48.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.394+0000 7f84c2c61b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a86800 0x557717a8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:48.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.394+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557717a8bde0 con 0x557717a7b050 2023-10-08T13:56:48.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.394+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557717a8bf50 con 0x557717a7b970 2023-10-08T13:56:48.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.394+0000 7f84c2c61b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557717a8c0c0 con 0x557717a86800 2023-10-08T13:56:48.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.394+0000 7f84c2c61b80 10 client.0 _ll_register_callbacks cb 0x7f84a8009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.394+0000 7f84b4814700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 0x557717a8cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.394+0000 7f84b4814700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 0x557717a8cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58586/0 (socket says 172.21.15.57:58586) 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.394+0000 7f84b4814700 1 -- 172.21.15.57:0/2025914187 learned_addr learned my addr 172.21.15.57:0/2025914187 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.395+0000 7f84affff700 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a8b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.395+0000 7f84b5015700 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a86800 0x557717a8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.395+0000 7f84affff700 1 -- 172.21.15.57:0/2025914187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a86800 msgr2=0x557717a8b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.395+0000 7f84affff700 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a86800 0x557717a8b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.395+0000 7f84affff700 1 -- 172.21.15.57:0/2025914187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 msgr2=0x557717a8cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.395+0000 7f84affff700 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 0x557717a8cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.395+0000 7f84affff700 1 -- 172.21.15.57:0/2025914187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577177e2c60 con 0x557717a7b970 2023-10-08T13:56:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.396+0000 7f84affff700 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a8b360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f84a400afd0 tx=0x7f84a4018ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de729d1af3c21b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:48.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.688+0000 7f848ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:56:48.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.688+0000 7f848ffff700 1 -- 172.21.15.57:0/2025914187 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4005040 con 0x557717a7b970 2023-10-08T13:56:48.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.688+0000 7f848ffff700 1 -- 172.21.15.57:0/2025914187 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f84a40051a0 con 0x557717a7b970 2023-10-08T13:56:48.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.688+0000 7f84c2c61b80 1 -- 172.21.15.57:0/2025914187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84a4018900 con 0x557717a7b970 2023-10-08T13:56:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.701+0000 7f848ffff700 1 -- 172.21.15.57:0/2025914187 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84a4005360 con 0x557717a7b970 2023-10-08T13:56:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.701+0000 7f84c2c61b80 10 client.26303 fetch_fsmap learned FSMap version 38 2023-10-08T13:56:48.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.701+0000 7f84c2c61b80 1 -- 172.21.15.57:0/2025914187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5577177f82b0 con 0x557717a7b970 2023-10-08T13:56:48.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.711+0000 7f848ffff700 1 -- 172.21.15.57:0/2025914187 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a4018900 con 0x557717a7b970 2023-10-08T13:56:48.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.711+0000 7f848ffff700 1 -- 172.21.15.57:0/2025914187 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f84a402b440 con 0x557717a7b970 2023-10-08T13:56:48.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.712+0000 7f84c2c61b80 10 client.26303 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T13:56:48.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.712+0000 7f84c2c61b80 10 client.26303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:56:48.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.712+0000 7f84c2c61b80 20 client.26303 populate_metadata read hostname 'smithi057' 2023-10-08T13:56:48.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.712+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557717aeb480 0x557717aed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:48.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.712+0000 7f84c2c61b80 4 client.26303 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:56:48.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.712+0000 7f84c2c61b80 1 -- 172.21.15.57:0/2025914187 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577179c7b30 con 0x557717aeb480 2023-10-08T13:56:48.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.720+0000 7f84b4814700 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557717aeb480 0x557717aed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.742+0000 7f84b4814700 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557717aeb480 0x557717aed8a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d30b500d4b3f755f server_cookie=5bff66d949766c56 in_seq=0 out_seq=0 2023-10-08T13:56:48.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.752+0000 7f848ffff700 10 client.26303 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:48.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.752+0000 7f848ffff700 1 -- 172.21.15.57:0/2025914187 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5577179c7b30 con 0x557717aeb480 2023-10-08T13:56:48.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.752+0000 7f848ffff700 10 client.26303 handle_command_reply: tid=0 2023-10-08T13:56:48.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.752+0000 7f84c2c61b80 1 client.26303 shutdown 2023-10-08T13:56:48.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.758+0000 7f84c2c61b80 1 -- 172.21.15.57:0/2025914187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 msgr2=0x557717a8b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:48.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.758+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a8b360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f84a400afd0 tx=0x7f84a4018ca0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.758+0000 7f84c2c61b80 1 -- 172.21.15.57:0/2025914187 shutdown_connections 2023-10-08T13:56:48.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.758+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557717aeb480 0x557717aed8a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.758+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557717a86800 0x557717a8b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.758+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557717a7b050 0x557717a8cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.758+0000 7f84c2c61b80 1 --2- 172.21.15.57:0/2025914187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557717a7b970 0x557717a8b360 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:48.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.758+0000 7f84c2c61b80 1 -- 172.21.15.57:0/2025914187 >> 172.21.15.57:0/2025914187 conn(0x5577179f0260 msgr2=0x557717a703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:48.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.758+0000 7f848ffff700 0 client.26303 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:48.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.759+0000 7f84c2c61b80 1 -- 172.21.15.57:0/2025914187 shutdown_connections 2023-10-08T13:56:48.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.759+0000 7f84c2c61b80 1 -- 172.21.15.57:0/2025914187 wait complete. 2023-10-08T13:56:48.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:48.759+0000 7f84c2c61b80 20 client.26303 trim_cache size 0 max 16384 2023-10-08T13:56:48.760 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:56:48.760 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T13:56:48.761 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": { 2023-10-08T13:56:48.761 INFO:teuthology.orchestra.run.smithi057.stdout: "6db03f06-e888-4427-a580-5fa3643c8e0d": { 2023-10-08T13:56:48.761 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/", 2023-10-08T13:56:48.761 INFO:teuthology.orchestra.run.smithi057.stdout: "tag": "6db03f06-e888-4427-a580-5fa3643c8e0d", 2023-10-08T13:56:48.761 INFO:teuthology.orchestra.run.smithi057.stdout: "options": "recursive,force" 2023-10-08T13:56:48.761 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:56:48.761 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:56:48.762 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:56:49.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.174+0000 7f84b5816700 1 -- 172.21.15.57:0/1084195472 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84980a1800 msgr2=0x7f84980a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:49.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.174+0000 7f84b5816700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84980a1800 0x7f84980a3cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f84a0005950 tx=0x7f84a0012670 comp rx=0 tx=0).stop 2023-10-08T13:56:49.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.174+0000 7f84b5816700 1 -- 172.21.15.57:0/1084195472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b0119c80 msgr2=0x7f84b01a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:49.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.174+0000 7f84b5816700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b0119c80 0x7f84b01a5800 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f84b0069a50 tx=0x7f84a80077f0 comp rx=0 tx=0).stop 2023-10-08T13:56:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.185+0000 7f84b5816700 1 -- 172.21.15.57:0/1084195472 shutdown_connections 2023-10-08T13:56:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.185+0000 7f84b5816700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84b0119c80 0x7f84b01a5800 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:49.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.185+0000 7f84b5816700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84b010c8d0 0x7f84b01ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.185+0000 7f84b5816700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f84980a1800 0x7f84980a3cc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.185+0000 7f84b5816700 1 --2- 172.21.15.57:0/1084195472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b010d1f0 0x7f84b01abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:49.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.185+0000 7f84b5816700 1 -- 172.21.15.57:0/1084195472 >> 172.21.15.57:0/1084195472 conn(0x7f84b006b650 msgr2=0x7f84b0118720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:49.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.185+0000 7f84b5816700 1 -- 172.21.15.57:0/1084195472 shutdown_connections 2023-10-08T13:56:49.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:56:49.186+0000 7f84b5816700 1 -- 172.21.15.57:0/1084195472 wait complete. 2023-10-08T13:56:49.258 INFO:tasks.cephfs.filesystem:scrub status for tag:6db03f06-e888-4427-a580-5fa3643c8e0d - {'path': '/', 'tag': '6db03f06-e888-4427-a580-5fa3643c8e0d', 'options': 'recursive,force'} 2023-10-08T13:56:49.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:49 smithi115 ceph-mon[104408]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 307 op/s 2023-10-08T13:56:49.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:49 smithi057 ceph-mon[99476]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 307 op/s 2023-10-08T13:56:49.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:49 smithi099 ceph-mon[103849]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 307 op/s 2023-10-08T13:56:50.936 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 13:56:50 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-i[141877]: 2023-10-08T13:56:50.593+0000 7ff6325de700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b6 [...7,head] /volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-1/ auth{0=1,2=1,3=1,4=1} fragtree_t(*^3) v135665 ap=4 f(v13 m2023-10-08T13:56:47.334071+0000 209=209+0) n(v18 rc2023-10-08T13:56:47.334071+0000 b3445142 210=209+1) old_inodes=3 (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={25349=pAsLsXs/pAsLsXsFsxcral@24} | ptrwaiter=0 dirtyscattered=0 request=0 lock=3 importingcaps=0 importing=0 dirfrag=8 caps=1 stickydirs=0 dirtyrstat=0 exportingcaps=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 tempexporting=0 discoverbase=0 scrubqueue=0 0x559f15530b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000001b6:[<0x100000001b3/blog-1 v98506>,<0x10000000014/blogtest_in v1087>,<0x1000000000d/src v1679>,<0x1000000000b/blogbench-1.0 v1459>,<0x1000000000a/tmp v70>,<0x10000000007/client.0 v69>,<0x10000000006/9576dea1-484d-4e52-882d-96f1d9052725 v84>,<0x10000000001/sv_1 v107>,<0x10000000000/_nogroup v92>,<0x1/volumes v84>]//[]","memoryvalue":"(2)0x100000001b6:[<0x100000001b3/blog-1 v135571>,<0x10000000014/blogtest_in v1889>,<0x1000000000d/src v1711>,<0x1000000000b/blogbench-1.0 v1437>,<0x1000000000a/tmp v68>,<0x10000000007/client.0 v52>,<0x10000000006/9576dea1-484d-4e52-882d-96f1d9052725 v72>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v79>,<0x1/volumes v76>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-08T13:56:47.334071+0000 209=209+0)","ondisk_value.rstat":"n(v0 rc2023-10-08T13:56:47.334071+0000 b3445142 210=209+1)","memory_value.dirstat":"f(v13 m2023-10-08T13:56:47.334071+0000 209=209+0)","memory_value.rstat":"n(v18 rc2023-10-08T13:56:47.334071+0000 b3445142 210=209+1)","error_str":""},"return_code":0} 2023-10-08T13:56:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:51 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:51 smithi099 ceph-mon[103849]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 323 op/s 2023-10-08T13:56:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:51 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:56:51.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:51 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:51.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:51 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:56:51.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:51 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:51.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:51 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:51.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:51 smithi115 ceph-mon[104408]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 323 op/s 2023-10-08T13:56:51.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:51 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:56:51.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:51 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:51.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:51 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:56:51.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:51 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:51.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:51 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:51.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:51 smithi057 ceph-mon[99476]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 323 op/s 2023-10-08T13:56:51.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:51 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:56:51.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:51 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:51.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:51 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:56:51.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:51 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.031+0000 7fe3e179d700 1 -- 172.21.15.57:0/2155664450 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0ff5d0 msgr2=0x7fe3dc0ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.031+0000 7fe3e179d700 1 --2- 172.21.15.57:0/2155664450 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0ff5d0 0x7fe3dc0ffa50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009990 tx=0x7fe3d0009ca0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.031+0000 7fe3e179d700 1 -- 172.21.15.57:0/2155664450 shutdown_connections 2023-10-08T13:56:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.031+0000 7fe3e179d700 1 --2- 172.21.15.57:0/2155664450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3dc0fff90 0x7fe3dc10c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.031+0000 7fe3e179d700 1 --2- 172.21.15.57:0/2155664450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3dc1045f0 0x7fe3dc1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.031+0000 7fe3e179d700 1 --2- 172.21.15.57:0/2155664450 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0ff5d0 0x7fe3dc0ffa50 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.031+0000 7fe3e179d700 1 -- 172.21.15.57:0/2155664450 >> 172.21.15.57:0/2155664450 conn(0x7fe3dc0767f0 msgr2=0x7fe3dc076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.031+0000 7fe3e179d700 1 -- 172.21.15.57:0/2155664450 shutdown_connections 2023-10-08T13:56:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.031+0000 7fe3e179d700 1 -- 172.21.15.57:0/2155664450 wait complete. 2023-10-08T13:56:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.032+0000 7fe3e179d700 1 Processor -- start 2023-10-08T13:56:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3e179d700 1 -- start start 2023-10-08T13:56:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3e179d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3dc0ff5d0 0x7fe3dc19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3e179d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0fff90 0x7fe3dc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3e179d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3dc1045f0 0x7fe3dc196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3e179d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc197090 con 0x7fe3dc0fff90 2023-10-08T13:56:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3e179d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc197200 con 0x7fe3dc0ff5d0 2023-10-08T13:56:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3e179d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc1973c0 con 0x7fe3dc1045f0 2023-10-08T13:56:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0fff90 0x7fe3dc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3dc0ff5d0 0x7fe3dc19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0fff90 0x7fe3dc19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58602/0 (socket says 172.21.15.57:58602) 2023-10-08T13:56:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3da7fc700 1 -- 172.21.15.57:0/3668234047 learned_addr learned my addr 172.21.15.57:0/3668234047 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3db7fe700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3dc1045f0 0x7fe3dc196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3da7fc700 1 -- 172.21.15.57:0/3668234047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3dc1045f0 msgr2=0x7fe3dc196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3da7fc700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3dc1045f0 0x7fe3dc196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3da7fc700 1 -- 172.21.15.57:0/3668234047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3dc0ff5d0 msgr2=0x7fe3dc19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3da7fc700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3dc0ff5d0 0x7fe3dc19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.033+0000 7fe3da7fc700 1 -- 172.21.15.57:0/3668234047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d00095f0 con 0x7fe3dc0fff90 2023-10-08T13:56:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.034+0000 7fe3db7fe700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3dc1045f0 0x7fe3dc196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.034+0000 7fe3da7fc700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0fff90 0x7fe3dc19ce40 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0000c00 tx=0x7fe3d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.034+0000 7fe3c3fff700 1 -- 172.21.15.57:0/3668234047 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d001d070 con 0x7fe3dc0fff90 2023-10-08T13:56:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.034+0000 7fe3c3fff700 1 -- 172.21.15.57:0/3668234047 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d0022470 con 0x7fe3dc0fff90 2023-10-08T13:56:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.034+0000 7fe3c3fff700 1 -- 172.21.15.57:0/3668234047 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0017650 con 0x7fe3dc0fff90 2023-10-08T13:56:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.034+0000 7fe3e179d700 1 -- 172.21.15.57:0/3668234047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc197640 con 0x7fe3dc0fff90 2023-10-08T13:56:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.034+0000 7fe3e179d700 1 -- 172.21.15.57:0/3668234047 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc1a38f0 con 0x7fe3dc0fff90 2023-10-08T13:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.037+0000 7fe3c3fff700 1 -- 172.21.15.57:0/3668234047 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe3d00225e0 con 0x7fe3dc0fff90 2023-10-08T13:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.037+0000 7fe3c3fff700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3c40a1670 0x7fe3c40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.038+0000 7fe3daffd700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3c40a1670 0x7fe3c40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.038+0000 7fe3c3fff700 1 -- 172.21.15.57:0/3668234047 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe3d00ce5a0 con 0x7fe3dc0fff90 2023-10-08T13:56:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.038+0000 7fe3daffd700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3c40a1670 0x7fe3c40a3b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0689d0 tx=0x7fe3cc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.039+0000 7fe3ed986b80 1 Processor -- start 2023-10-08T13:56:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.039+0000 7fe3ed986b80 1 -- start start 2023-10-08T13:56:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564811325880 0x564811325c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x56481131fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113200b0 0x564811320510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564811288400 con 0x5648113200b0 2023-10-08T13:56:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564811279bc0 con 0x564811325880 2023-10-08T13:56:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648112774e0 con 0x5648113261a0 2023-10-08T13:56:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564811325880 0x564811325c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564811325880 0x564811325c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60034/0 (socket says 172.21.15.57:60034) 2023-10-08T13:56:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x56481131fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3db7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113200b0 0x564811320510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3daffd700 1 -- 172.21.15.57:0/1673302311 learned_addr learned my addr 172.21.15.57:0/1673302311 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3da7fc700 1 -- 172.21.15.57:0/1673302311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564811325880 msgr2=0x564811325c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3da7fc700 1 --2- 172.21.15.57:0/1673302311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564811325880 0x564811325c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3da7fc700 1 -- 172.21.15.57:0/1673302311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113200b0 msgr2=0x564811320510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3da7fc700 1 --2- 172.21.15.57:0/1673302311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113200b0 0x564811320510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3da7fc700 1 -- 172.21.15.57:0/1673302311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56481109dd80 con 0x5648113261a0 2023-10-08T13:56:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.040+0000 7fe3da7fc700 1 --2- 172.21.15.57:0/1673302311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x56481131fb70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00cb9f0 tx=0x7fe3d009aed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f82fc797b0f64aa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.041+0000 7fe3c17fa700 1 -- 172.21.15.57:0/1673302311 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d009b310 con 0x5648113261a0 2023-10-08T13:56:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.041+0000 7fe3c17fa700 1 -- 172.21.15.57:0/1673302311 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d00cd030 con 0x5648113261a0 2023-10-08T13:56:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.041+0000 7fe3c17fa700 1 -- 172.21.15.57:0/1673302311 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00228e0 con 0x5648113261a0 2023-10-08T13:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1673302311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 msgr2=0x56481131fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1673302311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x56481131fb70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00cb9f0 tx=0x7fe3d009aed0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1673302311 shutdown_connections 2023-10-08T13:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1673302311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113200b0 0x564811320510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1673302311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564811325880 0x564811325c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1673302311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x56481131fb70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1673302311 >> 172.21.15.57:0/1673302311 conn(0x564811294200 msgr2=0x56481131ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1673302311 shutdown_connections 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1673302311 wait complete. 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 -- start start 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.042+0000 7fe3ed986b80 1 Processor -- start 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811333730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x564811333c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x56481132daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564811272dd0 con 0x564811325880 2023-10-08T13:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481120f6b0 con 0x5648113200b0 2023-10-08T13:56:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481120f350 con 0x5648113261a0 2023-10-08T13:56:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3ed986b80 10 client.0 _ll_register_callbacks cb 0x56481106c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:56:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x564811333c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811333730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811333730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60040/0 (socket says 172.21.15.57:60040) 2023-10-08T13:56:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3daffd700 1 -- 172.21.15.57:0/1215546023 learned_addr learned my addr 172.21.15.57:0/1215546023 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x564811333c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58616/0 (socket says 172.21.15.57:58616) 2023-10-08T13:56:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3db7fe700 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x56481132daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3db7fe700 1 -- 172.21.15.57:0/1215546023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 msgr2=0x564811333730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3db7fe700 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811333730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3db7fe700 1 -- 172.21.15.57:0/1215546023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 msgr2=0x564811333c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.043+0000 7fe3db7fe700 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x564811333c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3db7fe700 1 -- 172.21.15.57:0/1215546023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc1a38f0 con 0x5648113261a0 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3daffd700 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811333730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3db7fe700 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x56481132daa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4004660 tx=0x7fe3d4004690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61f878be779dcda8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3beff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3beff5700 1 -- 172.21.15.57:0/1215546023 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4004db0 con 0x5648113261a0 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1215546023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56481123ae50 con 0x5648113261a0 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3beff5700 1 -- 172.21.15.57:0/1215546023 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe3d40077d0 con 0x5648113261a0 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3beff5700 1 -- 172.21.15.57:0/1215546023 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3d4007990 con 0x5648113261a0 2023-10-08T13:56:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3ed986b80 10 client.26368 fetch_fsmap learned FSMap version 39 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1215546023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564811086c00 con 0x5648113261a0 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.044+0000 7fe3beff5700 1 -- 172.21.15.57:0/1215546023 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56481123ae50 con 0x5648113261a0 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.045+0000 7fe3beff5700 1 -- 172.21.15.57:0/1215546023 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 39) v1 ==== 7958+0+0 (secure 0 0 0) 0x7fe3d4015390 con 0x5648113261a0 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.045+0000 7fe3ed986b80 10 client.26368 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.045+0000 7fe3ed986b80 10 client.26368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.045+0000 7fe3ed986b80 20 client.26368 populate_metadata read hostname 'smithi057' 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.045+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56481133de40 0x564811340260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.045+0000 7fe3ed986b80 4 client.26368 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.045+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1215546023 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56481126bdb0 con 0x56481133de40 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.045+0000 7fe3daffd700 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56481133de40 0x564811340260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.045+0000 7fe3beff5700 10 client.26368 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.046+0000 7fe3daffd700 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56481133de40 0x564811340260 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9661e1b18ecb779b server_cookie=f9400e3e53d1eb5f in_seq=0 out_seq=0 2023-10-08T13:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.046+0000 7fe3beff5700 1 -- 172.21.15.57:0/1215546023 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56481126bdb0 con 0x56481133de40 2023-10-08T13:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.046+0000 7fe3beff5700 10 client.26368 handle_command_reply: tid=0 2023-10-08T13:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 client.26368 shutdown 2023-10-08T13:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1215546023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 msgr2=0x56481132daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x56481132daa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4004660 tx=0x7fe3d4004690 comp rx=0 tx=0).stop 2023-10-08T13:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1215546023 shutdown_connections 2023-10-08T13:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56481133de40 0x564811340260 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x56481132daa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811333730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1215546023 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x564811333c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1215546023 >> 172.21.15.57:0/1215546023 conn(0x564811294200 msgr2=0x56481131a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3beff5700 0 client.26368 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1215546023 shutdown_connections 2023-10-08T13:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1215546023 wait complete. 2023-10-08T13:56:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.047+0000 7fe3ed986b80 20 client.26368 trim_cache size 0 max 16384 2023-10-08T13:56:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.065+0000 7fe3ed986b80 1 Processor -- start 2023-10-08T13:56:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.065+0000 7fe3ed986b80 1 -- start start 2023-10-08T13:56:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.065+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x56481131fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.065+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x56481132e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.065+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x564811333380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.065+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648110f38c0 con 0x564811325880 2023-10-08T13:56:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.065+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564811317830 con 0x5648113200b0 2023-10-08T13:56:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.065+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56481132ec30 con 0x5648113261a0 2023-10-08T13:56:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x56481132e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x56481131fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3db7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x564811333380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x56481132e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58626/0 (socket says 172.21.15.57:58626) 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x56481131fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60050/0 (socket says 172.21.15.57:60050) 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3da7fc700 1 -- 172.21.15.57:0/1805165361 learned_addr learned my addr 172.21.15.57:0/1805165361 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3daffd700 1 -- 172.21.15.57:0/1805165361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 msgr2=0x564811333380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3daffd700 1 --2- 172.21.15.57:0/1805165361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x564811333380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3daffd700 1 -- 172.21.15.57:0/1805165361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 msgr2=0x56481132e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3daffd700 1 --2- 172.21.15.57:0/1805165361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x56481132e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3daffd700 1 -- 172.21.15.57:0/1805165361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648110c41f0 con 0x5648113200b0 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3db7fe700 1 --2- 172.21.15.57:0/1805165361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x564811333380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3da7fc700 1 --2- 172.21.15.57:0/1805165361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x56481132e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.066+0000 7fe3daffd700 1 --2- 172.21.15.57:0/1805165361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x56481131fb70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009fd0 tx=0x7fe3cc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd7ee617f6a3051 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.067+0000 7fe3c1ffb700 1 -- 172.21.15.57:0/1805165361 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc022030 con 0x5648113200b0 2023-10-08T13:56:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.067+0000 7fe3c1ffb700 1 -- 172.21.15.57:0/1805165361 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3cc023040 con 0x5648113200b0 2023-10-08T13:56:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.067+0000 7fe3c1ffb700 1 -- 172.21.15.57:0/1805165361 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc02a6f0 con 0x5648113200b0 2023-10-08T13:56:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.067+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1805165361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 msgr2=0x56481131fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.067+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1805165361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x56481131fb70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009fd0 tx=0x7fe3cc014000 comp rx=0 tx=0).stop 2023-10-08T13:56:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.067+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1805165361 shutdown_connections 2023-10-08T13:56:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.067+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1805165361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648113261a0 0x564811333380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.067+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1805165361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x56481131fb70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.067+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1805165361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x564811325880 0x56481132e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1805165361 >> 172.21.15.57:0/1805165361 conn(0x564811294200 msgr2=0x564811320490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1805165361 shutdown_connections 2023-10-08T13:56:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1805165361 wait complete. 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 -- start start 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 Processor -- start 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811329770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564811325880 0x564811329cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113261a0 0x56481132a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564811339670 con 0x5648113261a0 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648113397e0 con 0x5648113200b0 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648113399a0 con 0x564811325880 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.068+0000 7fe3ed986b80 10 client.0 _ll_register_callbacks cb 0x56481106c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811329770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811329770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60054/0 (socket says 172.21.15.57:60054) 2023-10-08T13:56:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3daffd700 1 -- 172.21.15.57:0/1508624076 learned_addr learned my addr 172.21.15.57:0/1508624076 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:56:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3db7fe700 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113261a0 0x56481132a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3daffd700 1 -- 172.21.15.57:0/1508624076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564811325880 msgr2=0x564811329cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:56:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3daffd700 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564811325880 0x564811329cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3daffd700 1 -- 172.21.15.57:0/1508624076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113261a0 msgr2=0x56481132a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:56:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3daffd700 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113261a0 0x56481132a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3daffd700 1 -- 172.21.15.57:0/1508624076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d00b3040 con 0x5648113200b0 2023-10-08T13:56:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3db7fe700 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113261a0 0x56481132a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:56:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3daffd700 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811329770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0117d0 tx=0x7fe3cc02b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb2ecb27ed7ff4f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:56:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3beff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3beff5700 1 -- 172.21.15.57:0/1508624076 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc022040 con 0x5648113200b0 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1508624076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3d00cd030 con 0x5648113200b0 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.069+0000 7fe3beff5700 1 -- 172.21.15.57:0/1508624076 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fe3cc02ceb0 con 0x5648113200b0 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3beff5700 1 -- 172.21.15.57:0/1508624076 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3cc005040 con 0x5648113200b0 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3ed986b80 10 client.26321 fetch_fsmap learned FSMap version 39 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1508624076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564811086c00 con 0x5648113200b0 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3beff5700 1 -- 172.21.15.57:0/1508624076 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d00cd030 con 0x5648113200b0 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3beff5700 1 -- 172.21.15.57:0/1508624076 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 39) v1 ==== 7958+0+0 (secure 0 0 0) 0x7fe3cc03c430 con 0x5648113200b0 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3ed986b80 10 client.26321 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3ed986b80 10 client.26321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:56:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3ed986b80 20 client.26321 populate_metadata read hostname 'smithi057' 2023-10-08T13:56:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564811396c00 0x564811399020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:56:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3ed986b80 4 client.26321 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:56:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.070+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1508624076 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56481126bad0 con 0x564811396c00 2023-10-08T13:56:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.071+0000 7fe3da7fc700 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564811396c00 0x564811399020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:56:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.071+0000 7fe3beff5700 10 client.26321 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:52.071+0000 7fe3da7fc700 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564811396c00 0x564811399020 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a71b23ffb146def4 server_cookie=6c88b1f5d2960aaf in_seq=0 out_seq=0 2023-10-08T13:56:52.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:52 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:56:52.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:52 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:52.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:52 smithi115 ceph-mon[104408]: scrub summary: active paths [/] 2023-10-08T13:56:52.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:52 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:52.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:52 smithi115 ceph-mon[104408]: Scrub error on inode 0x100000001b6 (/volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-1) see mds.i log and `damage ls` output for details 2023-10-08T13:56:52.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:52 smithi115 ceph-mon[104408]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:52.770 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:52 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:56:52.770 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:52 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:52.770 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:52 smithi057 ceph-mon[99476]: scrub summary: active paths [/] 2023-10-08T13:56:52.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:52 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:52.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:52 smithi057 ceph-mon[99476]: Scrub error on inode 0x100000001b6 (/volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-1) see mds.i log and `damage ls` output for details 2023-10-08T13:56:52.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:52 smithi057 ceph-mon[99476]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:52.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:52 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:56:52.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:52 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:52.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:52 smithi099 ceph-mon[103849]: scrub summary: active paths [/] 2023-10-08T13:56:52.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:52 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:52.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:52 smithi099 ceph-mon[103849]: Scrub error on inode 0x100000001b6 (/volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-1) see mds.i log and `damage ls` output for details 2023-10-08T13:56:52.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:52 smithi099 ceph-mon[103849]: scrub summary: idle+waiting paths [/] 2023-10-08T13:56:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.681+0000 7fe3beff5700 1 -- 172.21.15.57:0/1508624076 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56481126bad0 con 0x564811396c00 2023-10-08T13:56:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.681+0000 7fe3beff5700 10 client.26321 handle_command_reply: tid=0 2023-10-08T13:56:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.681+0000 7fe3ed986b80 1 client.26321 shutdown 2023-10-08T13:56:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.691+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1508624076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 msgr2=0x564811329770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.691+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811329770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0117d0 tx=0x7fe3cc02b410 comp rx=0 tx=0).stop 2023-10-08T13:56:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.691+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1508624076 shutdown_connections 2023-10-08T13:56:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.691+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x564811396c00 0x564811399020 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.691+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5648113261a0 0x56481132a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.691+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648113200b0 0x564811329770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.691+0000 7fe3ed986b80 1 --2- 172.21.15.57:0/1508624076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564811325880 0x564811329cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.691+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1508624076 >> 172.21.15.57:0/1508624076 conn(0x564811294200 msgr2=0x5648113332f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.691+0000 7fe3beff5700 0 client.26321 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:56:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.702+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1508624076 shutdown_connections 2023-10-08T13:56:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.702+0000 7fe3ed986b80 1 -- 172.21.15.57:0/1508624076 wait complete. 2023-10-08T13:56:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.703+0000 7fe3ed986b80 20 client.26321 trim_cache size 0 max 16384 2023-10-08T13:56:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:56:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:56:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:56:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:56:53.752 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:53 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a[99452]: 2023-10-08T13:56:53.357+0000 7f3dce8c4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-08T13:56:53.753 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:53 smithi057 ceph-mon[99476]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 175 op/s 2023-10-08T13:56:53.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:53 smithi099 ceph-mon[103849]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 175 op/s 2023-10-08T13:56:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.845+0000 7fe3e179d700 1 -- 172.21.15.57:0/3668234047 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3c40a1670 msgr2=0x7fe3c40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.845+0000 7fe3e179d700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3c40a1670 0x7fe3c40a3b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0689d0 tx=0x7fe3cc009260 comp rx=0 tx=0).stop 2023-10-08T13:56:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.845+0000 7fe3e179d700 1 -- 172.21.15.57:0/3668234047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0fff90 msgr2=0x7fe3dc19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:56:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.845+0000 7fe3e179d700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0fff90 0x7fe3dc19ce40 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0000c00 tx=0x7fe3d0004730 comp rx=0 tx=0).stop 2023-10-08T13:56:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.850+0000 7fe3e179d700 1 -- 172.21.15.57:0/3668234047 shutdown_connections 2023-10-08T13:56:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.850+0000 7fe3e179d700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3dc1045f0 0x7fe3dc196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.850+0000 7fe3e179d700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe3c40a1670 0x7fe3c40a3b30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.850+0000 7fe3e179d700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3dc0ff5d0 0x7fe3dc19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.850+0000 7fe3e179d700 1 --2- 172.21.15.57:0/3668234047 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3dc0fff90 0x7fe3dc19ce40 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:56:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.850+0000 7fe3e179d700 1 -- 172.21.15.57:0/3668234047 >> 172.21.15.57:0/3668234047 conn(0x7fe3dc0767f0 msgr2=0x7fe3dc0692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:56:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.851+0000 7fe3e179d700 1 -- 172.21.15.57:0/3668234047 shutdown_connections 2023-10-08T13:56:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:56:53.851+0000 7fe3e179d700 1 -- 172.21.15.57:0/3668234047 wait complete. 2023-10-08T13:56:53.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:53 smithi115 ceph-mon[104408]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 175 op/s 2023-10-08T13:56:54.417 INFO:tasks.workunit.client.0.smithi057.stdout: 459 2420272 537 1670506 368 387998 678 2023-10-08T13:56:54.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:54 smithi057 ceph-mon[99476]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-08T13:56:54.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:54 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] up:active 2023-10-08T13:56:54.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:54 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:54.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:54 smithi057 ceph-mon[99476]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2023-10-08T13:56:54.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:54 smithi099 ceph-mon[103849]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-08T13:56:54.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:54 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] up:active 2023-10-08T13:56:54.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:54 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:54.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:54 smithi099 ceph-mon[103849]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2023-10-08T13:56:54.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:54 smithi115 ceph-mon[104408]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-08T13:56:54.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:54 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] up:active 2023-10-08T13:56:54.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:54 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:54.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:54 smithi115 ceph-mon[104408]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2023-10-08T13:56:56.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:56 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:56.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:56 smithi115 ceph-mon[104408]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2023-10-08T13:56:56.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:56 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:56.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:56 smithi057 ceph-mon[99476]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2023-10-08T13:56:57.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:56 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:56:57.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:56 smithi099 ceph-mon[103849]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2023-10-08T13:56:59.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:56:59 smithi115 ceph-mon[104408]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 227 op/s 2023-10-08T13:56:59.519 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:56:59 smithi057 ceph-mon[99476]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 227 op/s 2023-10-08T13:56:59.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:56:59 smithi099 ceph-mon[103849]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 227 op/s 2023-10-08T13:57:00.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:00 smithi115 ceph-mon[104408]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2023-10-08T13:57:00.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:00 smithi057 ceph-mon[99476]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2023-10-08T13:57:01.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:00 smithi099 ceph-mon[103849]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2023-10-08T13:57:02.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:02 smithi115 ceph-mon[104408]: pgmap v464: 129 pgs: 129 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 381 op/s 2023-10-08T13:57:02.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:02 smithi057 ceph-mon[99476]: pgmap v464: 129 pgs: 129 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 381 op/s 2023-10-08T13:57:03.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:02 smithi099 ceph-mon[103849]: pgmap v464: 129 pgs: 129 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 381 op/s 2023-10-08T13:57:04.415 INFO:tasks.workunit.client.0.smithi057.stdout: 464 2190041 397 1507479 274 364900 561 2023-10-08T13:57:04.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:04 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:57:04.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:04 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:57:04.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:04 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:57:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:05 smithi099 ceph-mon[103849]: scrub summary: idle 2023-10-08T13:57:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:05 smithi099 ceph-mon[103849]: scrub summary: idle 2023-10-08T13:57:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:05 smithi099 ceph-mon[103849]: scrub summary: idle 2023-10-08T13:57:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:05 smithi099 ceph-mon[103849]: scrub summary: idle 2023-10-08T13:57:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:05 smithi099 ceph-mon[103849]: pgmap v465: 129 pgs: 129 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.5 MiB/s wr, 443 op/s 2023-10-08T13:57:05.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:05 smithi115 ceph-mon[104408]: scrub summary: idle 2023-10-08T13:57:05.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:05 smithi115 ceph-mon[104408]: scrub summary: idle 2023-10-08T13:57:05.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:05 smithi115 ceph-mon[104408]: scrub summary: idle 2023-10-08T13:57:05.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:05 smithi115 ceph-mon[104408]: scrub summary: idle 2023-10-08T13:57:05.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:05 smithi115 ceph-mon[104408]: pgmap v465: 129 pgs: 129 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.5 MiB/s wr, 443 op/s 2023-10-08T13:57:05.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:05 smithi057 ceph-mon[99476]: scrub summary: idle 2023-10-08T13:57:05.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:05 smithi057 ceph-mon[99476]: scrub summary: idle 2023-10-08T13:57:05.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:05 smithi057 ceph-mon[99476]: scrub summary: idle 2023-10-08T13:57:05.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:05 smithi057 ceph-mon[99476]: scrub summary: idle 2023-10-08T13:57:05.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:05 smithi057 ceph-mon[99476]: pgmap v465: 129 pgs: 129 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.5 MiB/s wr, 443 op/s 2023-10-08T13:57:06.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:06 smithi115 ceph-mon[104408]: scrub summary: idle 2023-10-08T13:57:06.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:06 smithi099 ceph-mon[103849]: scrub summary: idle 2023-10-08T13:57:06.801 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:06 smithi057 ceph-mon[99476]: scrub summary: idle 2023-10-08T13:57:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:07.929+0000 7f33bc4c4700 1 -- 172.21.15.57:0/3321855327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33b4107a60 msgr2=0x7f33b4107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:07.929+0000 7f33bc4c4700 1 --2- 172.21.15.57:0/3321855327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33b4107a60 0x7f33b4107e40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f33b0007570 tx=0x7f33b0007880 comp rx=0 tx=0).stop 2023-10-08T13:57:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:07.931+0000 7f33bc4c4700 1 -- 172.21.15.57:0/3321855327 shutdown_connections 2023-10-08T13:57:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:07.931+0000 7f33bc4c4700 1 --2- 172.21.15.57:0/3321855327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33b410c070 0x7f33b4078b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:07.931+0000 7f33bc4c4700 1 --2- 172.21.15.57:0/3321855327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33b4107a60 0x7f33b4107e40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:07.931+0000 7f33bc4c4700 1 --2- 172.21.15.57:0/3321855327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b4108380 0x7f33b410b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:07.931+0000 7f33bc4c4700 1 -- 172.21.15.57:0/3321855327 >> 172.21.15.57:0/3321855327 conn(0x7f33b406b7c0 msgr2=0x7f33b406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:07.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:07 smithi115 ceph-mon[104408]: pgmap v466: 129 pgs: 129 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.8 MiB/s wr, 449 op/s 2023-10-08T13:57:08.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:07 smithi099 ceph-mon[103849]: pgmap v466: 129 pgs: 129 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.8 MiB/s wr, 449 op/s 2023-10-08T13:57:08.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:07 smithi057 ceph-mon[99476]: pgmap v466: 129 pgs: 129 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.8 MiB/s wr, 449 op/s 2023-10-08T13:57:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.089+0000 7f33bc4c4700 1 -- 172.21.15.57:0/3321855327 shutdown_connections 2023-10-08T13:57:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.094+0000 7f33bc4c4700 1 -- 172.21.15.57:0/3321855327 wait complete. 2023-10-08T13:57:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.095+0000 7f33bc4c4700 1 Processor -- start 2023-10-08T13:57:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.096+0000 7f33bc4c4700 1 -- start start 2023-10-08T13:57:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.096+0000 7f33bc4c4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33b4107a60 0x7f33b4083300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.096+0000 7f33bc4c4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b410c070 0x7f33b407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.096+0000 7f33bc4c4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33b407d930 0x7f33b407dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.096+0000 7f33bc4c4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41baa10 con 0x7f33b4107a60 2023-10-08T13:57:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.096+0000 7f33bc4c4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41bab80 con 0x7f33b410c070 2023-10-08T13:57:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.096+0000 7f33bc4c4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41bad40 con 0x7f33b407d930 2023-10-08T13:57:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.096+0000 7f33baa61700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33b407d930 0x7f33b407dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.096+0000 7f33baa61700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33b407d930 0x7f33b407dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59456/0 (socket says 172.21.15.57:59456) 2023-10-08T13:57:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.096+0000 7f33baa61700 1 -- 172.21.15.57:0/2859296990 learned_addr learned my addr 172.21.15.57:0/2859296990 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.106+0000 7f33b9a5f700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b410c070 0x7f33b407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.106+0000 7f33ba260700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33b4107a60 0x7f33b4083300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.107+0000 7f33baa61700 1 -- 172.21.15.57:0/2859296990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b410c070 msgr2=0x7f33b407d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.107+0000 7f33baa61700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b410c070 0x7f33b407d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.107+0000 7f33baa61700 1 -- 172.21.15.57:0/2859296990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33b4107a60 msgr2=0x7f33b4083300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.107+0000 7f33baa61700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33b4107a60 0x7f33b4083300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.107+0000 7f33baa61700 1 -- 172.21.15.57:0/2859296990 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b0007220 con 0x7f33b407d930 2023-10-08T13:57:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.116+0000 7f33baa61700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33b407d930 0x7f33b407dde0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00b970 tx=0x7f33ac00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.117+0000 7f33ab7fe700 1 -- 172.21.15.57:0/2859296990 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00c9b0 con 0x7f33b407d930 2023-10-08T13:57:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.117+0000 7f33ab7fe700 1 -- 172.21.15.57:0/2859296990 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33ac014440 con 0x7f33b407d930 2023-10-08T13:57:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.117+0000 7f33ab7fe700 1 -- 172.21.15.57:0/2859296990 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0075d0 con 0x7f33b407d930 2023-10-08T13:57:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.117+0000 7f33bc4c4700 1 -- 172.21.15.57:0/2859296990 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b41bb020 con 0x7f33b407d930 2023-10-08T13:57:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.117+0000 7f33bc4c4700 1 -- 172.21.15.57:0/2859296990 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b41bb4e0 con 0x7f33b407d930 2023-10-08T13:57:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.139+0000 7f33c86adb80 1 Processor -- start 2023-10-08T13:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33c86adb80 1 -- start start 2023-10-08T13:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af56880 0x558e8af56c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af50b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af51510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33c86adb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8aeb9400 con 0x558e8af56880 2023-10-08T13:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33c86adb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8aeaabc0 con 0x558e8af571a0 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33c86adb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8aea84e0 con 0x558e8af510b0 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33ab7fe700 1 -- 172.21.15.57:0/2859296990 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f33ac007730 con 0x7f33b407d930 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33ba260700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af50b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33ba260700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af50b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60074/0 (socket says 172.21.15.57:60074) 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.142+0000 7f33ba260700 1 -- 172.21.15.57:0/340380664 learned_addr learned my addr 172.21.15.57:0/340380664 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.143+0000 7f33ba260700 1 -- 172.21.15.57:0/340380664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 msgr2=0x558e8af51510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.143+0000 7f33ba260700 1 --2- 172.21.15.57:0/340380664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af51510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.143+0000 7f33ba260700 1 -- 172.21.15.57:0/340380664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af56880 msgr2=0x558e8af56c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.143+0000 7f33ba260700 1 --2- 172.21.15.57:0/340380664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af56880 0x558e8af56c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.143+0000 7f33ba260700 1 -- 172.21.15.57:0/340380664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e8acced80 con 0x558e8af571a0 2023-10-08T13:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.143+0000 7f33ba260700 1 --2- 172.21.15.57:0/340380664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af50b70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f33b000a210 tx=0x7f33b000a240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c59032cfe72445 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.143+0000 7f33ab7fe700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33a00a17c0 0x7f33a00a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.144+0000 7f33ab7fe700 1 -- 172.21.15.57:0/2859296990 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f33ac006e40 con 0x7f33b407d930 2023-10-08T13:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.164+0000 7f33b9a5f700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33a00a17c0 0x7f33a00a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.168+0000 7f33b9a5f700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33a00a17c0 0x7f33a00a3c80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f33b407ead0 tx=0x7f33a4008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.169+0000 7f33a8d82700 1 -- 172.21.15.57:0/340380664 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b000f040 con 0x558e8af571a0 2023-10-08T13:57:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.169+0000 7f33a8d82700 1 -- 172.21.15.57:0/340380664 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33b0007e60 con 0x558e8af571a0 2023-10-08T13:57:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.169+0000 7f33a8d82700 1 -- 172.21.15.57:0/340380664 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0003680 con 0x558e8af571a0 2023-10-08T13:57:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.171+0000 7f33c86adb80 1 -- 172.21.15.57:0/340380664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 msgr2=0x558e8af50b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.171+0000 7f33c86adb80 1 --2- 172.21.15.57:0/340380664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af50b70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f33b000a210 tx=0x7f33b000a240 comp rx=0 tx=0).stop 2023-10-08T13:57:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.171+0000 7f33c86adb80 1 -- 172.21.15.57:0/340380664 shutdown_connections 2023-10-08T13:57:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.171+0000 7f33c86adb80 1 --2- 172.21.15.57:0/340380664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af51510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.171+0000 7f33c86adb80 1 --2- 172.21.15.57:0/340380664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af56880 0x558e8af56c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.171+0000 7f33c86adb80 1 --2- 172.21.15.57:0/340380664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af50b70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.171+0000 7f33c86adb80 1 -- 172.21.15.57:0/340380664 >> 172.21.15.57:0/340380664 conn(0x558e8aec5200 msgr2=0x558e8af4bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.175+0000 7f33c86adb80 1 -- 172.21.15.57:0/340380664 shutdown_connections 2023-10-08T13:57:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.191+0000 7f33c86adb80 1 -- 172.21.15.57:0/340380664 wait complete. 2023-10-08T13:57:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.192+0000 7f33c86adb80 1 -- start start 2023-10-08T13:57:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.202+0000 7f33c86adb80 1 Processor -- start 2023-10-08T13:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.202+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af5d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af56880 0x558e8af5a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af571a0 0x558e8af5ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33c86adb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8aea3dd0 con 0x558e8af571a0 2023-10-08T13:57:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33c86adb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8ae406b0 con 0x558e8af56880 2023-10-08T13:57:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33c86adb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8ae40350 con 0x558e8af510b0 2023-10-08T13:57:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33c86adb80 10 client.0 _ll_register_callbacks cb 0x558e8ac9d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33ba260700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af5d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33ba260700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af5d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59464/0 (socket says 172.21.15.57:59464) 2023-10-08T13:57:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33ba260700 1 -- 172.21.15.57:0/3676931899 learned_addr learned my addr 172.21.15.57:0/3676931899 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33ba260700 1 -- 172.21.15.57:0/3676931899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af56880 msgr2=0x558e8af5a470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:57:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33ba260700 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af56880 0x558e8af5a470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33ba260700 1 -- 172.21.15.57:0/3676931899 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af571a0 msgr2=0x558e8af5ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33ba260700 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af571a0 0x558e8af5ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.203+0000 7f33ba260700 1 -- 172.21.15.57:0/3676931899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b40793c0 con 0x558e8af510b0 2023-10-08T13:57:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.213+0000 7f33ba260700 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af5d180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f33b0000c00 tx=0x7f33b000ab60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aebb7f607463f3d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.227+0000 7f339dffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:57:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.227+0000 7f339dffb700 1 -- 172.21.15.57:0/3676931899 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b000f040 con 0x558e8af510b0 2023-10-08T13:57:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.227+0000 7f339dffb700 1 -- 172.21.15.57:0/3676931899 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f33b0005450 con 0x558e8af510b0 2023-10-08T13:57:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.227+0000 7f33c86adb80 1 -- 172.21.15.57:0/3676931899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e8ae6be50 con 0x558e8af510b0 2023-10-08T13:57:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.227+0000 7f339dffb700 1 -- 172.21.15.57:0/3676931899 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33b0007ce0 con 0x558e8af510b0 2023-10-08T13:57:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.227+0000 7f33c86adb80 10 client.26377 fetch_fsmap learned FSMap version 41 2023-10-08T13:57:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.228+0000 7f33c86adb80 1 -- 172.21.15.57:0/3676931899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558e8acb7c00 con 0x558e8af510b0 2023-10-08T13:57:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.237+0000 7f339dffb700 1 -- 172.21.15.57:0/3676931899 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e8ae6be50 con 0x558e8af510b0 2023-10-08T13:57:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.237+0000 7f339dffb700 1 -- 172.21.15.57:0/3676931899 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 41) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f33b0023070 con 0x558e8af510b0 2023-10-08T13:57:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.238+0000 7f33c86adb80 10 client.26377 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T13:57:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.238+0000 7f33c86adb80 10 client.26377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.238+0000 7f33c86adb80 20 client.26377 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.238+0000 7f33c86adb80 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558e8af6a910 0x558e8af6cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.238+0000 7f33c86adb80 4 client.26377 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:57:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.238+0000 7f33c86adb80 1 -- 172.21.15.57:0/3676931899 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e8ae9cdb0 con 0x558e8af6a910 2023-10-08T13:57:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.247+0000 7f33b9a5f700 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558e8af6a910 0x558e8af6cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.258+0000 7f33b9a5f700 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558e8af6a910 0x558e8af6cd50 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f604ef5ba5c00d server_cookie=1217951ccc1f4598 in_seq=0 out_seq=0 2023-10-08T13:57:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f339dffb700 10 client.26377 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f339dffb700 1 -- 172.21.15.57:0/3676931899 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558e8ae9cdb0 con 0x558e8af6a910 2023-10-08T13:57:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f339dffb700 10 client.26377 handle_command_reply: tid=0 2023-10-08T13:57:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f33c86adb80 1 client.26377 shutdown 2023-10-08T13:57:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f33c86adb80 1 -- 172.21.15.57:0/3676931899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 msgr2=0x558e8af5d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f33c86adb80 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af5d180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f33b0000c00 tx=0x7f33b000ab60 comp rx=0 tx=0).stop 2023-10-08T13:57:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f33c86adb80 1 -- 172.21.15.57:0/3676931899 shutdown_connections 2023-10-08T13:57:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f33c86adb80 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558e8af6a910 0x558e8af6cd50 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f33c86adb80 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af571a0 0x558e8af5ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f33c86adb80 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af5d180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f33c86adb80 1 --2- 172.21.15.57:0/3676931899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af56880 0x558e8af5a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.268+0000 7f33c86adb80 1 -- 172.21.15.57:0/3676931899 >> 172.21.15.57:0/3676931899 conn(0x558e8aec5200 msgr2=0x558e8af4b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.269+0000 7f339dffb700 0 client.26377 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.277+0000 7f33c86adb80 1 -- 172.21.15.57:0/3676931899 shutdown_connections 2023-10-08T13:57:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.277+0000 7f33c86adb80 1 -- 172.21.15.57:0/3676931899 wait complete. 2023-10-08T13:57:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.277+0000 7f33c86adb80 20 client.26377 trim_cache size 0 max 16384 2023-10-08T13:57:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.307+0000 7f33c86adb80 1 Processor -- start 2023-10-08T13:57:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.307+0000 7f33c86adb80 1 -- start start 2023-10-08T13:57:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.307+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af510b0 0x558e8af50b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.307+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af56880 0x558e8af4b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.307+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af4bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.307+0000 7f33c86adb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8ad248c0 con 0x558e8af510b0 2023-10-08T13:57:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.307+0000 7f33c86adb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8af5d840 con 0x558e8af571a0 2023-10-08T13:57:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.307+0000 7f33c86adb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8af5a240 con 0x558e8af56880 2023-10-08T13:57:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.308+0000 7f33baa61700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af4bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.308+0000 7f33baa61700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af4bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60086/0 (socket says 172.21.15.57:60086) 2023-10-08T13:57:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.308+0000 7f33baa61700 1 -- 172.21.15.57:0/2503593361 learned_addr learned my addr 172.21.15.57:0/2503593361 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.308+0000 7f33baa61700 1 -- 172.21.15.57:0/2503593361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af56880 msgr2=0x558e8af4b790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:57:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.308+0000 7f33baa61700 1 --2- 172.21.15.57:0/2503593361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af56880 0x558e8af4b790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.308+0000 7f33baa61700 1 -- 172.21.15.57:0/2503593361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af510b0 msgr2=0x558e8af50b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.308+0000 7f33baa61700 1 --2- 172.21.15.57:0/2503593361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af510b0 0x558e8af50b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.308+0000 7f33baa61700 1 -- 172.21.15.57:0/2503593361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e8acf51f0 con 0x558e8af571a0 2023-10-08T13:57:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.308+0000 7f33baa61700 1 --2- 172.21.15.57:0/2503593361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af4bcd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f33ac096bd0 tx=0x7f33ac09a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39a229a6a4627a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.323+0000 7f33a9583700 1 -- 172.21.15.57:0/2503593361 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac09add0 con 0x558e8af571a0 2023-10-08T13:57:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.323+0000 7f33a9583700 1 -- 172.21.15.57:0/2503593361 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33ac09a6d0 con 0x558e8af571a0 2023-10-08T13:57:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.323+0000 7f33a9583700 1 -- 172.21.15.57:0/2503593361 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac099e20 con 0x558e8af571a0 2023-10-08T13:57:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.325+0000 7f33c86adb80 1 -- 172.21.15.57:0/2503593361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 msgr2=0x558e8af4bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.325+0000 7f33c86adb80 1 --2- 172.21.15.57:0/2503593361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af4bcd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f33ac096bd0 tx=0x7f33ac09a210 comp rx=0 tx=0).stop 2023-10-08T13:57:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.334+0000 7f33c86adb80 1 -- 172.21.15.57:0/2503593361 shutdown_connections 2023-10-08T13:57:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.334+0000 7f33c86adb80 1 --2- 172.21.15.57:0/2503593361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af571a0 0x558e8af4bcd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.334+0000 7f33c86adb80 1 --2- 172.21.15.57:0/2503593361 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af510b0 0x558e8af50b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.334+0000 7f33c86adb80 1 --2- 172.21.15.57:0/2503593361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af56880 0x558e8af4b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.334+0000 7f33c86adb80 1 -- 172.21.15.57:0/2503593361 >> 172.21.15.57:0/2503593361 conn(0x558e8aec5200 msgr2=0x558e8af545c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.334+0000 7f33c86adb80 1 -- 172.21.15.57:0/2503593361 shutdown_connections 2023-10-08T13:57:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.334+0000 7f33c86adb80 1 -- 172.21.15.57:0/2503593361 wait complete. 2023-10-08T13:57:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.335+0000 7f33c86adb80 1 -- start start 2023-10-08T13:57:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.335+0000 7f33c86adb80 1 Processor -- start 2023-10-08T13:57:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.335+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af53560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.335+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af56880 0x558e8af53aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.335+0000 7f33c86adb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af571a0 0x558e8af52390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.335+0000 7f33c86adb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8af54180 con 0x558e8af571a0 2023-10-08T13:57:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.335+0000 7f33c86adb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8af528d0 con 0x558e8af56880 2023-10-08T13:57:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.335+0000 7f33c86adb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8af52a90 con 0x558e8af510b0 2023-10-08T13:57:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.335+0000 7f33c86adb80 10 client.0 _ll_register_callbacks cb 0x7f33b0007440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.336+0000 7f33baa61700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af571a0 0x558e8af52390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.336+0000 7f33baa61700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af571a0 0x558e8af52390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58664/0 (socket says 172.21.15.57:58664) 2023-10-08T13:57:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.336+0000 7f33baa61700 1 -- 172.21.15.57:0/2493936856 learned_addr learned my addr 172.21.15.57:0/2493936856 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.336+0000 7f33ba260700 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af53560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.336+0000 7f33ba260700 1 -- 172.21.15.57:0/2493936856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af56880 msgr2=0x558e8af53aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:57:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.336+0000 7f33ba260700 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af56880 0x558e8af53aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.336+0000 7f33ba260700 1 -- 172.21.15.57:0/2493936856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af571a0 msgr2=0x558e8af52390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.336+0000 7f33ba260700 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af571a0 0x558e8af52390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.336+0000 7f33ba260700 1 -- 172.21.15.57:0/2493936856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b41bb4e0 con 0x558e8af510b0 2023-10-08T13:57:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.337+0000 7f33ba260700 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af53560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f33b0000c00 tx=0x7f33b00048c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=640810e20afa362e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.337+0000 7f339dffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:57:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.337+0000 7f339dffb700 1 -- 172.21.15.57:0/2493936856 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b000f070 con 0x558e8af510b0 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.337+0000 7f339dffb700 1 -- 172.21.15.57:0/2493936856 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f33b0000e40 con 0x558e8af510b0 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.337+0000 7f33c86adb80 1 -- 172.21.15.57:0/2493936856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33b0007e60 con 0x558e8af510b0 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.337+0000 7f339dffb700 1 -- 172.21.15.57:0/2493936856 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33b000a190 con 0x558e8af510b0 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.337+0000 7f33c86adb80 10 client.26380 fetch_fsmap learned FSMap version 41 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.337+0000 7f33c86adb80 1 -- 172.21.15.57:0/2493936856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558e8accd250 con 0x558e8af510b0 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.338+0000 7f339dffb700 1 -- 172.21.15.57:0/2493936856 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33b0007e60 con 0x558e8af510b0 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.338+0000 7f339dffb700 1 -- 172.21.15.57:0/2493936856 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 41) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f33b00075a0 con 0x558e8af510b0 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.338+0000 7f33c86adb80 10 client.26380 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.338+0000 7f33c86adb80 10 client.26380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.338+0000 7f33c86adb80 20 client.26380 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.338+0000 7f33c86adb80 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558e8afbf2e0 0x558e8afc1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.338+0000 7f33c86adb80 4 client.26380 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:57:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.338+0000 7f33c86adb80 1 -- 172.21.15.57:0/2493936856 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e8ae9cad0 con 0x558e8afbf2e0 2023-10-08T13:57:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.377+0000 7f33b9a5f700 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558e8afbf2e0 0x558e8afc1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.415+0000 7f33b9a5f700 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558e8afbf2e0 0x558e8afc1700 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f840dfe37555f30b server_cookie=a3ef5df86049c708 in_seq=0 out_seq=0 2023-10-08T13:57:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:08.415+0000 7f339dffb700 10 client.26380 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:09.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:09 smithi115 ceph-mon[104408]: pgmap v467: 129 pgs: 129 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.2 MiB/s wr, 396 op/s 2023-10-08T13:57:09.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:09 smithi057 ceph-mon[99476]: pgmap v467: 129 pgs: 129 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.2 MiB/s wr, 396 op/s 2023-10-08T13:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.514+0000 7f339dffb700 1 -- 172.21.15.57:0/2493936856 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e8ae9cad0 con 0x558e8afbf2e0 2023-10-08T13:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.514+0000 7f339dffb700 10 client.26380 handle_command_reply: tid=0 2023-10-08T13:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.514+0000 7f33c86adb80 1 client.26380 shutdown 2023-10-08T13:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 1 -- 172.21.15.57:0/2493936856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 msgr2=0x558e8af53560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af53560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f33b0000c00 tx=0x7f33b00048c0 comp rx=0 tx=0).stop 2023-10-08T13:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 1 -- 172.21.15.57:0/2493936856 shutdown_connections 2023-10-08T13:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558e8afbf2e0 0x558e8afc1700 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558e8af571a0 0x558e8af52390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f339dffb700 0 client.26380 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e8af510b0 0x558e8af53560 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 1 --2- 172.21.15.57:0/2493936856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e8af56880 0x558e8af53aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 1 -- 172.21.15.57:0/2493936856 >> 172.21.15.57:0/2493936856 conn(0x558e8aec5200 msgr2=0x558e8af58cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 1 -- 172.21.15.57:0/2493936856 shutdown_connections 2023-10-08T13:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 1 -- 172.21.15.57:0/2493936856 wait complete. 2023-10-08T13:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33c86adb80 20 client.26380 trim_cache size 0 max 16384 2023-10-08T13:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:57:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:57:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:57:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33bc4c4700 1 -- 172.21.15.57:0/2859296990 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33a00a17c0 msgr2=0x7f33a00a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33bc4c4700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33a00a17c0 0x7f33a00a3c80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f33b407ead0 tx=0x7f33a4008040 comp rx=0 tx=0).stop 2023-10-08T13:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33bc4c4700 1 -- 172.21.15.57:0/2859296990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33b407d930 msgr2=0x7f33b407dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33bc4c4700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33b407d930 0x7f33b407dde0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00b970 tx=0x7f33ac00bd30 comp rx=0 tx=0).stop 2023-10-08T13:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33bc4c4700 1 -- 172.21.15.57:0/2859296990 shutdown_connections 2023-10-08T13:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33bc4c4700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33b407d930 0x7f33b407dde0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33bc4c4700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f33b4107a60 0x7f33b4083300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33bc4c4700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f33a00a17c0 0x7f33a00a3c80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.515+0000 7f33bc4c4700 1 --2- 172.21.15.57:0/2859296990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b410c070 0x7f33b407d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.516+0000 7f33bc4c4700 1 -- 172.21.15.57:0/2859296990 >> 172.21.15.57:0/2859296990 conn(0x7f33b406b7c0 msgr2=0x7f33b40702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.516+0000 7f33bc4c4700 1 -- 172.21.15.57:0/2859296990 shutdown_connections 2023-10-08T13:57:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:09.516+0000 7f33bc4c4700 1 -- 172.21.15.57:0/2859296990 wait complete. 2023-10-08T13:57:09.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:09 smithi099 ceph-mon[103849]: pgmap v467: 129 pgs: 129 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.2 MiB/s wr, 396 op/s 2023-10-08T13:57:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:57:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:57:10.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:57:10.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:57:10.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:57:10.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:57:10.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:57:10.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:57:10.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:57:11.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:11 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:57:11.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:11 smithi115 ceph-mon[104408]: pgmap v468: 129 pgs: 129 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 570 op/s 2023-10-08T13:57:11.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:11 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:57:11.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:11 smithi057 ceph-mon[99476]: pgmap v468: 129 pgs: 129 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 570 op/s 2023-10-08T13:57:11.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:11 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:57:11.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:11 smithi099 ceph-mon[103849]: pgmap v468: 129 pgs: 129 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 570 op/s 2023-10-08T13:57:13.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:13 smithi115 ceph-mon[104408]: pgmap v469: 129 pgs: 129 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.1 MiB/s wr, 537 op/s 2023-10-08T13:57:13.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:13 smithi057 ceph-mon[99476]: pgmap v469: 129 pgs: 129 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.1 MiB/s wr, 537 op/s 2023-10-08T13:57:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:13 smithi099 ceph-mon[103849]: pgmap v469: 129 pgs: 129 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.1 MiB/s wr, 537 op/s 2023-10-08T13:57:14.415 INFO:tasks.workunit.client.0.smithi057.stdout: 472 1025328 424 707633 553 171396 295 2023-10-08T13:57:14.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:14 smithi115 ceph-mon[104408]: pgmap v470: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 564 op/s 2023-10-08T13:57:14.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:14 smithi057 ceph-mon[99476]: pgmap v470: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 564 op/s 2023-10-08T13:57:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:14 smithi099 ceph-mon[103849]: pgmap v470: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 564 op/s 2023-10-08T13:57:16.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:16 smithi115 ceph-mon[104408]: pgmap v471: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 479 op/s 2023-10-08T13:57:16.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:16 smithi057 ceph-mon[99476]: pgmap v471: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 479 op/s 2023-10-08T13:57:17.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:16 smithi099 ceph-mon[103849]: pgmap v471: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 479 op/s 2023-10-08T13:57:19.259 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T13:57:19.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:19 smithi115 ceph-mon[104408]: pgmap v472: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2023-10-08T13:57:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:19 smithi057 ceph-mon[99476]: pgmap v472: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2023-10-08T13:57:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:19 smithi099 ceph-mon[103849]: pgmap v472: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 415 op/s 2023-10-08T13:57:20.668 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:20 smithi057 ceph-mon[99476]: pgmap v473: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 390 op/s 2023-10-08T13:57:20.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:20 smithi115 ceph-mon[104408]: pgmap v473: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 390 op/s 2023-10-08T13:57:21.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:20 smithi099 ceph-mon[103849]: pgmap v473: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 390 op/s 2023-10-08T13:57:21.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.662+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/3363627619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18108380 msgr2=0x7f3a1810b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:21.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.662+0000 7f3a1dc0a700 1 --2- 172.21.15.57:0/3363627619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18108380 0x7f3a1810b9b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000b0a0 tx=0x7f3a1000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.673+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/3363627619 shutdown_connections 2023-10-08T13:57:21.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.673+0000 7f3a1dc0a700 1 --2- 172.21.15.57:0/3363627619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a1810c0e0 0x7f3a18078b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.673+0000 7f3a1dc0a700 1 --2- 172.21.15.57:0/3363627619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a18107a60 0x7f3a18107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.673+0000 7f3a1dc0a700 1 --2- 172.21.15.57:0/3363627619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18108380 0x7f3a1810b9b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.673+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/3363627619 >> 172.21.15.57:0/3363627619 conn(0x7f3a1806b830 msgr2=0x7f3a1806bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:21.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.673+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/3363627619 shutdown_connections 2023-10-08T13:57:21.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.674+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/3363627619 wait complete. 2023-10-08T13:57:21.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.675+0000 7f3a1dc0a700 1 Processor -- start 2023-10-08T13:57:21.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.675+0000 7f3a1dc0a700 1 -- start start 2023-10-08T13:57:21.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.676+0000 7f3a1dc0a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18107a60 0x7f3a18083250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.676+0000 7f3a1dc0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a18108380 0x7f3a1807d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.676+0000 7f3a1dc0a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a1810c0e0 0x7f3a1807d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.676+0000 7f3a1dc0a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1807e000 con 0x7f3a18108380 2023-10-08T13:57:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.676+0000 7f3a1dc0a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1807e170 con 0x7f3a18107a60 2023-10-08T13:57:21.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.676+0000 7f3a1dc0a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a18135c20 con 0x7f3a1810c0e0 2023-10-08T13:57:21.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.680+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a18108380 0x7f3a1807d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.680+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a18108380 0x7f3a1807d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58678/0 (socket says 172.21.15.57:58678) 2023-10-08T13:57:21.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.680+0000 7f3a16ffd700 1 -- 172.21.15.57:0/1856919727 learned_addr learned my addr 172.21.15.57:0/1856919727 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.687+0000 7f3a17fff700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a1810c0e0 0x7f3a1807d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.687+0000 7f3a177fe700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18107a60 0x7f3a18083250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.692+0000 7f3a177fe700 1 -- 172.21.15.57:0/1856919727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a1810c0e0 msgr2=0x7f3a1807d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:21.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.692+0000 7f3a177fe700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a1810c0e0 0x7f3a1807d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.692+0000 7f3a177fe700 1 -- 172.21.15.57:0/1856919727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a18108380 msgr2=0x7f3a1807d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:21.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.692+0000 7f3a177fe700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a18108380 0x7f3a1807d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.692+0000 7f3a177fe700 1 -- 172.21.15.57:0/1856919727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009d00 con 0x7f3a18107a60 2023-10-08T13:57:21.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.701+0000 7f3a177fe700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18107a60 0x7f3a18083250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006fd0 tx=0x7f3a0c00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:21.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.704+0000 7f3a14ff9700 1 -- 172.21.15.57:0/1856919727 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c011830 con 0x7f3a18107a60 2023-10-08T13:57:21.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.704+0000 7f3a14ff9700 1 -- 172.21.15.57:0/1856919727 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a0c011e50 con 0x7f3a18107a60 2023-10-08T13:57:21.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.704+0000 7f3a14ff9700 1 -- 172.21.15.57:0/1856919727 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c010dc0 con 0x7f3a18107a60 2023-10-08T13:57:21.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.704+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/1856919727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a18135f00 con 0x7f3a18107a60 2023-10-08T13:57:21.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.704+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/1856919727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a181363c0 con 0x7f3a18107a60 2023-10-08T13:57:21.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.729+0000 7f3a29df3b80 1 Processor -- start 2023-10-08T13:57:21.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.730+0000 7f3a29df3b80 1 -- start start 2023-10-08T13:57:21.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.730+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.730+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.730+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.730+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcb02d460 con 0x55bdcb0c3e80 2023-10-08T13:57:21.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.730+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcb01ec20 con 0x55bdcb0c47a0 2023-10-08T13:57:21.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.730+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcb01c540 con 0x55bdcb0cf650 2023-10-08T13:57:21.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.735+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.735+0000 7f3a14ff9700 1 -- 172.21.15.57:0/1856919727 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3a0c011990 con 0x7f3a18107a60 2023-10-08T13:57:21.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.736+0000 7f3a14ff9700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a000a1800 0x7f3a000a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.737+0000 7f3a14ff9700 1 -- 172.21.15.57:0/1856919727 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3a0c0ccb30 con 0x7f3a18107a60 2023-10-08T13:57:21.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.737+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.737+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60112/0 (socket says 172.21.15.57:60112) 2023-10-08T13:57:21.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.737+0000 7f3a177fe700 1 -- 172.21.15.57:0/4166672105 learned_addr learned my addr 172.21.15.57:0/4166672105 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:21.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.752+0000 7f3a17fff700 1 --2- 172.21.15.57:0/4166672105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.752+0000 7f3a17fff700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a000a1800 0x7f3a000a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.757+0000 7f3a16ffd700 1 -- 172.21.15.57:0/4166672105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 msgr2=0x55bdcb0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:21.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.757+0000 7f3a16ffd700 1 --2- 172.21.15.57:0/4166672105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.757+0000 7f3a16ffd700 1 -- 172.21.15.57:0/4166672105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 msgr2=0x55bdcb0c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:21.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.757+0000 7f3a16ffd700 1 --2- 172.21.15.57:0/4166672105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.757+0000 7f3a16ffd700 1 -- 172.21.15.57:0/4166672105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdcae42de0 con 0x55bdcb0cf650 2023-10-08T13:57:21.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.760+0000 7f3a16ffd700 1 --2- 172.21.15.57:0/4166672105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d1a40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009fd0 tx=0x7f3a10009340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc95536b0cdd4045 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:21.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.838+0000 7f3a05ffb700 1 -- 172.21.15.57:0/4166672105 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000e040 con 0x55bdcb0cf650 2023-10-08T13:57:21.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.838+0000 7f3a05ffb700 1 -- 172.21.15.57:0/4166672105 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a100079a0 con 0x55bdcb0cf650 2023-10-08T13:57:21.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.838+0000 7f3a05ffb700 1 -- 172.21.15.57:0/4166672105 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100055b0 con 0x55bdcb0cf650 2023-10-08T13:57:21.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.850+0000 7f3a29df3b80 1 -- 172.21.15.57:0/4166672105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 msgr2=0x55bdcb0d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:21.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.850+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/4166672105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d1a40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009fd0 tx=0x7f3a10009340 comp rx=0 tx=0).stop 2023-10-08T13:57:21.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.853+0000 7f3a17fff700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a000a1800 0x7f3a000a3cc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009d00 tx=0x7f3a08009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:21.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.857+0000 7f3a29df3b80 1 -- 172.21.15.57:0/4166672105 shutdown_connections 2023-10-08T13:57:21.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.858+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/4166672105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d1a40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.858+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/4166672105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.858+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/4166672105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.858+0000 7f3a29df3b80 1 -- 172.21.15.57:0/4166672105 >> 172.21.15.57:0/4166672105 conn(0x55bdcb039260 msgr2=0x55bdcb0c02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:21.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.858+0000 7f3a29df3b80 1 -- 172.21.15.57:0/4166672105 shutdown_connections 2023-10-08T13:57:21.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.858+0000 7f3a29df3b80 1 -- 172.21.15.57:0/4166672105 wait complete. 2023-10-08T13:57:21.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.858+0000 7f3a29df3b80 1 -- start start 2023-10-08T13:57:21.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.859+0000 7f3a29df3b80 1 Processor -- start 2023-10-08T13:57:21.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcb017e30 con 0x55bdcb0c3e80 2023-10-08T13:57:21.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcafb4710 con 0x55bdcb0c47a0 2023-10-08T13:57:21.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcafb43b0 con 0x55bdcb0cf650 2023-10-08T13:57:21.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a29df3b80 10 client.0 _ll_register_callbacks cb 0x55bdcae11800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:21.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60118/0 (socket says 172.21.15.57:60118) 2023-10-08T13:57:21.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a16ffd700 1 -- 172.21.15.57:0/97715843 learned_addr learned my addr 172.21.15.57:0/97715843 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:21.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.860+0000 7f3a17fff700 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.867+0000 7f3a177fe700 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.867+0000 7f3a17fff700 1 -- 172.21.15.57:0/97715843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 msgr2=0x55bdcb0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:21.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.867+0000 7f3a17fff700 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.867+0000 7f3a17fff700 1 -- 172.21.15.57:0/97715843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 msgr2=0x55bdcb0cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:21.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.867+0000 7f3a17fff700 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.867+0000 7f3a17fff700 1 -- 172.21.15.57:0/97715843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a180793c0 con 0x55bdcb0cf650 2023-10-08T13:57:21.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.868+0000 7f3a16ffd700 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:21.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.875+0000 7f3a17fff700 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0cc940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3a08015a00 tx=0x7f3a08004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c0c2c4c7630a418 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:21.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.880+0000 7f39faffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:57:21.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.882+0000 7f39faffd700 1 -- 172.21.15.57:0/97715843 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08009de0 con 0x55bdcb0cf650 2023-10-08T13:57:21.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.882+0000 7f39faffd700 1 -- 172.21.15.57:0/97715843 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3a08014870 con 0x55bdcb0cf650 2023-10-08T13:57:21.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.882+0000 7f3a29df3b80 1 -- 172.21.15.57:0/97715843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdcafdfeb0 con 0x55bdcb0cf650 2023-10-08T13:57:21.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.888+0000 7f39faffd700 1 -- 172.21.15.57:0/97715843 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a08005a50 con 0x55bdcb0cf650 2023-10-08T13:57:21.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.888+0000 7f3a29df3b80 10 client.26389 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:21.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.888+0000 7f3a29df3b80 1 -- 172.21.15.57:0/97715843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bdcae2bc60 con 0x55bdcb0cf650 2023-10-08T13:57:21.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.899+0000 7f39faffd700 1 -- 172.21.15.57:0/97715843 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a0802d030 con 0x55bdcb0cf650 2023-10-08T13:57:21.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.899+0000 7f39faffd700 1 -- 172.21.15.57:0/97715843 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f3a08006260 con 0x55bdcb0cf650 2023-10-08T13:57:21.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.900+0000 7f3a29df3b80 10 client.26389 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:21.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.900+0000 7f3a29df3b80 10 client.26389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:21.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.901+0000 7f3a29df3b80 20 client.26389 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:21.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.901+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bdcb0df280 0x55bdcb0e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:21.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.901+0000 7f3a29df3b80 4 client.26389 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:57:21.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.901+0000 7f3a29df3b80 1 -- 172.21.15.57:0/97715843 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdcb010e10 con 0x55bdcb0df280 2023-10-08T13:57:21.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.907+0000 7f3a177fe700 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bdcb0df280 0x55bdcb0e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:21.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.923+0000 7f3a177fe700 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bdcb0df280 0x55bdcb0e16a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44173c2c928376ce server_cookie=5dbb6b510018df28 in_seq=0 out_seq=0 2023-10-08T13:57:21.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.924+0000 7f39faffd700 10 client.26389 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:21.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.937+0000 7f39faffd700 1 -- 172.21.15.57:0/97715843 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bdcb010e10 con 0x55bdcb0df280 2023-10-08T13:57:21.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.937+0000 7f39faffd700 10 client.26389 handle_command_reply: tid=0 2023-10-08T13:57:21.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.937+0000 7f3a29df3b80 1 client.26389 shutdown 2023-10-08T13:57:21.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.938+0000 7f3a29df3b80 1 -- 172.21.15.57:0/97715843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 msgr2=0x55bdcb0cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:21.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.938+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0cc940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3a08015a00 tx=0x7f3a08004750 comp rx=0 tx=0).stop 2023-10-08T13:57:21.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.938+0000 7f3a29df3b80 1 -- 172.21.15.57:0/97715843 shutdown_connections 2023-10-08T13:57:21.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.938+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bdcb0df280 0x55bdcb0e16a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.938+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0cc940 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.938+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.938+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/97715843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:21.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.938+0000 7f3a29df3b80 1 -- 172.21.15.57:0/97715843 >> 172.21.15.57:0/97715843 conn(0x55bdcb039260 msgr2=0x55bdcb0bfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:21.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.938+0000 7f39faffd700 0 client.26389 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:21.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.938+0000 7f3a29df3b80 1 -- 172.21.15.57:0/97715843 shutdown_connections 2023-10-08T13:57:21.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.945+0000 7f3a29df3b80 1 -- 172.21.15.57:0/97715843 wait complete. 2023-10-08T13:57:21.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:21.949+0000 7f3a29df3b80 20 client.26389 trim_cache size 0 max 16384 2023-10-08T13:57:22.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.010+0000 7f3a29df3b80 1 Processor -- start 2023-10-08T13:57:22.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.010+0000 7f3a29df3b80 1 -- start start 2023-10-08T13:57:22.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.010+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:22.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.011+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0bea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:22.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.011+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:22.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.011+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcae98920 con 0x55bdcb0c3e80 2023-10-08T13:57:22.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.011+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcb0bef70 con 0x55bdcb0c47a0 2023-10-08T13:57:22.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.011+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcb0d6720 con 0x55bdcb0cf650 2023-10-08T13:57:22.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.011+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:22.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.012+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59514/0 (socket says 172.21.15.57:59514) 2023-10-08T13:57:22.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.012+0000 7f3a17fff700 1 -- 172.21.15.57:0/3269251028 learned_addr learned my addr 172.21.15.57:0/3269251028 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:22.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.012+0000 7f3a16ffd700 1 --2- 172.21.15.57:0/3269251028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:22.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.012+0000 7f3a177fe700 1 --2- 172.21.15.57:0/3269251028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:22.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.012+0000 7f3a17fff700 1 -- 172.21.15.57:0/3269251028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 msgr2=0x55bdcb0bea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:22.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.012+0000 7f3a17fff700 1 --2- 172.21.15.57:0/3269251028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0bea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.012+0000 7f3a17fff700 1 -- 172.21.15.57:0/3269251028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 msgr2=0x55bdcb0cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:22.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.012+0000 7f3a17fff700 1 --2- 172.21.15.57:0/3269251028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.012+0000 7f3a17fff700 1 -- 172.21.15.57:0/3269251028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdcb0bc4f0 con 0x55bdcb0cf650 2023-10-08T13:57:22.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.013+0000 7f3a17fff700 1 --2- 172.21.15.57:0/3269251028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009320 tx=0x7f3a08020440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9dd2d5bc827aa60 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:22.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.051+0000 7f3a067fc700 1 -- 172.21.15.57:0/3269251028 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08009a90 con 0x55bdcb0cf650 2023-10-08T13:57:22.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.051+0000 7f3a067fc700 1 -- 172.21.15.57:0/3269251028 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a0802d070 con 0x55bdcb0cf650 2023-10-08T13:57:22.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.051+0000 7f3a067fc700 1 -- 172.21.15.57:0/3269251028 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08020ba0 con 0x55bdcb0cf650 2023-10-08T13:57:22.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.052+0000 7f3a29df3b80 1 -- 172.21.15.57:0/3269251028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 msgr2=0x55bdcb0d61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:22.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.052+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/3269251028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009320 tx=0x7f3a08020440 comp rx=0 tx=0).stop 2023-10-08T13:57:22.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.053+0000 7f3a29df3b80 1 -- 172.21.15.57:0/3269251028 shutdown_connections 2023-10-08T13:57:22.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.053+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/3269251028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.053+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/3269251028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.053+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/3269251028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0bea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.053+0000 7f3a29df3b80 1 -- 172.21.15.57:0/3269251028 >> 172.21.15.57:0/3269251028 conn(0x55bdcb039260 msgr2=0x55bdcb0cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:22.055 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.054+0000 7f3a29df3b80 1 -- 172.21.15.57:0/3269251028 shutdown_connections 2023-10-08T13:57:22.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.054+0000 7f3a29df3b80 1 -- 172.21.15.57:0/3269251028 wait complete. 2023-10-08T13:57:22.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.054+0000 7f3a29df3b80 1 -- start start 2023-10-08T13:57:22.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.055+0000 7f3a29df3b80 1 Processor -- start 2023-10-08T13:57:22.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.055+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0d5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:22.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.055+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0d42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:22.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.055+0000 7f3a29df3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:22.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.055+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcb0d4d60 con 0x55bdcb0cf650 2023-10-08T13:57:22.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.055+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcb0d4ed0 con 0x55bdcb0c47a0 2023-10-08T13:57:22.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.055+0000 7f3a29df3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcb0d5040 con 0x55bdcb0c3e80 2023-10-08T13:57:22.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.055+0000 7f3a29df3b80 10 client.0 _ll_register_callbacks cb 0x7f3a08012e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:22.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.057+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:22.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.058+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58708/0 (socket says 172.21.15.57:58708) 2023-10-08T13:57:22.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.058+0000 7f3a17fff700 1 -- 172.21.15.57:0/1576284521 learned_addr learned my addr 172.21.15.57:0/1576284521 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:22.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.068+0000 7f3a177fe700 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0d5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:22.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.068+0000 7f3a16ffd700 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0d42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:22.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.072+0000 7f3a17fff700 1 -- 172.21.15.57:0/1576284521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0c3e80 msgr2=0x55bdcb0d5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:22.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.072+0000 7f3a17fff700 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0d5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.072+0000 7f3a17fff700 1 -- 172.21.15.57:0/1576284521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 msgr2=0x55bdcb0d42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:22.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.072+0000 7f3a17fff700 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0d42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.072+0000 7f3a17fff700 1 -- 172.21.15.57:0/1576284521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdcae2bc60 con 0x55bdcb0cf650 2023-10-08T13:57:22.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.074+0000 7f3a17fff700 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d4820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b010 tx=0x7f3a08004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dd403bffd3fede server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:22.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.158+0000 7f39faffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:57:22.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.158+0000 7f39faffd700 1 -- 172.21.15.57:0/1576284521 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08020df0 con 0x55bdcb0cf650 2023-10-08T13:57:22.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.158+0000 7f39faffd700 1 -- 172.21.15.57:0/1576284521 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f3a080146c0 con 0x55bdcb0cf650 2023-10-08T13:57:22.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.158+0000 7f3a29df3b80 1 -- 172.21.15.57:0/1576284521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a100079a0 con 0x55bdcb0cf650 2023-10-08T13:57:22.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.170+0000 7f39faffd700 1 -- 172.21.15.57:0/1576284521 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a08014880 con 0x55bdcb0cf650 2023-10-08T13:57:22.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.170+0000 7f3a29df3b80 10 client.16785 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:22.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.170+0000 7f3a29df3b80 1 -- 172.21.15.57:0/1576284521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bdcae69250 con 0x55bdcb0cf650 2023-10-08T13:57:22.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.176+0000 7f39faffd700 1 -- 172.21.15.57:0/1576284521 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a0804c030 con 0x55bdcb0cf650 2023-10-08T13:57:22.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.176+0000 7f39faffd700 1 -- 172.21.15.57:0/1576284521 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f3a080061d0 con 0x55bdcb0cf650 2023-10-08T13:57:22.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.177+0000 7f3a29df3b80 10 client.16785 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:22.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.177+0000 7f3a29df3b80 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:22.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.177+0000 7f3a29df3b80 20 client.16785 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:22.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.177+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bdcb134190 0x55bdcb136580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:22.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.177+0000 7f3a29df3b80 4 client.16785 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T13:57:22.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.177+0000 7f3a29df3b80 1 -- 172.21.15.57:0/1576284521 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bdcb010b30 con 0x55bdcb134190 2023-10-08T13:57:22.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.189+0000 7f3a177fe700 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bdcb134190 0x55bdcb136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:22.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.234+0000 7f3a177fe700 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bdcb134190 0x55bdcb136580 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcd1e3b2fc33399d server_cookie=ff21ce580c60c6ec in_seq=0 out_seq=0 2023-10-08T13:57:22.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.234+0000 7f39faffd700 10 client.16785 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:22.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.243+0000 7f39faffd700 1 -- 172.21.15.57:0/1576284521 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bdcb010b30 con 0x55bdcb134190 2023-10-08T13:57:22.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.243+0000 7f39faffd700 10 client.16785 handle_command_reply: tid=0 2023-10-08T13:57:22.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.243+0000 7f3a29df3b80 1 client.16785 shutdown 2023-10-08T13:57:22.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.245+0000 7f3a29df3b80 1 -- 172.21.15.57:0/1576284521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0cf650 msgr2=0x55bdcb0d4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:22.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.245+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d4820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b010 tx=0x7f3a08004860 comp rx=0 tx=0).stop 2023-10-08T13:57:22.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.245+0000 7f3a29df3b80 1 -- 172.21.15.57:0/1576284521 shutdown_connections 2023-10-08T13:57:22.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.245+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55bdcb134190 0x55bdcb136580 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.245+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bdcb0cf650 0x55bdcb0d4820 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.245+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bdcb0c3e80 0x55bdcb0d5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.245+0000 7f3a29df3b80 1 --2- 172.21.15.57:0/1576284521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bdcb0c47a0 0x55bdcb0d42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.245+0000 7f3a29df3b80 1 -- 172.21.15.57:0/1576284521 >> 172.21.15.57:0/1576284521 conn(0x55bdcb039260 msgr2=0x55bdcb0bfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:22.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.245+0000 7f39faffd700 0 client.16785 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:22.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.246+0000 7f3a29df3b80 1 -- 172.21.15.57:0/1576284521 shutdown_connections 2023-10-08T13:57:22.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.246+0000 7f3a29df3b80 1 -- 172.21.15.57:0/1576284521 wait complete. 2023-10-08T13:57:22.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.246+0000 7f3a29df3b80 20 client.16785 trim_cache size 0 max 16384 2023-10-08T13:57:22.250 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T13:57:22.250 INFO:teuthology.orchestra.run.smithi057.stdout: "status": "no active scrubs running", 2023-10-08T13:57:22.250 INFO:teuthology.orchestra.run.smithi057.stdout: "scrubs": {} 2023-10-08T13:57:22.250 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T13:57:22.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.269+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/1856919727 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a000a1800 msgr2=0x7f3a000a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:22.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.269+0000 7f3a1dc0a700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a000a1800 0x7f3a000a3cc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009d00 tx=0x7f3a08009110 comp rx=0 tx=0).stop 2023-10-08T13:57:22.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.269+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/1856919727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18107a60 msgr2=0x7f3a18083250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:22.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.269+0000 7f3a1dc0a700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18107a60 0x7f3a18083250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006fd0 tx=0x7f3a0c00b730 comp rx=0 tx=0).stop 2023-10-08T13:57:22.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.269+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/1856919727 shutdown_connections 2023-10-08T13:57:22.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.269+0000 7f3a1dc0a700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a1810c0e0 0x7f3a1807d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.269+0000 7f3a1dc0a700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3a000a1800 0x7f3a000a3cc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.269+0000 7f3a1dc0a700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18107a60 0x7f3a18083250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.270+0000 7f3a1dc0a700 1 --2- 172.21.15.57:0/1856919727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a18108380 0x7f3a1807d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:22.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.270+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/1856919727 >> 172.21.15.57:0/1856919727 conn(0x7f3a1806b830 msgr2=0x7f3a180704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:22.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.270+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/1856919727 shutdown_connections 2023-10-08T13:57:22.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:22.270+0000 7f3a1dc0a700 1 -- 172.21.15.57:0/1856919727 wait complete. 2023-10-08T13:57:22.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-08T13:57:22.402 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T13:57:23.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:23 smithi115 ceph-mon[104408]: pgmap v474: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-10-08T13:57:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.950+0000 7f9918f7a700 1 -- 172.21.15.57:0/2148043512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f991410ef10 msgr2=0x7f99141077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.950+0000 7f9918f7a700 1 --2- 172.21.15.57:0/2148043512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f991410ef10 0x7f99141077f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9904009990 tx=0x7f9904009ca0 comp rx=0 tx=0).stop 2023-10-08T13:57:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.956+0000 7f9918f7a700 1 -- 172.21.15.57:0/2148043512 shutdown_connections 2023-10-08T13:57:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.956+0000 7f9918f7a700 1 --2- 172.21.15.57:0/2148043512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9914119fd0 0x7f991411c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.956+0000 7f9918f7a700 1 --2- 172.21.15.57:0/2148043512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f991410ef10 0x7f99141077f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.956+0000 7f9918f7a700 1 --2- 172.21.15.57:0/2148043512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f991410f380 0x7f9914107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.956+0000 7f9918f7a700 1 -- 172.21.15.57:0/2148043512 >> 172.21.15.57:0/2148043512 conn(0x7f991406b7c0 msgr2=0x7f991406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.956+0000 7f9918f7a700 1 -- 172.21.15.57:0/2148043512 shutdown_connections 2023-10-08T13:57:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.956+0000 7f9918f7a700 1 -- 172.21.15.57:0/2148043512 wait complete. 2023-10-08T13:57:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.957+0000 7f9918f7a700 1 Processor -- start 2023-10-08T13:57:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.958+0000 7f9918f7a700 1 -- start start 2023-10-08T13:57:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.958+0000 7f9918f7a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991410ef10 0x7f9914118340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.958+0000 7f9918f7a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f991410f380 0x7f99141188c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.958+0000 7f9918f7a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9914119fd0 0x7f9914111470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.958+0000 7f9918f7a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9914111c40 con 0x7f991410ef10 2023-10-08T13:57:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.958+0000 7f9918f7a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9914111db0 con 0x7f991410f380 2023-10-08T13:57:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.958+0000 7f9918f7a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9914111f70 con 0x7f9914119fd0 2023-10-08T13:57:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.964+0000 7f9911d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f991410f380 0x7f99141188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.964+0000 7f9912d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9914119fd0 0x7f9914111470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.964+0000 7f9911d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f991410f380 0x7f99141188c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60144/0 (socket says 172.21.15.57:60144) 2023-10-08T13:57:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.964+0000 7f9911d9b700 1 -- 172.21.15.57:0/896912659 learned_addr learned my addr 172.21.15.57:0/896912659 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.964+0000 7f991259c700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991410ef10 0x7f9914118340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.971+0000 7f991259c700 1 -- 172.21.15.57:0/896912659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9914119fd0 msgr2=0x7f9914111470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.971+0000 7f991259c700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9914119fd0 0x7f9914111470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.971+0000 7f991259c700 1 -- 172.21.15.57:0/896912659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f991410f380 msgr2=0x7f99141188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.971+0000 7f991259c700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f991410f380 0x7f99141188c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.971+0000 7f991259c700 1 -- 172.21.15.57:0/896912659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99040095f0 con 0x7f991410ef10 2023-10-08T13:57:23.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:23 smithi057 ceph-mon[99476]: pgmap v474: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-10-08T13:57:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.987+0000 7f991259c700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991410ef10 0x7f9914118340 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9904000c00 tx=0x7f9904005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.994+0000 7f99037fe700 1 -- 172.21.15.57:0/896912659 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990401d070 con 0x7f991410ef10 2023-10-08T13:57:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.994+0000 7f99037fe700 1 -- 172.21.15.57:0/896912659 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99040173f0 con 0x7f991410ef10 2023-10-08T13:57:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:23.994+0000 7f99037fe700 1 -- 172.21.15.57:0/896912659 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400e640 con 0x7f991410ef10 2023-10-08T13:57:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.002+0000 7f9918f7a700 1 -- 172.21.15.57:0/896912659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99141121f0 con 0x7f991410ef10 2023-10-08T13:57:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.002+0000 7f9918f7a700 1 -- 172.21.15.57:0/896912659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99141160b0 con 0x7f991410ef10 2023-10-08T13:57:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.014+0000 7f9925163b80 1 Processor -- start 2023-10-08T13:57:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.018+0000 7f9925163b80 1 -- start start 2023-10-08T13:57:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.018+0000 7f9925163b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188be880 0x5649188bec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.019+0000 7f9925163b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649188bf1a0 0x5649188b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.019+0000 7f9925163b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188b9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.019+0000 7f9925163b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564918821400 con 0x5649188be880 2023-10-08T13:57:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.019+0000 7f9925163b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564918812bc0 con 0x5649188bf1a0 2023-10-08T13:57:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.019+0000 7f9925163b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649188104e0 con 0x5649188b90b0 2023-10-08T13:57:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.019+0000 7f9911d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188b9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.019+0000 7f9911d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188b9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59536/0 (socket says 172.21.15.57:59536) 2023-10-08T13:57:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.019+0000 7f9911d9b700 1 -- 172.21.15.57:0/4032354512 learned_addr learned my addr 172.21.15.57:0/4032354512 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.020+0000 7f9911d9b700 1 -- 172.21.15.57:0/4032354512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649188bf1a0 msgr2=0x5649188b8b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:57:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.020+0000 7f9911d9b700 1 --2- 172.21.15.57:0/4032354512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649188bf1a0 0x5649188b8b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.020+0000 7f9911d9b700 1 -- 172.21.15.57:0/4032354512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188be880 msgr2=0x5649188bec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.020+0000 7f9911d9b700 1 --2- 172.21.15.57:0/4032354512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188be880 0x5649188bec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.020+0000 7f9911d9b700 1 -- 172.21.15.57:0/4032354512 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564918636d80 con 0x5649188b90b0 2023-10-08T13:57:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:23 smithi099 ceph-mon[103849]: pgmap v474: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-10-08T13:57:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.021+0000 7f9911d9b700 1 --2- 172.21.15.57:0/4032354512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188b9510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f990c00a6d0 tx=0x7f990c0079c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80171e48db302963 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.022+0000 7f99037fe700 1 -- 172.21.15.57:0/896912659 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9904017560 con 0x7f991410ef10 2023-10-08T13:57:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.023+0000 7f99037fe700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98fc0a1800 0x7f98fc0a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.023+0000 7f99037fe700 1 -- 172.21.15.57:0/896912659 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f99040ceab0 con 0x7f991410ef10 2023-10-08T13:57:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.025+0000 7f9912d9d700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98fc0a1800 0x7f98fc0a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.039+0000 7f9900ff9700 1 -- 172.21.15.57:0/4032354512 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00f050 con 0x5649188b90b0 2023-10-08T13:57:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.039+0000 7f9900ff9700 1 -- 172.21.15.57:0/4032354512 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990c010040 con 0x5649188b90b0 2023-10-08T13:57:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.039+0000 7f9900ff9700 1 -- 172.21.15.57:0/4032354512 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0089e0 con 0x5649188b90b0 2023-10-08T13:57:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.040+0000 7f9925163b80 1 -- 172.21.15.57:0/4032354512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 msgr2=0x5649188b9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.040+0000 7f9925163b80 1 --2- 172.21.15.57:0/4032354512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188b9510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f990c00a6d0 tx=0x7f990c0079c0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.041+0000 7f9925163b80 1 -- 172.21.15.57:0/4032354512 shutdown_connections 2023-10-08T13:57:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.041+0000 7f9925163b80 1 --2- 172.21.15.57:0/4032354512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188b9510 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.041+0000 7f9925163b80 1 --2- 172.21.15.57:0/4032354512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188be880 0x5649188bec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.041+0000 7f9925163b80 1 --2- 172.21.15.57:0/4032354512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649188bf1a0 0x5649188b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.041+0000 7f9925163b80 1 -- 172.21.15.57:0/4032354512 >> 172.21.15.57:0/4032354512 conn(0x56491882d200 msgr2=0x5649188b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.046+0000 7f9912d9d700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98fc0a1800 0x7f98fc0a3cc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9914069db0 tx=0x7f9908009850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.046+0000 7f9925163b80 1 -- 172.21.15.57:0/4032354512 shutdown_connections 2023-10-08T13:57:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.046+0000 7f9925163b80 1 -- 172.21.15.57:0/4032354512 wait complete. 2023-10-08T13:57:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.047+0000 7f9925163b80 1 -- start start 2023-10-08T13:57:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.047+0000 7f9925163b80 1 Processor -- start 2023-10-08T13:57:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.048+0000 7f9925163b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649188b90b0 0x5649188c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.048+0000 7f9925163b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 0x5649188c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.048+0000 7f9925163b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188bf1a0 0x5649188c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.048+0000 7f9925163b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491880bdd0 con 0x5649188bf1a0 2023-10-08T13:57:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.048+0000 7f9925163b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649187a86b0 con 0x5649188b90b0 2023-10-08T13:57:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.048+0000 7f9925163b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649187a8350 con 0x5649188be880 2023-10-08T13:57:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.048+0000 7f9925163b80 10 client.0 _ll_register_callbacks cb 0x5649186057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.056+0000 7f9912d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188bf1a0 0x5649188c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.056+0000 7f9912d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188bf1a0 0x5649188c30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58732/0 (socket says 172.21.15.57:58732) 2023-10-08T13:57:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.056+0000 7f9912d9d700 1 -- 172.21.15.57:0/4102286793 learned_addr learned my addr 172.21.15.57:0/4102286793 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.056+0000 7f9911d9b700 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 0x5649188c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.056+0000 7f991259c700 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649188b90b0 0x5649188c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.067+0000 7f9912d9d700 1 -- 172.21.15.57:0/4102286793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 msgr2=0x5649188c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.067+0000 7f9912d9d700 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 0x5649188c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.067+0000 7f9912d9d700 1 -- 172.21.15.57:0/4102286793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649188b90b0 msgr2=0x5649188c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.067+0000 7f9912d9d700 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649188b90b0 0x5649188c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.067+0000 7f9912d9d700 1 -- 172.21.15.57:0/4102286793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c00a410 con 0x5649188bf1a0 2023-10-08T13:57:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.068+0000 7f991259c700 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649188b90b0 0x5649188c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.077+0000 7f9912d9d700 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188bf1a0 0x5649188c30d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9908009b70 tx=0x7f9908009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5980f42adc627c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.092+0000 7f98f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:57:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.092+0000 7f98f9ffb700 1 -- 172.21.15.57:0/4102286793 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908027430 con 0x5649188bf1a0 2023-10-08T13:57:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.092+0000 7f98f9ffb700 1 -- 172.21.15.57:0/4102286793 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9908023070 con 0x5649188bf1a0 2023-10-08T13:57:24.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.105+0000 7f83105e8700 1 -- 172.21.15.57:0/2603337539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83081083a0 msgr2=0x7f830811bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:24.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.105+0000 7f83105e8700 1 --2- 172.21.15.57:0/2603337539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83081083a0 0x7f830811bfe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f830000b0a0 tx=0x7f830000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.100+0000 7f9925163b80 1 -- 172.21.15.57:0/4102286793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649187d3e50 con 0x5649188bf1a0 2023-10-08T13:57:24.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.106+0000 7f83105e8700 1 -- 172.21.15.57:0/2603337539 shutdown_connections 2023-10-08T13:57:24.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.106+0000 7f83105e8700 1 --2- 172.21.15.57:0/2603337539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83081083a0 0x7f830811bfe0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.106+0000 7f83105e8700 1 --2- 172.21.15.57:0/2603337539 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830810edf0 0x7f830810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.106+0000 7f83105e8700 1 --2- 172.21.15.57:0/2603337539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83081077f0 0x7f8308107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.106+0000 7f83105e8700 1 -- 172.21.15.57:0/2603337539 >> 172.21.15.57:0/2603337539 conn(0x7f830806b760 msgr2=0x7f830806bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.107+0000 7f83105e8700 1 -- 172.21.15.57:0/2603337539 shutdown_connections 2023-10-08T13:57:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.108+0000 7f83105e8700 1 -- 172.21.15.57:0/2603337539 wait complete. 2023-10-08T13:57:24.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.109+0000 7f83105e8700 1 Processor -- start 2023-10-08T13:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.110+0000 7f98f9ffb700 1 -- 172.21.15.57:0/4102286793 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9908027590 con 0x5649188bf1a0 2023-10-08T13:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.110+0000 7f9925163b80 10 client.16794 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.110+0000 7f9925163b80 1 -- 172.21.15.57:0/4102286793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56491861fc00 con 0x5649188bf1a0 2023-10-08T13:57:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.109+0000 7f83105e8700 1 -- start start 2023-10-08T13:57:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.109+0000 7f83105e8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83081077f0 0x7f830811b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.110+0000 7f83105e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83081083a0 0x7f830811bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.110+0000 7f83105e8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830810edf0 0x7f8308115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.110+0000 7f83105e8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081163b0 con 0x7f830810edf0 2023-10-08T13:57:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.110+0000 7f83105e8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8308116520 con 0x7f83081077f0 2023-10-08T13:57:24.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.110+0000 7f83105e8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081166e0 con 0x7f83081083a0 2023-10-08T13:57:24.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.113+0000 7f830db83700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83081083a0 0x7f830811bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.113+0000 7f830db83700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83081083a0 0x7f830811bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59554/0 (socket says 172.21.15.57:59554) 2023-10-08T13:57:24.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.113+0000 7f830db83700 1 -- 172.21.15.57:0/3636442660 learned_addr learned my addr 172.21.15.57:0/3636442660 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:24.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.113+0000 7f830db83700 1 -- 172.21.15.57:0/3636442660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83081077f0 msgr2=0x7f830811b9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T13:57:24.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.113+0000 7f830db83700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83081077f0 0x7f830811b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.113+0000 7f830db83700 1 -- 172.21.15.57:0/3636442660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830810edf0 msgr2=0x7f8308115be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:24.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.113+0000 7f830db83700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830810edf0 0x7f8308115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.113+0000 7f830db83700 1 -- 172.21.15.57:0/3636442660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8300009d00 con 0x7f83081083a0 2023-10-08T13:57:24.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.114+0000 7f830db83700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83081083a0 0x7f830811bf10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82f800b550 tx=0x7f82f800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:24.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.114+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3636442660 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8011830 con 0x7f83081083a0 2023-10-08T13:57:24.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.114+0000 7f83105e8700 1 -- 172.21.15.57:0/3636442660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83081baab0 con 0x7f83081083a0 2023-10-08T13:57:24.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.114+0000 7f83105e8700 1 -- 172.21.15.57:0/3636442660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081baf60 con 0x7f83081083a0 2023-10-08T13:57:24.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.115+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3636442660 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82f8011e50 con 0x7f83081083a0 2023-10-08T13:57:24.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.115+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3636442660 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8010e50 con 0x7f83081083a0 2023-10-08T13:57:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.119+0000 7f98f9ffb700 1 -- 172.21.15.57:0/4102286793 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649187d3e50 con 0x5649188bf1a0 2023-10-08T13:57:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.119+0000 7f98f9ffb700 1 -- 172.21.15.57:0/4102286793 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f9908031910 con 0x5649188bf1a0 2023-10-08T13:57:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.121+0000 7f9925163b80 10 client.16794 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.121+0000 7f9925163b80 10 client.16794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.121+0000 7f9925163b80 20 client.16794 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.121+0000 7f9925163b80 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5649188d2cc0 0x5649188d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.121+0000 7f9925163b80 4 client.16794 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:57:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.121+0000 7f9925163b80 1 -- 172.21.15.57:0/4102286793 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564918804db0 con 0x5649188d2cc0 2023-10-08T13:57:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.122+0000 7f991259c700 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5649188d2cc0 0x5649188d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.121+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3636442660 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f82f80104f0 con 0x7f83081083a0 2023-10-08T13:57:24.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.122+0000 7f82ff7fe700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82f40a1730 0x7f82f40a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.123+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3636442660 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f82f80ccd30 con 0x7f83081083a0 2023-10-08T13:57:24.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.124+0000 7f830e384700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82f40a1730 0x7f82f40a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.125+0000 7f830e384700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82f40a1730 0x7f82f40a3bf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8304009730 tx=0x7f8304006cb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:24.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.161+0000 7f83105e8700 1 -- 172.21.15.57:0/3636442660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ec005320 con 0x7f83081083a0 2023-10-08T13:57:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.171+0000 7f991259c700 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5649188d2cc0 0x5649188d50e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb146cd0a7998691 server_cookie=80488b4cb442b8be in_seq=0 out_seq=0 2023-10-08T13:57:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.171+0000 7f98f9ffb700 10 client.16794 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:24.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:24.171+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3636442660 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f82f8095e20 con 0x7f83081083a0 2023-10-08T13:57:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.183+0000 7f98f9ffb700 1 -- 172.21.15.57:0/4102286793 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564918804db0 con 0x5649188d2cc0 2023-10-08T13:57:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.183+0000 7f98f9ffb700 10 client.16794 handle_command_reply: tid=0 2023-10-08T13:57:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.183+0000 7f9925163b80 1 client.16794 shutdown 2023-10-08T13:57:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.183+0000 7f9925163b80 1 -- 172.21.15.57:0/4102286793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188bf1a0 msgr2=0x5649188c30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.183+0000 7f9925163b80 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188bf1a0 0x5649188c30d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9908009b70 tx=0x7f9908009260 comp rx=0 tx=0).stop 2023-10-08T13:57:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.184+0000 7f9925163b80 1 -- 172.21.15.57:0/4102286793 shutdown_connections 2023-10-08T13:57:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.184+0000 7f9925163b80 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5649188d2cc0 0x5649188d50e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.184+0000 7f9925163b80 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188bf1a0 0x5649188c30d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.184+0000 7f9925163b80 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649188b90b0 0x5649188c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.184+0000 7f9925163b80 1 --2- 172.21.15.57:0/4102286793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 0x5649188c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.184+0000 7f9925163b80 1 -- 172.21.15.57:0/4102286793 >> 172.21.15.57:0/4102286793 conn(0x56491882d200 msgr2=0x5649188b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.184+0000 7f98f9ffb700 0 client.16794 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.184+0000 7f9925163b80 1 -- 172.21.15.57:0/4102286793 shutdown_connections 2023-10-08T13:57:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.184+0000 7f9925163b80 1 -- 172.21.15.57:0/4102286793 wait complete. 2023-10-08T13:57:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.185+0000 7f9925163b80 20 client.16794 trim_cache size 0 max 16384 2023-10-08T13:57:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.361+0000 7f9925163b80 1 Processor -- start 2023-10-08T13:57:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.374+0000 7f9925163b80 1 -- start start 2023-10-08T13:57:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.374+0000 7f9925163b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 0x5649188b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.374+0000 7f9925163b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.374+0000 7f9925163b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188be880 0x5649188bddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.374+0000 7f9925163b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491868c8c0 con 0x5649188be880 2023-10-08T13:57:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.374+0000 7f9925163b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649188b4940 con 0x56491882d200 2023-10-08T13:57:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.374+0000 7f9925163b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649188b52a0 con 0x5649188b90b0 2023-10-08T13:57:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.377+0000 7f9911d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.377+0000 7f991259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 0x5649188b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.378+0000 7f9911d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59562/0 (socket says 172.21.15.57:59562) 2023-10-08T13:57:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.378+0000 7f9911d9b700 1 -- 172.21.15.57:0/1972673497 learned_addr learned my addr 172.21.15.57:0/1972673497 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.386+0000 7f9912d9d700 1 --2- 172.21.15.57:0/1972673497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188be880 0x5649188bddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.386+0000 7f991259c700 1 -- 172.21.15.57:0/1972673497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 msgr2=0x5649188c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.386+0000 7f991259c700 1 --2- 172.21.15.57:0/1972673497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.386+0000 7f991259c700 1 -- 172.21.15.57:0/1972673497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188be880 msgr2=0x5649188bddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.386+0000 7f991259c700 1 --2- 172.21.15.57:0/1972673497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188be880 0x5649188bddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.386+0000 7f991259c700 1 -- 172.21.15.57:0/1972673497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649188c3240 con 0x56491882d200 2023-10-08T13:57:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.405+0000 7f991259c700 1 --2- 172.21.15.57:0/1972673497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 0x5649188b8b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f99040cbf30 tx=0x7f990401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483a1276983d4a70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:24.428 INFO:tasks.workunit.client.0.smithi057.stdout: 477 2189362 375 1514347 207 371180 519 2023-10-08T13:57:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.428+0000 7f99017fa700 1 -- 172.21.15.57:0/1972673497 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990409bf50 con 0x56491882d200 2023-10-08T13:57:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.428+0000 7f99017fa700 1 -- 172.21.15.57:0/1972673497 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990409b850 con 0x56491882d200 2023-10-08T13:57:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.428+0000 7f99017fa700 1 -- 172.21.15.57:0/1972673497 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904099bf0 con 0x56491882d200 2023-10-08T13:57:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.430+0000 7f9925163b80 1 -- 172.21.15.57:0/1972673497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 msgr2=0x5649188b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.430+0000 7f9925163b80 1 --2- 172.21.15.57:0/1972673497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 0x5649188b8b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f99040cbf30 tx=0x7f990401a040 comp rx=0 tx=0).stop 2023-10-08T13:57:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.431+0000 7f9925163b80 1 -- 172.21.15.57:0/1972673497 shutdown_connections 2023-10-08T13:57:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.431+0000 7f9925163b80 1 --2- 172.21.15.57:0/1972673497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188be880 0x5649188bddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.431+0000 7f9925163b80 1 --2- 172.21.15.57:0/1972673497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 0x5649188b8b70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.431+0000 7f9925163b80 1 --2- 172.21.15.57:0/1972673497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188b90b0 0x5649188c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.431+0000 7f9925163b80 1 -- 172.21.15.57:0/1972673497 >> 172.21.15.57:0/1972673497 conn(0x5649188b97f0 msgr2=0x5649188bd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.466+0000 7f9925163b80 1 -- 172.21.15.57:0/1972673497 shutdown_connections 2023-10-08T13:57:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.466+0000 7f9925163b80 1 -- 172.21.15.57:0/1972673497 wait complete. 2023-10-08T13:57:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.466+0000 7f9925163b80 1 -- start start 2023-10-08T13:57:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.467+0000 7f9925163b80 1 Processor -- start 2023-10-08T13:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.467+0000 7f9925163b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 0x56491891aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.467+0000 7f9925163b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188b90b0 0x56491891af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.467+0000 7f9925163b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 0x56491891b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.467+0000 7f9925163b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x564918925ac0 con 0x5649188b90b0 2023-10-08T13:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.467+0000 7f9925163b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564918925c00 con 0x56491882d200 2023-10-08T13:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.467+0000 7f9925163b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x564918925d40 con 0x5649188be880 2023-10-08T13:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.467+0000 7f9925163b80 10 client.0 _ll_register_callbacks cb 0x7f9908011eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.468+0000 7f991259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 0x56491891aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.468+0000 7f991259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 0x56491891aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60182/0 (socket says 172.21.15.57:60182) 2023-10-08T13:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.468+0000 7f991259c700 1 -- 172.21.15.57:0/3811949408 learned_addr learned my addr 172.21.15.57:0/3811949408 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.469+0000 7f9912d9d700 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 0x56491891b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.479+0000 7f9912d9d700 1 -- 172.21.15.57:0/3811949408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 msgr2=0x56491891aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.479+0000 7f9912d9d700 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 0x56491891aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.479+0000 7f9912d9d700 1 -- 172.21.15.57:0/3811949408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188b90b0 msgr2=0x56491891af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.479+0000 7f9912d9d700 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188b90b0 0x56491891af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.479+0000 7f9912d9d700 1 -- 172.21.15.57:0/3811949408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99141160b0 con 0x5649188be880 2023-10-08T13:57:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.479+0000 7f9912d9d700 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 0x56491891b4c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9908013010 tx=0x7f99080057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11b719955105398a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.481+0000 7f98f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:57:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.481+0000 7f98f9ffb700 1 -- 172.21.15.57:0/3811949408 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990801d4d0 con 0x5649188be880 2023-10-08T13:57:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.481+0000 7f9925163b80 1 -- 172.21.15.57:0/3811949408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f990c010040 con 0x5649188be880 2023-10-08T13:57:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.491+0000 7f98f9ffb700 1 -- 172.21.15.57:0/3811949408 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9908023040 con 0x5649188be880 2023-10-08T13:57:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.491+0000 7f98f9ffb700 1 -- 172.21.15.57:0/3811949408 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9908031630 con 0x5649188be880 2023-10-08T13:57:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.492+0000 7f9925163b80 10 client.26413 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.492+0000 7f9925163b80 1 -- 172.21.15.57:0/3811949408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56491865d1f0 con 0x5649188be880 2023-10-08T13:57:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.498+0000 7f98f9ffb700 1 -- 172.21.15.57:0/3811949408 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990c010040 con 0x5649188be880 2023-10-08T13:57:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.498+0000 7f98f9ffb700 1 -- 172.21.15.57:0/3811949408 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f990801d6c0 con 0x5649188be880 2023-10-08T13:57:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.498+0000 7f9925163b80 10 client.26413 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.498+0000 7f9925163b80 10 client.26413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.498+0000 7f9925163b80 20 client.26413 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.498+0000 7f9925163b80 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5649188bf1a0 0x564918929540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.498+0000 7f9925163b80 4 client.26413 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:57:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.498+0000 7f9925163b80 1 -- 172.21.15.57:0/3811949408 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564918804ad0 con 0x5649188bf1a0 2023-10-08T13:57:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.500+0000 7f991259c700 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5649188bf1a0 0x564918929540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.523+0000 7f991259c700 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5649188bf1a0 0x564918929540 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85950873e0a0c14 server_cookie=6f154c4f02363606 in_seq=0 out_seq=0 2023-10-08T13:57:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:24.524+0000 7f98f9ffb700 10 client.26413 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:25.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:24 smithi099 ceph-mon[103849]: pgmap v475: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 161 op/s 2023-10-08T13:57:25.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:24 smithi115 ceph-mon[104408]: pgmap v475: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 161 op/s 2023-10-08T13:57:25.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:24 smithi057 ceph-mon[99476]: pgmap v475: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 161 op/s 2023-10-08T13:57:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.779+0000 7f98f9ffb700 1 -- 172.21.15.57:0/3811949408 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564918804ad0 con 0x5649188bf1a0 2023-10-08T13:57:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.779+0000 7f98f9ffb700 10 client.26413 handle_command_reply: tid=0 2023-10-08T13:57:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.787+0000 7f9925163b80 1 client.26413 shutdown 2023-10-08T13:57:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.787+0000 7f9925163b80 1 -- 172.21.15.57:0/3811949408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 msgr2=0x56491891b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.787+0000 7f9925163b80 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 0x56491891b4c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9908013010 tx=0x7f99080057a0 comp rx=0 tx=0).stop 2023-10-08T13:57:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.788+0000 7f9925163b80 1 -- 172.21.15.57:0/3811949408 shutdown_connections 2023-10-08T13:57:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.788+0000 7f9925163b80 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5649188bf1a0 0x564918929540 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.788+0000 7f9925163b80 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5649188be880 0x56491891b4c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.788+0000 7f9925163b80 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56491882d200 0x56491891aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.788+0000 7f9925163b80 1 --2- 172.21.15.57:0/3811949408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5649188b90b0 0x56491891af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.788+0000 7f9925163b80 1 -- 172.21.15.57:0/3811949408 >> 172.21.15.57:0/3811949408 conn(0x5649188b97f0 msgr2=0x5649188bdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.788+0000 7f98f9ffb700 0 client.26413 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.788+0000 7f9925163b80 1 -- 172.21.15.57:0/3811949408 shutdown_connections 2023-10-08T13:57:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.788+0000 7f9925163b80 1 -- 172.21.15.57:0/3811949408 wait complete. 2023-10-08T13:57:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.788+0000 7f9925163b80 20 client.26413 trim_cache size 0 max 16384 2023-10-08T13:57:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:57:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:57:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:57:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.854+0000 7f9918f7a700 1 -- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98fc0a1800 msgr2=0x7f98fc0a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.854+0000 7f9918f7a700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98fc0a1800 0x7f98fc0a3cc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9914069db0 tx=0x7f9908009850 comp rx=0 tx=0).stop 2023-10-08T13:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.854+0000 7f9918f7a700 1 -- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991410ef10 msgr2=0x7f9914118340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.854+0000 7f9918f7a700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991410ef10 0x7f9914118340 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9904000c00 tx=0x7f9904005740 comp rx=0 tx=0).stop 2023-10-08T13:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.854+0000 7f9918f7a700 1 -- 172.21.15.57:0/896912659 shutdown_connections 2023-10-08T13:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.854+0000 7f9918f7a700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9914119fd0 0x7f9914111470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.854+0000 7f9918f7a700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f991410ef10 0x7f9914118340 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.854+0000 7f9918f7a700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98fc0a1800 0x7f98fc0a3cc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.854+0000 7f9918f7a700 1 --2- 172.21.15.57:0/896912659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f991410f380 0x7f99141188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.854+0000 7f9918f7a700 1 -- 172.21.15.57:0/896912659 >> 172.21.15.57:0/896912659 conn(0x7f991406b7c0 msgr2=0x7f991406f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.859+0000 7f9918f7a700 1 -- 172.21.15.57:0/896912659 shutdown_connections 2023-10-08T13:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:25.859+0000 7f9918f7a700 1 -- 172.21.15.57:0/896912659 wait complete. 2023-10-08T13:57:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:26 smithi115 ceph-mon[104408]: pgmap v476: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 867 KiB/s wr, 114 op/s 2023-10-08T13:57:27.003 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:26 smithi057 ceph-mon[99476]: pgmap v476: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 867 KiB/s wr, 114 op/s 2023-10-08T13:57:27.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:26 smithi099 ceph-mon[103849]: pgmap v476: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 867 KiB/s wr, 114 op/s 2023-10-08T13:57:29.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:29 smithi115 ceph-mon[104408]: pgmap v477: 129 pgs: 129 active+clean; 1.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 140 op/s 2023-10-08T13:57:29.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:29 smithi057 ceph-mon[99476]: pgmap v477: 129 pgs: 129 active+clean; 1.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 140 op/s 2023-10-08T13:57:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:29 smithi099 ceph-mon[103849]: pgmap v477: 129 pgs: 129 active+clean; 1.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 140 op/s 2023-10-08T13:57:29.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.638+0000 7f83105e8700 1 -- 172.21.15.57:0/3636442660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82f0005180 con 0x7f83081083a0 2023-10-08T13:57:29.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.649+0000 7f82ff7fe700 1 -- 172.21.15.57:0/3636442660 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+11395 (secure 0 0 0) 0x7f82f8015020 con 0x7f83081083a0 2023-10-08T13:57:29.663 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T13:57:29.663 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T13:57:09.249648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":112,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T13:57:29.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.702+0000 7f83105e8700 1 -- 172.21.15.57:0/3636442660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82f40a1730 msgr2=0x7f82f40a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:29.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.702+0000 7f83105e8700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82f40a1730 0x7f82f40a3bf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8304009730 tx=0x7f8304006cb0 comp rx=0 tx=0).stop 2023-10-08T13:57:29.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.702+0000 7f83105e8700 1 -- 172.21.15.57:0/3636442660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83081083a0 msgr2=0x7f830811bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:29.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.702+0000 7f83105e8700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83081083a0 0x7f830811bf10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82f800b550 tx=0x7f82f800b910 comp rx=0 tx=0).stop 2023-10-08T13:57:29.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.706+0000 7f83105e8700 1 -- 172.21.15.57:0/3636442660 shutdown_connections 2023-10-08T13:57:29.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.706+0000 7f83105e8700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f830810edf0 0x7f8308115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.706+0000 7f83105e8700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82f40a1730 0x7f82f40a3bf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.706+0000 7f83105e8700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83081077f0 0x7f830811b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.706+0000 7f83105e8700 1 --2- 172.21.15.57:0/3636442660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83081083a0 0x7f830811bf10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.706+0000 7f83105e8700 1 -- 172.21.15.57:0/3636442660 >> 172.21.15.57:0/3636442660 conn(0x7f830806b760 msgr2=0x7f830806f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:29.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.706+0000 7f83105e8700 1 -- 172.21.15.57:0/3636442660 shutdown_connections 2023-10-08T13:57:29.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:29.706+0000 7f83105e8700 1 -- 172.21.15.57:0/3636442660 wait complete. 2023-10-08T13:57:29.728 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 42 2023-10-08T13:57:29.765 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-08T13:57:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.245+0000 7fa6f1522700 1 -- 172.21.15.57:0/700187080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6ec100740 msgr2=0x7fa6ec100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.245+0000 7fa6f1522700 1 --2- 172.21.15.57:0/700187080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6ec100740 0x7fa6ec100bc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009990 tx=0x7fa6d4009ca0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 -- 172.21.15.57:0/700187080 shutdown_connections 2023-10-08T13:57:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 --2- 172.21.15.57:0/700187080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec101100 0x7fa6ec10d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 --2- 172.21.15.57:0/700187080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106750 0x7fa6ec106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 --2- 172.21.15.57:0/700187080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6ec100740 0x7fa6ec100bc0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 -- 172.21.15.57:0/700187080 >> 172.21.15.57:0/700187080 conn(0x7fa6ec0fc230 msgr2=0x7fa6ec0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 -- 172.21.15.57:0/700187080 shutdown_connections 2023-10-08T13:57:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 -- 172.21.15.57:0/700187080 wait complete. 2023-10-08T13:57:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 Processor -- start 2023-10-08T13:57:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 -- start start 2023-10-08T13:57:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec100740 0x7fa6ec19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6ec101100 0x7fa6ec19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106750 0x7fa6ec1969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec197000 con 0x7fa6ec101100 2023-10-08T13:57:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec197170 con 0x7fa6ec106750 2023-10-08T13:57:30.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.246+0000 7fa6f1522700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec197330 con 0x7fa6ec100740 2023-10-08T13:57:30.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6ec101100 0x7fa6ec19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6eb7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106750 0x7fa6ec1969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6eaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec100740 0x7fa6ec19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6ec101100 0x7fa6ec19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58772/0 (socket says 172.21.15.57:58772) 2023-10-08T13:57:30.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6e3fff700 1 -- 172.21.15.57:0/767361027 learned_addr learned my addr 172.21.15.57:0/767361027 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:30.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6eb7fe700 1 -- 172.21.15.57:0/767361027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec100740 msgr2=0x7fa6ec19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6eb7fe700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec100740 0x7fa6ec19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6eb7fe700 1 -- 172.21.15.57:0/767361027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6ec101100 msgr2=0x7fa6ec19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6eb7fe700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6ec101100 0x7fa6ec19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6eb7fe700 1 -- 172.21.15.57:0/767361027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc009520 con 0x7fa6ec106750 2023-10-08T13:57:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6eb7fe700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106750 0x7fa6ec1969a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400efc0 tx=0x7fa6e400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6e8ff9700 1 -- 172.21.15.57:0/767361027 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4018070 con 0x7fa6ec106750 2023-10-08T13:57:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6e8ff9700 1 -- 172.21.15.57:0/767361027 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6e400f040 con 0x7fa6ec106750 2023-10-08T13:57:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6f1522700 1 -- 172.21.15.57:0/767361027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d40095f0 con 0x7fa6ec106750 2023-10-08T13:57:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.247+0000 7fa6e8ff9700 1 -- 172.21.15.57:0/767361027 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400cd40 con 0x7fa6ec106750 2023-10-08T13:57:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.248+0000 7fa6f1522700 1 -- 172.21.15.57:0/767361027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec1a3680 con 0x7fa6ec106750 2023-10-08T13:57:30.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.251+0000 7fa6e8ff9700 1 -- 172.21.15.57:0/767361027 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa6e4003a00 con 0x7fa6ec106750 2023-10-08T13:57:30.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.251+0000 7fa6e8ff9700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa6d80a15f0 0x7fa6d80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.251+0000 7fa6e8ff9700 1 -- 172.21.15.57:0/767361027 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa6e40ccff0 con 0x7fa6ec106750 2023-10-08T13:57:30.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.251+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa6d80a15f0 0x7fa6d80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.252+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa6d80a15f0 0x7fa6d80a3ab0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009d70 tx=0x7fa6dc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.252+0000 7fa6fd70bb80 1 Processor -- start 2023-10-08T13:57:30.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.252+0000 7fa6fd70bb80 1 -- start start 2023-10-08T13:57:30.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf67260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf677a0 0x55a8daf72110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf72650 0x55a8daf74a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8daed0460 con 0x55a8daf677a0 2023-10-08T13:57:30.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8daec1c20 con 0x55a8daf72650 2023-10-08T13:57:30.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8daebf540 con 0x55a8daf66e80 2023-10-08T13:57:30.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6eaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf67260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf677a0 0x55a8daf72110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6eaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf67260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59590/0 (socket says 172.21.15.57:59590) 2023-10-08T13:57:30.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf677a0 0x55a8daf72110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58780/0 (socket says 172.21.15.57:58780) 2023-10-08T13:57:30.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6eaffd700 1 -- 172.21.15.57:0/2995335439 learned_addr learned my addr 172.21.15.57:0/2995335439 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:30.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6eb7fe700 1 --2- 172.21.15.57:0/2995335439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf72650 0x55a8daf74a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6eaffd700 1 -- 172.21.15.57:0/2995335439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf72650 msgr2=0x55a8daf74a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/2995335439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf72650 0x55a8daf74a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6eaffd700 1 -- 172.21.15.57:0/2995335439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf677a0 msgr2=0x55a8daf72110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.253+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/2995335439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf677a0 0x55a8daf72110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.254+0000 7fa6eaffd700 1 -- 172.21.15.57:0/2995335439 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8dace5de0 con 0x55a8daf66e80 2023-10-08T13:57:30.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.254+0000 7fa6eb7fe700 1 --2- 172.21.15.57:0/2995335439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf72650 0x55a8daf74a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:30.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.254+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/2995335439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf67260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00e620 tx=0x7fa6dc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de7334e54185d4b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.254+0000 7fa6e17fa700 1 -- 172.21.15.57:0/2995335439 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc00ee00 con 0x55a8daf66e80 2023-10-08T13:57:30.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.254+0000 7fa6e17fa700 1 -- 172.21.15.57:0/2995335439 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6dc01e8e0 con 0x55a8daf66e80 2023-10-08T13:57:30.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.254+0000 7fa6e17fa700 1 -- 172.21.15.57:0/2995335439 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc026800 con 0x55a8daf66e80 2023-10-08T13:57:30.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/2995335439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 msgr2=0x55a8daf67260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/2995335439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf67260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00e620 tx=0x7fa6dc014000 comp rx=0 tx=0).stop 2023-10-08T13:57:30.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/2995335439 shutdown_connections 2023-10-08T13:57:30.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/2995335439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf72650 0x55a8daf74a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/2995335439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf67260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/2995335439 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf677a0 0x55a8daf72110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/2995335439 >> 172.21.15.57:0/2995335439 conn(0x55a8daedc260 msgr2=0x55a8daf632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/2995335439 shutdown_connections 2023-10-08T13:57:30.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/2995335439 wait complete. 2023-10-08T13:57:30.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 -- start start 2023-10-08T13:57:30.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.255+0000 7fa6fd70bb80 1 Processor -- start 2023-10-08T13:57:30.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf72060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf6f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 0x55a8daf6f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8daebae30 con 0x55a8daf72650 2023-10-08T13:57:30.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dae57710 con 0x55a8daf677a0 2023-10-08T13:57:30.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dae573b0 con 0x55a8daf66e80 2023-10-08T13:57:30.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6fd70bb80 10 client.0 _ll_register_callbacks cb 0x55a8dacb4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:30.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6eaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf72060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6e3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf6f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6eb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 0x55a8daf6f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6e3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf6f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60212/0 (socket says 172.21.15.57:60212) 2023-10-08T13:57:30.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6eaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf72060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59596/0 (socket says 172.21.15.57:59596) 2023-10-08T13:57:30.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6e3fff700 1 -- 172.21.15.57:0/860136844 learned_addr learned my addr 172.21.15.57:0/860136844 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:30.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6e3fff700 1 -- 172.21.15.57:0/860136844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 msgr2=0x55a8daf72060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6e3fff700 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf72060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6e3fff700 1 -- 172.21.15.57:0/860136844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 msgr2=0x55a8daf6f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6e3fff700 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 0x55a8daf6f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6e3fff700 1 -- 172.21.15.57:0/860136844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec1a3680 con 0x55a8daf677a0 2023-10-08T13:57:30.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6e3fff700 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf6f350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4000c00 tx=0x7fa6d4005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2528d77d2413ea8c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.256+0000 7fa6d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:57:30.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6d27fc700 1 -- 172.21.15.57:0/860136844 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d401d030 con 0x55a8daf677a0 2023-10-08T13:57:30.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/860136844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8dae82eb0 con 0x55a8daf677a0 2023-10-08T13:57:30.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6d27fc700 1 -- 172.21.15.57:0/860136844 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa6d4004500 con 0x55a8daf677a0 2023-10-08T13:57:30.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6d27fc700 1 -- 172.21.15.57:0/860136844 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6d40046c0 con 0x55a8daf677a0 2023-10-08T13:57:30.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6fd70bb80 10 client.26372 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:30.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/860136844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fa6dc017070 con 0x55a8daf677a0 2023-10-08T13:57:30.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6d27fc700 1 -- 172.21.15.57:0/860136844 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8dae82eb0 con 0x55a8daf677a0 2023-10-08T13:57:30.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6d27fc700 1 -- 172.21.15.57:0/860136844 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fa6d400f9f0 con 0x55a8daf677a0 2023-10-08T13:57:30.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6fd70bb80 10 client.26372 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:30.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6fd70bb80 10 client.26372 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-08T13:57:30.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6fd70bb80 20 client.26372 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:30.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55a8daf82360 0x55a8daf84780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6fd70bb80 4 client.26372 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:57:30.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.257+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/860136844 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8daeb3e10 con 0x55a8daf82360 2023-10-08T13:57:30.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.258+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55a8daf82360 0x55a8daf84780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.258+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55a8daf82360 0x55a8daf84780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0e20e876eca9e5c server_cookie=3b0ddfe4ab95a712 in_seq=0 out_seq=0 2023-10-08T13:57:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.258+0000 7fa6d27fc700 10 client.26372 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:57:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6d27fc700 1 -- 172.21.15.57:0/860136844 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a8daeb3e10 con 0x55a8daf82360 2023-10-08T13:57:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6d27fc700 10 client.26372 handle_command_reply: tid=0 2023-10-08T13:57:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 client.26372 shutdown 2023-10-08T13:57:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/860136844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 msgr2=0x55a8daf6f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf6f350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4000c00 tx=0x7fa6d4005c20 comp rx=0 tx=0).stop 2023-10-08T13:57:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/860136844 shutdown_connections 2023-10-08T13:57:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55a8daf82360 0x55a8daf84780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 0x55a8daf6f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf66e80 0x55a8daf72060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6d27fc700 0 client.26372 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:57:30.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/860136844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf6f350 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/860136844 >> 172.21.15.57:0/860136844 conn(0x55a8daedc260 msgr2=0x55a8daf62ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/860136844 shutdown_connections 2023-10-08T13:57:30.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/860136844 wait complete. 2023-10-08T13:57:30.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.259+0000 7fa6fd70bb80 20 client.26372 trim_cache size 0 max 16384 2023-10-08T13:57:30.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.274+0000 7fa6fd70bb80 1 Processor -- start 2023-10-08T13:57:30.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.274+0000 7fa6fd70bb80 1 -- start start 2023-10-08T13:57:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf66e80 0x55a8daf84750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf69d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf72650 0x55a8daf791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dad3b920 con 0x55a8daf66e80 2023-10-08T13:57:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8daf7ece0 con 0x55a8daf677a0 2023-10-08T13:57:30.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8daf689c0 con 0x55a8daf72650 2023-10-08T13:57:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6e3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf69d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6eaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf66e80 0x55a8daf84750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6eaffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf66e80 0x55a8daf84750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58794/0 (socket says 172.21.15.57:58794) 2023-10-08T13:57:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6eaffd700 1 -- 172.21.15.57:0/3386121026 learned_addr learned my addr 172.21.15.57:0/3386121026 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6eb7fe700 1 --2- 172.21.15.57:0/3386121026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf72650 0x55a8daf791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6e3fff700 1 -- 172.21.15.57:0/3386121026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf72650 msgr2=0x55a8daf791a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6e3fff700 1 --2- 172.21.15.57:0/3386121026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf72650 0x55a8daf791a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6e3fff700 1 -- 172.21.15.57:0/3386121026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf66e80 msgr2=0x55a8daf84750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6e3fff700 1 --2- 172.21.15.57:0/3386121026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf66e80 0x55a8daf84750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6e3fff700 1 -- 172.21.15.57:0/3386121026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8daccec60 con 0x55a8daf677a0 2023-10-08T13:57:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/3386121026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf66e80 0x55a8daf84750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.275+0000 7fa6eb7fe700 1 --2- 172.21.15.57:0/3386121026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf72650 0x55a8daf791a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.276+0000 7fa6e3fff700 1 --2- 172.21.15.57:0/3386121026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf69d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400f020 tx=0x7fa6d4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8092fc28642efa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.276+0000 7fa6e1ffb700 1 -- 172.21.15.57:0/3386121026 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d403a030 con 0x55a8daf677a0 2023-10-08T13:57:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.276+0000 7fa6e1ffb700 1 -- 172.21.15.57:0/3386121026 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6d401e040 con 0x55a8daf677a0 2023-10-08T13:57:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.276+0000 7fa6e1ffb700 1 -- 172.21.15.57:0/3386121026 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d401d070 con 0x55a8daf677a0 2023-10-08T13:57:30.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.277+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/3386121026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 msgr2=0x55a8daf69d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.277+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/3386121026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf69d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400f020 tx=0x7fa6d4009ca0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.279+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/3386121026 shutdown_connections 2023-10-08T13:57:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.279+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/3386121026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf72650 0x55a8daf791a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.279+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/3386121026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf66e80 0x55a8daf84750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.279+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/3386121026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf677a0 0x55a8daf69d50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.279+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/3386121026 >> 172.21.15.57:0/3386121026 conn(0x55a8daedc260 msgr2=0x55a8daf624d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.279+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/3386121026 shutdown_connections 2023-10-08T13:57:30.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.279+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/3386121026 wait complete. 2023-10-08T13:57:30.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.280+0000 7fa6fd70bb80 1 -- start start 2023-10-08T13:57:30.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.280+0000 7fa6fd70bb80 1 Processor -- start 2023-10-08T13:57:30.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.280+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf66e80 0x55a8daf790f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf677a0 0x55a8daf76780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6fd70bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 0x55a8daf76cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8daf77200 con 0x55a8daf72650 2023-10-08T13:57:30.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8daf77370 con 0x55a8daf66e80 2023-10-08T13:57:30.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6fd70bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8daf77530 con 0x55a8daf677a0 2023-10-08T13:57:30.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6fd70bb80 10 client.0 _ll_register_callbacks cb 0x7fa6d40053c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:30.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6eb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 0x55a8daf76cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6eb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 0x55a8daf76cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58798/0 (socket says 172.21.15.57:58798) 2023-10-08T13:57:30.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6eb7fe700 1 -- 172.21.15.57:0/1760260554 learned_addr learned my addr 172.21.15.57:0/1760260554 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:30.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf66e80 0x55a8daf790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6e3fff700 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf677a0 0x55a8daf76780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6eaffd700 1 -- 172.21.15.57:0/1760260554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf677a0 msgr2=0x55a8daf76780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf677a0 0x55a8daf76780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6eaffd700 1 -- 172.21.15.57:0/1760260554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 msgr2=0x55a8daf76cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 0x55a8daf76cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6eaffd700 1 -- 172.21.15.57:0/1760260554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8dace5de0 con 0x55a8daf66e80 2023-10-08T13:57:30.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.281+0000 7fa6eb7fe700 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 0x55a8daf76cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:30.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.284+0000 7fa6eaffd700 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf66e80 0x55a8daf790f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009fa0 tx=0x7fa6dc009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95bfd8c5f5646062 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.284+0000 7fa6d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:57:30.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.284+0000 7fa6d27fc700 1 -- 172.21.15.57:0/1760260554 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc02b070 con 0x55a8daf66e80 2023-10-08T13:57:30.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.284+0000 7fa6d27fc700 1 -- 172.21.15.57:0/1760260554 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fa6dc01ea50 con 0x55a8daf66e80 2023-10-08T13:57:30.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.284+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/1760260554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6dc01e8e0 con 0x55a8daf66e80 2023-10-08T13:57:30.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.284+0000 7fa6d27fc700 1 -- 172.21.15.57:0/1760260554 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6dc01ec50 con 0x55a8daf66e80 2023-10-08T13:57:30.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.284+0000 7fa6fd70bb80 10 client.26378 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:30.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.284+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/1760260554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fa6d40095f0 con 0x55a8daf66e80 2023-10-08T13:57:30.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.285+0000 7fa6d27fc700 1 -- 172.21.15.57:0/1760260554 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6dc01e8e0 con 0x55a8daf66e80 2023-10-08T13:57:30.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.285+0000 7fa6d27fc700 1 -- 172.21.15.57:0/1760260554 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fa6dc024020 con 0x55a8daf66e80 2023-10-08T13:57:30.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.285+0000 7fa6fd70bb80 10 client.26378 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:30.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.285+0000 7fa6fd70bb80 10 client.26378 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2023-10-08T13:57:30.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.285+0000 7fa6fd70bb80 20 client.26378 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:30.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.285+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55a8dafd09d0 0x55a8dafd2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.285+0000 7fa6fd70bb80 4 client.26378 mds_command: new command op to 24617 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:57:30.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.285+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/1760260554 --> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a8daeb3b30 con 0x55a8dafd09d0 2023-10-08T13:57:30.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.285+0000 7fa6e3fff700 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55a8dafd09d0 0x55a8dafd2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6e3fff700 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55a8dafd09d0 0x55a8dafd2df0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20d7e8427a438e10 server_cookie=5b3a7774ca08caff in_seq=0 out_seq=0 2023-10-08T13:57:30.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6d27fc700 10 client.26378 ms_handle_connect on v2:172.21.15.99:6836/822922696 2023-10-08T13:57:30.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6d27fc700 1 -- 172.21.15.57:0/1760260554 <== mds.4 v2:172.21.15.99:6836/822922696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a8daeb3b30 con 0x55a8dafd09d0 2023-10-08T13:57:30.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6d27fc700 10 client.26378 handle_command_reply: tid=0 2023-10-08T13:57:30.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 client.26378 shutdown 2023-10-08T13:57:30.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/1760260554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf66e80 msgr2=0x55a8daf790f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf66e80 0x55a8daf790f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009fa0 tx=0x7fa6dc009ab0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/1760260554 shutdown_connections 2023-10-08T13:57:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.99:6836/822922696,v1:172.21.15.99:6837/822922696] conn(0x55a8dafd09d0 0x55a8dafd2df0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a8daf72650 0x55a8daf76cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8daf66e80 0x55a8daf790f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6d27fc700 0 client.26378 ms_handle_reset on v2:172.21.15.99:6836/822922696 2023-10-08T13:57:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 --2- 172.21.15.57:0/1760260554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a8daf677a0 0x55a8daf76780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/1760260554 >> 172.21.15.57:0/1760260554 conn(0x55a8daedc260 msgr2=0x55a8daf6a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/1760260554 shutdown_connections 2023-10-08T13:57:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 1 -- 172.21.15.57:0/1760260554 wait complete. 2023-10-08T13:57:30.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.286+0000 7fa6fd70bb80 20 client.26378 trim_cache size 0 max 16384 2023-10-08T13:57:30.294 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:57:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 -- 172.21.15.57:0/767361027 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa6d80a15f0 msgr2=0x7fa6d80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa6d80a15f0 0x7fa6d80a3ab0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009d70 tx=0x7fa6dc009260 comp rx=0 tx=0).stop 2023-10-08T13:57:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 -- 172.21.15.57:0/767361027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106750 msgr2=0x7fa6ec1969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106750 0x7fa6ec1969a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400efc0 tx=0x7fa6e400c3a0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 -- 172.21.15.57:0/767361027 shutdown_connections 2023-10-08T13:57:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa6d80a15f0 0x7fa6d80a3ab0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106750 0x7fa6ec1969a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec100740 0x7fa6ec19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 --2- 172.21.15.57:0/767361027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6ec101100 0x7fa6ec19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 -- 172.21.15.57:0/767361027 >> 172.21.15.57:0/767361027 conn(0x7fa6ec0fc230 msgr2=0x7fa6ec0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 -- 172.21.15.57:0/767361027 shutdown_connections 2023-10-08T13:57:30.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.287+0000 7fa6f1522700 1 -- 172.21.15.57:0/767361027 wait complete. 2023-10-08T13:57:30.304 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-08T13:57:30.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:30 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3636442660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:57:30.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.480+0000 7fbffb7cd700 1 -- 172.21.15.57:0/1803375621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff4106770 msgr2=0x7fbff4106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.480+0000 7fbffb7cd700 1 --2- 172.21.15.57:0/1803375621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff4106770 0x7fbff4106b50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbff00098f0 tx=0x7fbff0009c00 comp rx=0 tx=0).stop 2023-10-08T13:57:30.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.480+0000 7fbffb7cd700 1 -- 172.21.15.57:0/1803375621 shutdown_connections 2023-10-08T13:57:30.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.480+0000 7fbffb7cd700 1 --2- 172.21.15.57:0/1803375621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff40696d0 0x7fbff410d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.480+0000 7fbffb7cd700 1 --2- 172.21.15.57:0/1803375621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff4106770 0x7fbff4106b50 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.480+0000 7fbffb7cd700 1 --2- 172.21.15.57:0/1803375621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbff4068d10 0x7fbff4069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.480+0000 7fbffb7cd700 1 -- 172.21.15.57:0/1803375621 >> 172.21.15.57:0/1803375621 conn(0x7fbff4076790 msgr2=0x7fbff4076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.481+0000 7fbffb7cd700 1 -- 172.21.15.57:0/1803375621 shutdown_connections 2023-10-08T13:57:30.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.481+0000 7fbffb7cd700 1 -- 172.21.15.57:0/1803375621 wait complete. 2023-10-08T13:57:30.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.481+0000 7fbffb7cd700 1 Processor -- start 2023-10-08T13:57:30.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.481+0000 7fbffb7cd700 1 -- start start 2023-10-08T13:57:30.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbffb7cd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbff4068d10 0x7fbff419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbffb7cd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff40696d0 0x7fbff419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbffb7cd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff4106770 0x7fbff4196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbffb7cd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff4197170 con 0x7fbff40696d0 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbffb7cd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff41972e0 con 0x7fbff4106770 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbffb7cd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff41974a0 con 0x7fbff4068d10 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff8d68700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff40696d0 0x7fbff419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff8d68700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff40696d0 0x7fbff419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58814/0 (socket says 172.21.15.57:58814) 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff8d68700 1 -- 172.21.15.57:0/3049473341 learned_addr learned my addr 172.21.15.57:0/3049473341 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff9569700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbff4068d10 0x7fbff419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff9d6a700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff4106770 0x7fbff4196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff8d68700 1 -- 172.21.15.57:0/3049473341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbff4068d10 msgr2=0x7fbff419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff8d68700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbff4068d10 0x7fbff419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff8d68700 1 -- 172.21.15.57:0/3049473341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff4106770 msgr2=0x7fbff4196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff8d68700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff4106770 0x7fbff4196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff8d68700 1 -- 172.21.15.57:0/3049473341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0009520 con 0x7fbff40696d0 2023-10-08T13:57:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.482+0000 7fbff8d68700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff40696d0 0x7fbff419cf20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400b550 tx=0x7fbfe400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.483+0000 7fbfea7fc700 1 -- 172.21.15.57:0/3049473341 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4011830 con 0x7fbff40696d0 2023-10-08T13:57:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.483+0000 7fbfea7fc700 1 -- 172.21.15.57:0/3049473341 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfe4011e50 con 0x7fbff40696d0 2023-10-08T13:57:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.483+0000 7fbffb7cd700 1 -- 172.21.15.57:0/3049473341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff4197780 con 0x7fbff40696d0 2023-10-08T13:57:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.483+0000 7fbfea7fc700 1 -- 172.21.15.57:0/3049473341 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4010dc0 con 0x7fbff40696d0 2023-10-08T13:57:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.483+0000 7fbffb7cd700 1 -- 172.21.15.57:0/3049473341 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff40ff5e0 con 0x7fbff40696d0 2023-10-08T13:57:30.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:30 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3636442660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:57:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fc0079b6b80 1 Processor -- start 2023-10-08T13:57:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fc0079b6b80 1 -- start start 2023-10-08T13:57:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4e7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 0x55872f4f2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4f2650 0x55872f4f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f450460 con 0x55872f4e77a0 2023-10-08T13:57:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f441c20 con 0x55872f4f2650 2023-10-08T13:57:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f43f540 con 0x55872f4e6e80 2023-10-08T13:57:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fbfea7fc700 1 -- 172.21.15.57:0/3049473341 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbfe4010380 con 0x7fbff40696d0 2023-10-08T13:57:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fbff9d6a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4e7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fbff9d6a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4e7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59630/0 (socket says 172.21.15.57:59630) 2023-10-08T13:57:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fbff9d6a700 1 -- 172.21.15.57:0/3099286384 learned_addr learned my addr 172.21.15.57:0/3099286384 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.487+0000 7fbff9569700 1 --2- 172.21.15.57:0/3099286384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 0x55872f4f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbff8d68700 1 --2- 172.21.15.57:0/3099286384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4f2650 0x55872f4f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbff9d6a700 1 -- 172.21.15.57:0/3099286384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4f2650 msgr2=0x55872f4f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbff9d6a700 1 --2- 172.21.15.57:0/3099286384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4f2650 0x55872f4f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbff9d6a700 1 -- 172.21.15.57:0/3099286384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 msgr2=0x55872f4f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbff9d6a700 1 --2- 172.21.15.57:0/3099286384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 0x55872f4f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbff9d6a700 1 -- 172.21.15.57:0/3099286384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55872f265de0 con 0x55872f4e6e80 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbff9d6a700 1 --2- 172.21.15.57:0/3099286384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4e7260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbfec003f80 tx=0x7fbfec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d691e56dae970862 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbfea7fc700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbfe00a15f0 0x7fbfe00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbfdf7fe700 1 -- 172.21.15.57:0/3099286384 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec007bd0 con 0x55872f4e6e80 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbff9569700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbfe00a15f0 0x7fbfe00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbfea7fc700 1 -- 172.21.15.57:0/3049473341 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fbfe40ccd20 con 0x7fbff40696d0 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbfdf7fe700 1 -- 172.21.15.57:0/3099286384 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfec00f040 con 0x55872f4e6e80 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.488+0000 7fbfdf7fe700 1 -- 172.21.15.57:0/3099286384 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec015750 con 0x55872f4e6e80 2023-10-08T13:57:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.489+0000 7fc0079b6b80 1 -- 172.21.15.57:0/3099286384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 msgr2=0x55872f4e7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.489+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/3099286384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4e7260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbfec003f80 tx=0x7fbfec009b80 comp rx=0 tx=0).stop 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.489+0000 7fbff9569700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbfe00a15f0 0x7fbfe00a3ab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbff00098c0 tx=0x7fbff0019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.489+0000 7fc0079b6b80 1 -- 172.21.15.57:0/3099286384 shutdown_connections 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.489+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/3099286384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4f2650 0x55872f4f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.489+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/3099286384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4e7260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.489+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/3099286384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 0x55872f4f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.489+0000 7fc0079b6b80 1 -- 172.21.15.57:0/3099286384 >> 172.21.15.57:0/3099286384 conn(0x55872f45c260 msgr2=0x55872f4e32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.489+0000 7fc0079b6b80 1 -- 172.21.15.57:0/3099286384 shutdown_connections 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.489+0000 7fc0079b6b80 1 -- 172.21.15.57:0/3099286384 wait complete. 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fc0079b6b80 1 -- start start 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fc0079b6b80 1 Processor -- start 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e77a0 0x55872f4ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 0x55872f4ef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f43ae30 con 0x55872f4f2650 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f3d7710 con 0x55872f4e6e80 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f3d73b0 con 0x55872f4e77a0 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fc0079b6b80 10 client.0 _ll_register_callbacks cb 0x55872f234800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fbff9d6a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 0x55872f4ef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fbff8d68700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e77a0 0x55872f4ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fbff9d6a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 0x55872f4ef9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58830/0 (socket says 172.21.15.57:58830) 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fbff9569700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.490+0000 7fbff9d6a700 1 -- 172.21.15.57:0/1052661067 learned_addr learned my addr 172.21.15.57:0/1052661067 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbff9569700 1 -- 172.21.15.57:0/1052661067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e77a0 msgr2=0x55872f4ef3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbff9569700 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e77a0 0x55872f4ef3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbff9569700 1 -- 172.21.15.57:0/1052661067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 msgr2=0x55872f4ef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbff9569700 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 0x55872f4ef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbff9569700 1 -- 172.21.15.57:0/1052661067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec003bb0 con 0x55872f4e6e80 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbff9d6a700 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 0x55872f4ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbff8d68700 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e77a0 0x55872f4ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbff9569700 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f2060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009810 tx=0x7fbff0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=153bb90f32f6c46a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbfdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbfdcff9700 1 -- 172.21.15.57:0/1052661067 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0024840 con 0x55872f4e6e80 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbfdcff9700 1 -- 172.21.15.57:0/1052661067 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fbff0029050 con 0x55872f4e6e80 2023-10-08T13:57:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fc0079b6b80 1 -- 172.21.15.57:0/1052661067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55872f402eb0 con 0x55872f4e6e80 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fbfdcff9700 1 -- 172.21.15.57:0/1052661067 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbff000bc70 con 0x55872f4e6e80 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.491+0000 7fc0079b6b80 10 client.26387 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.492+0000 7fc0079b6b80 1 -- 172.21.15.57:0/1052661067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55872f24ec60 con 0x55872f4e6e80 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.492+0000 7fbfdcff9700 1 -- 172.21.15.57:0/1052661067 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55872f402eb0 con 0x55872f4e6e80 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.492+0000 7fbfdcff9700 1 -- 172.21.15.57:0/1052661067 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fbff002ea70 con 0x55872f4e6e80 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.492+0000 7fc0079b6b80 10 client.26387 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.492+0000 7fc0079b6b80 10 client.26387 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.492+0000 7fc0079b6b80 20 client.26387 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.492+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x55872f5020e0 0x55872f504500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.492+0000 7fc0079b6b80 4 client.26387 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.492+0000 7fc0079b6b80 1 -- 172.21.15.57:0/1052661067 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55872f433e10 con 0x55872f5020e0 2023-10-08T13:57:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.492+0000 7fbff8d68700 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x55872f5020e0 0x55872f504500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.493+0000 7fbff8d68700 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x55872f5020e0 0x55872f504500 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e01e7ab661b8392 server_cookie=f5796f1c18f87b2 in_seq=0 out_seq=0 2023-10-08T13:57:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.493+0000 7fbfdcff9700 10 client.26387 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:57:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.493+0000 7fbfdcff9700 1 -- 172.21.15.57:0/1052661067 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55872f433e10 con 0x55872f5020e0 2023-10-08T13:57:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.493+0000 7fbfdcff9700 10 client.26387 handle_command_reply: tid=0 2023-10-08T13:57:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.493+0000 7fc0079b6b80 1 client.26387 shutdown 2023-10-08T13:57:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.493+0000 7fc0079b6b80 1 -- 172.21.15.57:0/1052661067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 msgr2=0x55872f4f2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.493+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f2060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009810 tx=0x7fbff0009c00 comp rx=0 tx=0).stop 2023-10-08T13:57:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.494+0000 7fc0079b6b80 1 -- 172.21.15.57:0/1052661067 shutdown_connections 2023-10-08T13:57:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.494+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x55872f5020e0 0x55872f504500 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.494+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 0x55872f4ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.494+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f2060 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.494+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/1052661067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e77a0 0x55872f4ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.494+0000 7fbfdcff9700 0 client.26387 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:57:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.494+0000 7fc0079b6b80 1 -- 172.21.15.57:0/1052661067 >> 172.21.15.57:0/1052661067 conn(0x55872f45c260 msgr2=0x55872f4e2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.494+0000 7fc0079b6b80 1 -- 172.21.15.57:0/1052661067 shutdown_connections 2023-10-08T13:57:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.494+0000 7fc0079b6b80 1 -- 172.21.15.57:0/1052661067 wait complete. 2023-10-08T13:57:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.494+0000 7fc0079b6b80 20 client.26387 trim_cache size 0 max 16384 2023-10-08T13:57:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fc0079b6b80 1 Processor -- start 2023-10-08T13:57:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fc0079b6b80 1 -- start start 2023-10-08T13:57:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 0x55872f4ea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4f2650 0x55872f4eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f2bb920 con 0x55872f4e77a0 2023-10-08T13:57:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f4e7620 con 0x55872f4e6e80 2023-10-08T13:57:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f4dd570 con 0x55872f4f2650 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff8d68700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 0x55872f4ea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff9569700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff9d6a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4f2650 0x55872f4eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff9569700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60260/0 (socket says 172.21.15.57:60260) 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff9569700 1 -- 172.21.15.57:0/2537234354 learned_addr learned my addr 172.21.15.57:0/2537234354 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff8d68700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 0x55872f4ea6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58834/0 (socket says 172.21.15.57:58834) 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff9569700 1 -- 172.21.15.57:0/2537234354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4f2650 msgr2=0x55872f4eabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff9569700 1 --2- 172.21.15.57:0/2537234354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4f2650 0x55872f4eabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff9569700 1 -- 172.21.15.57:0/2537234354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 msgr2=0x55872f4ea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff9569700 1 --2- 172.21.15.57:0/2537234354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 0x55872f4ea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.509+0000 7fbff9569700 1 -- 172.21.15.57:0/2537234354 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55872f28c250 con 0x55872f4e6e80 2023-10-08T13:57:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.510+0000 7fbff9569700 1 --2- 172.21.15.57:0/2537234354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f91a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbff002edd0 tx=0x7fbff0010e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=247d4cb72a7a7aa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.510+0000 7fbfdffff700 1 -- 172.21.15.57:0/2537234354 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff00174d0 con 0x55872f4e6e80 2023-10-08T13:57:30.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.510+0000 7fbfdffff700 1 -- 172.21.15.57:0/2537234354 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbff001d040 con 0x55872f4e6e80 2023-10-08T13:57:30.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.510+0000 7fbfdffff700 1 -- 172.21.15.57:0/2537234354 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff002e8e0 con 0x55872f4e6e80 2023-10-08T13:57:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 -- 172.21.15.57:0/2537234354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 msgr2=0x55872f4f91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/2537234354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f91a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbff002edd0 tx=0x7fbff0010e30 comp rx=0 tx=0).stop 2023-10-08T13:57:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 -- 172.21.15.57:0/2537234354 shutdown_connections 2023-10-08T13:57:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/2537234354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4f2650 0x55872f4eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/2537234354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e6e80 0x55872f4f91a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/2537234354 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4e77a0 0x55872f4ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 -- 172.21.15.57:0/2537234354 >> 172.21.15.57:0/2537234354 conn(0x55872f45c260 msgr2=0x55872f4ee8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 -- 172.21.15.57:0/2537234354 shutdown_connections 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 -- 172.21.15.57:0/2537234354 wait complete. 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 -- start start 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.511+0000 7fc0079b6b80 1 Processor -- start 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e77a0 0x55872f4f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fc0079b6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 0x55872f554f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f4f7bc0 con 0x55872f4f2650 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f555470 con 0x55872f4e77a0 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fc0079b6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55872f555600 con 0x55872f4e6e80 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fc0079b6b80 10 client.0 _ll_register_callbacks cb 0x7fbff000f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff8d68700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e77a0 0x55872f4f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff9569700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff8d68700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e77a0 0x55872f4f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60268/0 (socket says 172.21.15.57:60268) 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff9569700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4f6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59652/0 (socket says 172.21.15.57:59652) 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff8d68700 1 -- 172.21.15.57:0/583387620 learned_addr learned my addr 172.21.15.57:0/583387620 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff9d6a700 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 0x55872f554f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff8d68700 1 -- 172.21.15.57:0/583387620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 msgr2=0x55872f4f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff8d68700 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff8d68700 1 -- 172.21.15.57:0/583387620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 msgr2=0x55872f554f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff8d68700 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 0x55872f554f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff8d68700 1 -- 172.21.15.57:0/583387620 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55872f24ec60 con 0x55872f4e77a0 2023-10-08T13:57:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.512+0000 7fbff9569700 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.513+0000 7fbff8d68700 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e77a0 0x55872f4f7490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4096580 tx=0x7fbfe4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87cb4253c3b43d78 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.513+0000 7fbfdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.513+0000 7fbfdcff9700 1 -- 172.21.15.57:0/583387620 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4099b10 con 0x55872f4e77a0 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.513+0000 7fbfdcff9700 1 -- 172.21.15.57:0/583387620 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7fbfe409ae40 con 0x55872f4e77a0 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.513+0000 7fc0079b6b80 1 -- 172.21.15.57:0/583387620 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfec00f040 con 0x55872f4e77a0 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.513+0000 7fbfdcff9700 1 -- 172.21.15.57:0/583387620 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfe409dc60 con 0x55872f4e77a0 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.513+0000 7fc0079b6b80 10 client.26393 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.513+0000 7fc0079b6b80 1 -- 172.21.15.57:0/583387620 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55872f2642b0 con 0x55872f4e77a0 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.514+0000 7fbfdcff9700 1 -- 172.21.15.57:0/583387620 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfec00f040 con 0x55872f4e77a0 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.514+0000 7fbfdcff9700 1 -- 172.21.15.57:0/583387620 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fbfe40b2170 con 0x55872f4e77a0 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.514+0000 7fc0079b6b80 10 client.26393 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.514+0000 7fc0079b6b80 10 client.26393 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-08T13:57:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.514+0000 7fc0079b6b80 20 client.26393 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.514+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x55872f556f60 0x55872f559380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.514+0000 7fc0079b6b80 4 client.26393 mds_command: new command op to 24622 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:57:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.514+0000 7fc0079b6b80 1 -- 172.21.15.57:0/583387620 --> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55872f433b30 con 0x55872f556f60 2023-10-08T13:57:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.514+0000 7fbff9569700 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x55872f556f60 0x55872f559380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.514+0000 7fbfdcff9700 10 client.26393 ms_handle_connect on v2:172.21.15.115:6834/389121345 2023-10-08T13:57:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.515+0000 7fbff9569700 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x55872f556f60 0x55872f559380 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a562111afd1c79a3 server_cookie=8d3f4044b22dfdba in_seq=0 out_seq=0 2023-10-08T13:57:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fbfdcff9700 1 -- 172.21.15.57:0/583387620 <== mds.3 v2:172.21.15.115:6834/389121345 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55872f433b30 con 0x55872f556f60 2023-10-08T13:57:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fbfdcff9700 10 client.26393 handle_command_reply: tid=0 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 client.26393 shutdown 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 -- 172.21.15.57:0/583387620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e77a0 msgr2=0x55872f4f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e77a0 0x55872f4f7490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4096580 tx=0x7fbfe4004770 comp rx=0 tx=0).stop 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 -- 172.21.15.57:0/583387620 shutdown_connections 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.115:6834/389121345,v1:172.21.15.115:6835/389121345] conn(0x55872f556f60 0x55872f559380 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55872f4f2650 0x55872f554f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55872f4e6e80 0x55872f4f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fbfdcff9700 0 client.26393 ms_handle_reset on v2:172.21.15.115:6834/389121345 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 --2- 172.21.15.57:0/583387620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55872f4e77a0 0x55872f4f7490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 -- 172.21.15.57:0/583387620 >> 172.21.15.57:0/583387620 conn(0x55872f45c260 msgr2=0x55872f4edc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 -- 172.21.15.57:0/583387620 shutdown_connections 2023-10-08T13:57:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 1 -- 172.21.15.57:0/583387620 wait complete. 2023-10-08T13:57:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.517+0000 7fc0079b6b80 20 client.26393 trim_cache size 0 max 16384 2023-10-08T13:57:30.523 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:57:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 -- 172.21.15.57:0/3049473341 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbfe00a15f0 msgr2=0x7fbfe00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbfe00a15f0 0x7fbfe00a3ab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbff00098c0 tx=0x7fbff0019040 comp rx=0 tx=0).stop 2023-10-08T13:57:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 -- 172.21.15.57:0/3049473341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff40696d0 msgr2=0x7fbff419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff40696d0 0x7fbff419cf20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400b550 tx=0x7fbfe400b860 comp rx=0 tx=0).stop 2023-10-08T13:57:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 -- 172.21.15.57:0/3049473341 shutdown_connections 2023-10-08T13:57:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbfe00a15f0 0x7fbfe00a3ab0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff4106770 0x7fbff4196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbff4068d10 0x7fbff419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 --2- 172.21.15.57:0/3049473341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbff40696d0 0x7fbff419cf20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:30.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 -- 172.21.15.57:0/3049473341 >> 172.21.15.57:0/3049473341 conn(0x7fbff4076790 msgr2=0x7fbff40fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:30.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 -- 172.21.15.57:0/3049473341 shutdown_connections 2023-10-08T13:57:30.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:30.518+0000 7fbffb7cd700 1 -- 172.21.15.57:0/3049473341 wait complete. 2023-10-08T13:57:30.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:30 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3636442660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T13:57:30.535 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-08T13:57:31.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:31 smithi057 ceph-mon[99476]: pgmap v478: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 235 op/s 2023-10-08T13:57:31.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:31 smithi099 ceph-mon[103849]: pgmap v478: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 235 op/s 2023-10-08T13:57:31.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:31 smithi115 ceph-mon[104408]: pgmap v478: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 235 op/s 2023-10-08T13:57:31.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.684+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/1514734742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c24119c80 msgr2=0x7f9c2411c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:31.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.684+0000 7f9c2aa4e700 1 --2- 172.21.15.57:0/1514734742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c24119c80 0x7f9c2411c070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009f90 tx=0x7f9c1c00b330 comp rx=0 tx=0).stop 2023-10-08T13:57:31.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.685+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/1514734742 shutdown_connections 2023-10-08T13:57:31.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.685+0000 7f9c2aa4e700 1 --2- 172.21.15.57:0/1514734742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c24119c80 0x7f9c2411c070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:31.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.685+0000 7f9c2aa4e700 1 --2- 172.21.15.57:0/1514734742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c2410c8d0 0x7f9c2410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:31.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.685+0000 7f9c2aa4e700 1 --2- 172.21.15.57:0/1514734742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2410d1f0 0x7f9c24119740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:31.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.685+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/1514734742 >> 172.21.15.57:0/1514734742 conn(0x7f9c2406b650 msgr2=0x7f9c2406ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:31.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.686+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/1514734742 shutdown_connections 2023-10-08T13:57:31.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.686+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/1514734742 wait complete. 2023-10-08T13:57:31.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.687+0000 7f9c2aa4e700 1 Processor -- start 2023-10-08T13:57:31.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.687+0000 7f9c2aa4e700 1 -- start start 2023-10-08T13:57:31.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.687+0000 7f9c2aa4e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c2410c8d0 0x7f9c24119170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:31.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.687+0000 7f9c2aa4e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2410d1f0 0x7f9c241196f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:31.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.687+0000 7f9c2aa4e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c24119c80 0x7f9c24112470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:31.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.687+0000 7f9c2aa4e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24112c40 con 0x7f9c2410c8d0 2023-10-08T13:57:31.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.687+0000 7f9c2aa4e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24112db0 con 0x7f9c2410d1f0 2023-10-08T13:57:31.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.687+0000 7f9c2aa4e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24112f70 con 0x7f9c24119c80 2023-10-08T13:57:31.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.688+0000 7f9c2a24d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c24119c80 0x7f9c24112470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:31.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.688+0000 7f9c2a24d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c24119c80 0x7f9c24112470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59668/0 (socket says 172.21.15.57:59668) 2023-10-08T13:57:31.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.688+0000 7f9c2a24d700 1 -- 172.21.15.57:0/3674283255 learned_addr learned my addr 172.21.15.57:0/3674283255 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:31.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.688+0000 7f9c2924b700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2410d1f0 0x7f9c241196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:31.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.689+0000 7f9c29a4c700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c2410c8d0 0x7f9c24119170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:31.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.690+0000 7f9c2924b700 1 -- 172.21.15.57:0/3674283255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c24119c80 msgr2=0x7f9c24112470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:31.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.690+0000 7f9c2924b700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c24119c80 0x7f9c24112470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:31.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.690+0000 7f9c2924b700 1 -- 172.21.15.57:0/3674283255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c2410c8d0 msgr2=0x7f9c24119170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:31.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.690+0000 7f9c2924b700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c2410c8d0 0x7f9c24119170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:31.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.690+0000 7f9c2924b700 1 -- 172.21.15.57:0/3674283255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c009c40 con 0x7f9c2410d1f0 2023-10-08T13:57:31.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.691+0000 7f9c2924b700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2410d1f0 0x7f9c241196f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400d950 tx=0x7f9c1400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.982+0000 7f9c1affd700 1 -- 172.21.15.57:0/3674283255 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14009930 con 0x7f9c2410d1f0 2023-10-08T13:57:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.982+0000 7f9c1affd700 1 -- 172.21.15.57:0/3674283255 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c14010460 con 0x7f9c2410d1f0 2023-10-08T13:57:31.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.982+0000 7f9c1affd700 1 -- 172.21.15.57:0/3674283255 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1400b5d0 con 0x7f9c2410d1f0 2023-10-08T13:57:31.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.986+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/3674283255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c24116d00 con 0x7f9c2410d1f0 2023-10-08T13:57:31.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.987+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/3674283255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c24117190 con 0x7f9c2410d1f0 2023-10-08T13:57:31.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.993+0000 7f9c37e99b80 1 Processor -- start 2023-10-08T13:57:32.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.996+0000 7f9c37e99b80 1 -- start start 2023-10-08T13:57:32.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.996+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169c5050 0x55e1169c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.996+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.996+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169d0800 0x55e1169d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.996+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e11692e460 con 0x55e1169c5050 2023-10-08T13:57:32.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.996+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e11691fc20 con 0x55e1169d0800 2023-10-08T13:57:32.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.996+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e11691d540 con 0x55e1169c5970 2023-10-08T13:57:32.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.999+0000 7f9c29a4c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:32.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.999+0000 7f9c29a4c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59672/0 (socket says 172.21.15.57:59672) 2023-10-08T13:57:32.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:31.999+0000 7f9c29a4c700 1 -- 172.21.15.57:0/2393841132 learned_addr learned my addr 172.21.15.57:0/2393841132 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:32.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.003+0000 7f9c29a4c700 1 -- 172.21.15.57:0/2393841132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169d0800 msgr2=0x55e1169d2bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:57:32.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.003+0000 7f9c29a4c700 1 --2- 172.21.15.57:0/2393841132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169d0800 0x55e1169d2bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.003+0000 7f9c29a4c700 1 -- 172.21.15.57:0/2393841132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169c5050 msgr2=0x55e1169c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:32.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.003+0000 7f9c29a4c700 1 --2- 172.21.15.57:0/2393841132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169c5050 0x55e1169c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.003+0000 7f9c29a4c700 1 -- 172.21.15.57:0/2393841132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e116743de0 con 0x55e1169c5970 2023-10-08T13:57:32.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.004+0000 7f9c29a4c700 1 --2- 172.21.15.57:0/2393841132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169d02c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009ce0 tx=0x7f9c200113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0b8bfb99f861a95 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:32.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.005+0000 7f9c1affd700 1 -- 172.21.15.57:0/3674283255 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9c14010820 con 0x7f9c2410d1f0 2023-10-08T13:57:32.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.006+0000 7f9c1affd700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c100a1800 0x7f9c100a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.007+0000 7f9c1affd700 1 -- 172.21.15.57:0/3674283255 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9c140cc9c0 con 0x7f9c2410d1f0 2023-10-08T13:57:32.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.040+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c100a1800 0x7f9c100a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:32.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.125+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c100a1800 0x7f9c100a3cc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009550 tx=0x7f9c1c0093f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:32.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.127+0000 7f9c0ffff700 1 -- 172.21.15.57:0/2393841132 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20011d10 con 0x55e1169c5970 2023-10-08T13:57:32.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.127+0000 7f9c0ffff700 1 -- 172.21.15.57:0/2393841132 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c20011e70 con 0x55e1169c5970 2023-10-08T13:57:32.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.127+0000 7f9c0ffff700 1 -- 172.21.15.57:0/2393841132 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2000f8d0 con 0x55e1169c5970 2023-10-08T13:57:32.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.128+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2393841132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 msgr2=0x55e1169d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:32.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.128+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/2393841132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169d02c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009ce0 tx=0x7f9c200113e0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.129+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2393841132 shutdown_connections 2023-10-08T13:57:32.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.129+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/2393841132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169d0800 0x55e1169d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.129+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/2393841132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169c5050 0x55e1169c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.129+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/2393841132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169d02c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.129+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2393841132 >> 172.21.15.57:0/2393841132 conn(0x55e11693a260 msgr2=0x55e1169c1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:32.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.140+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2393841132 shutdown_connections 2023-10-08T13:57:32.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.140+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2393841132 wait complete. 2023-10-08T13:57:32.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.141+0000 7f9c37e99b80 1 -- start start 2023-10-08T13:57:32.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.141+0000 7f9c37e99b80 1 Processor -- start 2023-10-08T13:57:32.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.142+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5050 0x55e1169cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.142+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.143+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.143+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e116918e30 con 0x55e1169d0800 2023-10-08T13:57:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.143+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1168b5710 con 0x55e1169c5050 2023-10-08T13:57:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.143+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1168b53b0 con 0x55e1169c5970 2023-10-08T13:57:32.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.143+0000 7f9c37e99b80 10 client.0 _ll_register_callbacks cb 0x55e116712800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:32.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.145+0000 7f9c2a24d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:32.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.145+0000 7f9c2a24d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58866/0 (socket says 172.21.15.57:58866) 2023-10-08T13:57:32.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.145+0000 7f9c2a24d700 1 -- 172.21.15.57:0/2732737040 learned_addr learned my addr 172.21.15.57:0/2732737040 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:32.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.147+0000 7f9c2924b700 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:32.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.149+0000 7f9c2a24d700 1 -- 172.21.15.57:0/2732737040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 msgr2=0x55e1169cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:32.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.149+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.149+0000 7f9c2a24d700 1 -- 172.21.15.57:0/2732737040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5050 msgr2=0x55e1169cd620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:57:32.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.149+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5050 0x55e1169cd620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.149+0000 7f9c2a24d700 1 -- 172.21.15.57:0/2732737040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c20009940 con 0x55e1169d0800 2023-10-08T13:57:32.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.155+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169ce0a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00bca0 tx=0x7f9c1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15dd48c48e71fd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:32.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.155+0000 7f9c0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:57:32.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.159+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/2732737040 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c023040 con 0x55e1169d0800 2023-10-08T13:57:32.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.159+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/2732737040 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9c1c0077b0 con 0x55e1169d0800 2023-10-08T13:57:32.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.159+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2732737040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1168e0eb0 con 0x55e1169d0800 2023-10-08T13:57:32.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.162+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/2732737040 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c1c00e040 con 0x55e1169d0800 2023-10-08T13:57:32.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.162+0000 7f9c37e99b80 10 client.16842 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:32.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.162+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2732737040 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55e11672cc60 con 0x55e1169d0800 2023-10-08T13:57:32.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.164+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/2732737040 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1168e0eb0 con 0x55e1169d0800 2023-10-08T13:57:32.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.164+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/2732737040 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f9c1c0298e0 con 0x55e1169d0800 2023-10-08T13:57:32.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.164+0000 7f9c37e99b80 10 client.16842 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:32.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.164+0000 7f9c37e99b80 10 client.16842 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T13:57:32.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.165+0000 7f9c37e99b80 20 client.16842 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:32.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.165+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e1169e0480 0x55e1169e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.165+0000 7f9c37e99b80 4 client.16842 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:57:32.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.165+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2732737040 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e116911e10 con 0x55e1169e0480 2023-10-08T13:57:32.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.172+0000 7f9c29a4c700 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e1169e0480 0x55e1169e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:32.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.274+0000 7f9c29a4c700 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e1169e0480 0x55e1169e28a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d36c803022f9073b server_cookie=43ed4c321096e8ba in_seq=0 out_seq=0 2023-10-08T13:57:32.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.279+0000 7f9c0d7fa700 10 client.16842 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:57:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.320+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/2732737040 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e116911e10 con 0x55e1169e0480 2023-10-08T13:57:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.320+0000 7f9c0d7fa700 10 client.16842 handle_command_reply: tid=0 2023-10-08T13:57:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.320+0000 7f9c37e99b80 1 client.16842 shutdown 2023-10-08T13:57:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.320+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2732737040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 msgr2=0x55e1169ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.320+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169ce0a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00bca0 tx=0x7f9c1c00b330 comp rx=0 tx=0).stop 2023-10-08T13:57:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.321+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2732737040 shutdown_connections 2023-10-08T13:57:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.321+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e1169e0480 0x55e1169e28a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.321+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169ce0a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.321+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5050 0x55e1169cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.321+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/2732737040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.321+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2732737040 >> 172.21.15.57:0/2732737040 conn(0x55e11693a260 msgr2=0x55e1169c70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.321+0000 7f9c0d7fa700 0 client.16842 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:57:32.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.346+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2732737040 shutdown_connections 2023-10-08T13:57:32.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.346+0000 7f9c37e99b80 1 -- 172.21.15.57:0/2732737040 wait complete. 2023-10-08T13:57:32.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.347+0000 7f9c37e99b80 20 client.16842 trim_cache size 0 max 16384 2023-10-08T13:57:32.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.727+0000 7f9c37e99b80 1 Processor -- start 2023-10-08T13:57:32.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.740+0000 7f9c37e99b80 1 -- start start 2023-10-08T13:57:32.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.740+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5050 0x55e1169d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.740+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169d72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.740+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:32.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.740+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e116799920 con 0x55e1169d0800 2023-10-08T13:57:32.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.740+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1169d05f0 con 0x55e1169c5050 2023-10-08T13:57:32.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.740+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1169dda90 con 0x55e1169c5970 2023-10-08T13:57:32.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.750+0000 7f9c2a24d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:32.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.750+0000 7f9c2a24d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58878/0 (socket says 172.21.15.57:58878) 2023-10-08T13:57:32.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.750+0000 7f9c2a24d700 1 -- 172.21.15.57:0/3251867848 learned_addr learned my addr 172.21.15.57:0/3251867848 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:32.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.774+0000 7f9c29a4c700 1 --2- 172.21.15.57:0/3251867848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5050 0x55e1169d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:32.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.774+0000 7f9c2924b700 1 --2- 172.21.15.57:0/3251867848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169d72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:32.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.780+0000 7f9c2a24d700 1 -- 172.21.15.57:0/3251867848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 msgr2=0x55e1169d72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:32.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.780+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/3251867848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169d72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.780+0000 7f9c2a24d700 1 -- 172.21.15.57:0/3251867848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5050 msgr2=0x55e1169d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:32.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.780+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/3251867848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5050 0x55e1169d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:32.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.780+0000 7f9c2a24d700 1 -- 172.21.15.57:0/3251867848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e11676a250 con 0x55e1169d0800 2023-10-08T13:57:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.820+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/3251867848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169cdb90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00bc70 tx=0x7f9c1c029e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a097d97e20330a57 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:33.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.985+0000 7f9c18d82700 1 -- 172.21.15.57:0/3251867848 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c03f070 con 0x55e1169d0800 2023-10-08T13:57:33.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.985+0000 7f9c18d82700 1 -- 172.21.15.57:0/3251867848 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c1c005040 con 0x55e1169d0800 2023-10-08T13:57:33.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.985+0000 7f9c18d82700 1 -- 172.21.15.57:0/3251867848 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00e040 con 0x55e1169d0800 2023-10-08T13:57:33.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.986+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3251867848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 msgr2=0x55e1169cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:33.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:32.986+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/3251867848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169cdb90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00bc70 tx=0x7f9c1c029e60 comp rx=0 tx=0).stop 2023-10-08T13:57:33.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.059+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3251867848 shutdown_connections 2023-10-08T13:57:33.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.059+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/3251867848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169d0800 0x55e1169cdb90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.059+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/3251867848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5050 0x55e1169d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.059+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/3251867848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169c5970 0x55e1169d72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.059+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3251867848 >> 172.21.15.57:0/3251867848 conn(0x55e11693a260 msgr2=0x55e1169cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:33.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.060+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3251867848 shutdown_connections 2023-10-08T13:57:33.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.060+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3251867848 wait complete. 2023-10-08T13:57:33.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.060+0000 7f9c37e99b80 1 -- start start 2023-10-08T13:57:33.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.060+0000 7f9c37e99b80 1 Processor -- start 2023-10-08T13:57:33.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.061+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169c5050 0x55e1169d5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:33.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.061+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5970 0x55e1169d57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:33.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.061+0000 7f9c37e99b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169d0800 0x55e116a332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:33.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.061+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e116a337f0 con 0x55e1169c5050 2023-10-08T13:57:33.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.061+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e116a33930 con 0x55e1169c5970 2023-10-08T13:57:33.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.061+0000 7f9c37e99b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e116a33a70 con 0x55e1169d0800 2023-10-08T13:57:33.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.061+0000 7f9c37e99b80 10 client.0 _ll_register_callbacks cb 0x55e116712800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:33.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.066+0000 7f9c29a4c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169c5050 0x55e1169d5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:33.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.066+0000 7f9c29a4c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169c5050 0x55e1169d5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58882/0 (socket says 172.21.15.57:58882) 2023-10-08T13:57:33.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.066+0000 7f9c29a4c700 1 -- 172.21.15.57:0/3151767513 learned_addr learned my addr 172.21.15.57:0/3151767513 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:33.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.066+0000 7f9c2924b700 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5970 0x55e1169d57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:33.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.066+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169d0800 0x55e116a332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.067+0000 7f9c2a24d700 1 -- 172.21.15.57:0/3151767513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5970 msgr2=0x55e1169d57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.067+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5970 0x55e1169d57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.067+0000 7f9c2a24d700 1 -- 172.21.15.57:0/3151767513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169c5050 msgr2=0x55e1169d5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.067+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169c5050 0x55e1169d5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.067+0000 7f9c2a24d700 1 -- 172.21.15.57:0/3151767513 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e11672cc60 con 0x55e1169d0800 2023-10-08T13:57:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.067+0000 7f9c2a24d700 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169d0800 0x55e116a332b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00bca0 tx=0x7f9c1c0060c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a1072bcb379a716 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:33.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.112+0000 7f9c0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:57:33.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.112+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/3151767513 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c03f030 con 0x55e1169d0800 2023-10-08T13:57:33.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.112+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/3151767513 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9c1c040070 con 0x55e1169d0800 2023-10-08T13:57:33.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.112+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3151767513 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c20011e70 con 0x55e1169d0800 2023-10-08T13:57:33.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.123+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/3151767513 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c1c00e040 con 0x55e1169d0800 2023-10-08T13:57:33.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.126+0000 7f9c37e99b80 10 client.26461 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:33.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.126+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3151767513 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55e1167422b0 con 0x55e1169d0800 2023-10-08T13:57:33.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.138+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/3151767513 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c20011e70 con 0x55e1169d0800 2023-10-08T13:57:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.138+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/3151767513 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f9c1c03b460 con 0x55e1169d0800 2023-10-08T13:57:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.138+0000 7f9c37e99b80 10 client.26461 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.138+0000 7f9c37e99b80 10 client.26461 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-10-08T13:57:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.138+0000 7f9c37e99b80 20 client.26461 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.138+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e116a34f00 0x55e116a37320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.138+0000 7f9c37e99b80 4 client.26461 mds_command: new command op to 24634 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:57:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.138+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3151767513 --> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e116911b30 con 0x55e116a34f00 2023-10-08T13:57:33.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.138+0000 7f9c29a4c700 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e116a34f00 0x55e116a37320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.150+0000 7f9c29a4c700 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e116a34f00 0x55e116a37320 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b87c0a82cd5b12a server_cookie=efd7fa30d9792d52 in_seq=0 out_seq=0 2023-10-08T13:57:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.150+0000 7f9c0d7fa700 10 client.26461 ms_handle_connect on v2:172.21.15.115:6836/383062444 2023-10-08T13:57:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.150+0000 7f9c0d7fa700 1 -- 172.21.15.57:0/3151767513 <== mds.2 v2:172.21.15.115:6836/383062444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e116911b30 con 0x55e116a34f00 2023-10-08T13:57:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.150+0000 7f9c0d7fa700 10 client.26461 handle_command_reply: tid=0 2023-10-08T13:57:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.150+0000 7f9c37e99b80 1 client.26461 shutdown 2023-10-08T13:57:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.153+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3151767513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169d0800 msgr2=0x55e116a332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.153+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169d0800 0x55e116a332b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00bca0 tx=0x7f9c1c0060c0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.153+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3151767513 shutdown_connections 2023-10-08T13:57:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.153+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.115:6836/383062444,v1:172.21.15.115:6837/383062444] conn(0x55e116a34f00 0x55e116a37320 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.153+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e1169d0800 0x55e116a332b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.153+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e1169c5050 0x55e1169d5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.153+0000 7f9c37e99b80 1 --2- 172.21.15.57:0/3151767513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1169c5970 0x55e1169d57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.153+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3151767513 >> 172.21.15.57:0/3151767513 conn(0x55e11693a260 msgr2=0x55e1169c0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.153+0000 7f9c0d7fa700 0 client.26461 ms_handle_reset on v2:172.21.15.115:6836/383062444 2023-10-08T13:57:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.154+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3151767513 shutdown_connections 2023-10-08T13:57:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.154+0000 7f9c37e99b80 1 -- 172.21.15.57:0/3151767513 wait complete. 2023-10-08T13:57:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.154+0000 7f9c37e99b80 20 client.26461 trim_cache size 0 max 16384 2023-10-08T13:57:33.156 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:57:33.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.282+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/3674283255 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c100a1800 msgr2=0x7f9c100a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:33.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.282+0000 7f9c2aa4e700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c100a1800 0x7f9c100a3cc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009550 tx=0x7f9c1c0093f0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.282+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/3674283255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2410d1f0 msgr2=0x7f9c241196f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.282+0000 7f9c2aa4e700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2410d1f0 0x7f9c241196f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400d950 tx=0x7f9c1400dc60 comp rx=0 tx=0).stop 2023-10-08T13:57:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.282+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/3674283255 shutdown_connections 2023-10-08T13:57:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.282+0000 7f9c2aa4e700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c24119c80 0x7f9c24112470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.282+0000 7f9c2aa4e700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9c2410c8d0 0x7f9c24119170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.282+0000 7f9c2aa4e700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9c100a1800 0x7f9c100a3cc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.282+0000 7f9c2aa4e700 1 --2- 172.21.15.57:0/3674283255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2410d1f0 0x7f9c241196f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.282+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/3674283255 >> 172.21.15.57:0/3674283255 conn(0x7f9c2406b650 msgr2=0x7f9c2411b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:33.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.283+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/3674283255 shutdown_connections 2023-10-08T13:57:33.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:33.289+0000 7f9c2aa4e700 1 -- 172.21.15.57:0/3674283255 wait complete. 2023-10-08T13:57:33.373 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-08T13:57:33.751 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:33 smithi057 ceph-mon[99476]: pgmap v479: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2023-10-08T13:57:33.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:33 smithi115 ceph-mon[104408]: pgmap v479: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2023-10-08T13:57:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:33 smithi099 ceph-mon[103849]: pgmap v479: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2023-10-08T13:57:34.421 INFO:tasks.workunit.client.0.smithi057.stdout: 487 2334163 538 1612890 409 415662 595 2023-10-08T13:57:34.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:34 smithi057 ceph-mon[99476]: pgmap v480: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2023-10-08T13:57:35.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:34 smithi099 ceph-mon[103849]: pgmap v480: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2023-10-08T13:57:35.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:34 smithi115 ceph-mon[104408]: pgmap v480: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2023-10-08T13:57:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.250+0000 7f5751517700 1 -- 172.21.15.57:0/984266429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f574c107a60 msgr2=0x7f574c107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.250+0000 7f5751517700 1 --2- 172.21.15.57:0/984266429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f574c107a60 0x7f574c107e40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f573c0098f0 tx=0x7f573c009c00 comp rx=0 tx=0).stop 2023-10-08T13:57:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.251+0000 7f5751517700 1 -- 172.21.15.57:0/984266429 shutdown_connections 2023-10-08T13:57:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.251+0000 7f5751517700 1 --2- 172.21.15.57:0/984266429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c10c0e0 0x7f574c078b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.251+0000 7f5751517700 1 --2- 172.21.15.57:0/984266429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f574c107a60 0x7f574c107e40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.251+0000 7f5751517700 1 --2- 172.21.15.57:0/984266429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f574c108380 0x7f574c10b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.251+0000 7f5751517700 1 -- 172.21.15.57:0/984266429 >> 172.21.15.57:0/984266429 conn(0x7f574c06b830 msgr2=0x7f574c06bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:35.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.258+0000 7f5751517700 1 -- 172.21.15.57:0/984266429 shutdown_connections 2023-10-08T13:57:35.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.259+0000 7f5751517700 1 -- 172.21.15.57:0/984266429 wait complete. 2023-10-08T13:57:35.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.260+0000 7f5751517700 1 Processor -- start 2023-10-08T13:57:35.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.260+0000 7f5751517700 1 -- start start 2023-10-08T13:57:35.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.260+0000 7f5751517700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f574c107a60 0x7f574c0831a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.260+0000 7f5751517700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c108380 0x7f574c07d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.260+0000 7f5751517700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f574c10c0e0 0x7f574c07d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.260+0000 7f5751517700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c07de30 con 0x7f574c10c0e0 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.260+0000 7f5751517700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c07dfa0 con 0x7f574c107a60 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.260+0000 7f5751517700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c12fbf0 con 0x7f574c108380 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.261+0000 7f574b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f574c10c0e0 0x7f574c07d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.261+0000 7f574b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f574c10c0e0 0x7f574c07d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58894/0 (socket says 172.21.15.57:58894) 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.261+0000 7f574b7fe700 1 -- 172.21.15.57:0/1453133177 learned_addr learned my addr 172.21.15.57:0/1453133177 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.261+0000 7f574affd700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f574c107a60 0x7f574c0831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.261+0000 7f574b7fe700 1 -- 172.21.15.57:0/1453133177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c108380 msgr2=0x7f574c07d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.261+0000 7f574b7fe700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c108380 0x7f574c07d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.261+0000 7f574b7fe700 1 -- 172.21.15.57:0/1453133177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f574c107a60 msgr2=0x7f574c0831a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:35.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.261+0000 7f574b7fe700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f574c107a60 0x7f574c0831a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.261+0000 7f574b7fe700 1 -- 172.21.15.57:0/1453133177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c009520 con 0x7f574c10c0e0 2023-10-08T13:57:35.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.261+0000 7f574b7fe700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f574c10c0e0 0x7f574c07d780 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f574000c970 tx=0x7f574000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:35.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.262+0000 7f5733fff700 1 -- 172.21.15.57:0/1453133177 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740007b00 con 0x7f574c10c0e0 2023-10-08T13:57:35.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.262+0000 7f5733fff700 1 -- 172.21.15.57:0/1453133177 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5740004cf0 con 0x7f574c10c0e0 2023-10-08T13:57:35.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.262+0000 7f5733fff700 1 -- 172.21.15.57:0/1453133177 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57400054e0 con 0x7f574c10c0e0 2023-10-08T13:57:35.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.262+0000 7f5751517700 1 -- 172.21.15.57:0/1453133177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c12fed0 con 0x7f574c10c0e0 2023-10-08T13:57:35.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.262+0000 7f5751517700 1 -- 172.21.15.57:0/1453133177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c130390 con 0x7f574c10c0e0 2023-10-08T13:57:35.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.265+0000 7f5733fff700 1 -- 172.21.15.57:0/1453133177 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5740004750 con 0x7f574c10c0e0 2023-10-08T13:57:35.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.282+0000 7f575d700b80 1 Processor -- start 2023-10-08T13:57:35.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.283+0000 7f5733fff700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57340a1730 0x7f57340a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.283+0000 7f5733fff700 1 -- 172.21.15.57:0/1453133177 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f57400cd020 con 0x7f574c10c0e0 2023-10-08T13:57:35.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.284+0000 7f575d700b80 1 -- start start 2023-10-08T13:57:35.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.284+0000 7f575d700b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 0x565006d9f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.284+0000 7f575d700b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 0x565006daa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.284+0000 7f575d700b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006daca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.284+0000 7f575d700b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006d08460 con 0x565006d9f7a0 2023-10-08T13:57:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.284+0000 7f575d700b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006cf9c20 con 0x565006d9ee80 2023-10-08T13:57:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.284+0000 7f575d700b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006cf7540 con 0x565006daa650 2023-10-08T13:57:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.284+0000 7f574affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 0x565006d9f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.285+0000 7f574affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 0x565006d9f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60330/0 (socket says 172.21.15.57:60330) 2023-10-08T13:57:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.285+0000 7f574affd700 1 -- 172.21.15.57:0/1073553927 learned_addr learned my addr 172.21.15.57:0/1073553927 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:35.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.285+0000 7f574b7fe700 1 --2- 172.21.15.57:0/1073553927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006daca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.290+0000 7f574a7fc700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57340a1730 0x7f57340a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.290+0000 7f574a7fc700 1 --2- 172.21.15.57:0/1073553927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 0x565006daa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.294+0000 7f574affd700 1 -- 172.21.15.57:0/1073553927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 msgr2=0x565006daca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:35.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.294+0000 7f574affd700 1 --2- 172.21.15.57:0/1073553927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006daca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.294+0000 7f574affd700 1 -- 172.21.15.57:0/1073553927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 msgr2=0x565006daa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:35.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.294+0000 7f574affd700 1 --2- 172.21.15.57:0/1073553927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 0x565006daa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.294+0000 7f574affd700 1 -- 172.21.15.57:0/1073553927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565006b1dde0 con 0x565006d9ee80 2023-10-08T13:57:35.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.301+0000 7f574a7fc700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57340a1730 0x7f57340a3bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f57440060b0 tx=0x7f574400b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:35.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.304+0000 7f574affd700 1 --2- 172.21.15.57:0/1073553927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 0x565006d9f260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f573c016000 tx=0x7f573c010cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e6ed5f4838e8bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.310+0000 7f57317fa700 1 -- 172.21.15.57:0/1073553927 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c00f660 con 0x565006d9ee80 2023-10-08T13:57:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.310+0000 7f57317fa700 1 -- 172.21.15.57:0/1073553927 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f573c01d040 con 0x565006d9ee80 2023-10-08T13:57:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.310+0000 7f57317fa700 1 -- 172.21.15.57:0/1073553927 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c00b940 con 0x565006d9ee80 2023-10-08T13:57:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.311+0000 7f575d700b80 1 -- 172.21.15.57:0/1073553927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 msgr2=0x565006d9f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.311+0000 7f575d700b80 1 --2- 172.21.15.57:0/1073553927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 0x565006d9f260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f573c016000 tx=0x7f573c010cf0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.311+0000 7f575d700b80 1 -- 172.21.15.57:0/1073553927 shutdown_connections 2023-10-08T13:57:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.311+0000 7f575d700b80 1 --2- 172.21.15.57:0/1073553927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006daca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.311+0000 7f575d700b80 1 --2- 172.21.15.57:0/1073553927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 0x565006d9f260 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.311+0000 7f575d700b80 1 --2- 172.21.15.57:0/1073553927 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 0x565006daa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.311+0000 7f575d700b80 1 -- 172.21.15.57:0/1073553927 >> 172.21.15.57:0/1073553927 conn(0x565006d14260 msgr2=0x565006d9b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:35.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.314+0000 7f575d700b80 1 -- 172.21.15.57:0/1073553927 shutdown_connections 2023-10-08T13:57:35.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.314+0000 7f575d700b80 1 -- 172.21.15.57:0/1073553927 wait complete. 2023-10-08T13:57:35.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.314+0000 7f575d700b80 1 -- start start 2023-10-08T13:57:35.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.314+0000 7f575d700b80 1 Processor -- start 2023-10-08T13:57:35.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.315+0000 7f575d700b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 0x565006daa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.315+0000 7f575d700b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006da7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.315+0000 7f575d700b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006da79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.315+0000 7f575d700b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006cf2e30 con 0x565006d9ee80 2023-10-08T13:57:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.315+0000 7f575d700b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006c8f710 con 0x565006d9f7a0 2023-10-08T13:57:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.315+0000 7f575d700b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006c8f3b0 con 0x565006daa650 2023-10-08T13:57:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.315+0000 7f575d700b80 10 client.0 _ll_register_callbacks cb 0x565006aec800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.315+0000 7f574a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006da7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.315+0000 7f574a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006da7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60334/0 (socket says 172.21.15.57:60334) 2023-10-08T13:57:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.315+0000 7f574a7fc700 1 -- 172.21.15.57:0/3659681851 learned_addr learned my addr 172.21.15.57:0/3659681851 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.317+0000 7f574a7fc700 1 -- 172.21.15.57:0/3659681851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 msgr2=0x565006da79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.317+0000 7f574a7fc700 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006da79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.317+0000 7f574a7fc700 1 -- 172.21.15.57:0/3659681851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 msgr2=0x565006daa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.317+0000 7f574a7fc700 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 0x565006daa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.318+0000 7f574a7fc700 1 -- 172.21.15.57:0/3659681851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c0793c0 con 0x565006d9f7a0 2023-10-08T13:57:35.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.325+0000 7f574a7fc700 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006da7370 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5744012fc0 tx=0x7f5744004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daed1e1f9ef8864e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:35.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.326+0000 7f572eff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:57:35.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.330+0000 7f572eff5700 1 -- 172.21.15.57:0/3659681851 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744014950 con 0x565006d9f7a0 2023-10-08T13:57:35.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.330+0000 7f572eff5700 1 -- 172.21.15.57:0/3659681851 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5744014ab0 con 0x565006d9f7a0 2023-10-08T13:57:35.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.331+0000 7f575d700b80 1 -- 172.21.15.57:0/3659681851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565006cbaeb0 con 0x565006d9f7a0 2023-10-08T13:57:35.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.345+0000 7f572eff5700 1 -- 172.21.15.57:0/3659681851 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5744007e90 con 0x565006d9f7a0 2023-10-08T13:57:35.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.345+0000 7f575d700b80 10 client.26414 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:35.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.345+0000 7f575d700b80 1 -- 172.21.15.57:0/3659681851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x565006b06c60 con 0x565006d9f7a0 2023-10-08T13:57:35.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.354+0000 7f572eff5700 1 -- 172.21.15.57:0/3659681851 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565006cbaeb0 con 0x565006d9f7a0 2023-10-08T13:57:35.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.354+0000 7f572eff5700 1 -- 172.21.15.57:0/3659681851 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f5744017a70 con 0x565006d9f7a0 2023-10-08T13:57:35.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.356+0000 7f575d700b80 10 client.26414 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:35.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.356+0000 7f575d700b80 10 client.26414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T13:57:35.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.356+0000 7f575d700b80 20 client.26414 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:35.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.356+0000 7f575d700b80 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x565006dba1c0 0x565006dbc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.356+0000 7f575d700b80 4 client.26414 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:57:35.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.356+0000 7f575d700b80 1 -- 172.21.15.57:0/3659681851 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565006cebe10 con 0x565006dba1c0 2023-10-08T13:57:35.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.363+0000 7f574affd700 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x565006dba1c0 0x565006dbc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.383+0000 7f572eff5700 10 client.26414 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:57:35.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.383+0000 7f574affd700 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x565006dba1c0 0x565006dbc600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0a71f4ff5576d75 server_cookie=6518639c34074f09 in_seq=0 out_seq=0 2023-10-08T13:57:35.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.398+0000 7f572eff5700 1 -- 172.21.15.57:0/3659681851 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565006cebe10 con 0x565006dba1c0 2023-10-08T13:57:35.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.398+0000 7f572eff5700 10 client.26414 handle_command_reply: tid=0 2023-10-08T13:57:35.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.398+0000 7f575d700b80 1 client.26414 shutdown 2023-10-08T13:57:35.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.413+0000 7f575d700b80 1 -- 172.21.15.57:0/3659681851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 msgr2=0x565006da7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:35.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.413+0000 7f575d700b80 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006da7370 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5744012fc0 tx=0x7f5744004ad0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.439+0000 7f575d700b80 1 -- 172.21.15.57:0/3659681851 shutdown_connections 2023-10-08T13:57:35.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.439+0000 7f575d700b80 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x565006dba1c0 0x565006dbc600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.439+0000 7f575d700b80 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006da79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.439+0000 7f575d700b80 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 0x565006daa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.439+0000 7f575d700b80 1 --2- 172.21.15.57:0/3659681851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006da7370 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.439+0000 7f575d700b80 1 -- 172.21.15.57:0/3659681851 >> 172.21.15.57:0/3659681851 conn(0x565006d14260 msgr2=0x565006d9aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:35.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.439+0000 7f572eff5700 0 client.26414 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:57:35.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.455+0000 7f575d700b80 1 -- 172.21.15.57:0/3659681851 shutdown_connections 2023-10-08T13:57:35.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.455+0000 7f575d700b80 1 -- 172.21.15.57:0/3659681851 wait complete. 2023-10-08T13:57:35.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.465+0000 7f575d700b80 20 client.26414 trim_cache size 0 max 16384 2023-10-08T13:57:35.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.494+0000 7f575d700b80 1 Processor -- start 2023-10-08T13:57:35.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.495+0000 7f575d700b80 1 -- start start 2023-10-08T13:57:35.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.495+0000 7f575d700b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 0x565006da79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.495+0000 7f575d700b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006db1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.495+0000 7f575d700b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006da68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.495+0000 7f575d700b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006b73920 con 0x565006d9ee80 2023-10-08T13:57:35.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.495+0000 7f575d700b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006db16c0 con 0x565006d9f7a0 2023-10-08T13:57:35.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.495+0000 7f575d700b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006da58e0 con 0x565006daa650 2023-10-08T13:57:35.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.496+0000 7f574affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 0x565006da79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.496+0000 7f574affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 0x565006da79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58916/0 (socket says 172.21.15.57:58916) 2023-10-08T13:57:35.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.496+0000 7f574affd700 1 -- 172.21.15.57:0/1336891574 learned_addr learned my addr 172.21.15.57:0/1336891574 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:35.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.497+0000 7f574affd700 1 -- 172.21.15.57:0/1336891574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 msgr2=0x565006da68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:35.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.497+0000 7f574affd700 1 --2- 172.21.15.57:0/1336891574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006da68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.497+0000 7f574affd700 1 -- 172.21.15.57:0/1336891574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 msgr2=0x565006db1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:35.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.497+0000 7f574a7fc700 1 --2- 172.21.15.57:0/1336891574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006db1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.497+0000 7f574affd700 1 --2- 172.21.15.57:0/1336891574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006db1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.497+0000 7f574affd700 1 -- 172.21.15.57:0/1336891574 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565006b44250 con 0x565006d9ee80 2023-10-08T13:57:35.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.497+0000 7f574a7fc700 1 --2- 172.21.15.57:0/1336891574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006db1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T13:57:35.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.498+0000 7f574affd700 1 --2- 172.21.15.57:0/1336891574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 0x565006da79d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f573c00faf0 tx=0x7f573c00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545fbd80e1a7276f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:35.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.498+0000 7f5731ffb700 1 -- 172.21.15.57:0/1336891574 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c02d9b0 con 0x565006d9ee80 2023-10-08T13:57:35.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.498+0000 7f5731ffb700 1 -- 172.21.15.57:0/1336891574 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f573c02d030 con 0x565006d9ee80 2023-10-08T13:57:35.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.498+0000 7f5731ffb700 1 -- 172.21.15.57:0/1336891574 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c00b870 con 0x565006d9ee80 2023-10-08T13:57:35.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.499+0000 7f575d700b80 1 -- 172.21.15.57:0/1336891574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 msgr2=0x565006da79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:35.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.499+0000 7f575d700b80 1 --2- 172.21.15.57:0/1336891574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 0x565006da79d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f573c00faf0 tx=0x7f573c00fa80 comp rx=0 tx=0).stop 2023-10-08T13:57:35.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.500+0000 7f575d700b80 1 -- 172.21.15.57:0/1336891574 shutdown_connections 2023-10-08T13:57:35.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.500+0000 7f575d700b80 1 --2- 172.21.15.57:0/1336891574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006da68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.500+0000 7f575d700b80 1 --2- 172.21.15.57:0/1336891574 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9ee80 0x565006da79d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.500+0000 7f575d700b80 1 --2- 172.21.15.57:0/1336891574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9f7a0 0x565006db1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.500+0000 7f575d700b80 1 -- 172.21.15.57:0/1336891574 >> 172.21.15.57:0/1336891574 conn(0x565006d14260 msgr2=0x565006dbc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:35.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.505+0000 7f575d700b80 1 -- 172.21.15.57:0/1336891574 shutdown_connections 2023-10-08T13:57:35.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.505+0000 7f575d700b80 1 -- 172.21.15.57:0/1336891574 wait complete. 2023-10-08T13:57:35.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.505+0000 7f575d700b80 1 -- start start 2023-10-08T13:57:35.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.506+0000 7f575d700b80 1 Processor -- start 2023-10-08T13:57:35.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.506+0000 7f575d700b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 0x565006daf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.506+0000 7f575d700b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 0x565006daf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.506+0000 7f575d700b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006e0d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.506+0000 7f575d700b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006daff90 con 0x565006d9f7a0 2023-10-08T13:57:35.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.506+0000 7f575d700b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006e0d590 con 0x565006d9ee80 2023-10-08T13:57:35.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.506+0000 7f575d700b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x565006e0d6d0 con 0x565006daa650 2023-10-08T13:57:35.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.506+0000 7f575d700b80 10 client.0 _ll_register_callbacks cb 0x565006aec800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:35.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.509+0000 7f574b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006e0d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.509+0000 7f574a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 0x565006daf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.509+0000 7f574a7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 0x565006daf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58922/0 (socket says 172.21.15.57:58922) 2023-10-08T13:57:35.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.509+0000 7f574a7fc700 1 -- 172.21.15.57:0/1899369 learned_addr learned my addr 172.21.15.57:0/1899369 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:35.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.509+0000 7f574a7fc700 1 -- 172.21.15.57:0/1899369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 msgr2=0x565006e0d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:35.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.510+0000 7f574a7fc700 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006e0d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.510+0000 7f574a7fc700 1 -- 172.21.15.57:0/1899369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 msgr2=0x565006daf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:35.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.510+0000 7f574a7fc700 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 0x565006daf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.510+0000 7f574a7fc700 1 -- 172.21.15.57:0/1899369 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565006b1dde0 con 0x565006d9f7a0 2023-10-08T13:57:35.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.510+0000 7f574a7fc700 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 0x565006daf6a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x565006da8860 tx=0x7f5744009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196e6eb9dc958b81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:35.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.510+0000 7f572eff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:57:35.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.511+0000 7f572eff5700 1 -- 172.21.15.57:0/1899369 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744003c10 con 0x565006d9f7a0 2023-10-08T13:57:35.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.511+0000 7f572eff5700 1 -- 172.21.15.57:0/1899369 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f5744003d70 con 0x565006d9f7a0 2023-10-08T13:57:35.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.511+0000 7f575d700b80 1 -- 172.21.15.57:0/1899369 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f573c01d040 con 0x565006d9f7a0 2023-10-08T13:57:35.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.511+0000 7f572eff5700 1 -- 172.21.15.57:0/1899369 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5744022800 con 0x565006d9f7a0 2023-10-08T13:57:35.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.514+0000 7f575d700b80 10 client.16863 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:35.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.514+0000 7f575d700b80 1 -- 172.21.15.57:0/1899369 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x565006b1c2b0 con 0x565006d9f7a0 2023-10-08T13:57:35.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.515+0000 7f572eff5700 1 -- 172.21.15.57:0/1899369 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f573c01d040 con 0x565006d9f7a0 2023-10-08T13:57:35.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.515+0000 7f572eff5700 1 -- 172.21.15.57:0/1899369 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f5744014bc0 con 0x565006d9f7a0 2023-10-08T13:57:35.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.515+0000 7f575d700b80 10 client.16863 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:35.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.515+0000 7f575d700b80 10 client.16863 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T13:57:35.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.515+0000 7f575d700b80 20 client.16863 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:35.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.515+0000 7f575d700b80 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x565006e0ee60 0x565006e11280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:35.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.515+0000 7f575d700b80 4 client.16863 mds_command: new command op to 24652 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:57:35.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.515+0000 7f575d700b80 1 -- 172.21.15.57:0/1899369 --> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565006cebb30 con 0x565006e0ee60 2023-10-08T13:57:35.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.518+0000 7f574affd700 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x565006e0ee60 0x565006e11280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:35.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.555+0000 7f574affd700 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x565006e0ee60 0x565006e11280 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30e188fd0a4bc592 server_cookie=c74bcb6de50574b7 in_seq=0 out_seq=0 2023-10-08T13:57:35.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.562+0000 7f572eff5700 10 client.16863 ms_handle_connect on v2:172.21.15.115:6838/2892511272 2023-10-08T13:57:35.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.562+0000 7f572eff5700 1 -- 172.21.15.57:0/1899369 <== mds.1 v2:172.21.15.115:6838/2892511272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+236 (crc 0 0 0) 0x565006cebb30 con 0x565006e0ee60 2023-10-08T13:57:35.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.562+0000 7f572eff5700 10 client.16863 handle_command_reply: tid=0 2023-10-08T13:57:35.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.562+0000 7f575d700b80 1 client.16863 shutdown 2023-10-08T13:57:35.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.563+0000 7f575d700b80 1 -- 172.21.15.57:0/1899369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 msgr2=0x565006daf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:35.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.563+0000 7f575d700b80 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 0x565006daf6a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x565006da8860 tx=0x7f5744009c90 comp rx=0 tx=0).stop 2023-10-08T13:57:35.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.574+0000 7f575d700b80 1 -- 172.21.15.57:0/1899369 shutdown_connections 2023-10-08T13:57:35.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.574+0000 7f575d700b80 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.115:6838/2892511272,v1:172.21.15.115:6839/2892511272] conn(0x565006e0ee60 0x565006e11280 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.574+0000 7f575d700b80 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565006daa650 0x565006e0d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.574+0000 7f575d700b80 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565006d9ee80 0x565006daf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.574+0000 7f575d700b80 1 --2- 172.21.15.57:0/1899369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x565006d9f7a0 0x565006daf6a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.574+0000 7f575d700b80 1 -- 172.21.15.57:0/1899369 >> 172.21.15.57:0/1899369 conn(0x565006d14260 msgr2=0x565006da26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:35.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.583+0000 7f572eff5700 0 client.16863 ms_handle_reset on v2:172.21.15.115:6838/2892511272 2023-10-08T13:57:35.599 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-10-08T13:57:35.599 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T13:57:35.599 INFO:teuthology.orchestra.run.smithi057.stdout: "damage_type": "backtrace", 2023-10-08T13:57:35.599 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 1216618087, 2023-10-08T13:57:35.599 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 1099511628214, 2023-10-08T13:57:35.599 INFO:teuthology.orchestra.run.smithi057.stdout: "path": "/volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-1" 2023-10-08T13:57:35.599 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T13:57:35.599 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-10-08T13:57:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.592+0000 7f575d700b80 1 -- 172.21.15.57:0/1899369 shutdown_connections 2023-10-08T13:57:35.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.594+0000 7f575d700b80 1 -- 172.21.15.57:0/1899369 wait complete. 2023-10-08T13:57:35.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.594+0000 7f575d700b80 20 client.16863 trim_cache size 0 max 16384 2023-10-08T13:57:35.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.612+0000 7f5751517700 1 -- 172.21.15.57:0/1453133177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57340a1730 msgr2=0x7f57340a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:35.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.612+0000 7f5751517700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57340a1730 0x7f57340a3bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f57440060b0 tx=0x7f574400b040 comp rx=0 tx=0).stop 2023-10-08T13:57:35.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.612+0000 7f5751517700 1 -- 172.21.15.57:0/1453133177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f574c10c0e0 msgr2=0x7f574c07d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:35.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.612+0000 7f5751517700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f574c10c0e0 0x7f574c07d780 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f574000c970 tx=0x7f574000cd30 comp rx=0 tx=0).stop 2023-10-08T13:57:35.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.619+0000 7f5751517700 1 -- 172.21.15.57:0/1453133177 shutdown_connections 2023-10-08T13:57:35.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.619+0000 7f5751517700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f574c10c0e0 0x7f574c07d780 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.619+0000 7f5751517700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f57340a1730 0x7f57340a3bf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.619+0000 7f5751517700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f574c107a60 0x7f574c0831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.619+0000 7f5751517700 1 --2- 172.21.15.57:0/1453133177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c108380 0x7f574c07d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:35.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.619+0000 7f5751517700 1 -- 172.21.15.57:0/1453133177 >> 172.21.15.57:0/1453133177 conn(0x7f574c06b830 msgr2=0x7f574c0709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:35.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.631+0000 7f5751517700 1 -- 172.21.15.57:0/1453133177 shutdown_connections 2023-10-08T13:57:35.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:35.632+0000 7f5751517700 1 -- 172.21.15.57:0/1453133177 wait complete. 2023-10-08T13:57:35.679 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-08T13:57:36.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.110+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/4167231000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bf8107d30 msgr2=0x7f1bf81081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:36.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.110+0000 7f1bfe4eb700 1 --2- 172.21.15.57:0/4167231000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bf8107d30 0x7f1bf81081b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1be801a9d0 tx=0x7f1be801ace0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.111+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/4167231000 shutdown_connections 2023-10-08T13:57:36.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.111+0000 7f1bfe4eb700 1 --2- 172.21.15.57:0/4167231000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bf8119fd0 0x7f1bf811c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.111+0000 7f1bfe4eb700 1 --2- 172.21.15.57:0/4167231000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf810ef80 0x7f1bf81077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.111+0000 7f1bfe4eb700 1 --2- 172.21.15.57:0/4167231000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bf8107d30 0x7f1bf81081b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.111+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/4167231000 >> 172.21.15.57:0/4167231000 conn(0x7f1bf806b830 msgr2=0x7f1bf806bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:36.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.131+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/4167231000 shutdown_connections 2023-10-08T13:57:36.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.131+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/4167231000 wait complete. 2023-10-08T13:57:36.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.132+0000 7f1bfe4eb700 1 Processor -- start 2023-10-08T13:57:36.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.132+0000 7f1bfe4eb700 1 -- start start 2023-10-08T13:57:36.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.132+0000 7f1bfe4eb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bf8107d30 0x7f1bf8115740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.132+0000 7f1bfe4eb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf810ef80 0x7f1bf8115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.132+0000 7f1bfe4eb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bf8119fd0 0x7f1bf81a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.132+0000 7f1bfe4eb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf81a2280 con 0x7f1bf8107d30 2023-10-08T13:57:36.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.132+0000 7f1bfe4eb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf81a23f0 con 0x7f1bf810ef80 2023-10-08T13:57:36.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.132+0000 7f1bfe4eb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf81a25b0 con 0x7f1bf8119fd0 2023-10-08T13:57:36.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.133+0000 7f1bf77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf810ef80 0x7f1bf8115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.133+0000 7f1bf77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf810ef80 0x7f1bf8115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60360/0 (socket says 172.21.15.57:60360) 2023-10-08T13:57:36.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.133+0000 7f1bf77fe700 1 -- 172.21.15.57:0/749185414 learned_addr learned my addr 172.21.15.57:0/749185414 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:36.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.138+0000 7f1bf7fff700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bf8107d30 0x7f1bf8115740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.138+0000 7f1bfca88700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bf8119fd0 0x7f1bf81a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.141+0000 7f1bf77fe700 1 -- 172.21.15.57:0/749185414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bf8119fd0 msgr2=0x7f1bf81a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:36.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.141+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bf8119fd0 0x7f1bf81a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.141+0000 7f1bf77fe700 1 -- 172.21.15.57:0/749185414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bf8107d30 msgr2=0x7f1bf8115740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:36.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.141+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bf8107d30 0x7f1bf8115740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.141+0000 7f1bf77fe700 1 -- 172.21.15.57:0/749185414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be801a600 con 0x7f1bf810ef80 2023-10-08T13:57:36.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.141+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf810ef80 0x7f1bf8115c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1be801a8f0 tx=0x7f1be8005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:36.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.141+0000 7f1bf57fa700 1 -- 172.21.15.57:0/749185414 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be802d070 con 0x7f1bf810ef80 2023-10-08T13:57:36.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.141+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/749185414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf81a2830 con 0x7f1bf810ef80 2023-10-08T13:57:36.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.142+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/749185414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf81a9fe0 con 0x7f1bf810ef80 2023-10-08T13:57:36.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.142+0000 7f1bf57fa700 1 -- 172.21.15.57:0/749185414 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be80283f0 con 0x7f1bf810ef80 2023-10-08T13:57:36.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.142+0000 7f1bf57fa700 1 -- 172.21.15.57:0/749185414 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be80313c0 con 0x7f1bf810ef80 2023-10-08T13:57:36.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.151+0000 7f1c0a6d4b80 1 Processor -- start 2023-10-08T13:57:36.154 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.152+0000 7f1bf57fa700 1 -- 172.21.15.57:0/749185414 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1be8031520 con 0x7f1bf810ef80 2023-10-08T13:57:36.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.153+0000 7f1bf57fa700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1be00a1730 0x7f1be00a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.153+0000 7f1bf57fa700 1 -- 172.21.15.57:0/749185414 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1be80df680 con 0x7f1bf810ef80 2023-10-08T13:57:36.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.154+0000 7f1c0a6d4b80 1 -- start start 2023-10-08T13:57:36.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.154+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27de80 0x56353d27e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.154+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27e7a0 0x56353d289110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.154+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d28ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.154+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d1e7460 con 0x56353d289650 2023-10-08T13:57:36.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.154+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d1d8c20 con 0x56353d27de80 2023-10-08T13:57:36.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.154+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d1d6540 con 0x56353d27e7a0 2023-10-08T13:57:36.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.155+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27de80 0x56353d27e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.155+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27de80 0x56353d27e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60368/0 (socket says 172.21.15.57:60368) 2023-10-08T13:57:36.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.155+0000 7f1bf7fff700 1 -- 172.21.15.57:0/405207754 learned_addr learned my addr 172.21.15.57:0/405207754 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:36.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.155+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/405207754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27e7a0 0x56353d289110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.155+0000 7f1bf7fff700 1 -- 172.21.15.57:0/405207754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27e7a0 msgr2=0x56353d289110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:36.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.155+0000 7f1bf7fff700 1 --2- 172.21.15.57:0/405207754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27e7a0 0x56353d289110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.155+0000 7f1bf7fff700 1 -- 172.21.15.57:0/405207754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 msgr2=0x56353d28ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:36.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.155+0000 7f1bf7fff700 1 --2- 172.21.15.57:0/405207754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d28ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.155+0000 7f1bf7fff700 1 -- 172.21.15.57:0/405207754 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56353cffcde0 con 0x56353d27de80 2023-10-08T13:57:36.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.156+0000 7f1bf7fff700 1 --2- 172.21.15.57:0/405207754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27de80 0x56353d27e260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1bec007840 tx=0x7f1bec007b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e01c7d82dc79c73d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:36.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.160+0000 7f1bfca88700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1be00a1730 0x7f1be00a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.170+0000 7f1bde7fc700 1 -- 172.21.15.57:0/405207754 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec0125f0 con 0x56353d27de80 2023-10-08T13:57:36.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.170+0000 7f1bde7fc700 1 -- 172.21.15.57:0/405207754 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bec012c10 con 0x56353d27de80 2023-10-08T13:57:36.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.170+0000 7f1bde7fc700 1 -- 172.21.15.57:0/405207754 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec013de0 con 0x56353d27de80 2023-10-08T13:57:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.171+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/405207754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27de80 msgr2=0x56353d27e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.171+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/405207754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27de80 0x56353d27e260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1bec007840 tx=0x7f1bec007b50 comp rx=0 tx=0).stop 2023-10-08T13:57:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.171+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/405207754 shutdown_connections 2023-10-08T13:57:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.171+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/405207754 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d28ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.171+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/405207754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27de80 0x56353d27e260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.171+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/405207754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27e7a0 0x56353d289110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.171+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/405207754 >> 172.21.15.57:0/405207754 conn(0x56353d1f3260 msgr2=0x56353d27a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:36.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.193+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/405207754 shutdown_connections 2023-10-08T13:57:36.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.193+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/405207754 wait complete. 2023-10-08T13:57:36.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.193+0000 7f1c0a6d4b80 1 -- start start 2023-10-08T13:57:36.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.194+0000 7f1c0a6d4b80 1 Processor -- start 2023-10-08T13:57:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.194+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d288f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.194+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d2862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.194+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d286890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.194+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d1d1e30 con 0x56353d289650 2023-10-08T13:57:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.194+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d16e710 con 0x56353d27e7a0 2023-10-08T13:57:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.194+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d16e3b0 con 0x56353d27de80 2023-10-08T13:57:36.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.194+0000 7f1c0a6d4b80 10 client.0 _ll_register_callbacks cb 0x56353cfcb800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.206+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d288f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.206+0000 7f1bf77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d2862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.206+0000 7f1bf77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d2862d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60378/0 (socket says 172.21.15.57:60378) 2023-10-08T13:57:36.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.206+0000 7f1bf77fe700 1 -- 172.21.15.57:0/3734622578 learned_addr learned my addr 172.21.15.57:0/3734622578 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:36.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.208+0000 7f1bf77fe700 1 -- 172.21.15.57:0/3734622578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 msgr2=0x56353d288f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:36.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.208+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d288f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.208+0000 7f1bf77fe700 1 -- 172.21.15.57:0/3734622578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 msgr2=0x56353d286890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:36.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.208+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d286890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.208+0000 7f1bf77fe700 1 -- 172.21.15.57:0/3734622578 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec00e040 con 0x56353d27e7a0 2023-10-08T13:57:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.208+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d2862d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1be8000c00 tx=0x7f1be80acb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fba4dbe41a3b93c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.208+0000 7f1bd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:57:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.208+0000 7f1bd77fe700 1 -- 172.21.15.57:0/3734622578 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be80b4690 con 0x56353d27e7a0 2023-10-08T13:57:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.208+0000 7f1bd77fe700 1 -- 172.21.15.57:0/3734622578 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1be80ac5f0 con 0x56353d27e7a0 2023-10-08T13:57:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.218+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/3734622578 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56353d199eb0 con 0x56353d27e7a0 2023-10-08T13:57:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.218+0000 7f1bfca88700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1be00a1730 0x7f1be00a3bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009ed0 tx=0x7f1bf0009540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.218+0000 7f1bd77fe700 1 -- 172.21.15.57:0/3734622578 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1be80ac7b0 con 0x56353d27e7a0 2023-10-08T13:57:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.220+0000 7f1c0a6d4b80 10 client.26423 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.220+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/3734622578 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56353cfe5c60 con 0x56353d27e7a0 2023-10-08T13:57:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.220+0000 7f1bd77fe700 1 -- 172.21.15.57:0/3734622578 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56353d199eb0 con 0x56353d27e7a0 2023-10-08T13:57:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.220+0000 7f1bd77fe700 1 -- 172.21.15.57:0/3734622578 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f1be80b4300 con 0x56353d27e7a0 2023-10-08T13:57:36.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.220+0000 7f1c0a6d4b80 10 client.26423 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:36.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.220+0000 7f1c0a6d4b80 10 client.26423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:36.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.220+0000 7f1c0a6d4b80 20 client.26423 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:36.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.220+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56353d298d60 0x56353d29b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.220+0000 7f1c0a6d4b80 4 client.26423 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:57:36.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.220+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/3734622578 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56353d1cae10 con 0x56353d298d60 2023-10-08T13:57:36.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.221+0000 7f1bf7fff700 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56353d298d60 0x56353d29b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.221+0000 7f1bd77fe700 10 client.26423 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:36.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.221+0000 7f1bf7fff700 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56353d298d60 0x56353d29b180 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7a7dbc67c18895 server_cookie=6a8652c4072fee2b in_seq=0 out_seq=0 2023-10-08T13:57:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.224+0000 7f1bd77fe700 1 -- 172.21.15.57:0/3734622578 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56353d1cae10 con 0x56353d298d60 2023-10-08T13:57:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.224+0000 7f1bd77fe700 10 client.26423 handle_command_reply: tid=0 2023-10-08T13:57:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.224+0000 7f1c0a6d4b80 1 client.26423 shutdown 2023-10-08T13:57:36.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.229+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/3734622578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 msgr2=0x56353d2862d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:36.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.229+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d2862d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1be8000c00 tx=0x7f1be80acb10 comp rx=0 tx=0).stop 2023-10-08T13:57:36.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.239+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/3734622578 shutdown_connections 2023-10-08T13:57:36.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.239+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56353d298d60 0x56353d29b180 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.239+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d286890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.239+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d288f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.239+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/3734622578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d2862d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.240+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/3734622578 >> 172.21.15.57:0/3734622578 conn(0x56353d1f3260 msgr2=0x56353d279ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:36.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.240+0000 7f1bd77fe700 0 client.26423 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:36.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.240+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/3734622578 shutdown_connections 2023-10-08T13:57:36.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.240+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/3734622578 wait complete. 2023-10-08T13:57:36.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.243+0000 7f1c0a6d4b80 20 client.26423 trim_cache size 0 max 16384 2023-10-08T13:57:36.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.279+0000 7f1c0a6d4b80 1 Processor -- start 2023-10-08T13:57:36.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.292+0000 7f1c0a6d4b80 1 -- start start 2023-10-08T13:57:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.292+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d290240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.292+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d29ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.292+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d2815b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.292+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d052920 con 0x56353d289650 2023-10-08T13:57:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.292+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d199eb0 con 0x56353d27e7a0 2023-10-08T13:57:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.292+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d279be0 con 0x56353d27de80 2023-10-08T13:57:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.292+0000 7f1bf77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d29ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.292+0000 7f1bf77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d29ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60382/0 (socket says 172.21.15.57:60382) 2023-10-08T13:57:36.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.292+0000 7f1bf77fe700 1 -- 172.21.15.57:0/1713120415 learned_addr learned my addr 172.21.15.57:0/1713120415 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.293+0000 7f1bf7fff700 1 --2- 172.21.15.57:0/1713120415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d290240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.293+0000 7f1bf77fe700 1 -- 172.21.15.57:0/1713120415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 msgr2=0x56353d290240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.293+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/1713120415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d290240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.293+0000 7f1bf77fe700 1 -- 172.21.15.57:0/1713120415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 msgr2=0x56353d2815b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.293+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/1713120415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d2815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.293+0000 7f1bf77fe700 1 -- 172.21.15.57:0/1713120415 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56353d023250 con 0x56353d27e7a0 2023-10-08T13:57:36.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.296+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/1713120415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d29ad70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1be80a8890 tx=0x7f1be8005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c232fb403951b830 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:36.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.298+0000 7f1bdeffd700 1 -- 172.21.15.57:0/1713120415 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be80b00d0 con 0x56353d27e7a0 2023-10-08T13:57:36.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.298+0000 7f1bdeffd700 1 -- 172.21.15.57:0/1713120415 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be80a86d0 con 0x56353d27e7a0 2023-10-08T13:57:36.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.298+0000 7f1bdeffd700 1 -- 172.21.15.57:0/1713120415 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be80ab880 con 0x56353d27e7a0 2023-10-08T13:57:36.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.299+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1713120415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 msgr2=0x56353d29ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:36.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.299+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/1713120415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d29ad70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1be80a8890 tx=0x7f1be8005fb0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.307+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1713120415 shutdown_connections 2023-10-08T13:57:36.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.307+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/1713120415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d2815b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.307+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/1713120415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d290240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.307+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/1713120415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d29ad70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.307+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1713120415 >> 172.21.15.57:0/1713120415 conn(0x56353d1f3260 msgr2=0x56353d2794d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.308+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1713120415 shutdown_connections 2023-10-08T13:57:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.308+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1713120415 wait complete. 2023-10-08T13:57:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.309+0000 7f1c0a6d4b80 1 -- start start 2023-10-08T13:57:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.309+0000 7f1c0a6d4b80 1 Processor -- start 2023-10-08T13:57:36.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d28db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d28e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1c0a6d4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d28e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d2ebd40 con 0x56353d289650 2023-10-08T13:57:36.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d2ebeb0 con 0x56353d27e7a0 2023-10-08T13:57:36.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1c0a6d4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353d2ec070 con 0x56353d27de80 2023-10-08T13:57:36.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1c0a6d4b80 10 client.0 _ll_register_callbacks cb 0x7f1be80abed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:36.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d28db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d28db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59774/0 (socket says 172.21.15.57:59774) 2023-10-08T13:57:36.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1bf7fff700 1 -- 172.21.15.57:0/1195052159 learned_addr learned my addr 172.21.15.57:0/1195052159 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:36.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1bfca88700 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d28e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1bf7fff700 1 -- 172.21.15.57:0/1195052159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 msgr2=0x56353d28e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1bf7fff700 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d28e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1bf7fff700 1 -- 172.21.15.57:0/1195052159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 msgr2=0x56353d28e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.310+0000 7f1bf7fff700 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d28e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.311+0000 7f1bf7fff700 1 -- 172.21.15.57:0/1195052159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be80c9070 con 0x56353d27de80 2023-10-08T13:57:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.311+0000 7f1bf7fff700 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d28db60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1bec006f00 tx=0x7f1bec007360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41724d46689671f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:36.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.314+0000 7f1bd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:57:36.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.314+0000 7f1bd77fe700 1 -- 172.21.15.57:0/1195052159 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec023040 con 0x56353d27de80 2023-10-08T13:57:36.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.314+0000 7f1bd77fe700 1 -- 172.21.15.57:0/1195052159 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f1bec026290 con 0x56353d27de80 2023-10-08T13:57:36.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.314+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1195052159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bec012c10 con 0x56353d27de80 2023-10-08T13:57:36.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.326+0000 7f1bd77fe700 1 -- 172.21.15.57:0/1195052159 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bec0135c0 con 0x56353d27de80 2023-10-08T13:57:36.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.335+0000 7f1c0a6d4b80 10 client.26476 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:36.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.335+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1195052159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56353cffb2b0 con 0x56353d27de80 2023-10-08T13:57:36.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.335+0000 7f1bd77fe700 1 -- 172.21.15.57:0/1195052159 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bec012c10 con 0x56353d27de80 2023-10-08T13:57:36.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.336+0000 7f1bd77fe700 1 -- 172.21.15.57:0/1195052159 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f1bec02c020 con 0x56353d27de80 2023-10-08T13:57:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.336+0000 7f1c0a6d4b80 10 client.26476 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.336+0000 7f1c0a6d4b80 10 client.26476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.336+0000 7f1c0a6d4b80 20 client.26476 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.337+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56353d2ed840 0x56353d2efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.337+0000 7f1c0a6d4b80 4 client.26476 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T13:57:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.337+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1195052159 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56353d1cab30 con 0x56353d2ed840 2023-10-08T13:57:36.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.337+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56353d2ed840 0x56353d2efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:36.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.358+0000 7f1bd77fe700 10 client.26476 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:36.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.358+0000 7f1bf77fe700 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56353d2ed840 0x56353d2efc60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f19410f8999772f server_cookie=b7dca2360ccb4f65 in_seq=0 out_seq=0 2023-10-08T13:57:36.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.371+0000 7f1bd77fe700 1 -- 172.21.15.57:0/1195052159 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56353d1cab30 con 0x56353d2ed840 2023-10-08T13:57:36.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.371+0000 7f1bd77fe700 10 client.26476 handle_command_reply: tid=0 2023-10-08T13:57:36.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.372+0000 7f1c0a6d4b80 1 client.26476 shutdown 2023-10-08T13:57:36.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.402+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1195052159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 msgr2=0x56353d28db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:36.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.402+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d28db60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1bec006f00 tx=0x7f1bec007360 comp rx=0 tx=0).stop 2023-10-08T13:57:36.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.407+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1195052159 shutdown_connections 2023-10-08T13:57:36.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.407+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56353d2ed840 0x56353d2efc60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.407+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56353d289650 0x56353d28e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.407+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56353d27de80 0x56353d28db60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.407+0000 7f1c0a6d4b80 1 --2- 172.21.15.57:0/1195052159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56353d27e7a0 0x56353d28e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.407+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1195052159 >> 172.21.15.57:0/1195052159 conn(0x56353d1f3260 msgr2=0x56353d29ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:36.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.407+0000 7f1bd77fe700 0 client.26476 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:36.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.416+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1195052159 shutdown_connections 2023-10-08T13:57:36.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.416+0000 7f1c0a6d4b80 1 -- 172.21.15.57:0/1195052159 wait complete. 2023-10-08T13:57:36.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.416+0000 7f1c0a6d4b80 20 client.26476 trim_cache size 0 max 16384 2023-10-08T13:57:36.418 INFO:teuthology.orchestra.run.smithi057.stdout:[] 2023-10-08T13:57:36.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/749185414 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1be00a1730 msgr2=0x7f1be00a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:36.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1be00a1730 0x7f1be00a3bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009ed0 tx=0x7f1bf0009540 comp rx=0 tx=0).stop 2023-10-08T13:57:36.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/749185414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf810ef80 msgr2=0x7f1bf8115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:36.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf810ef80 0x7f1bf8115c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1be801a8f0 tx=0x7f1be8005800 comp rx=0 tx=0).stop 2023-10-08T13:57:36.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/749185414 shutdown_connections 2023-10-08T13:57:36.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bf8119fd0 0x7f1bf81a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1bf8107d30 0x7f1bf8115740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1be00a1730 0x7f1be00a3bf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 --2- 172.21.15.57:0/749185414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf810ef80 0x7f1bf8115c80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:36.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/749185414 >> 172.21.15.57:0/749185414 conn(0x7f1bf806b830 msgr2=0x7f1bf811a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:36.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.417+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/749185414 shutdown_connections 2023-10-08T13:57:36.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T13:57:36.418+0000 7f1bfe4eb700 1 -- 172.21.15.57:0/749185414 wait complete. 2023-10-08T13:57:36.526 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 1216618087, 'ino': 1099511628214, 'path': '/volumes/_nogroup/sv_1/9576dea1-484d-4e52-882d-96f1d9052725/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-1'}] 2023-10-08T13:57:36.527 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-10-08T13:57:37.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:37 smithi115 ceph-mon[104408]: pgmap v481: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 320 op/s 2023-10-08T13:57:37.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:37 smithi057 ceph-mon[99476]: pgmap v481: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 320 op/s 2023-10-08T13:57:38.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:37 smithi099 ceph-mon[103849]: pgmap v481: 129 pgs: 129 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 320 op/s 2023-10-08T13:57:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.162+0000 7f9dd6037700 1 -- 172.21.15.57:0/2268912341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd01010a0 msgr2=0x7f9dd010d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.162+0000 7f9dd6037700 1 --2- 172.21.15.57:0/2268912341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd01010a0 0x7f9dd010d5b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009f90 tx=0x7f9dc800b330 comp rx=0 tx=0).stop 2023-10-08T13:57:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.162+0000 7f9dd6037700 1 -- 172.21.15.57:0/2268912341 shutdown_connections 2023-10-08T13:57:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.162+0000 7f9dd6037700 1 --2- 172.21.15.57:0/2268912341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd01010a0 0x7f9dd010d5b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.162+0000 7f9dd6037700 1 --2- 172.21.15.57:0/2268912341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dd01066f0 0x7f9dd0106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.162+0000 7f9dd6037700 1 --2- 172.21.15.57:0/2268912341 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd01006e0 0x7f9dd0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.162+0000 7f9dd6037700 1 -- 172.21.15.57:0/2268912341 >> 172.21.15.57:0/2268912341 conn(0x7f9dd00fc1b0 msgr2=0x7f9dd00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.162+0000 7f9dd6037700 1 -- 172.21.15.57:0/2268912341 shutdown_connections 2023-10-08T13:57:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.162+0000 7f9dd6037700 1 -- 172.21.15.57:0/2268912341 wait complete. 2023-10-08T13:57:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.163+0000 7f9dd6037700 1 Processor -- start 2023-10-08T13:57:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.163+0000 7f9dd6037700 1 -- start start 2023-10-08T13:57:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dd6037700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dd01006e0 0x7f9dd019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dd6037700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd01010a0 0x7f9dd019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dd6037700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd01066f0 0x7f9dd0198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dd6037700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd0199180 con 0x7f9dd01010a0 2023-10-08T13:57:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dd6037700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd01992f0 con 0x7f9dd01066f0 2023-10-08T13:57:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dd6037700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd01994b0 con 0x7f9dd01006e0 2023-10-08T13:57:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd01010a0 0x7f9dd019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd01010a0 0x7f9dd019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58976/0 (socket says 172.21.15.57:58976) 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dceffd700 1 -- 172.21.15.57:0/2500432999 learned_addr learned my addr 172.21.15.57:0/2500432999 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dd01006e0 0x7f9dd019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dcffff700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd01066f0 0x7f9dd0198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dceffd700 1 -- 172.21.15.57:0/2500432999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dd01006e0 msgr2=0x7f9dd019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dceffd700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dd01006e0 0x7f9dd019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dceffd700 1 -- 172.21.15.57:0/2500432999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd01066f0 msgr2=0x7f9dd0198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dceffd700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd01066f0 0x7f9dd0198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.164+0000 7f9dceffd700 1 -- 172.21.15.57:0/2500432999 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc40095f0 con 0x7f9dd01010a0 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.165+0000 7f9dceffd700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd01010a0 0x7f9dd019ef30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400ec10 tx=0x7f9dc400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.165+0000 7f9dccff9700 1 -- 172.21.15.57:0/2500432999 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400cc90 con 0x7f9dd01010a0 2023-10-08T13:57:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.165+0000 7f9dccff9700 1 -- 172.21.15.57:0/2500432999 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc40044e0 con 0x7f9dd01010a0 2023-10-08T13:57:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.165+0000 7f9dccff9700 1 -- 172.21.15.57:0/2500432999 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4010430 con 0x7f9dd01010a0 2023-10-08T13:57:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.165+0000 7f9dd6037700 1 -- 172.21.15.57:0/2500432999 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc8009c40 con 0x7f9dd01010a0 2023-10-08T13:57:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.165+0000 7f9dd6037700 1 -- 172.21.15.57:0/2500432999 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd01a5800 con 0x7f9dd01010a0 2023-10-08T13:57:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.167+0000 7f9dccff9700 1 -- 172.21.15.57:0/2500432999 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9dc4003fb0 con 0x7f9dd01010a0 2023-10-08T13:57:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.168+0000 7f9dccff9700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9db80a1670 0x7f9db80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.169+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9db80a1670 0x7f9db80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.169+0000 7f9dccff9700 1 -- 172.21.15.57:0/2500432999 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9dc4014070 con 0x7f9dd01010a0 2023-10-08T13:57:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.169+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9db80a1670 0x7f9db80a3b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009be0 tx=0x7f9dc0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9de2220b80 1 Processor -- start 2023-10-08T13:57:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9de2220b80 1 -- start start 2023-10-08T13:57:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf9396c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf9390b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93910b0 0x55daf9391510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9de2220b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf92f9400 con 0x55daf9396880 2023-10-08T13:57:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9de2220b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf92eabc0 con 0x55daf93910b0 2023-10-08T13:57:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9de2220b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf92e84e0 con 0x55daf93971a0 2023-10-08T13:57:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9dcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf9396c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9dcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf9396c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58982/0 (socket says 172.21.15.57:58982) 2023-10-08T13:57:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.170+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3596727265 learned_addr learned my addr 172.21.15.57:0/3596727265 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9dcffff700 1 --2- 172.21.15.57:0/3596727265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93910b0 0x55daf9391510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9dceffd700 1 --2- 172.21.15.57:0/3596727265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf9390b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3596727265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 msgr2=0x55daf9390b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/3596727265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf9390b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3596727265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93910b0 msgr2=0x55daf9391510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/3596727265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93910b0 0x55daf9391510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3596727265 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daf910ed80 con 0x55daf9396880 2023-10-08T13:57:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9dcffff700 1 --2- 172.21.15.57:0/3596727265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93910b0 0x55daf9391510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/3596727265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf9396c60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0011d70 tx=0x7f9dc0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d210261d7c2f84fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9db5ffb700 1 -- 172.21.15.57:0/3596727265 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0011da0 con 0x55daf9396880 2023-10-08T13:57:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9db5ffb700 1 -- 172.21.15.57:0/3596727265 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc0004020 con 0x55daf9396880 2023-10-08T13:57:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.171+0000 7f9db5ffb700 1 -- 172.21.15.57:0/3596727265 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0003cb0 con 0x55daf9396880 2023-10-08T13:57:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.172+0000 7f9de2220b80 1 -- 172.21.15.57:0/3596727265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 msgr2=0x55daf9396c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.172+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3596727265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf9396c60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0011d70 tx=0x7f9dc0014000 comp rx=0 tx=0).stop 2023-10-08T13:57:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.172+0000 7f9de2220b80 1 -- 172.21.15.57:0/3596727265 shutdown_connections 2023-10-08T13:57:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.172+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3596727265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93910b0 0x55daf9391510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.172+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3596727265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf9396c60 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.172+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3596727265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf9390b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.172+0000 7f9de2220b80 1 -- 172.21.15.57:0/3596727265 >> 172.21.15.57:0/3596727265 conn(0x55daf9305200 msgr2=0x55daf938bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.172+0000 7f9de2220b80 1 -- 172.21.15.57:0/3596727265 shutdown_connections 2023-10-08T13:57:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.172+0000 7f9de2220b80 1 -- 172.21.15.57:0/3596727265 wait complete. 2023-10-08T13:57:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.172+0000 7f9de2220b80 1 -- start start 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9de2220b80 1 Processor -- start 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf939a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 0x55daf939aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf939afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9de2220b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf92e3dd0 con 0x55daf93910b0 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9de2220b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf92806b0 con 0x55daf9396880 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9de2220b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf9280350 con 0x55daf93971a0 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9de2220b80 10 client.0 _ll_register_callbacks cb 0x55daf90dd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf939a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf939a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58988/0 (socket says 172.21.15.57:58988) 2023-10-08T13:57:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3497710570 learned_addr learned my addr 172.21.15.57:0/3497710570 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dceffd700 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 0x55daf939aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dcffff700 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf939afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3497710570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 msgr2=0x55daf939afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf939afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3497710570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 msgr2=0x55daf939aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 0x55daf939aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3497710570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd010deb0 con 0x55daf93910b0 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.173+0000 7f9dceffd700 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 0x55daf939aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf939a530 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009210 tx=0x7f9dc000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e4f413bd17918d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9daeffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9daeffd700 1 -- 172.21.15.57:0/3497710570 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0003c60 con 0x55daf93910b0 2023-10-08T13:57:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9daeffd700 1 -- 172.21.15.57:0/3497710570 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9dc00056a0 con 0x55daf93910b0 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9de2220b80 1 -- 172.21.15.57:0/3497710570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55daf92abe50 con 0x55daf93910b0 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9daeffd700 1 -- 172.21.15.57:0/3497710570 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dc0003dc0 con 0x55daf93910b0 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9de2220b80 10 client.16881 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9de2220b80 1 -- 172.21.15.57:0/3497710570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f9dc0017070 con 0x55daf93910b0 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9daeffd700 1 -- 172.21.15.57:0/3497710570 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55daf92abe50 con 0x55daf93910b0 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9daeffd700 1 -- 172.21.15.57:0/3497710570 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f9dc0024030 con 0x55daf93910b0 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9de2220b80 10 client.16881 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.174+0000 7f9de2220b80 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.175+0000 7f9de2220b80 20 client.16881 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.175+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55daf93aac30 0x55daf93ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.175+0000 7f9de2220b80 4 client.16881 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.175+0000 7f9de2220b80 1 -- 172.21.15.57:0/3497710570 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daf92dcdb0 con 0x55daf93aac30 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.175+0000 7f9dceffd700 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55daf93aac30 0x55daf93ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.175+0000 7f9daeffd700 10 client.16881 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.175+0000 7f9dceffd700 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55daf93aac30 0x55daf93ad050 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801f5f9016fa9e97 server_cookie=d967ca87f4a792ce in_seq=0 out_seq=0 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.176+0000 7f9daeffd700 1 -- 172.21.15.57:0/3497710570 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55daf92dcdb0 con 0x55daf93aac30 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.176+0000 7f9daeffd700 10 client.16881 handle_command_reply: tid=0 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.176+0000 7f9de2220b80 1 client.16881 shutdown 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 1 -- 172.21.15.57:0/3497710570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 msgr2=0x55daf939a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf939a530 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009210 tx=0x7f9dc000ed50 comp rx=0 tx=0).stop 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 1 -- 172.21.15.57:0/3497710570 shutdown_connections 2023-10-08T13:57:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55daf93aac30 0x55daf93ad050 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf939afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9daeffd700 0 client.16881 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf939a530 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3497710570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 0x55daf939aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 1 -- 172.21.15.57:0/3497710570 >> 172.21.15.57:0/3497710570 conn(0x55daf9305200 msgr2=0x55daf938b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 1 -- 172.21.15.57:0/3497710570 shutdown_connections 2023-10-08T13:57:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 1 -- 172.21.15.57:0/3497710570 wait complete. 2023-10-08T13:57:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.177+0000 7f9de2220b80 20 client.16881 trim_cache size 0 max 16384 2023-10-08T13:57:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9de2220b80 1 Processor -- start 2023-10-08T13:57:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9de2220b80 1 -- start start 2023-10-08T13:57:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf93a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 0x55daf93a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf939add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9de2220b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf91648c0 con 0x55daf93910b0 2023-10-08T13:57:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9de2220b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf938d100 con 0x55daf9396880 2023-10-08T13:57:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9de2220b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf938ca40 con 0x55daf93971a0 2023-10-08T13:57:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9dcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf93a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9dcf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf93a6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58996/0 (socket says 172.21.15.57:58996) 2023-10-08T13:57:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3985178795 learned_addr learned my addr 172.21.15.57:0/3985178795 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9dceffd700 1 --2- 172.21.15.57:0/3985178795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 0x55daf93a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.192+0000 7f9dcffff700 1 --2- 172.21.15.57:0/3985178795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf939add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3985178795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 msgr2=0x55daf939add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/3985178795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf939add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3985178795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 msgr2=0x55daf93a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/3985178795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 0x55daf93a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/3985178795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daf90f7c00 con 0x55daf93910b0 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9dcffff700 1 --2- 172.21.15.57:0/3985178795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf939add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/3985178795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf93a6d40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00264e0 tx=0x7f9dc0009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab2559d995e0db7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9db67fc700 1 -- 172.21.15.57:0/3985178795 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00268d0 con 0x55daf93910b0 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9db67fc700 1 -- 172.21.15.57:0/3985178795 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc000e970 con 0x55daf93910b0 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9db67fc700 1 -- 172.21.15.57:0/3985178795 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0005330 con 0x55daf93910b0 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9de2220b80 1 -- 172.21.15.57:0/3985178795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 msgr2=0x55daf93a6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3985178795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf93a6d40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00264e0 tx=0x7f9dc0009350 comp rx=0 tx=0).stop 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.193+0000 7f9de2220b80 1 -- 172.21.15.57:0/3985178795 shutdown_connections 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3985178795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93971a0 0x55daf939add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3985178795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf93910b0 0x55daf93a6d40 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 --2- 172.21.15.57:0/3985178795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf9396880 0x55daf93a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 -- 172.21.15.57:0/3985178795 >> 172.21.15.57:0/3985178795 conn(0x55daf9305200 msgr2=0x55daf9391490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 -- 172.21.15.57:0/3985178795 shutdown_connections 2023-10-08T13:57:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 -- 172.21.15.57:0/3985178795 wait complete. 2023-10-08T13:57:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 -- start start 2023-10-08T13:57:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 Processor -- start 2023-10-08T13:57:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93910b0 0x55daf938baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf939dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93971a0 0x55daf939e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf939e640 con 0x55daf9396880 2023-10-08T13:57:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf939e7b0 con 0x55daf93971a0 2023-10-08T13:57:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf939e970 con 0x55daf93910b0 2023-10-08T13:57:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9de2220b80 10 client.0 _ll_register_callbacks cb 0x7f9dc40091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9dceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf939dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.194+0000 7f9dcf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93910b0 0x55daf938baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9dceffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf939dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59004/0 (socket says 172.21.15.57:59004) 2023-10-08T13:57:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9dceffd700 1 -- 172.21.15.57:0/287705494 learned_addr learned my addr 172.21.15.57:0/287705494 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9dcffff700 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93971a0 0x55daf939e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/287705494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93971a0 msgr2=0x55daf939e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93971a0 0x55daf939e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/287705494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 msgr2=0x55daf939dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf939dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9dcf7fe700 1 -- 172.21.15.57:0/287705494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc0017070 con 0x55daf93910b0 2023-10-08T13:57:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9dceffd700 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf939dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:57:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9dcf7fe700 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93910b0 0x55daf938baa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0016040 tx=0x7f9dc0009950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=828251a569755a1b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9daeffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:57:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9daeffd700 1 -- 172.21.15.57:0/287705494 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0003a60 con 0x55daf93910b0 2023-10-08T13:57:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9daeffd700 1 -- 172.21.15.57:0/287705494 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f9dc0026ca0 con 0x55daf93910b0 2023-10-08T13:57:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9de2220b80 1 -- 172.21.15.57:0/287705494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9dc0004020 con 0x55daf93910b0 2023-10-08T13:57:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9daeffd700 1 -- 172.21.15.57:0/287705494 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dc0026e80 con 0x55daf93910b0 2023-10-08T13:57:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9de2220b80 10 client.26491 fetch_fsmap learned FSMap version 42 2023-10-08T13:57:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.195+0000 7f9de2220b80 1 -- 172.21.15.57:0/287705494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55daf91351f0 con 0x55daf93910b0 2023-10-08T13:57:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9daeffd700 1 -- 172.21.15.57:0/287705494 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc0004020 con 0x55daf93910b0 2023-10-08T13:57:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9daeffd700 1 -- 172.21.15.57:0/287705494 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f9dc003e070 con 0x55daf93910b0 2023-10-08T13:57:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9de2220b80 10 client.26491 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T13:57:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9de2220b80 10 client.26491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9de2220b80 20 client.26491 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9de2220b80 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55daf93febd0 0x55daf9400ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9de2220b80 4 client.26491 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:57:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9de2220b80 1 -- 172.21.15.57:0/287705494 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55daf92dcad0 con 0x55daf93febd0 2023-10-08T13:57:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9dceffd700 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55daf93febd0 0x55daf9400ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9daeffd700 10 client.26491 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:39.196+0000 7f9dceffd700 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55daf93febd0 0x55daf9400ff0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16aff44c18dfdae4 server_cookie=d4223d60d751cb20 in_seq=0 out_seq=0 2023-10-08T13:57:39.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:39 smithi115 ceph-mon[104408]: pgmap v482: 129 pgs: 129 active+clean; 1.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 327 op/s 2023-10-08T13:57:39.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:39 smithi057 ceph-mon[99476]: pgmap v482: 129 pgs: 129 active+clean; 1.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 327 op/s 2023-10-08T13:57:39.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:39 smithi099 ceph-mon[103849]: pgmap v482: 129 pgs: 129 active+clean; 1.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 327 op/s 2023-10-08T13:57:41.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:41 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:57:41.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:41 smithi057 ceph-mon[99476]: pgmap v483: 129 pgs: 129 active+clean; 1.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.0 MiB/s wr, 588 op/s 2023-10-08T13:57:41.530 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:41 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:57:41.530 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:41 smithi099 ceph-mon[103849]: pgmap v483: 129 pgs: 129 active+clean; 1.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.0 MiB/s wr, 588 op/s 2023-10-08T13:57:41.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:41 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:57:41.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:41 smithi115 ceph-mon[104408]: pgmap v483: 129 pgs: 129 active+clean; 1.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.0 MiB/s wr, 588 op/s 2023-10-08T13:57:43.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:42 smithi099 ceph-mon[103849]: pgmap v484: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.2 MiB/s wr, 791 op/s 2023-10-08T13:57:43.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:42 smithi115 ceph-mon[104408]: pgmap v484: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.2 MiB/s wr, 791 op/s 2023-10-08T13:57:43.275 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:42 smithi057 ceph-mon[99476]: pgmap v484: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.2 MiB/s wr, 791 op/s 2023-10-08T13:57:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.043+0000 7f9daeffd700 1 -- 172.21.15.57:0/287705494 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55daf92dcad0 con 0x55daf93febd0 2023-10-08T13:57:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.043+0000 7f9daeffd700 10 client.26491 handle_command_reply: tid=0 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.043+0000 7f9de2220b80 1 client.26491 shutdown 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 1 -- 172.21.15.57:0/287705494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93910b0 msgr2=0x55daf938baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93910b0 0x55daf938baa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0016040 tx=0x7f9dc0009950 comp rx=0 tx=0).stop 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 1 -- 172.21.15.57:0/287705494 shutdown_connections 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55daf93febd0 0x55daf9400ff0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf93971a0 0x55daf939e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9daeffd700 0 client.26491 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55daf93910b0 0x55daf938baa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 1 --2- 172.21.15.57:0/287705494 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55daf9396880 0x55daf939dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 1 -- 172.21.15.57:0/287705494 >> 172.21.15.57:0/287705494 conn(0x55daf9305200 msgr2=0x55daf9391490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 1 -- 172.21.15.57:0/287705494 shutdown_connections 2023-10-08T13:57:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 1 -- 172.21.15.57:0/287705494 wait complete. 2023-10-08T13:57:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9de2220b80 20 client.26491 trim_cache size 0 max 16384 2023-10-08T13:57:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:57:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:57:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:57:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:57:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9dd6037700 1 -- 172.21.15.57:0/2500432999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9db80a1670 msgr2=0x7f9db80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9dd6037700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9db80a1670 0x7f9db80a3b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009be0 tx=0x7f9dc0009260 comp rx=0 tx=0).stop 2023-10-08T13:57:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9dd6037700 1 -- 172.21.15.57:0/2500432999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd01010a0 msgr2=0x7f9dd019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.044+0000 7f9dd6037700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd01010a0 0x7f9dd019ef30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400ec10 tx=0x7f9dc400ef20 comp rx=0 tx=0).stop 2023-10-08T13:57:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.045+0000 7f9dd6037700 1 -- 172.21.15.57:0/2500432999 shutdown_connections 2023-10-08T13:57:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.045+0000 7f9dd6037700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9db80a1670 0x7f9db80a3b30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.045+0000 7f9dd6037700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd01066f0 0x7f9dd0198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.045+0000 7f9dd6037700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dd01006e0 0x7f9dd019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.045+0000 7f9dd6037700 1 --2- 172.21.15.57:0/2500432999 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9dd01010a0 0x7f9dd019ef30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.045+0000 7f9dd6037700 1 -- 172.21.15.57:0/2500432999 >> 172.21.15.57:0/2500432999 conn(0x7f9dd00fc1b0 msgr2=0x7f9dd0072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.045+0000 7f9dd6037700 1 -- 172.21.15.57:0/2500432999 shutdown_connections 2023-10-08T13:57:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:44.045+0000 7f9dd6037700 1 -- 172.21.15.57:0/2500432999 wait complete. 2023-10-08T13:57:44.421 INFO:tasks.workunit.client.0.smithi057.stdout: 498 2956884 561 429883 526 116453 298 2023-10-08T13:57:45.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:45 smithi115 ceph-mon[104408]: pgmap v485: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.2 MiB/s wr, 1.55k op/s 2023-10-08T13:57:45.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:45 smithi057 ceph-mon[99476]: pgmap v485: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.2 MiB/s wr, 1.55k op/s 2023-10-08T13:57:46.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:45 smithi099 ceph-mon[103849]: pgmap v485: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.2 MiB/s wr, 1.55k op/s 2023-10-08T13:57:47.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:47 smithi115 ceph-mon[104408]: pgmap v486: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.4 MiB/s wr, 1.80k op/s 2023-10-08T13:57:47.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:47 smithi057 ceph-mon[99476]: pgmap v486: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.4 MiB/s wr, 1.80k op/s 2023-10-08T13:57:48.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:47 smithi099 ceph-mon[103849]: pgmap v486: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.4 MiB/s wr, 1.80k op/s 2023-10-08T13:57:49.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:49 smithi115 ceph-mon[104408]: pgmap v487: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 1.82k op/s 2023-10-08T13:57:49.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:49 smithi057 ceph-mon[99476]: pgmap v487: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 1.82k op/s 2023-10-08T13:57:49.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:49 smithi099 ceph-mon[103849]: pgmap v487: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 1.82k op/s 2023-10-08T13:57:51.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:51 smithi115 ceph-mon[104408]: pgmap v488: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 1.79k op/s 2023-10-08T13:57:51.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:51 smithi057 ceph-mon[99476]: pgmap v488: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 1.79k op/s 2023-10-08T13:57:52.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:51 smithi099 ceph-mon[103849]: pgmap v488: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 1.79k op/s 2023-10-08T13:57:53.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:53 smithi115 ceph-mon[104408]: pgmap v489: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 1.51k op/s 2023-10-08T13:57:53.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:53 smithi057 ceph-mon[99476]: pgmap v489: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 1.51k op/s 2023-10-08T13:57:54.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:53 smithi099 ceph-mon[103849]: pgmap v489: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 1.51k op/s 2023-10-08T13:57:55.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:55 smithi115 ceph-mon[104408]: pgmap v490: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 1.21k op/s 2023-10-08T13:57:55.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:55 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:57:55.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:55 smithi057 ceph-mon[99476]: pgmap v490: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 1.21k op/s 2023-10-08T13:57:55.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:55 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:57:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:55 smithi099 ceph-mon[103849]: pgmap v490: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 1.21k op/s 2023-10-08T13:57:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:55 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:57:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.169+0000 7f8c733d2700 1 -- 172.21.15.57:0/1099504298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c6c1000d0 msgr2=0x7f8c6c1004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.169+0000 7f8c733d2700 1 --2- 172.21.15.57:0/1099504298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c6c1000d0 0x7f8c6c1004b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009990 tx=0x7f8c68009ca0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.169+0000 7f8c733d2700 1 -- 172.21.15.57:0/1099504298 shutdown_connections 2023-10-08T13:57:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.169+0000 7f8c733d2700 1 --2- 172.21.15.57:0/1099504298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c6c10d140 0x7f8c6c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.169+0000 7f8c733d2700 1 --2- 172.21.15.57:0/1099504298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c6c1000d0 0x7f8c6c1004b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.169+0000 7f8c733d2700 1 --2- 172.21.15.57:0/1099504298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c6c1009f0 0x7f8c6c10cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.169+0000 7f8c733d2700 1 -- 172.21.15.57:0/1099504298 >> 172.21.15.57:0/1099504298 conn(0x7f8c6c076790 msgr2=0x7f8c6c076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.169+0000 7f8c733d2700 1 -- 172.21.15.57:0/1099504298 shutdown_connections 2023-10-08T13:57:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.169+0000 7f8c733d2700 1 -- 172.21.15.57:0/1099504298 wait complete. 2023-10-08T13:57:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c733d2700 1 Processor -- start 2023-10-08T13:57:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c733d2700 1 -- start start 2023-10-08T13:57:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c733d2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c6c1000d0 0x7f8c6c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c733d2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c6c1009f0 0x7f8c6c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c733d2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c6c10d140 0x7f8c6c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c733d2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c19dcd0 con 0x7f8c6c1009f0 2023-10-08T13:57:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c733d2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c19de40 con 0x7f8c6c10d140 2023-10-08T13:57:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c733d2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c19e000 con 0x7f8c6c1000d0 2023-10-08T13:57:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c7096d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c6c1009f0 0x7f8c6c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c7096d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c6c1009f0 0x7f8c6c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59018/0 (socket says 172.21.15.57:59018) 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c7096d700 1 -- 172.21.15.57:0/1265959890 learned_addr learned my addr 172.21.15.57:0/1265959890 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.170+0000 7f8c7116e700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c6c1000d0 0x7f8c6c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c7196f700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c6c10d140 0x7f8c6c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c7096d700 1 -- 172.21.15.57:0/1265959890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c6c1000d0 msgr2=0x7f8c6c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c7096d700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c6c1000d0 0x7f8c6c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c7096d700 1 -- 172.21.15.57:0/1265959890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c6c10d140 msgr2=0x7f8c6c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c7096d700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c6c10d140 0x7f8c6c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c7096d700 1 -- 172.21.15.57:0/1265959890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c680095f0 con 0x7f8c6c1009f0 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c7096d700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c6c1009f0 0x7f8c6c1992f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00d980 tx=0x7f8c5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c627fc700 1 -- 172.21.15.57:0/1265959890 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c00b950 con 0x7f8c6c1009f0 2023-10-08T13:57:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c627fc700 1 -- 172.21.15.57:0/1265959890 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c5c007620 con 0x7f8c6c1009f0 2023-10-08T13:57:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c627fc700 1 -- 172.21.15.57:0/1265959890 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c00f3c0 con 0x7f8c6c1009f0 2023-10-08T13:57:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c733d2700 1 -- 172.21.15.57:0/1265959890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c19e2e0 con 0x7f8c6c1009f0 2023-10-08T13:57:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.171+0000 7f8c733d2700 1 -- 172.21.15.57:0/1265959890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c1a5b20 con 0x7f8c6c1009f0 2023-10-08T13:57:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.174+0000 7f8c627fc700 1 -- 172.21.15.57:0/1265959890 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8c5c010640 con 0x7f8c6c1009f0 2023-10-08T13:57:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.174+0000 7f8c627fc700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8c580a15f0 0x7f8c580a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.174+0000 7f8c627fc700 1 -- 172.21.15.57:0/1265959890 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8c5c0cd230 con 0x7f8c6c1009f0 2023-10-08T13:57:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.174+0000 7f8c7116e700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8c580a15f0 0x7f8c580a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.175+0000 7f8c7116e700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8c580a15f0 0x7f8c580a3ab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800b2e0 tx=0x7f8c68005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.175+0000 7f8c7f5bbb80 1 Processor -- start 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.175+0000 7f8c7f5bbb80 1 -- start start 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.175+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d23a880 0x55e38d23ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.175+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23b1a0 0x55e38d234b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.175+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d2350b0 0x55e38d235510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.175+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d19d400 con 0x55e38d23b1a0 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.175+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d18ebc0 con 0x55e38d23a880 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.175+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d18c4e0 con 0x55e38d2350b0 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7096d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23b1a0 0x55e38d234b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7096d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23b1a0 0x55e38d234b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59026/0 (socket says 172.21.15.57:59026) 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7096d700 1 -- 172.21.15.57:0/1708209307 learned_addr learned my addr 172.21.15.57:0/1708209307 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7116e700 1 --2- 172.21.15.57:0/1708209307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d23a880 0x55e38d23ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7196f700 1 --2- 172.21.15.57:0/1708209307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d2350b0 0x55e38d235510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7096d700 1 -- 172.21.15.57:0/1708209307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d2350b0 msgr2=0x55e38d235510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7096d700 1 --2- 172.21.15.57:0/1708209307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d2350b0 0x55e38d235510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7096d700 1 -- 172.21.15.57:0/1708209307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d23a880 msgr2=0x55e38d23ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7096d700 1 --2- 172.21.15.57:0/1708209307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d23a880 0x55e38d23ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7096d700 1 -- 172.21.15.57:0/1708209307 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e38cfb2d80 con 0x55e38d23b1a0 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c7096d700 1 --2- 172.21.15.57:0/1708209307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23b1a0 0x55e38d234b70 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c096410 tx=0x7f8c5c010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa730e2225813e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c577fe700 1 -- 172.21.15.57:0/1708209307 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c09e060 con 0x55e38d23b1a0 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c577fe700 1 -- 172.21.15.57:0/1708209307 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c5c0a22a0 con 0x55e38d23b1a0 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.176+0000 7f8c577fe700 1 -- 172.21.15.57:0/1708209307 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c099660 con 0x55e38d23b1a0 2023-10-08T13:57:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.177+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/1708209307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23b1a0 msgr2=0x55e38d234b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.177+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/1708209307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23b1a0 0x55e38d234b70 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c096410 tx=0x7f8c5c010530 comp rx=0 tx=0).stop 2023-10-08T13:57:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.177+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/1708209307 shutdown_connections 2023-10-08T13:57:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.177+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/1708209307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d2350b0 0x55e38d235510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.177+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/1708209307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d23a880 0x55e38d23ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.177+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/1708209307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23b1a0 0x55e38d234b70 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.177+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/1708209307 >> 172.21.15.57:0/1708209307 conn(0x55e38d1a9200 msgr2=0x55e38d22fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.177+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/1708209307 shutdown_connections 2023-10-08T13:57:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.177+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/1708209307 wait complete. 2023-10-08T13:57:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.177+0000 7f8c7f5bbb80 1 -- start start 2023-10-08T13:57:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7f5bbb80 1 Processor -- start 2023-10-08T13:57:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d2350b0 0x55e38d23e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d23e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d23ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d187dd0 con 0x55e38d23a880 2023-10-08T13:57:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d1246b0 con 0x55e38d2350b0 2023-10-08T13:57:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d124350 con 0x55e38d23b1a0 2023-10-08T13:57:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7f5bbb80 10 client.0 _ll_register_callbacks cb 0x55e38cf817a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7096d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d23e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7096d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d23e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59032/0 (socket says 172.21.15.57:59032) 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7096d700 1 -- 172.21.15.57:0/2226262167 learned_addr learned my addr 172.21.15.57:0/2226262167 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7196f700 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d23ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7116e700 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d2350b0 0x55e38d23e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.178+0000 7f8c7096d700 1 -- 172.21.15.57:0/2226262167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 msgr2=0x55e38d23ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c7096d700 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d23ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c7096d700 1 -- 172.21.15.57:0/2226262167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d2350b0 msgr2=0x55e38d23e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c7096d700 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d2350b0 0x55e38d23e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c7096d700 1 -- 172.21.15.57:0/2226262167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c1a5b20 con 0x55e38d23a880 2023-10-08T13:57:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c7096d700 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d23e9f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0cac00 tx=0x7f8c5c0ca690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359c5423fe3f78b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2226262167 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c099610 con 0x55e38d23a880 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2226262167 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8c5c0b8120 con 0x55e38d23a880 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2226262167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e38d14fe50 con 0x55e38d23a880 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2226262167 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c5c065610 con 0x55e38d23a880 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c7f5bbb80 10 client.16899 fetch_fsmap learned FSMap version 44 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2226262167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55e38cf9bc00 con 0x55e38d23a880 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2226262167 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e38d14fe50 con 0x55e38d23a880 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.179+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2226262167 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 44) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f8c5c0a1e10 con 0x55e38d23a880 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.180+0000 7f8c7f5bbb80 10 client.16899 fetch_fsmap finished waiting for FSMap version 44 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.180+0000 7f8c7f5bbb80 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.180+0000 7f8c7f5bbb80 20 client.16899 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.180+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e38d24e7d0 0x55e38d250bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.180+0000 7f8c7f5bbb80 4 client.16899 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.180+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2226262167 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e38d180db0 con 0x55e38d24e7d0 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.180+0000 7f8c7116e700 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e38d24e7d0 0x55e38d250bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.181+0000 7f8c54ff9700 10 client.16899 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.181+0000 7f8c7116e700 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e38d24e7d0 0x55e38d250bf0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d69b7cd79d53f1 server_cookie=167aa8ec43e63293 in_seq=0 out_seq=0 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2226262167 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e38d180db0 con 0x55e38d24e7d0 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c54ff9700 10 client.16899 handle_command_reply: tid=0 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 client.16899 shutdown 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2226262167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 msgr2=0x55e38d23e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d23e9f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0cac00 tx=0x7f8c5c0ca690 comp rx=0 tx=0).stop 2023-10-08T13:57:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2226262167 shutdown_connections 2023-10-08T13:57:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e38d24e7d0 0x55e38d250bf0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d23ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c54ff9700 0 client.16899 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d2350b0 0x55e38d23e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2226262167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d23e9f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2226262167 >> 172.21.15.57:0/2226262167 conn(0x55e38d1a9200 msgr2=0x55e38d22f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2226262167 shutdown_connections 2023-10-08T13:57:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2226262167 wait complete. 2023-10-08T13:57:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.182+0000 7f8c7f5bbb80 20 client.16899 trim_cache size 0 max 16384 2023-10-08T13:57:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.199+0000 7f8c7f5bbb80 1 Processor -- start 2023-10-08T13:57:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.199+0000 7f8c7f5bbb80 1 -- start start 2023-10-08T13:57:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.199+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d2350b0 0x55e38d234b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.199+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d250bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.199+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d23ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.199+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d0088c0 con 0x55e38d23a880 2023-10-08T13:57:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.199+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d23fb20 con 0x55e38d2350b0 2023-10-08T13:57:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.199+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d23afe0 con 0x55e38d23b1a0 2023-10-08T13:57:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.199+0000 7f8c7096d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d250bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c7096d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d250bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59038/0 (socket says 172.21.15.57:59038) 2023-10-08T13:57:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c7096d700 1 -- 172.21.15.57:0/3693310893 learned_addr learned my addr 172.21.15.57:0/3693310893 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c7116e700 1 --2- 172.21.15.57:0/3693310893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d2350b0 0x55e38d234b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c7196f700 1 --2- 172.21.15.57:0/3693310893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d23ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c7096d700 1 -- 172.21.15.57:0/3693310893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 msgr2=0x55e38d23ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c7096d700 1 --2- 172.21.15.57:0/3693310893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d23ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c7096d700 1 -- 172.21.15.57:0/3693310893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d2350b0 msgr2=0x55e38d234b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c7096d700 1 --2- 172.21.15.57:0/3693310893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d2350b0 0x55e38d234b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c7096d700 1 -- 172.21.15.57:0/3693310893 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e38cfd91f0 con 0x55e38d23a880 2023-10-08T13:57:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c7096d700 1 --2- 172.21.15.57:0/3693310893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d250bc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c096980 tx=0x7f8c5c096380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12645045a613191b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c57fff700 1 -- 172.21.15.57:0/3693310893 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c099a50 con 0x55e38d23a880 2023-10-08T13:57:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c57fff700 1 -- 172.21.15.57:0/3693310893 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c5c010460 con 0x55e38d23a880 2023-10-08T13:57:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.200+0000 7f8c57fff700 1 -- 172.21.15.57:0/3693310893 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c098340 con 0x55e38d23a880 2023-10-08T13:57:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/3693310893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 msgr2=0x55e38d250bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/3693310893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d250bc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c096980 tx=0x7f8c5c096380 comp rx=0 tx=0).stop 2023-10-08T13:57:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/3693310893 shutdown_connections 2023-10-08T13:57:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/3693310893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d23ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/3693310893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d2350b0 0x55e38d234b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/3693310893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d23a880 0x55e38d250bc0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/3693310893 >> 172.21.15.57:0/3693310893 conn(0x55e38d1a9200 msgr2=0x55e38d235490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/3693310893 shutdown_connections 2023-10-08T13:57:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/3693310893 wait complete. 2023-10-08T13:57:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 -- start start 2023-10-08T13:57:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.201+0000 7f8c7f5bbb80 1 Processor -- start 2023-10-08T13:57:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d2350b0 0x55e38d245ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d23a880 0x55e38d243ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7f5bbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d243ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d244530 con 0x55e38d2350b0 2023-10-08T13:57:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d2446a0 con 0x55e38d23a880 2023-10-08T13:57:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7f5bbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38d244860 con 0x55e38d23b1a0 2023-10-08T13:57:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7f5bbb80 10 client.0 _ll_register_callbacks cb 0x7f8c68005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:57:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7116e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d2350b0 0x55e38d245ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7116e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d2350b0 0x55e38d245ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59042/0 (socket says 172.21.15.57:59042) 2023-10-08T13:57:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7116e700 1 -- 172.21.15.57:0/2109803122 learned_addr learned my addr 172.21.15.57:0/2109803122 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:57:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7196f700 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d243ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7096d700 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d23a880 0x55e38d243ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7116e700 1 -- 172.21.15.57:0/2109803122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 msgr2=0x55e38d243ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7116e700 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d243ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7116e700 1 -- 172.21.15.57:0/2109803122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d23a880 msgr2=0x55e38d243ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:57:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7116e700 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d23a880 0x55e38d243ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.202+0000 7f8c7116e700 1 -- 172.21.15.57:0/2109803122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e38cf9bc00 con 0x55e38d2350b0 2023-10-08T13:57:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.203+0000 7f8c7116e700 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d2350b0 0x55e38d245ad0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009990 tx=0x7f8c68009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc65554ecff5eade server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:57:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.203+0000 7f8c54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:57:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.203+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2109803122 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800bee0 con 0x55e38d2350b0 2023-10-08T13:57:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.203+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2109803122 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9293+0+0 (secure 0 0 0) 0x7f8c68035ca0 con 0x55e38d2350b0 2023-10-08T13:57:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.203+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2109803122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c5c0a22a0 con 0x55e38d2350b0 2023-10-08T13:57:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.203+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2109803122 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c68035e60 con 0x55e38d2350b0 2023-10-08T13:57:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.203+0000 7f8c7f5bbb80 10 client.16905 fetch_fsmap learned FSMap version 44 2023-10-08T13:57:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.203+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2109803122 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55e38cfb1250 con 0x55e38d2350b0 2023-10-08T13:57:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.203+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2109803122 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c5c0a22a0 con 0x55e38d2350b0 2023-10-08T13:57:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.203+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2109803122 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 44) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f8c6803b020 con 0x55e38d2350b0 2023-10-08T13:57:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.204+0000 7f8c7f5bbb80 10 client.16905 fetch_fsmap finished waiting for FSMap version 44 2023-10-08T13:57:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.204+0000 7f8c7f5bbb80 10 client.16905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:57:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.204+0000 7f8c7f5bbb80 20 client.16905 populate_metadata read hostname 'smithi057' 2023-10-08T13:57:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.204+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e38d2a2aa0 0x55e38d2a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:57:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.204+0000 7f8c7f5bbb80 4 client.16905 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:57:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.204+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2109803122 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e38d180ad0 con 0x55e38d2a2aa0 2023-10-08T13:57:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.204+0000 7f8c7096d700 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e38d2a2aa0 0x55e38d2a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:57:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.204+0000 7f8c54ff9700 10 client.16905 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.204+0000 7f8c7096d700 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e38d2a2aa0 0x55e38d2a4ec0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18654061dd371d01 server_cookie=ae6218419ac7477f in_seq=0 out_seq=0 2023-10-08T13:57:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c54ff9700 1 -- 172.21.15.57:0/2109803122 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e38d180ad0 con 0x55e38d2a2aa0 2023-10-08T13:57:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c54ff9700 10 client.16905 handle_command_reply: tid=0 2023-10-08T13:57:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 client.16905 shutdown 2023-10-08T13:57:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2109803122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d2350b0 msgr2=0x55e38d245ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d2350b0 0x55e38d245ad0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009990 tx=0x7f8c68009ca0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2109803122 shutdown_connections 2023-10-08T13:57:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e38d2a2aa0 0x55e38d2a4ec0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e38d23b1a0 0x55e38d243ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e38d2350b0 0x55e38d245ad0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c54ff9700 0 client.16905 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:57:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 --2- 172.21.15.57:0/2109803122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e38d23a880 0x55e38d243ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2109803122 >> 172.21.15.57:0/2109803122 conn(0x55e38d1a9200 msgr2=0x55e38d23ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2109803122 shutdown_connections 2023-10-08T13:57:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 1 -- 172.21.15.57:0/2109803122 wait complete. 2023-10-08T13:57:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.424+0000 7f8c7f5bbb80 20 client.16905 trim_cache size 0 max 16384 2023-10-08T13:57:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:57:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:57:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:57:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 -- 172.21.15.57:0/1265959890 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8c580a15f0 msgr2=0x7f8c580a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8c580a15f0 0x7f8c580a3ab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800b2e0 tx=0x7f8c68005960 comp rx=0 tx=0).stop 2023-10-08T13:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 -- 172.21.15.57:0/1265959890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c6c1009f0 msgr2=0x7f8c6c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c6c1009f0 0x7f8c6c1992f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00d980 tx=0x7f8c5c00dc90 comp rx=0 tx=0).stop 2023-10-08T13:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 -- 172.21.15.57:0/1265959890 shutdown_connections 2023-10-08T13:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f8c580a15f0 0x7f8c580a3ab0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c6c10d140 0x7f8c6c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c6c1000d0 0x7f8c6c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 --2- 172.21.15.57:0/1265959890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c6c1009f0 0x7f8c6c1992f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 -- 172.21.15.57:0/1265959890 >> 172.21.15.57:0/1265959890 conn(0x7f8c6c076790 msgr2=0x7f8c6c10e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 -- 172.21.15.57:0/1265959890 shutdown_connections 2023-10-08T13:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:57:57.425+0000 7f8c733d2700 1 -- 172.21.15.57:0/1265959890 wait complete. 2023-10-08T13:57:57.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:57 smithi115 ceph-mon[104408]: pgmap v491: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 444 op/s 2023-10-08T13:57:57.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:57 smithi057 ceph-mon[99476]: pgmap v491: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 444 op/s 2023-10-08T13:57:58.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:57 smithi099 ceph-mon[103849]: pgmap v491: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 444 op/s 2023-10-08T13:57:59.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:57:59 smithi115 ceph-mon[104408]: pgmap v492: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2023-10-08T13:57:59.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:57:59 smithi057 ceph-mon[99476]: pgmap v492: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2023-10-08T13:57:59.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:57:59 smithi099 ceph-mon[103849]: pgmap v492: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2023-10-08T13:58:01.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:01 smithi115 ceph-mon[104408]: pgmap v493: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2023-10-08T13:58:01.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:01 smithi057 ceph-mon[99476]: pgmap v493: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2023-10-08T13:58:02.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:01 smithi099 ceph-mon[103849]: pgmap v493: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 142 op/s 2023-10-08T13:58:03.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:03 smithi115 ceph-mon[104408]: pgmap v494: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2023-10-08T13:58:03.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:03 smithi057 ceph-mon[99476]: pgmap v494: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2023-10-08T13:58:04.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:03 smithi099 ceph-mon[103849]: pgmap v494: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2023-10-08T13:58:05.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:05 smithi115 ceph-mon[104408]: pgmap v495: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 900 KiB/s wr, 163 op/s 2023-10-08T13:58:05.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:05 smithi057 ceph-mon[99476]: pgmap v495: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 900 KiB/s wr, 163 op/s 2023-10-08T13:58:06.029 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:05 smithi099 ceph-mon[103849]: pgmap v495: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 900 KiB/s wr, 163 op/s 2023-10-08T13:58:07.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:07 smithi115 ceph-mon[104408]: pgmap v496: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 567 KiB/s wr, 152 op/s 2023-10-08T13:58:07.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:07 smithi057 ceph-mon[99476]: pgmap v496: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 567 KiB/s wr, 152 op/s 2023-10-08T13:58:08.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:07 smithi099 ceph-mon[103849]: pgmap v496: 129 pgs: 129 active+clean; 1.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 567 KiB/s wr, 152 op/s 2023-10-08T13:58:09.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:09 smithi115 ceph-mon[104408]: pgmap v497: 129 pgs: 129 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2023-10-08T13:58:09.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:09 smithi057 ceph-mon[99476]: pgmap v497: 129 pgs: 129 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2023-10-08T13:58:09.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:09 smithi099 ceph-mon[103849]: pgmap v497: 129 pgs: 129 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2023-10-08T13:58:10.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:10 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:58:10.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:10 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:58:10.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:10 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:58:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.622+0000 7f284519a700 1 -- 172.21.15.57:0/3911733508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2840108110 msgr2=0x7f2840113400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.622+0000 7f284519a700 1 --2- 172.21.15.57:0/3911733508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2840108110 0x7f2840113400 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f28300099c0 tx=0x7f2830009cd0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.627+0000 7f284519a700 1 -- 172.21.15.57:0/3911733508 shutdown_connections 2023-10-08T13:58:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.627+0000 7f284519a700 1 --2- 172.21.15.57:0/3911733508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2840113940 0x7f2840115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.627+0000 7f284519a700 1 --2- 172.21.15.57:0/3911733508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28401077f0 0x7f2840107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.627+0000 7f284519a700 1 --2- 172.21.15.57:0/3911733508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2840108110 0x7f2840113400 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.627+0000 7f284519a700 1 -- 172.21.15.57:0/3911733508 >> 172.21.15.57:0/3911733508 conn(0x7f284006b7c0 msgr2=0x7f284006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.631+0000 7f284519a700 1 -- 172.21.15.57:0/3911733508 shutdown_connections 2023-10-08T13:58:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.635+0000 7f284519a700 1 -- 172.21.15.57:0/3911733508 wait complete. 2023-10-08T13:58:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.636+0000 7f284519a700 1 Processor -- start 2023-10-08T13:58:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.644+0000 7f284519a700 1 -- start start 2023-10-08T13:58:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.644+0000 7f284519a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28401077f0 0x7f28401a4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.644+0000 7f284519a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2840108110 0x7f28401a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.644+0000 7f284519a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2840113940 0x7f284019d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.644+0000 7f284519a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284019dab0 con 0x7f2840108110 2023-10-08T13:58:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.644+0000 7f284519a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284019dc20 con 0x7f28401077f0 2023-10-08T13:58:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.644+0000 7f284519a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284019dde0 con 0x7f2840113940 2023-10-08T13:58:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.645+0000 7f283e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2840108110 0x7f28401a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.645+0000 7f283e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2840108110 0x7f28401a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59058/0 (socket says 172.21.15.57:59058) 2023-10-08T13:58:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.645+0000 7f283e59c700 1 -- 172.21.15.57:0/1355118655 learned_addr learned my addr 172.21.15.57:0/1355118655 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.647+0000 7f283ed9d700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28401077f0 0x7f28401a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.647+0000 7f283f59e700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2840113940 0x7f284019d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.647+0000 7f283e59c700 1 -- 172.21.15.57:0/1355118655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2840113940 msgr2=0x7f284019d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.647+0000 7f283e59c700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2840113940 0x7f284019d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.647+0000 7f283e59c700 1 -- 172.21.15.57:0/1355118655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28401077f0 msgr2=0x7f28401a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.647+0000 7f283e59c700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28401077f0 0x7f28401a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.647+0000 7f283e59c700 1 -- 172.21.15.57:0/1355118655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300095f0 con 0x7f2840108110 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.647+0000 7f283f59e700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2840113940 0x7f284019d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.648+0000 7f283e59c700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2840108110 0x7f28401a45f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f28300098e0 tx=0x7f2830003750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.648+0000 7f2827fff700 1 -- 172.21.15.57:0/1355118655 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283001d070 con 0x7f2840108110 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.648+0000 7f2827fff700 1 -- 172.21.15.57:0/1355118655 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f283000fbb0 con 0x7f2840108110 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.648+0000 7f284519a700 1 -- 172.21.15.57:0/1355118655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28401a1c00 con 0x7f2840108110 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.648+0000 7f284519a700 1 -- 172.21.15.57:0/1355118655 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28401a2100 con 0x7f2840108110 2023-10-08T13:58:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.648+0000 7f2827fff700 1 -- 172.21.15.57:0/1355118655 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830003fe0 con 0x7f2840108110 2023-10-08T13:58:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.653+0000 7f2851383b80 1 Processor -- start 2023-10-08T13:58:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.655+0000 7f2827fff700 1 -- 172.21.15.57:0/1355118655 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f283000fd20 con 0x7f2840108110 2023-10-08T13:58:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.655+0000 7f2851383b80 1 -- start start 2023-10-08T13:58:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.655+0000 7f2851383b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4e880 0x563e18c4ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.656+0000 7f2851383b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4f1a0 0x563e18c48b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.656+0000 7f2851383b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 0x563e18c49510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.656+0000 7f2851383b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18bb1400 con 0x563e18c4f1a0 2023-10-08T13:58:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.656+0000 7f2851383b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18ba2bc0 con 0x563e18c4e880 2023-10-08T13:58:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.656+0000 7f2851383b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18ba04e0 con 0x563e18c490b0 2023-10-08T13:58:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.656+0000 7f2827fff700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f28280a16a0 0x7f28280a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.656+0000 7f283f59e700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f28280a16a0 0x7f28280a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.656+0000 7f2827fff700 1 -- 172.21.15.57:0/1355118655 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f28300ce310 con 0x7f2840108110 2023-10-08T13:58:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.656+0000 7f283f59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4e880 0x563e18c4ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.657+0000 7f283f59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4e880 0x563e18c4ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60486/0 (socket says 172.21.15.57:60486) 2023-10-08T13:58:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.657+0000 7f283f59e700 1 -- 172.21.15.57:0/268340294 learned_addr learned my addr 172.21.15.57:0/268340294 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.657+0000 7f283f59e700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f28280a16a0 0x7f28280a3b60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2834006e50 tx=0x7f283400a130 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.658+0000 7f283ed9d700 1 --2- 172.21.15.57:0/268340294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4f1a0 0x563e18c48b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.659+0000 7f283f59e700 1 -- 172.21.15.57:0/268340294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 msgr2=0x563e18c49510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.659+0000 7f283f59e700 1 --2- 172.21.15.57:0/268340294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 0x563e18c49510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.659+0000 7f283f59e700 1 -- 172.21.15.57:0/268340294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4f1a0 msgr2=0x563e18c48b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.659+0000 7f283f59e700 1 --2- 172.21.15.57:0/268340294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4f1a0 0x563e18c48b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.659+0000 7f283f59e700 1 -- 172.21.15.57:0/268340294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e189c6d80 con 0x563e18c4e880 2023-10-08T13:58:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.659+0000 7f283f59e700 1 --2- 172.21.15.57:0/268340294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4e880 0x563e18c4ec60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f283400f970 tx=0x7f2834018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73cee7245a49fcf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.662+0000 7f28257fa700 1 -- 172.21.15.57:0/268340294 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283400a380 con 0x563e18c4e880 2023-10-08T13:58:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.662+0000 7f28257fa700 1 -- 172.21.15.57:0/268340294 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2834014b50 con 0x563e18c4e880 2023-10-08T13:58:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.662+0000 7f28257fa700 1 -- 172.21.15.57:0/268340294 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283401dd80 con 0x563e18c4e880 2023-10-08T13:58:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.663+0000 7f2851383b80 1 -- 172.21.15.57:0/268340294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4e880 msgr2=0x563e18c4ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.663+0000 7f2851383b80 1 --2- 172.21.15.57:0/268340294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4e880 0x563e18c4ec60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f283400f970 tx=0x7f2834018000 comp rx=0 tx=0).stop 2023-10-08T13:58:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.668+0000 7f2851383b80 1 -- 172.21.15.57:0/268340294 shutdown_connections 2023-10-08T13:58:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.668+0000 7f2851383b80 1 --2- 172.21.15.57:0/268340294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 0x563e18c49510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.668+0000 7f2851383b80 1 --2- 172.21.15.57:0/268340294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4e880 0x563e18c4ec60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.668+0000 7f2851383b80 1 --2- 172.21.15.57:0/268340294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4f1a0 0x563e18c48b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.668+0000 7f2851383b80 1 -- 172.21.15.57:0/268340294 >> 172.21.15.57:0/268340294 conn(0x563e18bbd200 msgr2=0x563e18c43b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.672+0000 7f2851383b80 1 -- 172.21.15.57:0/268340294 shutdown_connections 2023-10-08T13:58:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.680+0000 7f2851383b80 1 -- 172.21.15.57:0/268340294 wait complete. 2023-10-08T13:58:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.681+0000 7f2851383b80 1 -- start start 2023-10-08T13:58:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.681+0000 7f2851383b80 1 Processor -- start 2023-10-08T13:58:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f2851383b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 0x563e18c525a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f2851383b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 0x563e18c52ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f2851383b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 0x563e18c53020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f2851383b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18b9bdd0 con 0x563e18c4e880 2023-10-08T13:58:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f2851383b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18b386b0 con 0x563e18c4f1a0 2023-10-08T13:58:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f2851383b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18b38350 con 0x563e18c490b0 2023-10-08T13:58:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f2851383b80 10 client.0 _ll_register_callbacks cb 0x563e189957a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:58:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f283e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 0x563e18c52ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f283e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 0x563e18c52ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59068/0 (socket says 172.21.15.57:59068) 2023-10-08T13:58:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f283e59c700 1 -- 172.21.15.57:0/3184526607 learned_addr learned my addr 172.21.15.57:0/3184526607 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f283e59c700 1 -- 172.21.15.57:0/3184526607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 msgr2=0x563e18c525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f283e59c700 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 0x563e18c525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f283e59c700 1 -- 172.21.15.57:0/3184526607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 msgr2=0x563e18c53020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:58:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f283e59c700 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 0x563e18c53020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.682+0000 7f283e59c700 1 -- 172.21.15.57:0/3184526607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283400e040 con 0x563e18c4e880 2023-10-08T13:58:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.683+0000 7f283e59c700 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 0x563e18c52ae0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f283000ffd0 tx=0x7f283009b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238bc6064cbcf77e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.683+0000 7f2822ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:58:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.690+0000 7f2822ff5700 1 -- 172.21.15.57:0/3184526607 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830099370 con 0x563e18c4e880 2023-10-08T13:58:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.690+0000 7f2822ff5700 1 -- 172.21.15.57:0/3184526607 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f28300ce020 con 0x563e18c4e880 2023-10-08T13:58:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.690+0000 7f2851383b80 1 -- 172.21.15.57:0/3184526607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e18b63e50 con 0x563e18c4e880 2023-10-08T13:58:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.690+0000 7f2822ff5700 1 -- 172.21.15.57:0/3184526607 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2830095be0 con 0x563e18c4e880 2023-10-08T13:58:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.691+0000 7f2851383b80 10 client.16917 fetch_fsmap learned FSMap version 44 2023-10-08T13:58:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.691+0000 7f2851383b80 1 -- 172.21.15.57:0/3184526607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x563e189afc00 con 0x563e18c4e880 2023-10-08T13:58:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.692+0000 7f2822ff5700 1 -- 172.21.15.57:0/3184526607 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28300b7030 con 0x563e18c4e880 2023-10-08T13:58:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.692+0000 7f2822ff5700 1 -- 172.21.15.57:0/3184526607 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 44) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f28300aed20 con 0x563e18c4e880 2023-10-08T13:58:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.692+0000 7f2851383b80 10 client.16917 fetch_fsmap finished waiting for FSMap version 44 2023-10-08T13:58:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.692+0000 7f2851383b80 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:58:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.692+0000 7f2851383b80 20 client.16917 populate_metadata read hostname 'smithi057' 2023-10-08T13:58:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.692+0000 7f2851383b80 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563e18c62ae0 0x563e18c64f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.692+0000 7f2851383b80 4 client.16917 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:58:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.692+0000 7f2851383b80 1 -- 172.21.15.57:0/3184526607 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e18b94db0 con 0x563e18c62ae0 2023-10-08T13:58:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.696+0000 7f283ed9d700 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563e18c62ae0 0x563e18c64f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.702+0000 7f283ed9d700 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563e18c62ae0 0x563e18c64f00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a24d677a8672c39 server_cookie=a6e792d3bf156407 in_seq=0 out_seq=0 2023-10-08T13:58:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.702+0000 7f2822ff5700 10 client.16917 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.707+0000 7f2822ff5700 1 -- 172.21.15.57:0/3184526607 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563e18b94db0 con 0x563e18c62ae0 2023-10-08T13:58:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.707+0000 7f2822ff5700 10 client.16917 handle_command_reply: tid=0 2023-10-08T13:58:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.707+0000 7f2851383b80 1 client.16917 shutdown 2023-10-08T13:58:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.709+0000 7f2851383b80 1 -- 172.21.15.57:0/3184526607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 msgr2=0x563e18c52ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.709+0000 7f2851383b80 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 0x563e18c52ae0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f283000ffd0 tx=0x7f283009b800 comp rx=0 tx=0).stop 2023-10-08T13:58:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.709+0000 7f2851383b80 1 -- 172.21.15.57:0/3184526607 shutdown_connections 2023-10-08T13:58:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.709+0000 7f2851383b80 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563e18c62ae0 0x563e18c64f00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.709+0000 7f2851383b80 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 0x563e18c53020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.709+0000 7f2851383b80 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 0x563e18c525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.709+0000 7f2851383b80 1 --2- 172.21.15.57:0/3184526607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 0x563e18c52ae0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.709+0000 7f2851383b80 1 -- 172.21.15.57:0/3184526607 >> 172.21.15.57:0/3184526607 conn(0x563e18bbd200 msgr2=0x563e18c43010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.709+0000 7f2822ff5700 0 client.16917 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.710+0000 7f2851383b80 1 -- 172.21.15.57:0/3184526607 shutdown_connections 2023-10-08T13:58:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.710+0000 7f2851383b80 1 -- 172.21.15.57:0/3184526607 wait complete. 2023-10-08T13:58:10.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.714+0000 7f2851383b80 20 client.16917 trim_cache size 0 max 16384 2023-10-08T13:58:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.741+0000 7f2851383b80 1 Processor -- start 2023-10-08T13:58:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.741+0000 7f2851383b80 1 -- start start 2023-10-08T13:58:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.742+0000 7f2851383b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c490b0 0x563e18c43790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.742+0000 7f2851383b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c4e880 0x563e18c43cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.742+0000 7f2851383b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4f1a0 0x563e18c52af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.742+0000 7f2851383b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18c49a10 con 0x563e18c4f1a0 2023-10-08T13:58:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.742+0000 7f2851383b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18a1c8c0 con 0x563e18c490b0 2023-10-08T13:58:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.742+0000 7f2851383b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18c40860 con 0x563e18c4e880 2023-10-08T13:58:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.743+0000 7f283e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c4e880 0x563e18c43cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.743+0000 7f283e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c4e880 0x563e18c43cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59890/0 (socket says 172.21.15.57:59890) 2023-10-08T13:58:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.743+0000 7f283e59c700 1 -- 172.21.15.57:0/3407650951 learned_addr learned my addr 172.21.15.57:0/3407650951 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.746+0000 7f283f59e700 1 --2- 172.21.15.57:0/3407650951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4f1a0 0x563e18c52af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.747+0000 7f283ed9d700 1 --2- 172.21.15.57:0/3407650951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c490b0 0x563e18c43790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.747+0000 7f283e59c700 1 -- 172.21.15.57:0/3407650951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c490b0 msgr2=0x563e18c43790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.747+0000 7f283e59c700 1 --2- 172.21.15.57:0/3407650951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c490b0 0x563e18c43790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.747+0000 7f283e59c700 1 -- 172.21.15.57:0/3407650951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4f1a0 msgr2=0x563e18c52af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.747+0000 7f283e59c700 1 --2- 172.21.15.57:0/3407650951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4f1a0 0x563e18c52af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.747+0000 7f283e59c700 1 -- 172.21.15.57:0/3407650951 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e189ed1f0 con 0x563e18c4e880 2023-10-08T13:58:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.759+0000 7f283e59c700 1 --2- 172.21.15.57:0/3407650951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c4e880 0x563e18c43cd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f283000c010 tx=0x7f2830009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bc72614b00f8848 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.763+0000 7f2825ffb700 1 -- 172.21.15.57:0/3407650951 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283009b230 con 0x563e18c4e880 2023-10-08T13:58:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.763+0000 7f2825ffb700 1 -- 172.21.15.57:0/3407650951 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e18b63e50 con 0x563e18c4e880 2023-10-08T13:58:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.763+0000 7f2825ffb700 1 -- 172.21.15.57:0/3407650951 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830095be0 con 0x563e18c4e880 2023-10-08T13:58:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.764+0000 7f2851383b80 1 -- 172.21.15.57:0/3407650951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c4e880 msgr2=0x563e18c43cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.764+0000 7f2851383b80 1 --2- 172.21.15.57:0/3407650951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c4e880 0x563e18c43cd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f283000c010 tx=0x7f2830009cd0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.765+0000 7f2851383b80 1 -- 172.21.15.57:0/3407650951 shutdown_connections 2023-10-08T13:58:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.765+0000 7f2851383b80 1 --2- 172.21.15.57:0/3407650951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4f1a0 0x563e18c52af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.765+0000 7f2851383b80 1 --2- 172.21.15.57:0/3407650951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c490b0 0x563e18c43790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.765+0000 7f2851383b80 1 --2- 172.21.15.57:0/3407650951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c4e880 0x563e18c43cd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.765+0000 7f2851383b80 1 -- 172.21.15.57:0/3407650951 >> 172.21.15.57:0/3407650951 conn(0x563e18bbd200 msgr2=0x563e18c4c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.765+0000 7f2851383b80 1 -- 172.21.15.57:0/3407650951 shutdown_connections 2023-10-08T13:58:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.765+0000 7f2851383b80 1 -- 172.21.15.57:0/3407650951 wait complete. 2023-10-08T13:58:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.766+0000 7f2851383b80 1 -- start start 2023-10-08T13:58:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.766+0000 7f2851383b80 1 Processor -- start 2023-10-08T13:58:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.768+0000 7f2851383b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 0x563e18c59c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.768+0000 7f2851383b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 0x563e18c57c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.768+0000 7f2851383b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 0x563e18c58140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.768+0000 7f2851383b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18c58720 con 0x563e18c4e880 2023-10-08T13:58:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.768+0000 7f2851383b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18c58890 con 0x563e18c4f1a0 2023-10-08T13:58:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.768+0000 7f2851383b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e18c58a50 con 0x563e18c490b0 2023-10-08T13:58:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.768+0000 7f2851383b80 10 client.0 _ll_register_callbacks cb 0x563e189957a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:58:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.768+0000 7f283f59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 0x563e18c58140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.768+0000 7f283f59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 0x563e18c58140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60510/0 (socket says 172.21.15.57:60510) 2023-10-08T13:58:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.768+0000 7f283f59e700 1 -- 172.21.15.57:0/911577928 learned_addr learned my addr 172.21.15.57:0/911577928 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.769+0000 7f283f59e700 1 -- 172.21.15.57:0/911577928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 msgr2=0x563e18c59c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.769+0000 7f283f59e700 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 0x563e18c59c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.769+0000 7f283f59e700 1 -- 172.21.15.57:0/911577928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 msgr2=0x563e18c57c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T13:58:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.769+0000 7f283f59e700 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 0x563e18c57c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.769+0000 7f283f59e700 1 -- 172.21.15.57:0/911577928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e189afc00 con 0x563e18c4f1a0 2023-10-08T13:58:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.769+0000 7f283f59e700 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 0x563e18c58140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f283400f940 tx=0x7f2834004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b106b9961cec2a22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.769+0000 7f2822ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:58:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.770+0000 7f2822ff5700 1 -- 172.21.15.57:0/911577928 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283400f5c0 con 0x563e18c4f1a0 2023-10-08T13:58:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.770+0000 7f2851383b80 1 -- 172.21.15.57:0/911577928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2834014b50 con 0x563e18c4f1a0 2023-10-08T13:58:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.770+0000 7f2822ff5700 1 -- 172.21.15.57:0/911577928 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f28340045c0 con 0x563e18c4f1a0 2023-10-08T13:58:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.770+0000 7f2822ff5700 1 -- 172.21.15.57:0/911577928 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28340149b0 con 0x563e18c4f1a0 2023-10-08T13:58:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.770+0000 7f2851383b80 10 client.26474 fetch_fsmap learned FSMap version 44 2023-10-08T13:58:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.770+0000 7f2851383b80 1 -- 172.21.15.57:0/911577928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x563e189c5250 con 0x563e18c4f1a0 2023-10-08T13:58:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.771+0000 7f2822ff5700 1 -- 172.21.15.57:0/911577928 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2834014b50 con 0x563e18c4f1a0 2023-10-08T13:58:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.771+0000 7f2822ff5700 1 -- 172.21.15.57:0/911577928 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 44) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f283401d710 con 0x563e18c4f1a0 2023-10-08T13:58:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.771+0000 7f2851383b80 10 client.26474 fetch_fsmap finished waiting for FSMap version 44 2023-10-08T13:58:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.771+0000 7f2851383b80 10 client.26474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:58:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.771+0000 7f2851383b80 20 client.26474 populate_metadata read hostname 'smithi057' 2023-10-08T13:58:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.771+0000 7f2851383b80 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563e18cb6fc0 0x563e18cb93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.771+0000 7f2851383b80 4 client.26474 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:58:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.771+0000 7f2851383b80 1 -- 172.21.15.57:0/911577928 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e18b94ad0 con 0x563e18cb6fc0 2023-10-08T13:58:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.772+0000 7f283ed9d700 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563e18cb6fc0 0x563e18cb93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.773+0000 7f283ed9d700 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563e18cb6fc0 0x563e18cb93e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5196eb93d02e2f12 server_cookie=971df13f811da168 in_seq=0 out_seq=0 2023-10-08T13:58:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.774+0000 7f2822ff5700 10 client.26474 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.939+0000 7f2822ff5700 1 -- 172.21.15.57:0/911577928 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e18b94ad0 con 0x563e18cb6fc0 2023-10-08T13:58:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.939+0000 7f2822ff5700 10 client.26474 handle_command_reply: tid=0 2023-10-08T13:58:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.939+0000 7f2851383b80 1 client.26474 shutdown 2023-10-08T13:58:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.943+0000 7f2851383b80 1 -- 172.21.15.57:0/911577928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 msgr2=0x563e18c58140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.943+0000 7f2851383b80 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 0x563e18c58140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f283400f940 tx=0x7f2834004320 comp rx=0 tx=0).stop 2023-10-08T13:58:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.948+0000 7f2851383b80 1 -- 172.21.15.57:0/911577928 shutdown_connections 2023-10-08T13:58:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.948+0000 7f2851383b80 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x563e18cb6fc0 0x563e18cb93e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.948+0000 7f2851383b80 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e18c4f1a0 0x563e18c58140 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.948+0000 7f2851383b80 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e18c490b0 0x563e18c59c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.948+0000 7f2851383b80 1 --2- 172.21.15.57:0/911577928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563e18c4e880 0x563e18c57c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.948+0000 7f2851383b80 1 -- 172.21.15.57:0/911577928 >> 172.21.15.57:0/911577928 conn(0x563e18bbd200 msgr2=0x563e18c50e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.949+0000 7f2822ff5700 0 client.26474 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.951+0000 7f2851383b80 1 -- 172.21.15.57:0/911577928 shutdown_connections 2023-10-08T13:58:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.952+0000 7f2851383b80 1 -- 172.21.15.57:0/911577928 wait complete. 2023-10-08T13:58:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.956+0000 7f2851383b80 20 client.26474 trim_cache size 0 max 16384 2023-10-08T13:58:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:58:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:58:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:58:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:58:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.964+0000 7f284519a700 1 -- 172.21.15.57:0/1355118655 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f28280a16a0 msgr2=0x7f28280a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.964+0000 7f284519a700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f28280a16a0 0x7f28280a3b60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2834006e50 tx=0x7f283400a130 comp rx=0 tx=0).stop 2023-10-08T13:58:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.964+0000 7f284519a700 1 -- 172.21.15.57:0/1355118655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2840108110 msgr2=0x7f28401a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.965+0000 7f284519a700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2840108110 0x7f28401a45f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f28300098e0 tx=0x7f2830003750 comp rx=0 tx=0).stop 2023-10-08T13:58:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.968+0000 7f284519a700 1 -- 172.21.15.57:0/1355118655 shutdown_connections 2023-10-08T13:58:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.968+0000 7f284519a700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2840113940 0x7f284019d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.968+0000 7f284519a700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f28280a16a0 0x7f28280a3b60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.968+0000 7f284519a700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28401077f0 0x7f28401a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.968+0000 7f284519a700 1 --2- 172.21.15.57:0/1355118655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2840108110 0x7f28401a45f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.968+0000 7f284519a700 1 -- 172.21.15.57:0/1355118655 >> 172.21.15.57:0/1355118655 conn(0x7f284006b7c0 msgr2=0x7f2840070730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.974+0000 7f284519a700 1 -- 172.21.15.57:0/1355118655 shutdown_connections 2023-10-08T13:58:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:10.974+0000 7f284519a700 1 -- 172.21.15.57:0/1355118655 wait complete. 2023-10-08T13:58:11.449 INFO:tasks.workunit.client.0.smithi057.stdout: 2023-10-08T13:58:11.449 INFO:tasks.workunit.client.0.smithi057.stdout:Final score for writes: 498 2023-10-08T13:58:11.449 INFO:tasks.workunit.client.0.smithi057.stdout:Final score for reads : 723900 2023-10-08T13:58:11.450 INFO:tasks.workunit.client.0.smithi057.stdout: 2023-10-08T13:58:11.451 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-08T13:58:11.451 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-08T13:58:11.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:11 smithi057 ceph-mon[99476]: osdmap e93: 12 total, 12 up, 12 in 2023-10-08T13:58:11.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:11 smithi057 ceph-mon[99476]: pgmap v499: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.9 MiB/s wr, 259 op/s 2023-10-08T13:58:11.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:58:11.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:58:11.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:58:11.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:11 smithi099 ceph-mon[103849]: osdmap e93: 12 total, 12 up, 12 in 2023-10-08T13:58:11.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:11 smithi099 ceph-mon[103849]: pgmap v499: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.9 MiB/s wr, 259 op/s 2023-10-08T13:58:11.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:58:11.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:58:11.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:58:11.691 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:11 smithi115 ceph-mon[104408]: osdmap e93: 12 total, 12 up, 12 in 2023-10-08T13:58:11.691 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:11 smithi115 ceph-mon[104408]: pgmap v499: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.9 MiB/s wr, 259 op/s 2023-10-08T13:58:11.691 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:58:11.691 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:58:11.691 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:58:13.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:13 smithi115 ceph-mon[104408]: pgmap v500: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 238 op/s 2023-10-08T13:58:13.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:13 smithi057 ceph-mon[99476]: pgmap v500: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 238 op/s 2023-10-08T13:58:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:13 smithi099 ceph-mon[103849]: pgmap v500: 129 pgs: 10 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 85 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 238 op/s 2023-10-08T13:58:15.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:15 smithi115 ceph-mon[104408]: pgmap v501: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 232 op/s 2023-10-08T13:58:15.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:15 smithi057 ceph-mon[99476]: pgmap v501: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 232 op/s 2023-10-08T13:58:16.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:15 smithi099 ceph-mon[103849]: pgmap v501: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 232 op/s 2023-10-08T13:58:16.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:16 smithi115 ceph-mon[104408]: pgmap v502: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 239 op/s 2023-10-08T13:58:16.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:16 smithi057 ceph-mon[99476]: pgmap v502: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 239 op/s 2023-10-08T13:58:17.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:16 smithi099 ceph-mon[103849]: pgmap v502: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 1.8 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 239 op/s 2023-10-08T13:58:19.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:19 smithi115 ceph-mon[104408]: pgmap v503: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 1.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 198 op/s 2023-10-08T13:58:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:19 smithi057 ceph-mon[99476]: pgmap v503: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 1.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 198 op/s 2023-10-08T13:58:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:19 smithi099 ceph-mon[103849]: pgmap v503: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 1.8 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 198 op/s 2023-10-08T13:58:21.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:21 smithi115 ceph-mon[104408]: pgmap v504: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 143 op/s 2023-10-08T13:58:21.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:21 smithi057 ceph-mon[99476]: pgmap v504: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 143 op/s 2023-10-08T13:58:22.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:21 smithi099 ceph-mon[103849]: pgmap v504: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 143 op/s 2023-10-08T13:58:23.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:23 smithi115 ceph-mon[104408]: pgmap v505: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 116 op/s 2023-10-08T13:58:23.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:23 smithi115 ceph-mon[104408]: osdmap e94: 12 total, 12 up, 12 in 2023-10-08T13:58:23.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:23 smithi057 ceph-mon[99476]: pgmap v505: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 116 op/s 2023-10-08T13:58:23.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:23 smithi057 ceph-mon[99476]: osdmap e94: 12 total, 12 up, 12 in 2023-10-08T13:58:24.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:23 smithi099 ceph-mon[103849]: pgmap v505: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 116 op/s 2023-10-08T13:58:24.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:23 smithi099 ceph-mon[103849]: osdmap e94: 12 total, 12 up, 12 in 2023-10-08T13:58:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.139+0000 7ff28f424700 1 -- 172.21.15.57:0/732603049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2881045e0 msgr2=0x7ff2881049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.139+0000 7ff28f424700 1 --2- 172.21.15.57:0/732603049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2881045e0 0x7ff2881049c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff284009990 tx=0x7ff284009ca0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.139+0000 7ff28f424700 1 -- 172.21.15.57:0/732603049 shutdown_connections 2023-10-08T13:58:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.139+0000 7ff28f424700 1 --2- 172.21.15.57:0/732603049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2880fff90 0x7ff28810c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.139+0000 7ff28f424700 1 --2- 172.21.15.57:0/732603049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2881045e0 0x7ff2881049c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.139+0000 7ff28f424700 1 --2- 172.21.15.57:0/732603049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2880ff5d0 0x7ff2880ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.139+0000 7ff28f424700 1 -- 172.21.15.57:0/732603049 >> 172.21.15.57:0/732603049 conn(0x7ff288076790 msgr2=0x7ff288076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.139+0000 7ff28f424700 1 -- 172.21.15.57:0/732603049 shutdown_connections 2023-10-08T13:58:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.139+0000 7ff28f424700 1 -- 172.21.15.57:0/732603049 wait complete. 2023-10-08T13:58:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28f424700 1 Processor -- start 2023-10-08T13:58:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28f424700 1 -- start start 2023-10-08T13:58:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28f424700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2880ff5d0 0x7ff288195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28f424700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2880fff90 0x7ff288195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28f424700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2881045e0 0x7ff28819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28f424700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28819a910 con 0x7ff2881045e0 2023-10-08T13:58:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28f424700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28819aa80 con 0x7ff2880fff90 2023-10-08T13:58:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28f424700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28819ac40 con 0x7ff2880ff5d0 2023-10-08T13:58:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28d1c0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2880ff5d0 0x7ff288195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28d9c1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2881045e0 0x7ff28819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28c9bf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2880fff90 0x7ff288195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28d1c0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2880ff5d0 0x7ff288195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59906/0 (socket says 172.21.15.57:59906) 2023-10-08T13:58:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28d1c0700 1 -- 172.21.15.57:0/220508152 learned_addr learned my addr 172.21.15.57:0/220508152 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28d9c1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2881045e0 0x7ff28819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59098/0 (socket says 172.21.15.57:59098) 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28c9bf700 1 -- 172.21.15.57:0/220508152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2880ff5d0 msgr2=0x7ff288195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28c9bf700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2880ff5d0 0x7ff288195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28c9bf700 1 -- 172.21.15.57:0/220508152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2881045e0 msgr2=0x7ff28819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28c9bf700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2881045e0 0x7ff28819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.140+0000 7ff28c9bf700 1 -- 172.21.15.57:0/220508152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840095f0 con 0x7ff2880fff90 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.141+0000 7ff28d9c1700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2881045e0 0x7ff28819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.141+0000 7ff28c9bf700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2880fff90 0x7ff288195f70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff27800e9f0 tx=0x7ff27800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.141+0000 7ff27e7fc700 1 -- 172.21.15.57:0/220508152 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800c9c0 con 0x7ff2880fff90 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.141+0000 7ff27e7fc700 1 -- 172.21.15.57:0/220508152 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff278010430 con 0x7ff2880fff90 2023-10-08T13:58:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.141+0000 7ff28f424700 1 -- 172.21.15.57:0/220508152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28819af20 con 0x7ff2880fff90 2023-10-08T13:58:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.141+0000 7ff27e7fc700 1 -- 172.21.15.57:0/220508152 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278018680 con 0x7ff2880fff90 2023-10-08T13:58:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.141+0000 7ff28f424700 1 -- 172.21.15.57:0/220508152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2881a27d0 con 0x7ff2880fff90 2023-10-08T13:58:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.143+0000 7ff27e7fc700 1 -- 172.21.15.57:0/220508152 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff278010860 con 0x7ff2880fff90 2023-10-08T13:58:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.144+0000 7ff27e7fc700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2740a15f0 0x7ff2740a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.144+0000 7ff27e7fc700 1 -- 172.21.15.57:0/220508152 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9393+0+0 (secure 0 0 0) 0x7ff278014070 con 0x7ff2880fff90 2023-10-08T13:58:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.144+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2740a15f0 0x7ff2740a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.144+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2740a15f0 0x7ff2740a3ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff28400b6f0 tx=0x7ff284005bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff29b60db80 1 Processor -- start 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff29b60db80 1 -- start start 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a91c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a921a0 0x55fde4a8bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 0x55fde4a8c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff29b60db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde49f4400 con 0x55fde4a91880 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff29b60db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde49e5bc0 con 0x55fde4a921a0 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff29b60db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde49e34e0 con 0x55fde4a8c0b0 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28d1c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a91c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28d1c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a91c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59102/0 (socket says 172.21.15.57:59102) 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28d1c0700 1 -- 172.21.15.57:0/1556483467 learned_addr learned my addr 172.21.15.57:0/1556483467 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28d9c1700 1 --2- 172.21.15.57:0/1556483467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 0x55fde4a8c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28c9bf700 1 --2- 172.21.15.57:0/1556483467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a921a0 0x55fde4a8bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28d1c0700 1 -- 172.21.15.57:0/1556483467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 msgr2=0x55fde4a8c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/1556483467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 0x55fde4a8c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28d1c0700 1 -- 172.21.15.57:0/1556483467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a921a0 msgr2=0x55fde4a8bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/1556483467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a921a0 0x55fde4a8bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28d1c0700 1 -- 172.21.15.57:0/1556483467 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fde4809d80 con 0x55fde4a91880 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.145+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/1556483467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a91c60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff284000c00 tx=0x7ff284020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69875df946394977 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff2737fe700 1 -- 172.21.15.57:0/1556483467 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284005720 con 0x55fde4a91880 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff2737fe700 1 -- 172.21.15.57:0/1556483467 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff28401e040 con 0x55fde4a91880 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff2737fe700 1 -- 172.21.15.57:0/1556483467 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28402ea40 con 0x55fde4a91880 2023-10-08T13:58:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff29b60db80 1 -- 172.21.15.57:0/1556483467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 msgr2=0x55fde4a91c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff29b60db80 1 --2- 172.21.15.57:0/1556483467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a91c60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff284000c00 tx=0x7ff284020000 comp rx=0 tx=0).stop 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff29b60db80 1 -- 172.21.15.57:0/1556483467 shutdown_connections 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff29b60db80 1 --2- 172.21.15.57:0/1556483467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 0x55fde4a8c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff29b60db80 1 --2- 172.21.15.57:0/1556483467 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a91c60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff29b60db80 1 --2- 172.21.15.57:0/1556483467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a921a0 0x55fde4a8bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff29b60db80 1 -- 172.21.15.57:0/1556483467 >> 172.21.15.57:0/1556483467 conn(0x55fde4a00200 msgr2=0x55fde4a86b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff29b60db80 1 -- 172.21.15.57:0/1556483467 shutdown_connections 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.146+0000 7ff29b60db80 1 -- 172.21.15.57:0/1556483467 wait complete. 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff29b60db80 1 -- start start 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff29b60db80 1 Processor -- start 2023-10-08T13:58:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a8c0b0 0x55fde4a95580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a91880 0x55fde4a95ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 0x55fde4a96000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff29b60db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde49dedd0 con 0x55fde4a8c0b0 2023-10-08T13:58:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff29b60db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde497b6b0 con 0x55fde4a91880 2023-10-08T13:58:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff29b60db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde497b350 con 0x55fde4a921a0 2023-10-08T13:58:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff29b60db80 10 client.0 _ll_register_callbacks cb 0x55fde47d87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:58:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff28d1c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a8c0b0 0x55fde4a95580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff28d1c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a8c0b0 0x55fde4a95580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59108/0 (socket says 172.21.15.57:59108) 2023-10-08T13:58:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff28d1c0700 1 -- 172.21.15.57:0/2758528891 learned_addr learned my addr 172.21.15.57:0/2758528891 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff28d9c1700 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 0x55fde4a96000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.147+0000 7ff28c9bf700 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a91880 0x55fde4a95ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff28d9c1700 1 -- 172.21.15.57:0/2758528891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a91880 msgr2=0x55fde4a95ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff28d9c1700 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a91880 0x55fde4a95ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff28d9c1700 1 -- 172.21.15.57:0/2758528891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a8c0b0 msgr2=0x55fde4a95580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff28d9c1700 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a8c0b0 0x55fde4a95580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff28d9c1700 1 -- 172.21.15.57:0/2758528891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28810cd90 con 0x55fde4a921a0 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a8c0b0 0x55fde4a95580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff28d9c1700 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 0x55fde4a96000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff280004660 tx=0x7ff280004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f68f9f7cc428b8f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff270ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff270ff9700 1 -- 172.21.15.57:0/2758528891 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28001e400 con 0x55fde4a921a0 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff29b60db80 1 -- 172.21.15.57:0/2758528891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fde49a6e50 con 0x55fde4a921a0 2023-10-08T13:58:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff270ff9700 1 -- 172.21.15.57:0/2758528891 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9393+0+0 (secure 0 0 0) 0x7ff280016400 con 0x55fde4a921a0 2023-10-08T13:58:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff270ff9700 1 -- 172.21.15.57:0/2758528891 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2800165c0 con 0x55fde4a921a0 2023-10-08T13:58:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff29b60db80 10 client.26527 fetch_fsmap learned FSMap version 44 2023-10-08T13:58:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.148+0000 7ff29b60db80 1 -- 172.21.15.57:0/2758528891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55fde47f2c00 con 0x55fde4a921a0 2023-10-08T13:58:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff270ff9700 1 -- 172.21.15.57:0/2758528891 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fde49a6e50 con 0x55fde4a921a0 2023-10-08T13:58:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff270ff9700 1 -- 172.21.15.57:0/2758528891 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 44) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff28001c070 con 0x55fde4a921a0 2023-10-08T13:58:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff29b60db80 10 client.26527 fetch_fsmap finished waiting for FSMap version 44 2023-10-08T13:58:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff29b60db80 10 client.26527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:58:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff29b60db80 20 client.26527 populate_metadata read hostname 'smithi057' 2023-10-08T13:58:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff29b60db80 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fde4aa5c80 0x55fde4aa80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff29b60db80 4 client.26527 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff29b60db80 1 -- 172.21.15.57:0/2758528891 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fde49d7db0 con 0x55fde4aa5c80 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fde4aa5c80 0x55fde4aa80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff270ff9700 10 client.26527 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.149+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fde4aa5c80 0x55fde4aa80a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c9dabead4b0a51 server_cookie=ec81b19fbb65bb56 in_seq=0 out_seq=0 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff270ff9700 1 -- 172.21.15.57:0/2758528891 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fde49d7db0 con 0x55fde4aa5c80 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff270ff9700 10 client.26527 handle_command_reply: tid=0 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 client.26527 shutdown 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 -- 172.21.15.57:0/2758528891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 msgr2=0x55fde4a96000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 0x55fde4a96000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff280004660 tx=0x7ff280004970 comp rx=0 tx=0).stop 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 -- 172.21.15.57:0/2758528891 shutdown_connections 2023-10-08T13:58:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fde4aa5c80 0x55fde4aa80a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 0x55fde4a96000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff270ff9700 0 client.26527 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a8c0b0 0x55fde4a95580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 --2- 172.21.15.57:0/2758528891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a91880 0x55fde4a95ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 -- 172.21.15.57:0/2758528891 >> 172.21.15.57:0/2758528891 conn(0x55fde4a00200 msgr2=0x55fde4a86010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 -- 172.21.15.57:0/2758528891 shutdown_connections 2023-10-08T13:58:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 1 -- 172.21.15.57:0/2758528891 wait complete. 2023-10-08T13:58:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.150+0000 7ff29b60db80 20 client.26527 trim_cache size 0 max 16384 2023-10-08T13:58:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.165+0000 7ff29b60db80 1 Processor -- start 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.165+0000 7ff29b60db80 1 -- start start 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 0x55fde4a8bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a91880 0x55fde4a94e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a921a0 0x55fde4a9ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff29b60db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde485f8c0 con 0x55fde4a921a0 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff29b60db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde4a953a0 con 0x55fde4a91880 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff29b60db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde4a96b30 con 0x55fde4a8c0b0 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d9c1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a921a0 0x55fde4a9ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d1c0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 0x55fde4a8bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d9c1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a921a0 0x55fde4a9ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59120/0 (socket says 172.21.15.57:59120) 2023-10-08T13:58:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d1c0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 0x55fde4a8bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59928/0 (socket says 172.21.15.57:59928) 2023-10-08T13:58:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28c9bf700 1 --2- 172.21.15.57:0/4154802680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a91880 0x55fde4a94e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d9c1700 1 -- 172.21.15.57:0/4154802680 learned_addr learned my addr 172.21.15.57:0/4154802680 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d9c1700 1 -- 172.21.15.57:0/4154802680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 msgr2=0x55fde4a8bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d9c1700 1 --2- 172.21.15.57:0/4154802680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 0x55fde4a8bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d9c1700 1 -- 172.21.15.57:0/4154802680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a91880 msgr2=0x55fde4a94e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d9c1700 1 --2- 172.21.15.57:0/4154802680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a91880 0x55fde4a94e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d9c1700 1 -- 172.21.15.57:0/4154802680 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fde48301f0 con 0x55fde4a921a0 2023-10-08T13:58:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/4154802680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 0x55fde4a8bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.166+0000 7ff28d9c1700 1 --2- 172.21.15.57:0/4154802680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a921a0 0x55fde4a9ca80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff280000f80 tx=0x7ff280003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff948c6664cce16d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff273fff700 1 -- 172.21.15.57:0/4154802680 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28001eb50 con 0x55fde4a921a0 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff273fff700 1 -- 172.21.15.57:0/4154802680 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff28000f040 con 0x55fde4a921a0 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff273fff700 1 -- 172.21.15.57:0/4154802680 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28002c680 con 0x55fde4a921a0 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff29b60db80 1 -- 172.21.15.57:0/4154802680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a921a0 msgr2=0x55fde4a9ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff29b60db80 1 --2- 172.21.15.57:0/4154802680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a921a0 0x55fde4a9ca80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff280000f80 tx=0x7ff280003a00 comp rx=0 tx=0).stop 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff29b60db80 1 -- 172.21.15.57:0/4154802680 shutdown_connections 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff29b60db80 1 --2- 172.21.15.57:0/4154802680 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a921a0 0x55fde4a9ca80 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff29b60db80 1 --2- 172.21.15.57:0/4154802680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a8c0b0 0x55fde4a8bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff29b60db80 1 --2- 172.21.15.57:0/4154802680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a91880 0x55fde4a94e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff29b60db80 1 -- 172.21.15.57:0/4154802680 >> 172.21.15.57:0/4154802680 conn(0x55fde4a00200 msgr2=0x55fde4a90650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff29b60db80 1 -- 172.21.15.57:0/4154802680 shutdown_connections 2023-10-08T13:58:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.167+0000 7ff29b60db80 1 -- 172.21.15.57:0/4154802680 wait complete. 2023-10-08T13:58:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff29b60db80 1 -- start start 2023-10-08T13:58:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff29b60db80 1 Processor -- start 2023-10-08T13:58:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a8c0b0 0x55fde4a9aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a9afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff29b60db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 0x55fde4a9b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff29b60db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde4af8a90 con 0x55fde4a91880 2023-10-08T13:58:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff29b60db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde4af8bd0 con 0x55fde4a8c0b0 2023-10-08T13:58:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff29b60db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde4af8d10 con 0x55fde4a921a0 2023-10-08T13:58:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff29b60db80 10 client.0 _ll_register_callbacks cb 0x7ff280003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:58:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff28c9bf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a9afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff28c9bf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a9afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59124/0 (socket says 172.21.15.57:59124) 2023-10-08T13:58:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff28c9bf700 1 -- 172.21.15.57:0/262281441 learned_addr learned my addr 172.21.15.57:0/262281441 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff28d9c1700 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 0x55fde4a9b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.168+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a8c0b0 0x55fde4a9aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff28c9bf700 1 -- 172.21.15.57:0/262281441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 msgr2=0x55fde4a9b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff28c9bf700 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 0x55fde4a9b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff28c9bf700 1 -- 172.21.15.57:0/262281441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a8c0b0 msgr2=0x55fde4a9aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff28c9bf700 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a8c0b0 0x55fde4a9aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff28c9bf700 1 -- 172.21.15.57:0/262281441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fde4809d80 con 0x55fde4a91880 2023-10-08T13:58:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff28c9bf700 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a9afb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff278096c80 tx=0x7ff27809a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee91d44495ca3026 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff270ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:58:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff270ff9700 1 -- 172.21.15.57:0/262281441 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27809a670 con 0x55fde4a91880 2023-10-08T13:58:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff270ff9700 1 -- 172.21.15.57:0/262281441 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9393+0+0 (secure 0 0 0) 0x7ff27809a210 con 0x55fde4a91880 2023-10-08T13:58:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff29b60db80 1 -- 172.21.15.57:0/262281441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff28401e040 con 0x55fde4a91880 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff270ff9700 1 -- 172.21.15.57:0/262281441 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff27809e360 con 0x55fde4a91880 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff29b60db80 10 client.16938 fetch_fsmap learned FSMap version 44 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff29b60db80 1 -- 172.21.15.57:0/262281441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55fde47f2c00 con 0x55fde4a91880 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff270ff9700 1 -- 172.21.15.57:0/262281441 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff278063080 con 0x55fde4a91880 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff270ff9700 1 -- 172.21.15.57:0/262281441 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 44) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff278099820 con 0x55fde4a91880 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff29b60db80 10 client.16938 fetch_fsmap finished waiting for FSMap version 44 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff29b60db80 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff29b60db80 20 client.16938 populate_metadata read hostname 'smithi057' 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff29b60db80 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fde4afa520 0x55fde4afc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff29b60db80 4 client.16938 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:58:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.169+0000 7ff29b60db80 1 -- 172.21.15.57:0/262281441 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fde49d7ad0 con 0x55fde4afa520 2023-10-08T13:58:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.170+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fde4afa520 0x55fde4afc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.170+0000 7ff270ff9700 10 client.16938 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.170+0000 7ff28d1c0700 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fde4afa520 0x55fde4afc940 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c57c5fa5bb031b3 server_cookie=226eebe136c4f8a8 in_seq=0 out_seq=0 2023-10-08T13:58:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff270ff9700 1 -- 172.21.15.57:0/262281441 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fde49d7ad0 con 0x55fde4afa520 2023-10-08T13:58:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff270ff9700 10 client.16938 handle_command_reply: tid=0 2023-10-08T13:58:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 client.16938 shutdown 2023-10-08T13:58:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 -- 172.21.15.57:0/262281441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 msgr2=0x55fde4a9afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a9afb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff278096c80 tx=0x7ff27809a9e0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 -- 172.21.15.57:0/262281441 shutdown_connections 2023-10-08T13:58:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fde4afa520 0x55fde4afc940 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fde4a921a0 0x55fde4a9b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff270ff9700 0 client.16938 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fde4a8c0b0 0x55fde4a9aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 --2- 172.21.15.57:0/262281441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fde4a91880 0x55fde4a9afb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 -- 172.21.15.57:0/262281441 >> 172.21.15.57:0/262281441 conn(0x55fde4a00200 msgr2=0x55fde4a90ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 -- 172.21.15.57:0/262281441 shutdown_connections 2023-10-08T13:58:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.730+0000 7ff29b60db80 1 -- 172.21.15.57:0/262281441 wait complete. 2023-10-08T13:58:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff29b60db80 20 client.16938 trim_cache size 0 max 16384 2023-10-08T13:58:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:58:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:58:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:58:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:58:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 -- 172.21.15.57:0/220508152 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2740a15f0 msgr2=0x7ff2740a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2740a15f0 0x7ff2740a3ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff28400b6f0 tx=0x7ff284005bc0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 -- 172.21.15.57:0/220508152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2880fff90 msgr2=0x7ff288195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2880fff90 0x7ff288195f70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff27800e9f0 tx=0x7ff27800ed00 comp rx=0 tx=0).stop 2023-10-08T13:58:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 -- 172.21.15.57:0/220508152 shutdown_connections 2023-10-08T13:58:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2881045e0 0x7ff28819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2880ff5d0 0x7ff288195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff2740a15f0 0x7ff2740a3ab0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 --2- 172.21.15.57:0/220508152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2880fff90 0x7ff288195f70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 -- 172.21.15.57:0/220508152 >> 172.21.15.57:0/220508152 conn(0x7ff288076790 msgr2=0x7ff2880fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 -- 172.21.15.57:0/220508152 shutdown_connections 2023-10-08T13:58:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:24.731+0000 7ff28f424700 1 -- 172.21.15.57:0/220508152 wait complete. 2023-10-08T13:58:25.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:25 smithi115 ceph-mon[104408]: pgmap v507: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 165 op/s 2023-10-08T13:58:25.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:25 smithi057 ceph-mon[99476]: pgmap v507: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 165 op/s 2023-10-08T13:58:26.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:25 smithi099 ceph-mon[103849]: pgmap v507: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 165 op/s 2023-10-08T13:58:27.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:27 smithi115 ceph-mon[104408]: pgmap v508: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 280 op/s 2023-10-08T13:58:27.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:27 smithi057 ceph-mon[99476]: pgmap v508: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 280 op/s 2023-10-08T13:58:28.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:27 smithi099 ceph-mon[103849]: pgmap v508: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 280 op/s 2023-10-08T13:58:29.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:29 smithi115 ceph-mon[104408]: osdmap e95: 12 total, 12 up, 12 in 2023-10-08T13:58:29.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:29 smithi115 ceph-mon[104408]: pgmap v510: 129 pgs: 129 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 416 op/s 2023-10-08T13:58:29.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:29 smithi057 ceph-mon[99476]: osdmap e95: 12 total, 12 up, 12 in 2023-10-08T13:58:29.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:29 smithi057 ceph-mon[99476]: pgmap v510: 129 pgs: 129 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 416 op/s 2023-10-08T13:58:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:29 smithi099 ceph-mon[103849]: osdmap e95: 12 total, 12 up, 12 in 2023-10-08T13:58:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:29 smithi099 ceph-mon[103849]: pgmap v510: 129 pgs: 129 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 416 op/s 2023-10-08T13:58:31.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:31 smithi115 ceph-mon[104408]: pgmap v511: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 550 op/s 2023-10-08T13:58:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:31 smithi057 ceph-mon[99476]: pgmap v511: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 550 op/s 2023-10-08T13:58:32.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:31 smithi099 ceph-mon[103849]: pgmap v511: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 550 op/s 2023-10-08T13:58:33.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:33 smithi115 ceph-mon[104408]: pgmap v512: 129 pgs: 129 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 472 op/s 2023-10-08T13:58:33.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:33 smithi057 ceph-mon[99476]: pgmap v512: 129 pgs: 129 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 472 op/s 2023-10-08T13:58:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:33 smithi099 ceph-mon[103849]: pgmap v512: 129 pgs: 129 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.4 MiB/s wr, 472 op/s 2023-10-08T13:58:35.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:35 smithi115 ceph-mon[104408]: pgmap v513: 129 pgs: 129 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.4 MiB/s wr, 437 op/s 2023-10-08T13:58:35.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:35 smithi057 ceph-mon[99476]: pgmap v513: 129 pgs: 129 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.4 MiB/s wr, 437 op/s 2023-10-08T13:58:36.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:35 smithi099 ceph-mon[103849]: pgmap v513: 129 pgs: 129 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.4 MiB/s wr, 437 op/s 2023-10-08T13:58:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.845+0000 7f2a899e2700 1 -- 172.21.15.57:0/2363189137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a841045e0 msgr2=0x7f2a841049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.845+0000 7f2a899e2700 1 --2- 172.21.15.57:0/2363189137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a841045e0 0x7f2a841049c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009990 tx=0x7f2a78009ca0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.845+0000 7f2a899e2700 1 -- 172.21.15.57:0/2363189137 shutdown_connections 2023-10-08T13:58:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.845+0000 7f2a899e2700 1 --2- 172.21.15.57:0/2363189137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a840fff90 0x7f2a8410c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.845+0000 7f2a899e2700 1 --2- 172.21.15.57:0/2363189137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a841045e0 0x7f2a841049c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.845+0000 7f2a899e2700 1 --2- 172.21.15.57:0/2363189137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a840ff5d0 0x7f2a840ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.845+0000 7f2a899e2700 1 -- 172.21.15.57:0/2363189137 >> 172.21.15.57:0/2363189137 conn(0x7f2a84076790 msgr2=0x7f2a84076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.845+0000 7f2a899e2700 1 -- 172.21.15.57:0/2363189137 shutdown_connections 2023-10-08T13:58:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.845+0000 7f2a899e2700 1 -- 172.21.15.57:0/2363189137 wait complete. 2023-10-08T13:58:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.845+0000 7f2a899e2700 1 Processor -- start 2023-10-08T13:58:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a899e2700 1 -- start start 2023-10-08T13:58:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a899e2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a840ff5d0 0x7f2a84101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a899e2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a840fff90 0x7f2a84101760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a899e2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a841045e0 0x7f2a84101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a899e2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841a4fa0 con 0x7f2a840fff90 2023-10-08T13:58:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a899e2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841a5110 con 0x7f2a840ff5d0 2023-10-08T13:58:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a899e2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841a52d0 con 0x7f2a841045e0 2023-10-08T13:58:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a840fff90 0x7f2a84101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a840fff90 0x7f2a84101760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59138/0 (socket says 172.21.15.57:59138) 2023-10-08T13:58:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a827fc700 1 -- 172.21.15.57:0/1166506624 learned_addr learned my addr 172.21.15.57:0/1166506624 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a837fe700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a841045e0 0x7f2a84101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a840ff5d0 0x7f2a84101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a827fc700 1 -- 172.21.15.57:0/1166506624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a841045e0 msgr2=0x7f2a84101ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a827fc700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a841045e0 0x7f2a84101ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a827fc700 1 -- 172.21.15.57:0/1166506624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a840ff5d0 msgr2=0x7f2a84101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a827fc700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a840ff5d0 0x7f2a84101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.846+0000 7f2a827fc700 1 -- 172.21.15.57:0/1166506624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a74009520 con 0x7f2a840fff90 2023-10-08T13:58:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.847+0000 7f2a827fc700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a840fff90 0x7f2a84101760 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400eb30 tx=0x7f2a7400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.847+0000 7f2a889e0700 1 -- 172.21.15.57:0/1166506624 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74009880 con 0x7f2a840fff90 2023-10-08T13:58:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.847+0000 7f2a889e0700 1 -- 172.21.15.57:0/1166506624 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a740044e0 con 0x7f2a840fff90 2023-10-08T13:58:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.847+0000 7f2a899e2700 1 -- 172.21.15.57:0/1166506624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a780095f0 con 0x7f2a840fff90 2023-10-08T13:58:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.847+0000 7f2a889e0700 1 -- 172.21.15.57:0/1166506624 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74003d80 con 0x7f2a840fff90 2023-10-08T13:58:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.847+0000 7f2a899e2700 1 -- 172.21.15.57:0/1166506624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a841a5980 con 0x7f2a840fff90 2023-10-08T13:58:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.849+0000 7f2a889e0700 1 -- 172.21.15.57:0/1166506624 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2a74003f20 con 0x7f2a840fff90 2023-10-08T13:58:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.850+0000 7f2a889e0700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a6c0a1670 0x7f2a6c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.850+0000 7f2a889e0700 1 -- 172.21.15.57:0/1166506624 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2a74014070 con 0x7f2a840fff90 2023-10-08T13:58:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.850+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a6c0a1670 0x7f2a6c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.850+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a6c0a1670 0x7f2a6c0a3b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800b2e0 tx=0x7f2a78005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.850+0000 7f2a95bcbb80 1 Processor -- start 2023-10-08T13:58:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.850+0000 7f2a95bcbb80 1 -- start start 2023-10-08T13:58:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb8880 0x5579a0cb8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb91a0 0x5579a0cb2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0cb3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0c1b400 con 0x5579a0cb30b0 2023-10-08T13:58:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0c0cbc0 con 0x5579a0cb8880 2023-10-08T13:58:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0c0a4e0 con 0x5579a0cb91a0 2023-10-08T13:58:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0cb3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0cb3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59148/0 (socket says 172.21.15.57:59148) 2023-10-08T13:58:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a837fe700 1 -- 172.21.15.57:0/2235628201 learned_addr learned my addr 172.21.15.57:0/2235628201 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a827fc700 1 --2- 172.21.15.57:0/2235628201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb91a0 0x5579a0cb2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/2235628201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb8880 0x5579a0cb8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a827fc700 1 -- 172.21.15.57:0/2235628201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb8880 msgr2=0x5579a0cb8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a827fc700 1 --2- 172.21.15.57:0/2235628201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb8880 0x5579a0cb8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a827fc700 1 -- 172.21.15.57:0/2235628201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 msgr2=0x5579a0cb3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a827fc700 1 --2- 172.21.15.57:0/2235628201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0cb3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a827fc700 1 -- 172.21.15.57:0/2235628201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579a0a30d80 con 0x5579a0cb91a0 2023-10-08T13:58:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.851+0000 7f2a837fe700 1 --2- 172.21.15.57:0/2235628201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0cb3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a827fc700 1 --2- 172.21.15.57:0/2235628201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb91a0 0x5579a0cb2b70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2a740cad20 tx=0x7f2a7409a4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6fead85386d4895 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a69ffb700 1 -- 172.21.15.57:0/2235628201 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7409acb0 con 0x5579a0cb91a0 2023-10-08T13:58:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a69ffb700 1 -- 172.21.15.57:0/2235628201 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a7409a5b0 con 0x5579a0cb91a0 2023-10-08T13:58:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a69ffb700 1 -- 172.21.15.57:0/2235628201 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7401fec0 con 0x5579a0cb91a0 2023-10-08T13:58:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/2235628201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb91a0 msgr2=0x5579a0cb2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/2235628201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb91a0 0x5579a0cb2b70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2a740cad20 tx=0x7f2a7409a4b0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/2235628201 shutdown_connections 2023-10-08T13:58:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/2235628201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0cb3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/2235628201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb8880 0x5579a0cb8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/2235628201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb91a0 0x5579a0cb2b70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.852+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/2235628201 >> 172.21.15.57:0/2235628201 conn(0x5579a0c27200 msgr2=0x5579a0cadb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.853+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/2235628201 shutdown_connections 2023-10-08T13:58:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.853+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/2235628201 wait complete. 2023-10-08T13:58:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.853+0000 7f2a95bcbb80 1 -- start start 2023-10-08T13:58:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.853+0000 7f2a95bcbb80 1 Processor -- start 2023-10-08T13:58:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.853+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb30b0 0x5579a0cbc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.853+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb8880 0x5579a0cbca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.853+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 0x5579a0cbcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.853+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0c05dd0 con 0x5579a0cb8880 2023-10-08T13:58:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.853+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0ba26b0 con 0x5579a0cb91a0 2023-10-08T13:58:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.853+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0ba2350 con 0x5579a0cb30b0 2023-10-08T13:58:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a95bcbb80 10 client.0 _ll_register_callbacks cb 0x5579a09ff7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:58:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a82ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb30b0 0x5579a0cbc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb8880 0x5579a0cbca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a82ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb30b0 0x5579a0cbc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:59962/0 (socket says 172.21.15.57:59962) 2023-10-08T13:58:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb8880 0x5579a0cbca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59152/0 (socket says 172.21.15.57:59152) 2023-10-08T13:58:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a82ffd700 1 -- 172.21.15.57:0/1794588197 learned_addr learned my addr 172.21.15.57:0/1794588197 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a837fe700 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 0x5579a0cbcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a82ffd700 1 -- 172.21.15.57:0/1794588197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 msgr2=0x5579a0cbcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 0x5579a0cbcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a82ffd700 1 -- 172.21.15.57:0/1794588197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb8880 msgr2=0x5579a0cbca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb8880 0x5579a0cbca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a82ffd700 1 -- 172.21.15.57:0/1794588197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c011330 con 0x5579a0cb30b0 2023-10-08T13:58:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a827fc700 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb8880 0x5579a0cbca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb30b0 0x5579a0cbc550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009990 tx=0x7f2a7801a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb716e0e3a1f0fbf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T13:58:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a62ffd700 1 -- 172.21.15.57:0/1794588197 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78004c40 con 0x5579a0cb30b0 2023-10-08T13:58:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/1794588197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579a0bcde50 con 0x5579a0cb30b0 2023-10-08T13:58:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a62ffd700 1 -- 172.21.15.57:0/1794588197 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2a78004da0 con 0x5579a0cb30b0 2023-10-08T13:58:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a62ffd700 1 -- 172.21.15.57:0/1794588197 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a7800f460 con 0x5579a0cb30b0 2023-10-08T13:58:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.854+0000 7f2a95bcbb80 10 client.26545 fetch_fsmap learned FSMap version 44 2023-10-08T13:58:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/1794588197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f2a841a5980 con 0x5579a0cb30b0 2023-10-08T13:58:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a62ffd700 1 -- 172.21.15.57:0/1794588197 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5579a0bcde50 con 0x5579a0cb30b0 2023-10-08T13:58:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a62ffd700 1 -- 172.21.15.57:0/1794588197 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 44) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f2a780305e0 con 0x5579a0cb30b0 2023-10-08T13:58:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a95bcbb80 10 client.26545 fetch_fsmap finished waiting for FSMap version 44 2023-10-08T13:58:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a95bcbb80 10 client.26545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:58:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a95bcbb80 20 client.26545 populate_metadata read hostname 'smithi057' 2023-10-08T13:58:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5579a0cccd70 0x5579a0ccf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a95bcbb80 4 client.26545 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:58:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/1794588197 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579a0bfedb0 con 0x5579a0cccd70 2023-10-08T13:58:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a827fc700 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5579a0cccd70 0x5579a0ccf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a62ffd700 10 client.26545 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.855+0000 7f2a827fc700 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5579a0cccd70 0x5579a0ccf190 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9216e2a5f09ad761 server_cookie=af14648b0b5af1f5 in_seq=0 out_seq=0 2023-10-08T13:58:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a62ffd700 1 -- 172.21.15.57:0/1794588197 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5579a0bfedb0 con 0x5579a0cccd70 2023-10-08T13:58:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a62ffd700 10 client.26545 handle_command_reply: tid=0 2023-10-08T13:58:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 client.26545 shutdown 2023-10-08T13:58:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/1794588197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb30b0 msgr2=0x5579a0cbc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb30b0 0x5579a0cbc550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009990 tx=0x7f2a7801a050 comp rx=0 tx=0).stop 2023-10-08T13:58:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/1794588197 shutdown_connections 2023-10-08T13:58:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5579a0cccd70 0x5579a0ccf190 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 0x5579a0cbcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb30b0 0x5579a0cbc550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/1794588197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb8880 0x5579a0cbca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a62ffd700 0 client.26545 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/1794588197 >> 172.21.15.57:0/1794588197 conn(0x5579a0c27200 msgr2=0x5579a0cad010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/1794588197 shutdown_connections 2023-10-08T13:58:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.856+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/1794588197 wait complete. 2023-10-08T13:58:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.857+0000 7f2a95bcbb80 20 client.26545 trim_cache size 0 max 16384 2023-10-08T13:58:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.871+0000 7f2a95bcbb80 1 Processor -- start 2023-10-08T13:58:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.871+0000 7f2a95bcbb80 1 -- start start 2023-10-08T13:58:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.871+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0ccf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.871+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 0x5579a0cc3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.871+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 0x5579a0cad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.871+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0a868c0 con 0x5579a0cb30b0 2023-10-08T13:58:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.871+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0ca82b0 con 0x5579a0cb91a0 2023-10-08T13:58:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0cc9820 con 0x5579a0cb8880 2023-10-08T13:58:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a82ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0ccf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a82ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0ccf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59158/0 (socket says 172.21.15.57:59158) 2023-10-08T13:58:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a82ffd700 1 -- 172.21.15.57:0/623582259 learned_addr learned my addr 172.21.15.57:0/623582259 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a827fc700 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 0x5579a0cc3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a837fe700 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 0x5579a0cad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a82ffd700 1 -- 172.21.15.57:0/623582259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 msgr2=0x5579a0cc3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 0x5579a0cc3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a82ffd700 1 -- 172.21.15.57:0/623582259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 msgr2=0x5579a0cad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 0x5579a0cad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a82ffd700 1 -- 172.21.15.57:0/623582259 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579a0a19c00 con 0x5579a0cb30b0 2023-10-08T13:58:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a827fc700 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 0x5579a0cc3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a837fe700 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 0x5579a0cad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0ccf160 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006010 tx=0x7f2a78009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e14ec5ea732f00 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a6a7fc700 1 -- 172.21.15.57:0/623582259 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78030b60 con 0x5579a0cb30b0 2023-10-08T13:58:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a6a7fc700 1 -- 172.21.15.57:0/623582259 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a7801e040 con 0x5579a0cb30b0 2023-10-08T13:58:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.872+0000 7f2a6a7fc700 1 -- 172.21.15.57:0/623582259 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800f6a0 con 0x5579a0cb30b0 2023-10-08T13:58:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/623582259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 msgr2=0x5579a0ccf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0ccf160 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006010 tx=0x7f2a78009290 comp rx=0 tx=0).stop 2023-10-08T13:58:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/623582259 shutdown_connections 2023-10-08T13:58:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb91a0 0x5579a0cad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb30b0 0x5579a0ccf160 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/623582259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 0x5579a0cc3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/623582259 >> 172.21.15.57:0/623582259 conn(0x5579a0c27200 msgr2=0x5579a0cbce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/623582259 shutdown_connections 2023-10-08T13:58:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/623582259 wait complete. 2023-10-08T13:58:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 -- start start 2023-10-08T13:58:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.873+0000 7f2a95bcbb80 1 Processor -- start 2023-10-08T13:58:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb30b0 0x5579a0d192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 0x5579a0d197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a95bcbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb91a0 0x5579a0cc2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0cc3490 con 0x5579a0cb91a0 2023-10-08T13:58:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0cc3600 con 0x5579a0cb30b0 2023-10-08T13:58:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a95bcbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a0cc37c0 con 0x5579a0cb8880 2023-10-08T13:58:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a95bcbb80 10 client.0 _ll_register_callbacks cb 0x7f2a7800ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:58:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb91a0 0x5579a0cc2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb91a0 0x5579a0cc2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59168/0 (socket says 172.21.15.57:59168) 2023-10-08T13:58:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a837fe700 1 -- 172.21.15.57:0/4233111900 learned_addr learned my addr 172.21.15.57:0/4233111900 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a827fc700 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 0x5579a0d197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb30b0 0x5579a0d192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a837fe700 1 -- 172.21.15.57:0/4233111900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 msgr2=0x5579a0d197f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a837fe700 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 0x5579a0d197f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a837fe700 1 -- 172.21.15.57:0/4233111900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb30b0 msgr2=0x5579a0d192b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a837fe700 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb30b0 0x5579a0d192b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a837fe700 1 -- 172.21.15.57:0/4233111900 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579a0a30d80 con 0x5579a0cb91a0 2023-10-08T13:58:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a827fc700 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 0x5579a0d197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a837fe700 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb91a0 0x5579a0cc2eb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c018a40 tx=0x7f2a7c018d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2099b2c8131f7e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.874+0000 7f2a62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:58:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a62ffd700 1 -- 172.21.15.57:0/4233111900 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c01d070 con 0x5579a0cb91a0 2023-10-08T13:58:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a62ffd700 1 -- 172.21.15.57:0/4233111900 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2a7c003a00 con 0x5579a0cb91a0 2023-10-08T13:58:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/4233111900 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a7409a5b0 con 0x5579a0cb91a0 2023-10-08T13:58:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a62ffd700 1 -- 172.21.15.57:0/4233111900 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a7c00e040 con 0x5579a0cb91a0 2023-10-08T13:58:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a95bcbb80 10 client.16956 fetch_fsmap learned FSMap version 44 2023-10-08T13:58:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/4233111900 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f2a7c011650 con 0x5579a0cb91a0 2023-10-08T13:58:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a62ffd700 1 -- 172.21.15.57:0/4233111900 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a7409a5b0 con 0x5579a0cb91a0 2023-10-08T13:58:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a62ffd700 1 -- 172.21.15.57:0/4233111900 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 44) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f2a7c0040e0 con 0x5579a0cb91a0 2023-10-08T13:58:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a95bcbb80 10 client.16956 fetch_fsmap finished waiting for FSMap version 44 2023-10-08T13:58:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a95bcbb80 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:58:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a95bcbb80 20 client.16956 populate_metadata read hostname 'smithi057' 2023-10-08T13:58:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5579a0d21380 0x5579a0d237a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a95bcbb80 4 client.16956 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:58:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/4233111900 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5579a0bfead0 con 0x5579a0d21380 2023-10-08T13:58:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.875+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5579a0d21380 0x5579a0d237a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.876+0000 7f2a62ffd700 10 client.16956 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:37.876+0000 7f2a82ffd700 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5579a0d21380 0x5579a0d237a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec0c5cb68f1a6b4 server_cookie=225cd836717ac827 in_seq=0 out_seq=0 2023-10-08T13:58:37.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:37 smithi115 ceph-mon[104408]: pgmap v514: 129 pgs: 129 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 356 op/s 2023-10-08T13:58:37.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:37 smithi057 ceph-mon[99476]: pgmap v514: 129 pgs: 129 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 356 op/s 2023-10-08T13:58:38.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:37 smithi099 ceph-mon[103849]: pgmap v514: 129 pgs: 129 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.3 MiB/s wr, 356 op/s 2023-10-08T13:58:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.296+0000 7f2a62ffd700 1 -- 172.21.15.57:0/4233111900 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5579a0bfead0 con 0x5579a0d21380 2023-10-08T13:58:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.296+0000 7f2a62ffd700 10 client.16956 handle_command_reply: tid=0 2023-10-08T13:58:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.296+0000 7f2a95bcbb80 1 client.16956 shutdown 2023-10-08T13:58:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.297+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/4233111900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb91a0 msgr2=0x5579a0cc2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.297+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb91a0 0x5579a0cc2eb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c018a40 tx=0x7f2a7c018d50 comp rx=0 tx=0).stop 2023-10-08T13:58:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.298+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/4233111900 shutdown_connections 2023-10-08T13:58:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.298+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5579a0d21380 0x5579a0d237a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.298+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5579a0cb91a0 0x5579a0cc2eb0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.298+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579a0cb30b0 0x5579a0d192b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.298+0000 7f2a95bcbb80 1 --2- 172.21.15.57:0/4233111900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579a0cb8880 0x5579a0d197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.298+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/4233111900 >> 172.21.15.57:0/4233111900 conn(0x5579a0c27200 msgr2=0x5579a0cb7630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.298+0000 7f2a62ffd700 0 client.16956 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:58:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:58:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:58:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:58:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.300+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/4233111900 shutdown_connections 2023-10-08T13:58:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.301+0000 7f2a95bcbb80 1 -- 172.21.15.57:0/4233111900 wait complete. 2023-10-08T13:58:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.301+0000 7f2a95bcbb80 20 client.16956 trim_cache size 0 max 16384 2023-10-08T13:58:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.305+0000 7f2a899e2700 1 -- 172.21.15.57:0/1166506624 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a6c0a1670 msgr2=0x7f2a6c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.305+0000 7f2a899e2700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a6c0a1670 0x7f2a6c0a3b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800b2e0 tx=0x7f2a78005960 comp rx=0 tx=0).stop 2023-10-08T13:58:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.305+0000 7f2a899e2700 1 -- 172.21.15.57:0/1166506624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a840fff90 msgr2=0x7f2a84101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.305+0000 7f2a899e2700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a840fff90 0x7f2a84101760 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400eb30 tx=0x7f2a7400ee40 comp rx=0 tx=0).stop 2023-10-08T13:58:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.307+0000 7f2a899e2700 1 -- 172.21.15.57:0/1166506624 shutdown_connections 2023-10-08T13:58:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.307+0000 7f2a899e2700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a841045e0 0x7f2a84101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.307+0000 7f2a899e2700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a6c0a1670 0x7f2a6c0a3b30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.307+0000 7f2a899e2700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a840ff5d0 0x7f2a84101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.307+0000 7f2a899e2700 1 --2- 172.21.15.57:0/1166506624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a840fff90 0x7f2a84101760 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.307+0000 7f2a899e2700 1 -- 172.21.15.57:0/1166506624 >> 172.21.15.57:0/1166506624 conn(0x7f2a84076790 msgr2=0x7f2a84067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.310+0000 7f2a899e2700 1 -- 172.21.15.57:0/1166506624 shutdown_connections 2023-10-08T13:58:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:38.312+0000 7f2a899e2700 1 -- 172.21.15.57:0/1166506624 wait complete. 2023-10-08T13:58:39.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:39 smithi115 ceph-mon[104408]: pgmap v515: 129 pgs: 129 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 297 op/s 2023-10-08T13:58:39.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:39 smithi057 ceph-mon[99476]: pgmap v515: 129 pgs: 129 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 297 op/s 2023-10-08T13:58:39.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:39 smithi099 ceph-mon[103849]: pgmap v515: 129 pgs: 129 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 297 op/s 2023-10-08T13:58:41.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:41 smithi115 ceph-mon[104408]: pgmap v516: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.7 MiB/s wr, 404 op/s 2023-10-08T13:58:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:41 smithi057 ceph-mon[99476]: pgmap v516: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.7 MiB/s wr, 404 op/s 2023-10-08T13:58:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:41 smithi099 ceph-mon[103849]: pgmap v516: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.7 MiB/s wr, 404 op/s 2023-10-08T13:58:43.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:43 smithi115 ceph-mon[104408]: pgmap v517: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 354 op/s 2023-10-08T13:58:43.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:43 smithi057 ceph-mon[99476]: pgmap v517: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 354 op/s 2023-10-08T13:58:44.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:43 smithi099 ceph-mon[103849]: pgmap v517: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 354 op/s 2023-10-08T13:58:45.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:45 smithi115 ceph-mon[104408]: pgmap v518: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 393 op/s 2023-10-08T13:58:45.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:45 smithi057 ceph-mon[99476]: pgmap v518: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 393 op/s 2023-10-08T13:58:46.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:45 smithi099 ceph-mon[103849]: pgmap v518: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 393 op/s 2023-10-08T13:58:47.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:47 smithi115 ceph-mon[104408]: pgmap v519: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 402 op/s 2023-10-08T13:58:47.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:47 smithi057 ceph-mon[99476]: pgmap v519: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 402 op/s 2023-10-08T13:58:48.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:47 smithi099 ceph-mon[103849]: pgmap v519: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 402 op/s 2023-10-08T13:58:49.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:49 smithi115 ceph-mon[104408]: pgmap v520: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 406 op/s 2023-10-08T13:58:49.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:49 smithi057 ceph-mon[99476]: pgmap v520: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 406 op/s 2023-10-08T13:58:49.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:49 smithi099 ceph-mon[103849]: pgmap v520: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 406 op/s 2023-10-08T13:58:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.437+0000 7f2930335700 1 -- 172.21.15.57:0/3042866086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29280ffd70 msgr2=0x7f29281001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.437+0000 7f2930335700 1 --2- 172.21.15.57:0/3042866086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29280ffd70 0x7f29281001f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2918009990 tx=0x7f2918009ca0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.437+0000 7f2930335700 1 -- 172.21.15.57:0/3042866086 shutdown_connections 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.437+0000 7f2930335700 1 --2- 172.21.15.57:0/3042866086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2928100730 0x7f292810c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.437+0000 7f2930335700 1 --2- 172.21.15.57:0/3042866086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29281045b0 0x7f2928104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.437+0000 7f2930335700 1 --2- 172.21.15.57:0/3042866086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29280ffd70 0x7f29281001f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.437+0000 7f2930335700 1 -- 172.21.15.57:0/3042866086 >> 172.21.15.57:0/3042866086 conn(0x7f2928075bf0 msgr2=0x7f29280fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.437+0000 7f2930335700 1 -- 172.21.15.57:0/3042866086 shutdown_connections 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.437+0000 7f2930335700 1 -- 172.21.15.57:0/3042866086 wait complete. 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f2930335700 1 Processor -- start 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f2930335700 1 -- start start 2023-10-08T13:58:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f2930335700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29280ffd70 0x7f292819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f2930335700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100730 0x7f292819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f2930335700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29281045b0 0x7f2928196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f2930335700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2928196fa0 con 0x7f29281045b0 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f2930335700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2928197110 con 0x7f29280ffd70 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f2930335700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29281972d0 con 0x7f2928100730 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f292e8d2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29281045b0 0x7f2928196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f292e8d2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29281045b0 0x7f2928196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59182/0 (socket says 172.21.15.57:59182) 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f292e8d2700 1 -- 172.21.15.57:0/3811220314 learned_addr learned my addr 172.21.15.57:0/3811220314 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f292e0d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29280ffd70 0x7f292819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f292d8d0700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100730 0x7f292819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f292e8d2700 1 -- 172.21.15.57:0/3811220314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100730 msgr2=0x7f292819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f292e8d2700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100730 0x7f292819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.438+0000 7f292e8d2700 1 -- 172.21.15.57:0/3811220314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29280ffd70 msgr2=0x7f292819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.439+0000 7f292e8d2700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29280ffd70 0x7f292819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.439+0000 7f292e8d2700 1 -- 172.21.15.57:0/3811220314 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180095f0 con 0x7f29281045b0 2023-10-08T13:58:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.439+0000 7f292e8d2700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29281045b0 0x7f2928196940 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f292000efc0 tx=0x7f292000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.439+0000 7f291f7fe700 1 -- 172.21.15.57:0/3811220314 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920018070 con 0x7f29281045b0 2023-10-08T13:58:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.439+0000 7f291f7fe700 1 -- 172.21.15.57:0/3811220314 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f292000f040 con 0x7f29281045b0 2023-10-08T13:58:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.439+0000 7f2930335700 1 -- 172.21.15.57:0/3811220314 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2928197550 con 0x7f29281045b0 2023-10-08T13:58:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.439+0000 7f291f7fe700 1 -- 172.21.15.57:0/3811220314 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292000cc10 con 0x7f29281045b0 2023-10-08T13:58:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.439+0000 7f2930335700 1 -- 172.21.15.57:0/3811220314 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29281a3800 con 0x7f29281045b0 2023-10-08T13:58:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.441+0000 7f291f7fe700 1 -- 172.21.15.57:0/3811220314 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2920003a00 con 0x7f29281045b0 2023-10-08T13:58:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.441+0000 7f291f7fe700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f29140a15f0 0x7f29140a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.442+0000 7f291f7fe700 1 -- 172.21.15.57:0/3811220314 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f29200ccf20 con 0x7f29281045b0 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.442+0000 7f292e0d1700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f29140a15f0 0x7f29140a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.442+0000 7f292e0d1700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f29140a15f0 0x7f29140a3ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2924004630 tx=0x7f292400b3c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.442+0000 7f293c51eb80 1 Processor -- start 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f293c51eb80 1 -- start start 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1a880 0x556fdaa1ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1b1a0 0x556fdaa14b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa15510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f293c51eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fda97d400 con 0x556fdaa1b1a0 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f293c51eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fda96ebc0 con 0x556fdaa1a880 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f293c51eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fda96c4e0 con 0x556fdaa150b0 2023-10-08T13:58:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292d8d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1b1a0 0x556fdaa14b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292e0d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1a880 0x556fdaa1ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292e0d1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1a880 0x556fdaa1ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60612/0 (socket says 172.21.15.57:60612) 2023-10-08T13:58:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292d8d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1b1a0 0x556fdaa14b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59188/0 (socket says 172.21.15.57:59188) 2023-10-08T13:58:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292e0d1700 1 -- 172.21.15.57:0/2330859253 learned_addr learned my addr 172.21.15.57:0/2330859253 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292e8d2700 1 --2- 172.21.15.57:0/2330859253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa15510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292e0d1700 1 -- 172.21.15.57:0/2330859253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 msgr2=0x556fdaa15510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292e0d1700 1 --2- 172.21.15.57:0/2330859253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa15510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292e0d1700 1 -- 172.21.15.57:0/2330859253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1b1a0 msgr2=0x556fdaa14b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292e0d1700 1 --2- 172.21.15.57:0/2330859253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1b1a0 0x556fdaa14b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292e0d1700 1 -- 172.21.15.57:0/2330859253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fda792d80 con 0x556fdaa1a880 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.443+0000 7f292d8d0700 1 --2- 172.21.15.57:0/2330859253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1b1a0 0x556fdaa14b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.444+0000 7f292e0d1700 1 --2- 172.21.15.57:0/2330859253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1a880 0x556fdaa1ac60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2924003b10 tx=0x7f2924013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76144d70058073bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.444+0000 7f291cd82700 1 -- 172.21.15.57:0/2330859253 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29240042e0 con 0x556fdaa1a880 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.444+0000 7f291cd82700 1 -- 172.21.15.57:0/2330859253 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2924015070 con 0x556fdaa1a880 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.444+0000 7f291cd82700 1 -- 172.21.15.57:0/2330859253 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292401ed90 con 0x556fdaa1a880 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 -- 172.21.15.57:0/2330859253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1a880 msgr2=0x556fdaa1ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 --2- 172.21.15.57:0/2330859253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1a880 0x556fdaa1ac60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2924003b10 tx=0x7f2924013000 comp rx=0 tx=0).stop 2023-10-08T13:58:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 -- 172.21.15.57:0/2330859253 shutdown_connections 2023-10-08T13:58:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 --2- 172.21.15.57:0/2330859253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa15510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 --2- 172.21.15.57:0/2330859253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1a880 0x556fdaa1ac60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 --2- 172.21.15.57:0/2330859253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1b1a0 0x556fdaa14b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 -- 172.21.15.57:0/2330859253 >> 172.21.15.57:0/2330859253 conn(0x556fda989200 msgr2=0x556fdaa0fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 -- 172.21.15.57:0/2330859253 shutdown_connections 2023-10-08T13:58:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 -- 172.21.15.57:0/2330859253 wait complete. 2023-10-08T13:58:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 -- start start 2023-10-08T13:58:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 Processor -- start 2023-10-08T13:58:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa212a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa1e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa1ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fda967dd0 con 0x556fdaa1a880 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fda9046b0 con 0x556fdaa1b1a0 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.445+0000 7f293c51eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fda904350 con 0x556fdaa150b0 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f293c51eb80 10 client.0 _ll_register_callbacks cb 0x556fda7617a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292d8d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa1e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292d8d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa1e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59194/0 (socket says 172.21.15.57:59194) 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292d8d0700 1 -- 172.21.15.57:0/1786530693 learned_addr learned my addr 172.21.15.57:0/1786530693 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292e0d1700 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa212a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292e8d2700 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa1ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292d8d0700 1 -- 172.21.15.57:0/1786530693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 msgr2=0x556fdaa212a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292d8d0700 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa212a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292d8d0700 1 -- 172.21.15.57:0/1786530693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 msgr2=0x556fdaa1ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292d8d0700 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa1ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292d8d0700 1 -- 172.21.15.57:0/1786530693 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292810cd90 con 0x556fdaa1a880 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292e8d2700 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa1ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f292d8d0700 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa1e5e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2918009990 tx=0x7f2918005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e58417c6f060444 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f2911ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f2911ffb700 1 -- 172.21.15.57:0/1786530693 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291801d070 con 0x556fdaa1a880 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f293c51eb80 1 -- 172.21.15.57:0/1786530693 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fda92fe50 con 0x556fdaa1a880 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f2911ffb700 1 -- 172.21.15.57:0/1786530693 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2918004500 con 0x556fdaa1a880 2023-10-08T13:58:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f2911ffb700 1 -- 172.21.15.57:0/1786530693 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29180046c0 con 0x556fdaa1a880 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f293c51eb80 10 client.16968 fetch_fsmap learned FSMap version 45 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.446+0000 7f293c51eb80 1 -- 172.21.15.57:0/1786530693 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f29240037f0 con 0x556fdaa1a880 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f2911ffb700 1 -- 172.21.15.57:0/1786530693 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556fda92fe50 con 0x556fdaa1a880 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f2911ffb700 1 -- 172.21.15.57:0/1786530693 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f291800f860 con 0x556fdaa1a880 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f293c51eb80 10 client.16968 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f293c51eb80 10 client.16968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f293c51eb80 20 client.16968 populate_metadata read hostname 'smithi057' 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f293c51eb80 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556fdaa2ee20 0x556fdaa31240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f293c51eb80 4 client.16968 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f293c51eb80 1 -- 172.21.15.57:0/1786530693 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fda960db0 con 0x556fdaa2ee20 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f292e0d1700 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556fdaa2ee20 0x556fdaa31240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f2911ffb700 10 client.16968 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.447+0000 7f292e0d1700 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556fdaa2ee20 0x556fdaa31240 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcafa03f69c8a62f server_cookie=4b402bea1706c36d in_seq=0 out_seq=0 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f2911ffb700 1 -- 172.21.15.57:0/1786530693 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556fda960db0 con 0x556fdaa2ee20 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f2911ffb700 10 client.16968 handle_command_reply: tid=0 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 client.16968 shutdown 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 -- 172.21.15.57:0/1786530693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 msgr2=0x556fdaa1e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa1e5e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2918009990 tx=0x7f2918005800 comp rx=0 tx=0).stop 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 -- 172.21.15.57:0/1786530693 shutdown_connections 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556fdaa2ee20 0x556fdaa31240 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa1ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa212a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f2911ffb700 0 client.16968 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 --2- 172.21.15.57:0/1786530693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa1e5e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 -- 172.21.15.57:0/1786530693 >> 172.21.15.57:0/1786530693 conn(0x556fda989200 msgr2=0x556fdaa0f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 -- 172.21.15.57:0/1786530693 shutdown_connections 2023-10-08T13:58:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 1 -- 172.21.15.57:0/1786530693 wait complete. 2023-10-08T13:58:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.449+0000 7f293c51eb80 20 client.16968 trim_cache size 0 max 16384 2023-10-08T13:58:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f293c51eb80 1 Processor -- start 2023-10-08T13:58:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f293c51eb80 1 -- start start 2023-10-08T13:58:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa0f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa0fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa19660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f293c51eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fda7e88c0 con 0x556fdaa1a880 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f293c51eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fdaa11350 con 0x556fdaa1b1a0 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f293c51eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fdaa10610 con 0x556fdaa150b0 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292d8d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa0fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292d8d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa0fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59202/0 (socket says 172.21.15.57:59202) 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292d8d0700 1 -- 172.21.15.57:0/3449596910 learned_addr learned my addr 172.21.15.57:0/3449596910 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292e8d2700 1 --2- 172.21.15.57:0/3449596910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa19660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292e0d1700 1 --2- 172.21.15.57:0/3449596910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa0f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292d8d0700 1 -- 172.21.15.57:0/3449596910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 msgr2=0x556fdaa0f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292d8d0700 1 --2- 172.21.15.57:0/3449596910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa0f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292d8d0700 1 -- 172.21.15.57:0/3449596910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 msgr2=0x556fdaa19660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292d8d0700 1 --2- 172.21.15.57:0/3449596910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa19660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292d8d0700 1 -- 172.21.15.57:0/3449596910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fda77bc00 con 0x556fdaa1a880 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292e8d2700 1 --2- 172.21.15.57:0/3449596910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa19660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.464+0000 7f292d8d0700 1 --2- 172.21.15.57:0/3449596910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa0fcd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2918006010 tx=0x7f2918009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184a89fb2a27efc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f291d583700 1 -- 172.21.15.57:0/3449596910 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291801d070 con 0x556fdaa1a880 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f291d583700 1 -- 172.21.15.57:0/3449596910 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f291801e040 con 0x556fdaa1a880 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f291d583700 1 -- 172.21.15.57:0/3449596910 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918024a60 con 0x556fdaa1a880 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f293c51eb80 1 -- 172.21.15.57:0/3449596910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 msgr2=0x556fdaa0fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f293c51eb80 1 --2- 172.21.15.57:0/3449596910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa0fcd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2918006010 tx=0x7f2918009ca0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f293c51eb80 1 -- 172.21.15.57:0/3449596910 shutdown_connections 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f293c51eb80 1 --2- 172.21.15.57:0/3449596910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa19660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f293c51eb80 1 --2- 172.21.15.57:0/3449596910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa150b0 0x556fdaa0f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f293c51eb80 1 --2- 172.21.15.57:0/3449596910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa1a880 0x556fdaa0fcd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f293c51eb80 1 -- 172.21.15.57:0/3449596910 >> 172.21.15.57:0/3449596910 conn(0x556fda989200 msgr2=0x556fdaa1ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f293c51eb80 1 -- 172.21.15.57:0/3449596910 shutdown_connections 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.465+0000 7f293c51eb80 1 -- 172.21.15.57:0/3449596910 wait complete. 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f293c51eb80 1 -- start start 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f293c51eb80 1 Processor -- start 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa150b0 0x556fdaa16400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa1a880 0x556fdaa16940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f293c51eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa16e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f293c51eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fdaa17400 con 0x556fdaa150b0 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f293c51eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fdaa17540 con 0x556fdaa1b1a0 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f293c51eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fdaa17700 con 0x556fdaa1a880 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f293c51eb80 10 client.0 _ll_register_callbacks cb 0x7f2918005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f292e0d1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa150b0 0x556fdaa16400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f292e0d1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa150b0 0x556fdaa16400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59206/0 (socket says 172.21.15.57:59206) 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f292e0d1700 1 -- 172.21.15.57:0/2851758710 learned_addr learned my addr 172.21.15.57:0/2851758710 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f292d8d0700 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa1a880 0x556fdaa16940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f292e8d2700 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa16e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f292e0d1700 1 -- 172.21.15.57:0/2851758710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa1a880 msgr2=0x556fdaa16940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f292e0d1700 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa1a880 0x556fdaa16940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f292e0d1700 1 -- 172.21.15.57:0/2851758710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 msgr2=0x556fdaa16e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:58:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f292e0d1700 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa16e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.466+0000 7f292e0d1700 1 -- 172.21.15.57:0/2851758710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29281a3800 con 0x556fdaa150b0 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f292e0d1700 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa150b0 0x556fdaa16400 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2924003ae0 tx=0x7f29240042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b88af70e74bf4bf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f2911ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f2911ffb700 1 -- 172.21.15.57:0/2851758710 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292402b040 con 0x556fdaa150b0 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f2911ffb700 1 -- 172.21.15.57:0/2851758710 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2924017070 con 0x556fdaa150b0 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f293c51eb80 1 -- 172.21.15.57:0/2851758710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2924015070 con 0x556fdaa150b0 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f2911ffb700 1 -- 172.21.15.57:0/2851758710 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2924032980 con 0x556fdaa150b0 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f293c51eb80 10 client.16974 fetch_fsmap learned FSMap version 45 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f293c51eb80 1 -- 172.21.15.57:0/2851758710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f29240037f0 con 0x556fdaa150b0 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f2911ffb700 1 -- 172.21.15.57:0/2851758710 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2924015070 con 0x556fdaa150b0 2023-10-08T13:58:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f2911ffb700 1 -- 172.21.15.57:0/2851758710 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f2924011860 con 0x556fdaa150b0 2023-10-08T13:58:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f293c51eb80 10 client.16974 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T13:58:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f293c51eb80 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:58:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f293c51eb80 20 client.16974 populate_metadata read hostname 'smithi057' 2023-10-08T13:58:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f293c51eb80 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556fdaa836b0 0x556fdaa85ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:58:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f293c51eb80 4 client.16974 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:58:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.467+0000 7f293c51eb80 1 -- 172.21.15.57:0/2851758710 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556fda960ad0 con 0x556fdaa836b0 2023-10-08T13:58:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.468+0000 7f292d8d0700 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556fdaa836b0 0x556fdaa85ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:58:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.468+0000 7f2911ffb700 10 client.16974 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.468+0000 7f292d8d0700 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556fdaa836b0 0x556fdaa85ad0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f60012512581f1 server_cookie=8577e755700bf3aa in_seq=0 out_seq=0 2023-10-08T13:58:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.579+0000 7f2911ffb700 1 -- 172.21.15.57:0/2851758710 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556fda960ad0 con 0x556fdaa836b0 2023-10-08T13:58:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.579+0000 7f2911ffb700 10 client.16974 handle_command_reply: tid=0 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.579+0000 7f293c51eb80 1 client.16974 shutdown 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 1 -- 172.21.15.57:0/2851758710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa150b0 msgr2=0x556fdaa16400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa150b0 0x556fdaa16400 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2924003ae0 tx=0x7f29240042c0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 1 -- 172.21.15.57:0/2851758710 shutdown_connections 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556fdaa836b0 0x556fdaa85ad0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fdaa1b1a0 0x556fdaa16e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556fdaa150b0 0x556fdaa16400 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 1 --2- 172.21.15.57:0/2851758710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556fdaa1a880 0x556fdaa16940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f2911ffb700 0 client.16974 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 1 -- 172.21.15.57:0/2851758710 >> 172.21.15.57:0/2851758710 conn(0x556fda989200 msgr2=0x556fdaa19660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 1 -- 172.21.15.57:0/2851758710 shutdown_connections 2023-10-08T13:58:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 1 -- 172.21.15.57:0/2851758710 wait complete. 2023-10-08T13:58:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f293c51eb80 20 client.16974 trim_cache size 0 max 16384 2023-10-08T13:58:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:58:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:58:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:58:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:58:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f2930335700 1 -- 172.21.15.57:0/3811220314 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f29140a15f0 msgr2=0x7f29140a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f2930335700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f29140a15f0 0x7f29140a3ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2924004630 tx=0x7f292400b3c0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f2930335700 1 -- 172.21.15.57:0/3811220314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29281045b0 msgr2=0x7f2928196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:58:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f2930335700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29281045b0 0x7f2928196940 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f292000efc0 tx=0x7f292000c3a0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f2930335700 1 -- 172.21.15.57:0/3811220314 shutdown_connections 2023-10-08T13:58:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f2930335700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29281045b0 0x7f2928196940 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f2930335700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f29140a15f0 0x7f29140a3ab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f2930335700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29280ffd70 0x7f292819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.580+0000 7f2930335700 1 --2- 172.21.15.57:0/3811220314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100730 0x7f292819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:58:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.581+0000 7f2930335700 1 -- 172.21.15.57:0/3811220314 >> 172.21.15.57:0/3811220314 conn(0x7f2928075bf0 msgr2=0x7f29280ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:58:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.581+0000 7f2930335700 1 -- 172.21.15.57:0/3811220314 shutdown_connections 2023-10-08T13:58:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:58:51.581+0000 7f2930335700 1 -- 172.21.15.57:0/3811220314 wait complete. 2023-10-08T13:58:51.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:51 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:58:51.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:51 smithi099 ceph-mon[103849]: pgmap v521: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 424 op/s 2023-10-08T13:58:51.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:51 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:58:51.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:51 smithi115 ceph-mon[104408]: pgmap v521: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 424 op/s 2023-10-08T13:58:51.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:51 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T13:58:51.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:51 smithi057 ceph-mon[99476]: pgmap v521: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 424 op/s 2023-10-08T13:58:53.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:53 smithi115 ceph-mon[104408]: pgmap v522: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 312 op/s 2023-10-08T13:58:53.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:53 smithi057 ceph-mon[99476]: pgmap v522: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 312 op/s 2023-10-08T13:58:54.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:53 smithi099 ceph-mon[103849]: pgmap v522: 129 pgs: 129 active+clean; 1.9 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 312 op/s 2023-10-08T13:58:55.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:55 smithi115 ceph-mon[104408]: pgmap v523: 129 pgs: 129 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 309 op/s 2023-10-08T13:58:55.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:55 smithi057 ceph-mon[99476]: pgmap v523: 129 pgs: 129 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 309 op/s 2023-10-08T13:58:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:55 smithi099 ceph-mon[103849]: pgmap v523: 129 pgs: 129 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 309 op/s 2023-10-08T13:58:57.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:57 smithi115 ceph-mon[104408]: pgmap v524: 129 pgs: 129 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-10-08T13:58:57.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:57 smithi057 ceph-mon[99476]: pgmap v524: 129 pgs: 129 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-10-08T13:58:58.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:57 smithi099 ceph-mon[103849]: pgmap v524: 129 pgs: 129 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2023-10-08T13:58:59.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:58:59 smithi057 ceph-mon[99476]: pgmap v525: 129 pgs: 129 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 292 op/s 2023-10-08T13:58:59.526 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:58:59 smithi099 ceph-mon[103849]: pgmap v525: 129 pgs: 129 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 292 op/s 2023-10-08T13:58:59.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:58:59 smithi115 ceph-mon[104408]: pgmap v525: 129 pgs: 129 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 292 op/s 2023-10-08T13:59:01.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:01 smithi115 ceph-mon[104408]: pgmap v526: 129 pgs: 129 active+clean; 2.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.2 MiB/s wr, 292 op/s 2023-10-08T13:59:01.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:01 smithi057 ceph-mon[99476]: pgmap v526: 129 pgs: 129 active+clean; 2.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.2 MiB/s wr, 292 op/s 2023-10-08T13:59:02.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:01 smithi099 ceph-mon[103849]: pgmap v526: 129 pgs: 129 active+clean; 2.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.2 MiB/s wr, 292 op/s 2023-10-08T13:59:03.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:03 smithi115 ceph-mon[104408]: pgmap v527: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 241 op/s 2023-10-08T13:59:03.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:03 smithi057 ceph-mon[99476]: pgmap v527: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 241 op/s 2023-10-08T13:59:04.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:03 smithi099 ceph-mon[103849]: pgmap v527: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 241 op/s 2023-10-08T13:59:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.742+0000 7fda4ef66700 1 -- 172.21.15.57:0/3948446367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda4810f380 msgr2=0x7fda48107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.742+0000 7fda4ef66700 1 --2- 172.21.15.57:0/3948446367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda4810f380 0x7fda48107d30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fda380099c0 tx=0x7fda38009cd0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.742+0000 7fda4ef66700 1 -- 172.21.15.57:0/3948446367 shutdown_connections 2023-10-08T13:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.742+0000 7fda4ef66700 1 --2- 172.21.15.57:0/3948446367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda48119fd0 0x7fda4811c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.742+0000 7fda4ef66700 1 --2- 172.21.15.57:0/3948446367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4810ef10 0x7fda481077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.742+0000 7fda4ef66700 1 --2- 172.21.15.57:0/3948446367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda4810f380 0x7fda48107d30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.742+0000 7fda4ef66700 1 -- 172.21.15.57:0/3948446367 >> 172.21.15.57:0/3948446367 conn(0x7fda4806b7c0 msgr2=0x7fda4806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.744+0000 7fda4ef66700 1 -- 172.21.15.57:0/3948446367 shutdown_connections 2023-10-08T13:59:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.744+0000 7fda4ef66700 1 -- 172.21.15.57:0/3948446367 wait complete. 2023-10-08T13:59:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.745+0000 7fda4ef66700 1 Processor -- start 2023-10-08T13:59:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.745+0000 7fda4ef66700 1 -- start start 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4ef66700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4810ef10 0x7fda48117890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4ef66700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda4810f380 0x7fda4810f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4ef66700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda48119fd0 0x7fda4810fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4ef66700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda48110570 con 0x7fda4810f380 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4ef66700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda481106e0 con 0x7fda4810ef10 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4ef66700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda481ba830 con 0x7fda48119fd0 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4cd02700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4810ef10 0x7fda48117890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4cd02700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4810ef10 0x7fda48117890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60648/0 (socket says 172.21.15.57:60648) 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4cd02700 1 -- 172.21.15.57:0/2185292590 learned_addr learned my addr 172.21.15.57:0/2185292590 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4cd02700 1 -- 172.21.15.57:0/2185292590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda48119fd0 msgr2=0x7fda4810fde0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4cd02700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda48119fd0 0x7fda4810fde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4cd02700 1 -- 172.21.15.57:0/2185292590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda4810f380 msgr2=0x7fda4810f8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T13:59:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4cd02700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda4810f380 0x7fda4810f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.746+0000 7fda4cd02700 1 -- 172.21.15.57:0/2185292590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda380095f0 con 0x7fda4810ef10 2023-10-08T13:59:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.747+0000 7fda4cd02700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4810ef10 0x7fda48117890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda44007ee0 tx=0x7fda4402f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.747+0000 7fda3dffb700 1 -- 172.21.15.57:0/2185292590 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4402fea0 con 0x7fda4810ef10 2023-10-08T13:59:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.747+0000 7fda3dffb700 1 -- 172.21.15.57:0/2185292590 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda440378d0 con 0x7fda4810ef10 2023-10-08T13:59:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.747+0000 7fda4ef66700 1 -- 172.21.15.57:0/2185292590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda481a5860 con 0x7fda4810ef10 2023-10-08T13:59:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.747+0000 7fda4ef66700 1 -- 172.21.15.57:0/2185292590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda481a5d70 con 0x7fda4810ef10 2023-10-08T13:59:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.748+0000 7fda3dffb700 1 -- 172.21.15.57:0/2185292590 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440368f0 con 0x7fda4810ef10 2023-10-08T13:59:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.751+0000 7fda3dffb700 1 -- 172.21.15.57:0/2185292590 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fda44037410 con 0x7fda4810ef10 2023-10-08T13:59:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.752+0000 7fda3dffb700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fda300a1780 0x7fda300a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.752+0000 7fda3dffb700 1 -- 172.21.15.57:0/2185292590 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fda440ef350 con 0x7fda4810ef10 2023-10-08T13:59:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.754+0000 7fda3ffff700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fda300a1780 0x7fda300a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.755+0000 7fda5b14fb80 1 Processor -- start 2023-10-08T13:59:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.755+0000 7fda5b14fb80 1 -- start start 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.755+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde044880 0x556cde044c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.755+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde0451a0 0x556cde03eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.755+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde03f0b0 0x556cde03f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.755+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddfa7400 con 0x556cde044880 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.755+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddf98bc0 con 0x556cde03f0b0 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.755+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddf964e0 con 0x556cde0451a0 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.756+0000 7fda4d503700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde03f0b0 0x556cde03f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.756+0000 7fda4d503700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde03f0b0 0x556cde03f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60654/0 (socket says 172.21.15.57:60654) 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.756+0000 7fda4d503700 1 -- 172.21.15.57:0/1270431787 learned_addr learned my addr 172.21.15.57:0/1270431787 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.756+0000 7fda3ffff700 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde0451a0 0x556cde03eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.756+0000 7fda4cd02700 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde044880 0x556cde044c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.757+0000 7fda4d503700 1 -- 172.21.15.57:0/1270431787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde0451a0 msgr2=0x556cde03eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.757+0000 7fda4d503700 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde0451a0 0x556cde03eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.757+0000 7fda4d503700 1 -- 172.21.15.57:0/1270431787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde044880 msgr2=0x556cde044c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.757+0000 7fda4d503700 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde044880 0x556cde044c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.757+0000 7fda4d503700 1 -- 172.21.15.57:0/1270431787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cdddbcd80 con 0x556cde03f0b0 2023-10-08T13:59:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.757+0000 7fda3ffff700 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde0451a0 0x556cde03eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.757+0000 7fda4cd02700 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde044880 0x556cde044c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:59:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.757+0000 7fda4d503700 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde03f0b0 0x556cde03f510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda40009750 tx=0x7fda40009a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16a275b3ad317ae7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.757+0000 7fda3ffff700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fda300a1780 0x7fda300a3c40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fda48110fd0 tx=0x7fda38005da0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.760+0000 7fda2effd700 1 -- 172.21.15.57:0/1270431787 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40011830 con 0x556cde03f0b0 2023-10-08T13:59:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.760+0000 7fda2effd700 1 -- 172.21.15.57:0/1270431787 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda40011e50 con 0x556cde03f0b0 2023-10-08T13:59:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.760+0000 7fda2effd700 1 -- 172.21.15.57:0/1270431787 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4000f2e0 con 0x556cde03f0b0 2023-10-08T13:59:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.761+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1270431787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde03f0b0 msgr2=0x556cde03f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.761+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde03f0b0 0x556cde03f510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda40009750 tx=0x7fda40009a60 comp rx=0 tx=0).stop 2023-10-08T13:59:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.761+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1270431787 shutdown_connections 2023-10-08T13:59:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.761+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde03f0b0 0x556cde03f510 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.761+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde044880 0x556cde044c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.761+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/1270431787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde0451a0 0x556cde03eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.761+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1270431787 >> 172.21.15.57:0/1270431787 conn(0x556cddfb3200 msgr2=0x556cde039b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.762+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1270431787 shutdown_connections 2023-10-08T13:59:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.762+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1270431787 wait complete. 2023-10-08T13:59:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.762+0000 7fda5b14fb80 1 -- start start 2023-10-08T13:59:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.762+0000 7fda5b14fb80 1 Processor -- start 2023-10-08T13:59:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.763+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde03f0b0 0x556cde0484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.763+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde048a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.763+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 0x556cde048f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.763+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddf91dd0 con 0x556cde0451a0 2023-10-08T13:59:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.763+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddf2e6b0 con 0x556cde044880 2023-10-08T13:59:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.763+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddf2e350 con 0x556cde03f0b0 2023-10-08T13:59:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.763+0000 7fda5b14fb80 10 client.0 _ll_register_callbacks cb 0x556cddd8b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:59:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.764+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde048a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.764+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde048a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60664/0 (socket says 172.21.15.57:60664) 2023-10-08T13:59:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.764+0000 7fda3ffff700 1 -- 172.21.15.57:0/1022483234 learned_addr learned my addr 172.21.15.57:0/1022483234 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.765+0000 7fda4d503700 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 0x556cde048f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.765+0000 7fda3ffff700 1 -- 172.21.15.57:0/1022483234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde03f0b0 msgr2=0x556cde0484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.765+0000 7fda3ffff700 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde03f0b0 0x556cde0484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.765+0000 7fda3ffff700 1 -- 172.21.15.57:0/1022483234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 msgr2=0x556cde048f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.765+0000 7fda3ffff700 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 0x556cde048f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.765+0000 7fda3ffff700 1 -- 172.21.15.57:0/1022483234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4811cc50 con 0x556cde044880 2023-10-08T13:59:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.765+0000 7fda3ffff700 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde048a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda38006010 tx=0x7fda38005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff691ad71d21e0eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.765+0000 7fda27fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:59:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.769+0000 7fda27fff700 1 -- 172.21.15.57:0/1022483234 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3802a890 con 0x556cde044880 2023-10-08T13:59:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.769+0000 7fda27fff700 1 -- 172.21.15.57:0/1022483234 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fda3802a9f0 con 0x556cde044880 2023-10-08T13:59:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.769+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1022483234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cddf59e50 con 0x556cde044880 2023-10-08T13:59:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.770+0000 7fda27fff700 1 -- 172.21.15.57:0/1022483234 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda38005a00 con 0x556cde044880 2023-10-08T13:59:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.770+0000 7fda5b14fb80 10 client.26540 fetch_fsmap learned FSMap version 45 2023-10-08T13:59:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.770+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1022483234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fda40009380 con 0x556cde044880 2023-10-08T13:59:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.771+0000 7fda27fff700 1 -- 172.21.15.57:0/1022483234 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556cddf59e50 con 0x556cde044880 2023-10-08T13:59:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.771+0000 7fda27fff700 1 -- 172.21.15.57:0/1022483234 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fda3800f920 con 0x556cde044880 2023-10-08T13:59:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.771+0000 7fda5b14fb80 10 client.26540 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T13:59:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.771+0000 7fda5b14fb80 10 client.26540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:59:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.771+0000 7fda5b14fb80 20 client.26540 populate_metadata read hostname 'smithi057' 2023-10-08T13:59:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.771+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556cde058ce0 0x556cde05b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.771+0000 7fda5b14fb80 4 client.26540 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:59:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.771+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1022483234 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cddf8adb0 con 0x556cde058ce0 2023-10-08T13:59:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.771+0000 7fda4cd02700 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556cde058ce0 0x556cde05b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.772+0000 7fda4cd02700 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556cde058ce0 0x556cde05b100 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec72e8dca663e9e9 server_cookie=578b599a1d2314e9 in_seq=0 out_seq=0 2023-10-08T13:59:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.776+0000 7fda27fff700 10 client.26540 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.776+0000 7fda27fff700 1 -- 172.21.15.57:0/1022483234 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556cddf8adb0 con 0x556cde058ce0 2023-10-08T13:59:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.776+0000 7fda27fff700 10 client.26540 handle_command_reply: tid=0 2023-10-08T13:59:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.776+0000 7fda5b14fb80 1 client.26540 shutdown 2023-10-08T13:59:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.777+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1022483234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 msgr2=0x556cde048a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.777+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde048a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda38006010 tx=0x7fda38005c40 comp rx=0 tx=0).stop 2023-10-08T13:59:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.786+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1022483234 shutdown_connections 2023-10-08T13:59:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.786+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556cde058ce0 0x556cde05b100 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.786+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 0x556cde048f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.786+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde03f0b0 0x556cde0484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.786+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/1022483234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde048a10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.786+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1022483234 >> 172.21.15.57:0/1022483234 conn(0x556cddfb3200 msgr2=0x556cde039010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.786+0000 7fda27fff700 0 client.26540 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.797+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1022483234 shutdown_connections 2023-10-08T13:59:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.797+0000 7fda5b14fb80 1 -- 172.21.15.57:0/1022483234 wait complete. 2023-10-08T13:59:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.800+0000 7fda5b14fb80 20 client.26540 trim_cache size 0 max 16384 2023-10-08T13:59:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.828+0000 7fda5b14fb80 1 Processor -- start 2023-10-08T13:59:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.828+0000 7fda5b14fb80 1 -- start start 2023-10-08T13:59:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.828+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde03f0b0 0x556cde03eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.828+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde039790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.828+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde0451a0 0x556cde039cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.828+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cdde128c0 con 0x556cde03f0b0 2023-10-08T13:59:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.828+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cde046590 con 0x556cde044880 2023-10-08T13:59:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.828+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cde036830 con 0x556cde0451a0 2023-10-08T13:59:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.829+0000 7fda4cd02700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde03f0b0 0x556cde03eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.829+0000 7fda4cd02700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde03f0b0 0x556cde03eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59242/0 (socket says 172.21.15.57:59242) 2023-10-08T13:59:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.829+0000 7fda4cd02700 1 -- 172.21.15.57:0/3672163187 learned_addr learned my addr 172.21.15.57:0/3672163187 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.829+0000 7fda3ffff700 1 --2- 172.21.15.57:0/3672163187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde039790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.829+0000 7fda4cd02700 1 -- 172.21.15.57:0/3672163187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde0451a0 msgr2=0x556cde039cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T13:59:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.829+0000 7fda4cd02700 1 --2- 172.21.15.57:0/3672163187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde0451a0 0x556cde039cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.829+0000 7fda4cd02700 1 -- 172.21.15.57:0/3672163187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 msgr2=0x556cde039790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.829+0000 7fda4cd02700 1 --2- 172.21.15.57:0/3672163187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde039790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.829+0000 7fda4cd02700 1 -- 172.21.15.57:0/3672163187 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cde03f760 con 0x556cde03f0b0 2023-10-08T13:59:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.829+0000 7fda3ffff700 1 --2- 172.21.15.57:0/3672163187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde039790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.830+0000 7fda4cd02700 1 --2- 172.21.15.57:0/3672163187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde03f0b0 0x556cde03eb70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fda440b8530 tx=0x7fda44000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752831066eb63fcc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.832+0000 7fda2f7fe700 1 -- 172.21.15.57:0/3672163187 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440bc030 con 0x556cde03f0b0 2023-10-08T13:59:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.832+0000 7fda2f7fe700 1 -- 172.21.15.57:0/3672163187 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda440c0180 con 0x556cde03f0b0 2023-10-08T13:59:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.832+0000 7fda2f7fe700 1 -- 172.21.15.57:0/3672163187 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440bc290 con 0x556cde03f0b0 2023-10-08T13:59:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.833+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3672163187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde03f0b0 msgr2=0x556cde03eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.833+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/3672163187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde03f0b0 0x556cde03eb70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fda440b8530 tx=0x7fda44000f40 comp rx=0 tx=0).stop 2023-10-08T13:59:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.833+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3672163187 shutdown_connections 2023-10-08T13:59:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.833+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/3672163187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde0451a0 0x556cde039cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.833+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/3672163187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde03f0b0 0x556cde03eb70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.833+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/3672163187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde039790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.833+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3672163187 >> 172.21.15.57:0/3672163187 conn(0x556cddfb3200 msgr2=0x556cde042620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.834+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3672163187 shutdown_connections 2023-10-08T13:59:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.834+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3672163187 wait complete. 2023-10-08T13:59:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.834+0000 7fda5b14fb80 1 -- start start 2023-10-08T13:59:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.834+0000 7fda5b14fb80 1 Processor -- start 2023-10-08T13:59:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde03f0b0 0x556cde04dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde04e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda5b14fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 0x556cde04e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cde0abc40 con 0x556cde0451a0 2023-10-08T13:59:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cde0abdb0 con 0x556cde044880 2023-10-08T13:59:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda5b14fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cde0abf70 con 0x556cde03f0b0 2023-10-08T13:59:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda5b14fb80 10 client.0 _ll_register_callbacks cb 0x7fda3800bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:59:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde04e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde04e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60678/0 (socket says 172.21.15.57:60678) 2023-10-08T13:59:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda3ffff700 1 -- 172.21.15.57:0/3719001914 learned_addr learned my addr 172.21.15.57:0/3719001914 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.835+0000 7fda4d503700 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 0x556cde04e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.836+0000 7fda4cd02700 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde03f0b0 0x556cde04dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.836+0000 7fda4d503700 1 -- 172.21.15.57:0/3719001914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde03f0b0 msgr2=0x556cde04dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.836+0000 7fda4d503700 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde03f0b0 0x556cde04dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.836+0000 7fda4d503700 1 -- 172.21.15.57:0/3719001914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 msgr2=0x556cde04e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.836+0000 7fda4d503700 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde04e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.836+0000 7fda4d503700 1 -- 172.21.15.57:0/3719001914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda40009380 con 0x556cde0451a0 2023-10-08T13:59:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.837+0000 7fda4d503700 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 0x556cde04e6b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fda40009720 tx=0x7fda40010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd69992c6a220eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.837+0000 7fda27fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:59:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.837+0000 7fda27fff700 1 -- 172.21.15.57:0/3719001914 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40010580 con 0x556cde0451a0 2023-10-08T13:59:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.837+0000 7fda27fff700 1 -- 172.21.15.57:0/3719001914 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fda400106e0 con 0x556cde0451a0 2023-10-08T13:59:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.837+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3719001914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda40011e50 con 0x556cde0451a0 2023-10-08T13:59:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.837+0000 7fda27fff700 1 -- 172.21.15.57:0/3719001914 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda400108c0 con 0x556cde0451a0 2023-10-08T13:59:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.837+0000 7fda5b14fb80 10 client.16986 fetch_fsmap learned FSMap version 45 2023-10-08T13:59:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.837+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3719001914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fda481a5d70 con 0x556cde0451a0 2023-10-08T13:59:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.837+0000 7fda27fff700 1 -- 172.21.15.57:0/3719001914 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda40011e50 con 0x556cde0451a0 2023-10-08T13:59:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.838+0000 7fda27fff700 1 -- 172.21.15.57:0/3719001914 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fda400248f0 con 0x556cde0451a0 2023-10-08T13:59:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.838+0000 7fda5b14fb80 10 client.16986 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T13:59:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.838+0000 7fda5b14fb80 10 client.16986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:59:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.838+0000 7fda5b14fb80 20 client.16986 populate_metadata read hostname 'smithi057' 2023-10-08T13:59:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.838+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556cde0ad7c0 0x556cde0afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.838+0000 7fda5b14fb80 4 client.16986 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:59:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.838+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3719001914 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556cddf8aad0 con 0x556cde0ad7c0 2023-10-08T13:59:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.838+0000 7fda4cd02700 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556cde0ad7c0 0x556cde0afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.851+0000 7fda4cd02700 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556cde0ad7c0 0x556cde0afbe0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2f9beaee8db609 server_cookie=e5fde7dbe90e57a5 in_seq=0 out_seq=0 2023-10-08T13:59:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:04.851+0000 7fda27fff700 10 client.16986 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:05.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:05 smithi115 ceph-mon[104408]: pgmap v528: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 243 op/s 2023-10-08T13:59:05.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:05 smithi115 ceph-mon[104408]: osdmap e96: 12 total, 12 up, 12 in 2023-10-08T13:59:05.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:05 smithi057 ceph-mon[99476]: pgmap v528: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 243 op/s 2023-10-08T13:59:05.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:05 smithi057 ceph-mon[99476]: osdmap e96: 12 total, 12 up, 12 in 2023-10-08T13:59:06.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:05 smithi099 ceph-mon[103849]: pgmap v528: 129 pgs: 129 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 243 op/s 2023-10-08T13:59:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:05 smithi099 ceph-mon[103849]: osdmap e96: 12 total, 12 up, 12 in 2023-10-08T13:59:07.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:07 smithi115 ceph-mon[104408]: pgmap v530: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.1 MiB/s wr, 212 op/s 2023-10-08T13:59:07.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:07 smithi057 ceph-mon[99476]: pgmap v530: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.1 MiB/s wr, 212 op/s 2023-10-08T13:59:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:07 smithi099 ceph-mon[103849]: pgmap v530: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.1 MiB/s wr, 212 op/s 2023-10-08T13:59:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.374+0000 7fda27fff700 1 -- 172.21.15.57:0/3719001914 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556cddf8aad0 con 0x556cde0ad7c0 2023-10-08T13:59:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.374+0000 7fda27fff700 10 client.16986 handle_command_reply: tid=0 2023-10-08T13:59:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.379+0000 7fda5b14fb80 1 client.16986 shutdown 2023-10-08T13:59:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.382+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3719001914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 msgr2=0x556cde04e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.382+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 0x556cde04e6b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fda40009720 tx=0x7fda40010cc0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.387+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3719001914 shutdown_connections 2023-10-08T13:59:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.387+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556cde0ad7c0 0x556cde0afbe0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.387+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556cde0451a0 0x556cde04e6b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.387+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cde03f0b0 0x556cde04dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.387+0000 7fda5b14fb80 1 --2- 172.21.15.57:0/3719001914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cde044880 0x556cde04e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.387+0000 7fda27fff700 0 client.16986 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.387+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3719001914 >> 172.21.15.57:0/3719001914 conn(0x556cddfb3200 msgr2=0x556cde0435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.391+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3719001914 shutdown_connections 2023-10-08T13:59:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.391+0000 7fda5b14fb80 1 -- 172.21.15.57:0/3719001914 wait complete. 2023-10-08T13:59:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.391+0000 7fda5b14fb80 20 client.16986 trim_cache size 0 max 16384 2023-10-08T13:59:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:59:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:59:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:59:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:59:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.394+0000 7fda4ef66700 1 -- 172.21.15.57:0/2185292590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fda300a1780 msgr2=0x7fda300a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.394+0000 7fda4ef66700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fda300a1780 0x7fda300a3c40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fda48110fd0 tx=0x7fda38005da0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.394+0000 7fda4ef66700 1 -- 172.21.15.57:0/2185292590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4810ef10 msgr2=0x7fda48117890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.394+0000 7fda4ef66700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4810ef10 0x7fda48117890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda44007ee0 tx=0x7fda4402f3c0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.396+0000 7fda4ef66700 1 -- 172.21.15.57:0/2185292590 shutdown_connections 2023-10-08T13:59:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.396+0000 7fda4ef66700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda48119fd0 0x7fda4810fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.396+0000 7fda4ef66700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fda300a1780 0x7fda300a3c40 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fda48110fd0 tx=0x7fda38005da0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.396+0000 7fda4ef66700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4810ef10 0x7fda48117890 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda44007ee0 tx=0x7fda4402f3c0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.396+0000 7fda4ef66700 1 --2- 172.21.15.57:0/2185292590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda4810f380 0x7fda4810f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.396+0000 7fda4ef66700 1 -- 172.21.15.57:0/2185292590 >> 172.21.15.57:0/2185292590 conn(0x7fda4806b7c0 msgr2=0x7fda48117fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.401+0000 7fda4ef66700 1 -- 172.21.15.57:0/2185292590 shutdown_connections 2023-10-08T13:59:08.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:08.408+0000 7fda4ef66700 1 -- 172.21.15.57:0/2185292590 wait complete. 2023-10-08T13:59:09.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:09 smithi057 ceph-mon[99476]: pgmap v531: 129 pgs: 14 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 69 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.3 MiB/s wr, 212 op/s 2023-10-08T13:59:09.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:09 smithi099 ceph-mon[103849]: pgmap v531: 129 pgs: 14 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 69 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.3 MiB/s wr, 212 op/s 2023-10-08T13:59:09.688 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:09 smithi115 ceph-mon[104408]: pgmap v531: 129 pgs: 14 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 69 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.3 MiB/s wr, 212 op/s 2023-10-08T13:59:11.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:11 smithi115 ceph-mon[104408]: pgmap v532: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-10-08T13:59:11.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:11 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:59:11.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:11 smithi057 ceph-mon[99476]: pgmap v532: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-10-08T13:59:11.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:11 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:59:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:11 smithi099 ceph-mon[103849]: pgmap v532: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-10-08T13:59:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:11 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T13:59:12.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:59:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:59:12.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:59:12.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:59:12.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:59:12.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:59:13.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T13:59:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T13:59:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T13:59:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T13:59:13.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:13 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-10-08T13:59:13.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:13 smithi115 ceph-mon[104408]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-08T13:59:13.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:13 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi115 to 196.2M 2023-10-08T13:59:13.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:13 smithi115 ceph-mon[104408]: Adjusting osd_memory_target on smithi057 to 197.2M 2023-10-08T13:59:13.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:13 smithi115 ceph-mon[104408]: Unable to set osd_memory_target on smithi115 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-10-08T13:59:13.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:13 smithi115 ceph-mon[104408]: Unable to set osd_memory_target on smithi057 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-10-08T13:59:13.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:13 smithi115 ceph-mon[104408]: pgmap v533: 129 pgs: 17 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 69 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2023-10-08T13:59:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:13 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-10-08T13:59:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:13 smithi057 ceph-mon[99476]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-08T13:59:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:13 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi115 to 196.2M 2023-10-08T13:59:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:13 smithi057 ceph-mon[99476]: Adjusting osd_memory_target on smithi057 to 197.2M 2023-10-08T13:59:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:13 smithi057 ceph-mon[99476]: Unable to set osd_memory_target on smithi115 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-10-08T13:59:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:13 smithi057 ceph-mon[99476]: Unable to set osd_memory_target on smithi057 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-10-08T13:59:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:13 smithi057 ceph-mon[99476]: pgmap v533: 129 pgs: 17 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 69 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2023-10-08T13:59:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:13 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi099 to 196.2M 2023-10-08T13:59:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:13 smithi099 ceph-mon[103849]: Unable to set osd_memory_target on smithi099 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-08T13:59:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:13 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi115 to 196.2M 2023-10-08T13:59:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:13 smithi099 ceph-mon[103849]: Adjusting osd_memory_target on smithi057 to 197.2M 2023-10-08T13:59:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:13 smithi099 ceph-mon[103849]: Unable to set osd_memory_target on smithi115 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-10-08T13:59:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:13 smithi099 ceph-mon[103849]: Unable to set osd_memory_target on smithi057 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-10-08T13:59:14.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:13 smithi099 ceph-mon[103849]: pgmap v533: 129 pgs: 17 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 69 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2023-10-08T13:59:15.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:15 smithi115 ceph-mon[104408]: pgmap v534: 129 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 85 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.1 MiB/s wr, 186 op/s 2023-10-08T13:59:15.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:15 smithi057 ceph-mon[99476]: pgmap v534: 129 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 85 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.1 MiB/s wr, 186 op/s 2023-10-08T13:59:16.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:15 smithi099 ceph-mon[103849]: pgmap v534: 129 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 85 active+clean; 2.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.1 MiB/s wr, 186 op/s 2023-10-08T13:59:16.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:16 smithi057 ceph-mon[99476]: pgmap v535: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 191 op/s 2023-10-08T13:59:17.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:16 smithi099 ceph-mon[103849]: pgmap v535: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 191 op/s 2023-10-08T13:59:17.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:16 smithi115 ceph-mon[104408]: pgmap v535: 129 pgs: 8 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 108 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 191 op/s 2023-10-08T13:59:19.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:19 smithi115 ceph-mon[104408]: pgmap v536: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2023-10-08T13:59:19.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:19 smithi057 ceph-mon[99476]: pgmap v536: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2023-10-08T13:59:19.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:19 smithi099 ceph-mon[103849]: pgmap v536: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2023-10-08T13:59:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.537+0000 7f949f564700 1 -- 172.21.15.57:0/2425479305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94980fff90 msgr2=0x7f949810c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.537+0000 7f949f564700 1 --2- 172.21.15.57:0/2425479305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94980fff90 0x7f949810c4a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f949000b0a0 tx=0x7f949000b3b0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.538+0000 7f949f564700 1 -- 172.21.15.57:0/2425479305 shutdown_connections 2023-10-08T13:59:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.538+0000 7f949f564700 1 --2- 172.21.15.57:0/2425479305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94980fff90 0x7f949810c4a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.538+0000 7f949f564700 1 --2- 172.21.15.57:0/2425479305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9498105670 0x7f9498105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.538+0000 7f949f564700 1 --2- 172.21.15.57:0/2425479305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94980ff5d0 0x7f94980ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.538+0000 7f949f564700 1 -- 172.21.15.57:0/2425479305 >> 172.21.15.57:0/2425479305 conn(0x7f94980fb0c0 msgr2=0x7f94980fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.538+0000 7f949f564700 1 -- 172.21.15.57:0/2425479305 shutdown_connections 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.538+0000 7f949f564700 1 -- 172.21.15.57:0/2425479305 wait complete. 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.538+0000 7f949f564700 1 Processor -- start 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.538+0000 7f949f564700 1 -- start start 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949f564700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94980ff5d0 0x7f949819b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949f564700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94980fff90 0x7f949819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949f564700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9498105670 0x7f9498195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949f564700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9498195d70 con 0x7f94980ff5d0 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949d300700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94980ff5d0 0x7f949819b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949d300700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94980ff5d0 0x7f949819b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59270/0 (socket says 172.21.15.57:59270) 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949d300700 1 -- 172.21.15.57:0/3049967351 learned_addr learned my addr 172.21.15.57:0/3049967351 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949f564700 1 -- 172.21.15.57:0/3049967351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9498195ee0 con 0x7f94980fff90 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949f564700 1 -- 172.21.15.57:0/3049967351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94981960a0 con 0x7f9498105670 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949caff700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94980fff90 0x7f949819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949db01700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9498105670 0x7f9498195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949d300700 1 -- 172.21.15.57:0/3049967351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9498105670 msgr2=0x7f9498195710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949d300700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9498105670 0x7f9498195710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949d300700 1 -- 172.21.15.57:0/3049967351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94980fff90 msgr2=0x7f949819bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949d300700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94980fff90 0x7f949819bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949d300700 1 -- 172.21.15.57:0/3049967351 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9490009d00 con 0x7f94980ff5d0 2023-10-08T13:59:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f949d300700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94980ff5d0 0x7f949819b5e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f949400e970 tx=0x7f949400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.539+0000 7f948e7fc700 1 -- 172.21.15.57:0/3049967351 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494009790 con 0x7f94980ff5d0 2023-10-08T13:59:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.540+0000 7f949f564700 1 -- 172.21.15.57:0/3049967351 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9498196380 con 0x7f94980ff5d0 2023-10-08T13:59:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.540+0000 7f948e7fc700 1 -- 172.21.15.57:0/3049967351 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f949400c7e0 con 0x7f94980ff5d0 2023-10-08T13:59:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.540+0000 7f948e7fc700 1 -- 172.21.15.57:0/3049967351 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494003c70 con 0x7f94980ff5d0 2023-10-08T13:59:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.540+0000 7f949f564700 1 -- 172.21.15.57:0/3049967351 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94981a25d0 con 0x7f94980ff5d0 2023-10-08T13:59:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.542+0000 7f948e7fc700 1 -- 172.21.15.57:0/3049967351 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f949400cc50 con 0x7f94980ff5d0 2023-10-08T13:59:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.543+0000 7f948e7fc700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f94840a15f0 0x7f94840a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.543+0000 7f948e7fc700 1 -- 172.21.15.57:0/3049967351 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f9494014070 con 0x7f94980ff5d0 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.543+0000 7f949caff700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f94840a15f0 0x7f94840a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f949caff700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f94840a15f0 0x7f94840a3ab0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9498196e20 tx=0x7f9488009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f94ab74db80 1 Processor -- start 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f94ab74db80 1 -- start start 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcef0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dcef11a0 0x5609dceeab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dceeb0b0 0x5609dceeb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f94ab74db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dce53400 con 0x5609dcef11a0 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f94ab74db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dce44bc0 con 0x5609dcef0880 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f94ab74db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dce424e0 con 0x5609dceeb0b0 2023-10-08T13:59:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f949caff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dcef11a0 0x5609dceeab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f949d300700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcef0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f949db01700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dceeb0b0 0x5609dceeb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f949d300700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcef0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60704/0 (socket says 172.21.15.57:60704) 2023-10-08T13:59:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f949d300700 1 -- 172.21.15.57:0/2373567350 learned_addr learned my addr 172.21.15.57:0/2373567350 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f949db01700 1 --2- 172.21.15.57:0/2373567350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dceeb0b0 0x5609dceeb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60094/0 (socket says 172.21.15.57:60094) 2023-10-08T13:59:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f949caff700 1 -- 172.21.15.57:0/2373567350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dceeb0b0 msgr2=0x5609dceeb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f949caff700 1 --2- 172.21.15.57:0/2373567350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dceeb0b0 0x5609dceeb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.544+0000 7f949caff700 1 -- 172.21.15.57:0/2373567350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 msgr2=0x5609dcef0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.545+0000 7f949caff700 1 --2- 172.21.15.57:0/2373567350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcef0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.545+0000 7f949caff700 1 -- 172.21.15.57:0/2373567350 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609dcc68d80 con 0x5609dcef11a0 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.545+0000 7f949d300700 1 --2- 172.21.15.57:0/2373567350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcef0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.545+0000 7f949caff700 1 --2- 172.21.15.57:0/2373567350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dcef11a0 0x5609dceeab70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9488011c90 tx=0x7f9488014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9c0c300b5c8d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.545+0000 7f94837fe700 1 -- 172.21.15.57:0/2373567350 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488004230 con 0x5609dcef11a0 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.545+0000 7f94837fe700 1 -- 172.21.15.57:0/2373567350 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9488021070 con 0x5609dcef11a0 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.545+0000 7f94837fe700 1 -- 172.21.15.57:0/2373567350 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488003bf0 con 0x5609dcef11a0 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 -- 172.21.15.57:0/2373567350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dcef11a0 msgr2=0x5609dceeab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 --2- 172.21.15.57:0/2373567350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dcef11a0 0x5609dceeab70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9488011c90 tx=0x7f9488014000 comp rx=0 tx=0).stop 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 -- 172.21.15.57:0/2373567350 shutdown_connections 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 --2- 172.21.15.57:0/2373567350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dceeb0b0 0x5609dceeb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 --2- 172.21.15.57:0/2373567350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcef0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 --2- 172.21.15.57:0/2373567350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dcef11a0 0x5609dceeab70 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 -- 172.21.15.57:0/2373567350 >> 172.21.15.57:0/2373567350 conn(0x5609dce5f200 msgr2=0x5609dcee5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 -- 172.21.15.57:0/2373567350 shutdown_connections 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 -- 172.21.15.57:0/2373567350 wait complete. 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 -- start start 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.546+0000 7f94ab74db80 1 Processor -- start 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcefce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcef66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f94ab74db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dce3ddd0 con 0x5609dceeb0b0 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f94ab74db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dcdda6b0 con 0x5609dcef0880 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f94ab74db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dcdda350 con 0x5609dcef11a0 2023-10-08T13:59:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f94ab74db80 10 client.0 _ll_register_callbacks cb 0x5609dcc377a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949d300700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949d300700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59284/0 (socket says 172.21.15.57:59284) 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949d300700 1 -- 172.21.15.57:0/1438570725 learned_addr learned my addr 172.21.15.57:0/1438570725 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949db01700 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcef66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949caff700 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcefce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949d300700 1 -- 172.21.15.57:0/1438570725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 msgr2=0x5609dcef66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949d300700 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcef66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949d300700 1 -- 172.21.15.57:0/1438570725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 msgr2=0x5609dcefce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949d300700 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcefce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949d300700 1 -- 172.21.15.57:0/1438570725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94981a25d0 con 0x5609dceeb0b0 2023-10-08T13:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949caff700 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcefce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f949d300700 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefc8f0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f949400efd0 tx=0x7f9494003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d8f9fa84a73e2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f9480ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f9480ff9700 1 -- 172.21.15.57:0/1438570725 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949409e430 con 0x5609dceeb0b0 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f9480ff9700 1 -- 172.21.15.57:0/1438570725 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f94940a2670 con 0x5609dceeb0b0 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f94ab74db80 1 -- 172.21.15.57:0/1438570725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609dce05e50 con 0x5609dceeb0b0 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f9480ff9700 1 -- 172.21.15.57:0/1438570725 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94940cd290 con 0x5609dceeb0b0 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f94ab74db80 10 client.16998 fetch_fsmap learned FSMap version 45 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.547+0000 7f94ab74db80 1 -- 172.21.15.57:0/1438570725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f9488017070 con 0x5609dceeb0b0 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f9480ff9700 1 -- 172.21.15.57:0/1438570725 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609dce05e50 con 0x5609dceeb0b0 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f9480ff9700 1 -- 172.21.15.57:0/1438570725 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f94940997d0 con 0x5609dceeb0b0 2023-10-08T13:59:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f94ab74db80 10 client.16998 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T13:59:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f94ab74db80 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:59:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f94ab74db80 20 client.16998 populate_metadata read hostname 'smithi057' 2023-10-08T13:59:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f94ab74db80 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5609dcf06f20 0x5609dcf09340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f94ab74db80 4 client.16998 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:59:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f94ab74db80 1 -- 172.21.15.57:0/1438570725 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609dce36db0 con 0x5609dcf06f20 2023-10-08T13:59:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f949caff700 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5609dcf06f20 0x5609dcf09340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f949caff700 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5609dcf06f20 0x5609dcf09340 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa6fb09a0babad89 server_cookie=4c0005a1fae4db62 in_seq=0 out_seq=0 2023-10-08T13:59:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.548+0000 7f9480ff9700 10 client.16998 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f9480ff9700 1 -- 172.21.15.57:0/1438570725 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5609dce36db0 con 0x5609dcf06f20 2023-10-08T13:59:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f9480ff9700 10 client.16998 handle_command_reply: tid=0 2023-10-08T13:59:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 client.16998 shutdown 2023-10-08T13:59:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 -- 172.21.15.57:0/1438570725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 msgr2=0x5609dcefc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefc8f0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f949400efd0 tx=0x7f9494003dd0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 -- 172.21.15.57:0/1438570725 shutdown_connections 2023-10-08T13:59:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5609dcf06f20 0x5609dcf09340 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcef66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f9480ff9700 0 client.16998 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefc8f0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 --2- 172.21.15.57:0/1438570725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcefce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 -- 172.21.15.57:0/1438570725 >> 172.21.15.57:0/1438570725 conn(0x5609dce5f200 msgr2=0x5609dcee5010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 -- 172.21.15.57:0/1438570725 shutdown_connections 2023-10-08T13:59:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.613+0000 7f94ab74db80 1 -- 172.21.15.57:0/1438570725 wait complete. 2023-10-08T13:59:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.614+0000 7f94ab74db80 20 client.16998 trim_cache size 0 max 16384 2023-10-08T13:59:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.628+0000 7f94ab74db80 1 Processor -- start 2023-10-08T13:59:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.628+0000 7f94ab74db80 1 -- start start 2023-10-08T13:59:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.628+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcee5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.628+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcee5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.628+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcef2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.628+0000 7f94ab74db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dccbe8c0 con 0x5609dceeb0b0 2023-10-08T13:59:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.628+0000 7f94ab74db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dcedd670 con 0x5609dcef0880 2023-10-08T13:59:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.628+0000 7f94ab74db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dcef2a00 con 0x5609dcef11a0 2023-10-08T13:59:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949d300700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcee5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949d300700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcee5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59292/0 (socket says 172.21.15.57:59292) 2023-10-08T13:59:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949d300700 1 -- 172.21.15.57:0/3404820366 learned_addr learned my addr 172.21.15.57:0/3404820366 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949caff700 1 --2- 172.21.15.57:0/3404820366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcee5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949db01700 1 --2- 172.21.15.57:0/3404820366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcef2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949d300700 1 -- 172.21.15.57:0/3404820366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 msgr2=0x5609dcef2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949d300700 1 --2- 172.21.15.57:0/3404820366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcef2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949d300700 1 -- 172.21.15.57:0/3404820366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 msgr2=0x5609dcee5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949d300700 1 --2- 172.21.15.57:0/3404820366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcee5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949d300700 1 -- 172.21.15.57:0/3404820366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609dcc51c00 con 0x5609dceeb0b0 2023-10-08T13:59:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f949d300700 1 --2- 172.21.15.57:0/3404820366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcee5790 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9494099fe0 tx=0x7f949409e360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cbd5a5af400e0c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f9483fff700 1 -- 172.21.15.57:0/3404820366 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94940a2670 con 0x5609dceeb0b0 2023-10-08T13:59:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f9483fff700 1 -- 172.21.15.57:0/3404820366 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f949409e430 con 0x5609dceeb0b0 2023-10-08T13:59:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.629+0000 7f9483fff700 1 -- 172.21.15.57:0/3404820366 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94940cd4f0 con 0x5609dceeb0b0 2023-10-08T13:59:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 -- 172.21.15.57:0/3404820366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 msgr2=0x5609dcee5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 --2- 172.21.15.57:0/3404820366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcee5790 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9494099fe0 tx=0x7f949409e360 comp rx=0 tx=0).stop 2023-10-08T13:59:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 -- 172.21.15.57:0/3404820366 shutdown_connections 2023-10-08T13:59:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 --2- 172.21.15.57:0/3404820366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcef2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 --2- 172.21.15.57:0/3404820366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcee5790 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 --2- 172.21.15.57:0/3404820366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcee5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 -- 172.21.15.57:0/3404820366 >> 172.21.15.57:0/3404820366 conn(0x5609dce5f200 msgr2=0x5609dcef6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 -- 172.21.15.57:0/3404820366 shutdown_connections 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 -- 172.21.15.57:0/3404820366 wait complete. 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 -- start start 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.630+0000 7f94ab74db80 1 Processor -- start 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcefda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f94ab74db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcefdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f94ab74db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dcf02720 con 0x5609dceeb0b0 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f94ab74db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dcf02890 con 0x5609dcef0880 2023-10-08T13:59:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f94ab74db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609dcf02a50 con 0x5609dcef11a0 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f94ab74db80 10 client.0 _ll_register_callbacks cb 0x5609dcc377a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949d300700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949caff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcefda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949d300700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59298/0 (socket says 172.21.15.57:59298) 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949d300700 1 -- 172.21.15.57:0/897801266 learned_addr learned my addr 172.21.15.57:0/897801266 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949db01700 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcefdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949d300700 1 -- 172.21.15.57:0/897801266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 msgr2=0x5609dcefdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949d300700 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcefdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949d300700 1 -- 172.21.15.57:0/897801266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 msgr2=0x5609dcefda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949d300700 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcefda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949d300700 1 -- 172.21.15.57:0/897801266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9488017070 con 0x5609dceeb0b0 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.631+0000 7f949caff700 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcefda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f949d300700 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefd550 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9494098630 tx=0x7f9494003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20ddba48f6a7579 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f9480ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f9480ff9700 1 -- 172.21.15.57:0/897801266 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94940cd490 con 0x5609dceeb0b0 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f9480ff9700 1 -- 172.21.15.57:0/897801266 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f94940a2670 con 0x5609dceeb0b0 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f94ab74db80 1 -- 172.21.15.57:0/897801266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9488021070 con 0x5609dceeb0b0 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f9480ff9700 1 -- 172.21.15.57:0/897801266 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94940a21e0 con 0x5609dceeb0b0 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f94ab74db80 10 client.17004 fetch_fsmap learned FSMap version 45 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f94ab74db80 1 -- 172.21.15.57:0/897801266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5609dcc8f1f0 con 0x5609dceeb0b0 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f9480ff9700 1 -- 172.21.15.57:0/897801266 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9488021070 con 0x5609dceeb0b0 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f9480ff9700 1 -- 172.21.15.57:0/897801266 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f94940960d0 con 0x5609dceeb0b0 2023-10-08T13:59:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f94ab74db80 10 client.17004 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T13:59:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f94ab74db80 10 client.17004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:59:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f94ab74db80 20 client.17004 populate_metadata read hostname 'smithi057' 2023-10-08T13:59:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f94ab74db80 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5609dcf06f20 0x5609dcf5afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f94ab74db80 4 client.17004 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:59:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f94ab74db80 1 -- 172.21.15.57:0/897801266 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609dce36ad0 con 0x5609dcf06f20 2023-10-08T13:59:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.632+0000 7f949caff700 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5609dcf06f20 0x5609dcf5afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.633+0000 7f9480ff9700 10 client.17004 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:21.633+0000 7f949caff700 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5609dcf06f20 0x5609dcf5afb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148be5ef071a1ad0 server_cookie=71780ae4ba4811d5 in_seq=0 out_seq=0 2023-10-08T13:59:21.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:21 smithi115 ceph-mon[104408]: pgmap v537: 129 pgs: 129 active+clean; 2.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 201 op/s 2023-10-08T13:59:21.994 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:21 smithi057 ceph-mon[99476]: pgmap v537: 129 pgs: 129 active+clean; 2.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 201 op/s 2023-10-08T13:59:22.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:21 smithi099 ceph-mon[103849]: pgmap v537: 129 pgs: 129 active+clean; 2.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 201 op/s 2023-10-08T13:59:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f9480ff9700 1 -- 172.21.15.57:0/897801266 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609dce36ad0 con 0x5609dcf06f20 2023-10-08T13:59:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f9480ff9700 10 client.17004 handle_command_reply: tid=0 2023-10-08T13:59:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 client.17004 shutdown 2023-10-08T13:59:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 -- 172.21.15.57:0/897801266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 msgr2=0x5609dcefd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefd550 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9494098630 tx=0x7f9494003dd0 comp rx=0 tx=0).stop 2023-10-08T13:59:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 -- 172.21.15.57:0/897801266 shutdown_connections 2023-10-08T13:59:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5609dcf06f20 0x5609dcf5afb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5609dcef11a0 0x5609dcefdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5609dceeb0b0 0x5609dcefd550 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 --2- 172.21.15.57:0/897801266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dcef0880 0x5609dcefda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 -- 172.21.15.57:0/897801266 >> 172.21.15.57:0/897801266 conn(0x5609dce5f200 msgr2=0x5609dcef62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f9480ff9700 0 client.17004 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 -- 172.21.15.57:0/897801266 shutdown_connections 2023-10-08T13:59:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.422+0000 7f94ab74db80 1 -- 172.21.15.57:0/897801266 wait complete. 2023-10-08T13:59:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f94ab74db80 20 client.17004 trim_cache size 0 max 16384 2023-10-08T13:59:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:59:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:59:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:59:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 -- 172.21.15.57:0/3049967351 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f94840a15f0 msgr2=0x7f94840a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f94840a15f0 0x7f94840a3ab0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9498196e20 tx=0x7f9488009260 comp rx=0 tx=0).stop 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 -- 172.21.15.57:0/3049967351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94980ff5d0 msgr2=0x7f949819b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94980ff5d0 0x7f949819b5e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f949400e970 tx=0x7f949400ec80 comp rx=0 tx=0).stop 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 -- 172.21.15.57:0/3049967351 shutdown_connections 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9498105670 0x7f9498195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94980ff5d0 0x7f949819b5e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f94840a15f0 0x7f94840a3ab0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 --2- 172.21.15.57:0/3049967351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94980fff90 0x7f949819bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 -- 172.21.15.57:0/3049967351 >> 172.21.15.57:0/3049967351 conn(0x7f94980fb0c0 msgr2=0x7f94980fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 -- 172.21.15.57:0/3049967351 shutdown_connections 2023-10-08T13:59:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:22.423+0000 7f949f564700 1 -- 172.21.15.57:0/3049967351 wait complete. 2023-10-08T13:59:23.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:23 smithi115 ceph-mon[104408]: pgmap v538: 129 pgs: 129 active+clean; 2.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 229 op/s 2023-10-08T13:59:23.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:23 smithi115 ceph-mon[104408]: osdmap e97: 12 total, 12 up, 12 in 2023-10-08T13:59:23.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:23 smithi057 ceph-mon[99476]: pgmap v538: 129 pgs: 129 active+clean; 2.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 229 op/s 2023-10-08T13:59:23.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:23 smithi057 ceph-mon[99476]: osdmap e97: 12 total, 12 up, 12 in 2023-10-08T13:59:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:23 smithi099 ceph-mon[103849]: pgmap v538: 129 pgs: 129 active+clean; 2.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 229 op/s 2023-10-08T13:59:24.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:23 smithi099 ceph-mon[103849]: osdmap e97: 12 total, 12 up, 12 in 2023-10-08T13:59:25.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:25 smithi115 ceph-mon[104408]: pgmap v540: 129 pgs: 129 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 412 op/s 2023-10-08T13:59:25.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:25 smithi057 ceph-mon[99476]: pgmap v540: 129 pgs: 129 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 412 op/s 2023-10-08T13:59:26.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:25 smithi099 ceph-mon[103849]: pgmap v540: 129 pgs: 129 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 412 op/s 2023-10-08T13:59:27.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:27 smithi115 ceph-mon[104408]: pgmap v541: 129 pgs: 129 active+clean; 2.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 501 op/s 2023-10-08T13:59:27.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:27 smithi057 ceph-mon[99476]: pgmap v541: 129 pgs: 129 active+clean; 2.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 501 op/s 2023-10-08T13:59:28.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:27 smithi099 ceph-mon[103849]: pgmap v541: 129 pgs: 129 active+clean; 2.0 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 501 op/s 2023-10-08T13:59:29.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:29 smithi057 ceph-mon[99476]: pgmap v542: 129 pgs: 129 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 507 op/s 2023-10-08T13:59:29.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:29 smithi099 ceph-mon[103849]: pgmap v542: 129 pgs: 129 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 507 op/s 2023-10-08T13:59:29.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:29 smithi115 ceph-mon[104408]: pgmap v542: 129 pgs: 129 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 507 op/s 2023-10-08T13:59:31.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:31 smithi115 ceph-mon[104408]: pgmap v543: 129 pgs: 129 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 540 op/s 2023-10-08T13:59:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:31 smithi057 ceph-mon[99476]: pgmap v543: 129 pgs: 129 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 540 op/s 2023-10-08T13:59:32.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:31 smithi099 ceph-mon[103849]: pgmap v543: 129 pgs: 129 active+clean; 2.0 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 540 op/s 2023-10-08T13:59:33.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:33 smithi115 ceph-mon[104408]: pgmap v544: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 500 op/s 2023-10-08T13:59:33.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:33 smithi057 ceph-mon[99476]: pgmap v544: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 500 op/s 2023-10-08T13:59:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:33 smithi099 ceph-mon[103849]: pgmap v544: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 500 op/s 2023-10-08T13:59:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.555+0000 7f7379042700 1 -- 172.21.15.57:0/938893693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73740ff5d0 msgr2=0x7f73740ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.555+0000 7f7379042700 1 --2- 172.21.15.57:0/938893693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73740ff5d0 0x7f73740ffa50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7368009990 tx=0x7f7368009ca0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.555+0000 7f7379042700 1 -- 172.21.15.57:0/938893693 shutdown_connections 2023-10-08T13:59:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.555+0000 7f7379042700 1 --2- 172.21.15.57:0/938893693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73740fff90 0x7f737410c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.555+0000 7f7379042700 1 --2- 172.21.15.57:0/938893693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73741045e0 0x7f73741049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.555+0000 7f7379042700 1 --2- 172.21.15.57:0/938893693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73740ff5d0 0x7f73740ffa50 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.555+0000 7f7379042700 1 -- 172.21.15.57:0/938893693 >> 172.21.15.57:0/938893693 conn(0x7f7374076790 msgr2=0x7f7374076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.555+0000 7f7379042700 1 -- 172.21.15.57:0/938893693 shutdown_connections 2023-10-08T13:59:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.555+0000 7f7379042700 1 -- 172.21.15.57:0/938893693 wait complete. 2023-10-08T13:59:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.556+0000 7f7379042700 1 Processor -- start 2023-10-08T13:59:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.556+0000 7f7379042700 1 -- start start 2023-10-08T13:59:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f7379042700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73740ff5d0 0x7f73741979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f7379042700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73740fff90 0x7f7374197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f7379042700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73741045e0 0x7f7374198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f7379042700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73741a2cc0 con 0x7f73741045e0 2023-10-08T13:59:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f7379042700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73741a2e30 con 0x7f73740ff5d0 2023-10-08T13:59:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f7379042700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73741a2ff0 con 0x7f73740fff90 2023-10-08T13:59:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73741045e0 0x7f7374198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73741045e0 0x7f7374198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59318/0 (socket says 172.21.15.57:59318) 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737359e700 1 -- 172.21.15.57:0/3400068835 learned_addr learned my addr 172.21.15.57:0/3400068835 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f7372d9d700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73740ff5d0 0x7f73741979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737259c700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73740fff90 0x7f7374197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737359e700 1 -- 172.21.15.57:0/3400068835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73740fff90 msgr2=0x7f7374197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737359e700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73740fff90 0x7f7374197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737359e700 1 -- 172.21.15.57:0/3400068835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73740ff5d0 msgr2=0x7f73741979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737359e700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73740ff5d0 0x7f73741979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737359e700 1 -- 172.21.15.57:0/3400068835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73680095f0 con 0x7f73741045e0 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737259c700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73740fff90 0x7f7374197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f7372d9d700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73740ff5d0 0x7f73741979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f737359e700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73741045e0 0x7f7374198470 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f736c00ef30 tx=0x7f736c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.557+0000 7f735bfff700 1 -- 172.21.15.57:0/3400068835 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c018070 con 0x7f73741045e0 2023-10-08T13:59:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.558+0000 7f735bfff700 1 -- 172.21.15.57:0/3400068835 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f736c00f040 con 0x7f73741045e0 2023-10-08T13:59:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.558+0000 7f7379042700 1 -- 172.21.15.57:0/3400068835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73741a3270 con 0x7f73741045e0 2023-10-08T13:59:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.558+0000 7f735bfff700 1 -- 172.21.15.57:0/3400068835 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c00cdb0 con 0x7f73741045e0 2023-10-08T13:59:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.558+0000 7f7379042700 1 -- 172.21.15.57:0/3400068835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73741a3730 con 0x7f73741045e0 2023-10-08T13:59:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.560+0000 7f735bfff700 1 -- 172.21.15.57:0/3400068835 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f736c003a00 con 0x7f73741045e0 2023-10-08T13:59:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.560+0000 7f735bfff700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f735c0a15f0 0x7f735c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.561+0000 7f735bfff700 1 -- 172.21.15.57:0/3400068835 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f736c0cd060 con 0x7f73741045e0 2023-10-08T13:59:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.561+0000 7f7372d9d700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f735c0a15f0 0x7f735c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.561+0000 7f7372d9d700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f735c0a15f0 0x7f735c0a3ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7374069cf0 tx=0x7f7364009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.561+0000 7f738522bb80 1 Processor -- start 2023-10-08T13:59:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.561+0000 7f738522bb80 1 -- start start 2023-10-08T13:59:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f738522bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbedc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f738522bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbe7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f738522bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbe8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f738522bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cb50400 con 0x55ae1cbed880 2023-10-08T13:59:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f738522bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cb41bc0 con 0x55ae1cbee1a0 2023-10-08T13:59:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f738522bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cb3f4e0 con 0x55ae1cbe80b0 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbedc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbedc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59322/0 (socket says 172.21.15.57:59322) 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f7372d9d700 1 -- 172.21.15.57:0/1743121381 learned_addr learned my addr 172.21.15.57:0/1743121381 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f737259c700 1 --2- 172.21.15.57:0/1743121381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbe7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f737359e700 1 --2- 172.21.15.57:0/1743121381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbe8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f7372d9d700 1 -- 172.21.15.57:0/1743121381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbe80b0 msgr2=0x55ae1cbe8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f7372d9d700 1 --2- 172.21.15.57:0/1743121381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbe8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f7372d9d700 1 -- 172.21.15.57:0/1743121381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 msgr2=0x55ae1cbe7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f7372d9d700 1 --2- 172.21.15.57:0/1743121381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbe7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f7372d9d700 1 -- 172.21.15.57:0/1743121381 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae1c965d80 con 0x55ae1cbed880 2023-10-08T13:59:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f737259c700 1 --2- 172.21.15.57:0/1743121381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbe7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.562+0000 7f7372d9d700 1 --2- 172.21.15.57:0/1743121381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbedc60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f73640043d0 tx=0x7f7364014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6587d2e7dda138 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f73597fa700 1 -- 172.21.15.57:0/1743121381 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364003ce0 con 0x55ae1cbed880 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f73597fa700 1 -- 172.21.15.57:0/1743121381 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7364003e40 con 0x55ae1cbed880 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f73597fa700 1 -- 172.21.15.57:0/1743121381 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364026860 con 0x55ae1cbed880 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f738522bb80 1 -- 172.21.15.57:0/1743121381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 msgr2=0x55ae1cbedc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f738522bb80 1 --2- 172.21.15.57:0/1743121381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbedc60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f73640043d0 tx=0x7f7364014000 comp rx=0 tx=0).stop 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f738522bb80 1 -- 172.21.15.57:0/1743121381 shutdown_connections 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f738522bb80 1 --2- 172.21.15.57:0/1743121381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbe8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f738522bb80 1 --2- 172.21.15.57:0/1743121381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbedc60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f738522bb80 1 --2- 172.21.15.57:0/1743121381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbe7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f738522bb80 1 -- 172.21.15.57:0/1743121381 >> 172.21.15.57:0/1743121381 conn(0x55ae1cb5c200 msgr2=0x55ae1cbe2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f738522bb80 1 -- 172.21.15.57:0/1743121381 shutdown_connections 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.563+0000 7f738522bb80 1 -- 172.21.15.57:0/1743121381 wait complete. 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f738522bb80 1 -- start start 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f738522bb80 1 Processor -- start 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f738522bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f738522bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbed880 0x55ae1cbf1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f738522bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbf2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f738522bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cb3add0 con 0x55ae1cbe80b0 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f738522bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cad76b0 con 0x55ae1cbee1a0 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f738522bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cad7350 con 0x55ae1cbed880 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f738522bb80 10 client.0 _ll_register_callbacks cb 0x55ae1c9347a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:59:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59328/0 (socket says 172.21.15.57:59328) 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f7372d9d700 1 -- 172.21.15.57:0/3658198005 learned_addr learned my addr 172.21.15.57:0/3658198005 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f737359e700 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbf2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.564+0000 7f737259c700 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbed880 0x55ae1cbf1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7372d9d700 1 -- 172.21.15.57:0/3658198005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbed880 msgr2=0x55ae1cbf1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7372d9d700 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbed880 0x55ae1cbf1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7372d9d700 1 -- 172.21.15.57:0/3658198005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 msgr2=0x55ae1cbf2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7372d9d700 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbf2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7372d9d700 1 -- 172.21.15.57:0/3658198005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737410cd90 con 0x55ae1cbe80b0 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f737259c700 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbed880 0x55ae1cbf1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:59:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7372d9d700 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf15c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f73640041e0 tx=0x7f736402baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7986b3688ec01a9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7356ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7356ff5700 1 -- 172.21.15.57:0/3658198005 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364030070 con 0x55ae1cbe80b0 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7356ff5700 1 -- 172.21.15.57:0/3658198005 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f7364005040 con 0x55ae1cbe80b0 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f738522bb80 1 -- 172.21.15.57:0/3658198005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae1cb02e50 con 0x55ae1cbe80b0 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7356ff5700 1 -- 172.21.15.57:0/3658198005 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f736402bea0 con 0x55ae1cbe80b0 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f738522bb80 10 client.17016 fetch_fsmap learned FSMap version 45 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f738522bb80 1 -- 172.21.15.57:0/3658198005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f7364017070 con 0x55ae1cbe80b0 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.565+0000 7f7356ff5700 1 -- 172.21.15.57:0/3658198005 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae1cb02e50 con 0x55ae1cbe80b0 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.566+0000 7f7356ff5700 1 -- 172.21.15.57:0/3658198005 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f7364024020 con 0x55ae1cbe80b0 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.566+0000 7f738522bb80 10 client.17016 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T13:59:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.566+0000 7f738522bb80 10 client.17016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:59:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.566+0000 7f738522bb80 20 client.17016 populate_metadata read hostname 'smithi057' 2023-10-08T13:59:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.566+0000 7f738522bb80 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ae1cc01cc0 0x55ae1cc040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.566+0000 7f738522bb80 4 client.17016 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:59:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.566+0000 7f738522bb80 1 -- 172.21.15.57:0/3658198005 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae1cb33db0 con 0x55ae1cc01cc0 2023-10-08T13:59:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.566+0000 7f737259c700 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ae1cc01cc0 0x55ae1cc040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.566+0000 7f737259c700 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ae1cc01cc0 0x55ae1cc040e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab744982913dc448 server_cookie=c74e233c98a181ea in_seq=0 out_seq=0 2023-10-08T13:59:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.566+0000 7f7356ff5700 10 client.17016 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f7356ff5700 1 -- 172.21.15.57:0/3658198005 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae1cb33db0 con 0x55ae1cc01cc0 2023-10-08T13:59:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f7356ff5700 10 client.17016 handle_command_reply: tid=0 2023-10-08T13:59:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f738522bb80 1 client.17016 shutdown 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f738522bb80 1 -- 172.21.15.57:0/3658198005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 msgr2=0x55ae1cbf15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f738522bb80 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf15c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f73640041e0 tx=0x7f736402baf0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f738522bb80 1 -- 172.21.15.57:0/3658198005 shutdown_connections 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f738522bb80 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ae1cc01cc0 0x55ae1cc040e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f738522bb80 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbf2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f738522bb80 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf15c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f7356ff5700 0 client.17016 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f738522bb80 1 --2- 172.21.15.57:0/3658198005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbed880 0x55ae1cbf1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.568+0000 7f738522bb80 1 -- 172.21.15.57:0/3658198005 >> 172.21.15.57:0/3658198005 conn(0x55ae1cb5c200 msgr2=0x55ae1cbe2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.569+0000 7f738522bb80 1 -- 172.21.15.57:0/3658198005 shutdown_connections 2023-10-08T13:59:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.569+0000 7f738522bb80 1 -- 172.21.15.57:0/3658198005 wait complete. 2023-10-08T13:59:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.569+0000 7f738522bb80 20 client.17016 trim_cache size 0 max 16384 2023-10-08T13:59:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.583+0000 7f738522bb80 1 Processor -- start 2023-10-08T13:59:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.583+0000 7f738522bb80 1 -- start start 2023-10-08T13:59:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f738522bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f738522bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbf0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f738522bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbf1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f738522bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1c9bb8c0 con 0x55ae1cbed880 2023-10-08T13:59:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f738522bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cbef590 con 0x55ae1cbee1a0 2023-10-08T13:59:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f738522bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cbefb30 con 0x55ae1cbe80b0 2023-10-08T13:59:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbf0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbf0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59338/0 (socket says 172.21.15.57:59338) 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737259c700 1 -- 172.21.15.57:0/4197121088 learned_addr learned my addr 172.21.15.57:0/4197121088 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbf1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f7372d9d700 1 --2- 172.21.15.57:0/4197121088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737259c700 1 -- 172.21.15.57:0/4197121088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbe80b0 msgr2=0x55ae1cbecf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737259c700 1 --2- 172.21.15.57:0/4197121088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbecf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737259c700 1 -- 172.21.15.57:0/4197121088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 msgr2=0x55ae1cbf1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737259c700 1 --2- 172.21.15.57:0/4197121088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbf1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737259c700 1 -- 172.21.15.57:0/4197121088 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae1c94ec00 con 0x55ae1cbed880 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737359e700 1 --2- 172.21.15.57:0/4197121088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbf1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f737259c700 1 --2- 172.21.15.57:0/4197121088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbf0490 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7368009990 tx=0x7f736800ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cac1ac75b74333b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f7359ffb700 1 -- 172.21.15.57:0/4197121088 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736800bdd0 con 0x55ae1cbed880 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f7359ffb700 1 -- 172.21.15.57:0/4197121088 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f736801e040 con 0x55ae1cbed880 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.584+0000 7f7359ffb700 1 -- 172.21.15.57:0/4197121088 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736800f6c0 con 0x55ae1cbed880 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.585+0000 7f738522bb80 1 -- 172.21.15.57:0/4197121088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 msgr2=0x55ae1cbf0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.585+0000 7f738522bb80 1 --2- 172.21.15.57:0/4197121088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbf0490 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7368009990 tx=0x7f736800ba60 comp rx=0 tx=0).stop 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.585+0000 7f738522bb80 1 -- 172.21.15.57:0/4197121088 shutdown_connections 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.585+0000 7f738522bb80 1 --2- 172.21.15.57:0/4197121088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbee1a0 0x55ae1cbf1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.585+0000 7f738522bb80 1 --2- 172.21.15.57:0/4197121088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.585+0000 7f738522bb80 1 --2- 172.21.15.57:0/4197121088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbed880 0x55ae1cbf0490 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.585+0000 7f738522bb80 1 -- 172.21.15.57:0/4197121088 >> 172.21.15.57:0/4197121088 conn(0x55ae1cb5c200 msgr2=0x55ae1cbefd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.585+0000 7f738522bb80 1 -- 172.21.15.57:0/4197121088 shutdown_connections 2023-10-08T13:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.585+0000 7f738522bb80 1 -- 172.21.15.57:0/4197121088 wait complete. 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.585+0000 7f738522bb80 1 -- start start 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f738522bb80 1 Processor -- start 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f738522bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f738522bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbed880 0x55ae1cbf7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f738522bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbee1a0 0x55ae1cc54ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f738522bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cbf7aa0 con 0x55ae1cbe80b0 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f738522bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cc55020 con 0x55ae1cbed880 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f738522bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1cc551e0 con 0x55ae1cbee1a0 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f738522bb80 10 client.0 _ll_register_callbacks cb 0x7f7364005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59342/0 (socket says 172.21.15.57:59342) 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f7372d9d700 1 -- 172.21.15.57:0/2029298672 learned_addr learned my addr 172.21.15.57:0/2029298672 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f737259c700 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbed880 0x55ae1cbf7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f737359e700 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbee1a0 0x55ae1cc54ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f7372d9d700 1 -- 172.21.15.57:0/2029298672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbee1a0 msgr2=0x55ae1cc54ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f7372d9d700 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbee1a0 0x55ae1cc54ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f7372d9d700 1 -- 172.21.15.57:0/2029298672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbed880 msgr2=0x55ae1cbf7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f7372d9d700 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbed880 0x55ae1cbf7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f7372d9d700 1 -- 172.21.15.57:0/2029298672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73741a3730 con 0x55ae1cbe80b0 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f737359e700 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbee1a0 0x55ae1cc54ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f7372d9d700 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf6dc0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7364003e10 tx=0x7f736401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292342ddef699bab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.586+0000 7f7356ff5700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f7356ff5700 1 -- 172.21.15.57:0/2029298672 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364030070 con 0x55ae1cbe80b0 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f7356ff5700 1 -- 172.21.15.57:0/2029298672 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f73640048f0 con 0x55ae1cbe80b0 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f738522bb80 1 -- 172.21.15.57:0/2029298672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7364003e40 con 0x55ae1cbe80b0 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f7356ff5700 1 -- 172.21.15.57:0/2029298672 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7364011bb0 con 0x55ae1cbe80b0 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f738522bb80 10 client.17022 fetch_fsmap learned FSMap version 45 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f738522bb80 1 -- 172.21.15.57:0/2029298672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f736800b8d0 con 0x55ae1cbe80b0 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f7356ff5700 1 -- 172.21.15.57:0/2029298672 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7364003e40 con 0x55ae1cbe80b0 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f7356ff5700 1 -- 172.21.15.57:0/2029298672 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f7364003680 con 0x55ae1cbe80b0 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f738522bb80 10 client.17022 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f738522bb80 10 client.17022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:59:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f738522bb80 20 client.17022 populate_metadata read hostname 'smithi057' 2023-10-08T13:59:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f738522bb80 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ae1cc56680 0x55ae1cc58aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f738522bb80 4 client.17022 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:59:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f738522bb80 1 -- 172.21.15.57:0/2029298672 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ae1cb33ad0 con 0x55ae1cc56680 2023-10-08T13:59:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.587+0000 7f737259c700 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ae1cc56680 0x55ae1cc58aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.588+0000 7f7356ff5700 10 client.17022 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:35.588+0000 7f737259c700 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ae1cc56680 0x55ae1cc58aa0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc3a6b294dbc04c3 server_cookie=c423e55d10c23d90 in_seq=0 out_seq=0 2023-10-08T13:59:35.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:35 smithi115 ceph-mon[104408]: pgmap v545: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 461 op/s 2023-10-08T13:59:35.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:35 smithi057 ceph-mon[99476]: pgmap v545: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 461 op/s 2023-10-08T13:59:36.021 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:35 smithi099 ceph-mon[103849]: pgmap v545: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 461 op/s 2023-10-08T13:59:37.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:37 smithi115 ceph-mon[104408]: pgmap v546: 129 pgs: 129 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 355 op/s 2023-10-08T13:59:37.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:37 smithi057 ceph-mon[99476]: pgmap v546: 129 pgs: 129 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 355 op/s 2023-10-08T13:59:38.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:37 smithi099 ceph-mon[103849]: pgmap v546: 129 pgs: 129 active+clean; 2.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 355 op/s 2023-10-08T13:59:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f7356ff5700 1 -- 172.21.15.57:0/2029298672 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ae1cb33ad0 con 0x55ae1cc56680 2023-10-08T13:59:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f7356ff5700 10 client.17022 handle_command_reply: tid=0 2023-10-08T13:59:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 client.17022 shutdown 2023-10-08T13:59:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 -- 172.21.15.57:0/2029298672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 msgr2=0x55ae1cbf6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf6dc0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7364003e10 tx=0x7f736401ecd0 comp rx=0 tx=0).stop 2023-10-08T13:59:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 -- 172.21.15.57:0/2029298672 shutdown_connections 2023-10-08T13:59:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55ae1cc56680 0x55ae1cc58aa0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ae1cbee1a0 0x55ae1cc54ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ae1cbe80b0 0x55ae1cbf6dc0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 --2- 172.21.15.57:0/2029298672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae1cbed880 0x55ae1cbf7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 -- 172.21.15.57:0/2029298672 >> 172.21.15.57:0/2029298672 conn(0x55ae1cb5c200 msgr2=0x55ae1cbecd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f7356ff5700 0 client.17022 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 -- 172.21.15.57:0/2029298672 shutdown_connections 2023-10-08T13:59:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 1 -- 172.21.15.57:0/2029298672 wait complete. 2023-10-08T13:59:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.582+0000 7f738522bb80 20 client.17022 trim_cache size 0 max 16384 2023-10-08T13:59:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:59:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:59:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:59:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:59:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.583+0000 7f7379042700 1 -- 172.21.15.57:0/3400068835 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f735c0a15f0 msgr2=0x7f735c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.583+0000 7f7379042700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f735c0a15f0 0x7f735c0a3ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7374069cf0 tx=0x7f7364009260 comp rx=0 tx=0).stop 2023-10-08T13:59:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.583+0000 7f7379042700 1 -- 172.21.15.57:0/3400068835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73741045e0 msgr2=0x7f7374198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.583+0000 7f7379042700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73741045e0 0x7f7374198470 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f736c00ef30 tx=0x7f736c00c3a0 comp rx=0 tx=0).stop 2023-10-08T13:59:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.583+0000 7f7379042700 1 -- 172.21.15.57:0/3400068835 shutdown_connections 2023-10-08T13:59:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.583+0000 7f7379042700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73741045e0 0x7f7374198470 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.583+0000 7f7379042700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f735c0a15f0 0x7f735c0a3ab0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.583+0000 7f7379042700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73740ff5d0 0x7f73741979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.583+0000 7f7379042700 1 --2- 172.21.15.57:0/3400068835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73740fff90 0x7f7374197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.583+0000 7f7379042700 1 -- 172.21.15.57:0/3400068835 >> 172.21.15.57:0/3400068835 conn(0x7f7374076790 msgr2=0x7f73740fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.584+0000 7f7379042700 1 -- 172.21.15.57:0/3400068835 shutdown_connections 2023-10-08T13:59:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:38.584+0000 7f7379042700 1 -- 172.21.15.57:0/3400068835 wait complete. 2023-10-08T13:59:39.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:39 smithi115 ceph-mon[104408]: pgmap v547: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 264 op/s 2023-10-08T13:59:39.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:39 smithi057 ceph-mon[99476]: pgmap v547: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 264 op/s 2023-10-08T13:59:39.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:39 smithi099 ceph-mon[103849]: pgmap v547: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 264 op/s 2023-10-08T13:59:41.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:41 smithi115 ceph-mon[104408]: pgmap v548: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 265 op/s 2023-10-08T13:59:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:41 smithi057 ceph-mon[99476]: pgmap v548: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 265 op/s 2023-10-08T13:59:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:41 smithi099 ceph-mon[103849]: pgmap v548: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 265 op/s 2023-10-08T13:59:43.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:43 smithi115 ceph-mon[104408]: pgmap v549: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 224 op/s 2023-10-08T13:59:43.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:43 smithi057 ceph-mon[99476]: pgmap v549: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 224 op/s 2023-10-08T13:59:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:43 smithi099 ceph-mon[103849]: pgmap v549: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 224 op/s 2023-10-08T13:59:45.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:45 smithi115 ceph-mon[104408]: pgmap v550: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2023-10-08T13:59:45.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:45 smithi057 ceph-mon[99476]: pgmap v550: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2023-10-08T13:59:46.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:45 smithi099 ceph-mon[103849]: pgmap v550: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2023-10-08T13:59:47.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:47 smithi115 ceph-mon[104408]: pgmap v551: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2023-10-08T13:59:47.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:47 smithi057 ceph-mon[99476]: pgmap v551: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2023-10-08T13:59:48.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:47 smithi099 ceph-mon[103849]: pgmap v551: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2023-10-08T13:59:49.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:49 smithi057 ceph-mon[99476]: pgmap v552: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 214 op/s 2023-10-08T13:59:49.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:49 smithi099 ceph-mon[103849]: pgmap v552: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 214 op/s 2023-10-08T13:59:49.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:49 smithi115 ceph-mon[104408]: pgmap v552: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 214 op/s 2023-10-08T13:59:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.697+0000 7f73c70e3700 1 -- 172.21.15.57:0/159572740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73c01066c0 msgr2=0x7f73c0106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.697+0000 7f73c70e3700 1 --2- 172.21.15.57:0/159572740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73c01066c0 0x7f73c0106aa0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009990 tx=0x7f73a8009ca0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.697+0000 7f73c70e3700 1 -- 172.21.15.57:0/159572740 shutdown_connections 2023-10-08T13:59:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.697+0000 7f73c70e3700 1 --2- 172.21.15.57:0/159572740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c00684c0 0x7f73c010d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.697+0000 7f73c70e3700 1 --2- 172.21.15.57:0/159572740 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73c01066c0 0x7f73c0106aa0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.697+0000 7f73c70e3700 1 --2- 172.21.15.57:0/159572740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c0067b00 0x7f73c0067f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.697+0000 7f73c70e3700 1 -- 172.21.15.57:0/159572740 >> 172.21.15.57:0/159572740 conn(0x7f73c0076790 msgr2=0x7f73c0076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.697+0000 7f73c70e3700 1 -- 172.21.15.57:0/159572740 shutdown_connections 2023-10-08T13:59:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.697+0000 7f73c70e3700 1 -- 172.21.15.57:0/159572740 wait complete. 2023-10-08T13:59:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c70e3700 1 Processor -- start 2023-10-08T13:59:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c70e3700 1 -- start start 2023-10-08T13:59:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c70e3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73c0067b00 0x7f73c019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c70e3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c00684c0 0x7f73c019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c70e3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c01066c0 0x7f73c0196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c70e3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c0196f90 con 0x7f73c0067b00 2023-10-08T13:59:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c70e3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c0197100 con 0x7f73c01066c0 2023-10-08T13:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c70e3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c01972c0 con 0x7f73c00684c0 2023-10-08T13:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c4e7f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73c0067b00 0x7f73c019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c4e7f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73c0067b00 0x7f73c019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59356/0 (socket says 172.21.15.57:59356) 2023-10-08T13:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c4e7f700 1 -- 172.21.15.57:0/3282252629 learned_addr learned my addr 172.21.15.57:0/3282252629 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73bffff700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c00684c0 0x7f73c019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c5680700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c01066c0 0x7f73c0196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c4e7f700 1 -- 172.21.15.57:0/3282252629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c00684c0 msgr2=0x7f73c019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c00684c0 0x7f73c019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c4e7f700 1 -- 172.21.15.57:0/3282252629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c01066c0 msgr2=0x7f73c0196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c01066c0 0x7f73c0196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c4e7f700 1 -- 172.21.15.57:0/3282252629 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a80095f0 con 0x7f73c0067b00 2023-10-08T13:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c5680700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c01066c0 0x7f73c0196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T13:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.698+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73c0067b00 0x7f73c019c800 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f73a8000c00 tx=0x7f73a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.699+0000 7f73bdffb700 1 -- 172.21.15.57:0/3282252629 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a801d070 con 0x7f73c0067b00 2023-10-08T13:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.699+0000 7f73c70e3700 1 -- 172.21.15.57:0/3282252629 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c0197540 con 0x7f73c0067b00 2023-10-08T13:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.699+0000 7f73bdffb700 1 -- 172.21.15.57:0/3282252629 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a8022470 con 0x7f73c0067b00 2023-10-08T13:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.699+0000 7f73bdffb700 1 -- 172.21.15.57:0/3282252629 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8017650 con 0x7f73c0067b00 2023-10-08T13:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.699+0000 7f73c70e3700 1 -- 172.21.15.57:0/3282252629 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c01a37f0 con 0x7f73c0067b00 2023-10-08T13:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.701+0000 7f73bdffb700 1 -- 172.21.15.57:0/3282252629 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f73a80225e0 con 0x7f73c0067b00 2023-10-08T13:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.702+0000 7f73bdffb700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73ac0a15f0 0x7f73ac0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.702+0000 7f73bdffb700 1 -- 172.21.15.57:0/3282252629 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f73a80ce7e0 con 0x7f73c0067b00 2023-10-08T13:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.702+0000 7f73bffff700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73ac0a15f0 0x7f73ac0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.702+0000 7f73bffff700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73ac0a15f0 0x7f73ac0a3ab0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f73c0197af0 tx=0x7f73b0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73d32ccb80 1 Processor -- start 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73d32ccb80 1 -- start start 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc8e880 0x5620ffc8ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8f1a0 0x5620ffc88b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc890b0 0x5620ffc89510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffbf1400 con 0x5620ffc890b0 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffbe2bc0 con 0x5620ffc8e880 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffbe04e0 con 0x5620ffc8f1a0 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73c5680700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc890b0 0x5620ffc89510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73bffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8f1a0 0x5620ffc88b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73c5680700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc890b0 0x5620ffc89510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59368/0 (socket says 172.21.15.57:59368) 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73c5680700 1 -- 172.21.15.57:0/2534754535 learned_addr learned my addr 172.21.15.57:0/2534754535 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/2534754535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc8e880 0x5620ffc8ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73bffff700 1 -- 172.21.15.57:0/2534754535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc8e880 msgr2=0x5620ffc8ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73bffff700 1 --2- 172.21.15.57:0/2534754535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc8e880 0x5620ffc8ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73bffff700 1 -- 172.21.15.57:0/2534754535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc890b0 msgr2=0x5620ffc89510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73bffff700 1 --2- 172.21.15.57:0/2534754535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc890b0 0x5620ffc89510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.703+0000 7f73bffff700 1 -- 172.21.15.57:0/2534754535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620ffa06d80 con 0x5620ffc8f1a0 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73bffff700 1 --2- 172.21.15.57:0/2534754535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8f1a0 0x5620ffc88b70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f73b0011c90 tx=0x7f73b0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=588c7cdd7c82d1f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73b6ffd700 1 -- 172.21.15.57:0/2534754535 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0004230 con 0x5620ffc8f1a0 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73b6ffd700 1 -- 172.21.15.57:0/2534754535 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b0021070 con 0x5620ffc8f1a0 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73b6ffd700 1 -- 172.21.15.57:0/2534754535 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0003c40 con 0x5620ffc8f1a0 2023-10-08T13:59:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2534754535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8f1a0 msgr2=0x5620ffc88b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2534754535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8f1a0 0x5620ffc88b70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f73b0011c90 tx=0x7f73b0014000 comp rx=0 tx=0).stop 2023-10-08T13:59:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2534754535 shutdown_connections 2023-10-08T13:59:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2534754535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc890b0 0x5620ffc89510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2534754535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc8e880 0x5620ffc8ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2534754535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8f1a0 0x5620ffc88b70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.704+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2534754535 >> 172.21.15.57:0/2534754535 conn(0x5620ffbfd200 msgr2=0x5620ffc83b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.705+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2534754535 shutdown_connections 2023-10-08T13:59:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.705+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2534754535 wait complete. 2023-10-08T13:59:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.705+0000 7f73d32ccb80 1 -- start start 2023-10-08T13:59:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.705+0000 7f73d32ccb80 1 Processor -- start 2023-10-08T13:59:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc92530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 0x5620ffc92a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc92fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffbdbdd0 con 0x5620ffc8f1a0 2023-10-08T13:59:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffb786b0 con 0x5620ffc890b0 2023-10-08T13:59:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffb78350 con 0x5620ffc8e880 2023-10-08T13:59:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73d32ccb80 10 client.0 _ll_register_callbacks cb 0x5620ff9d57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c4e7f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc92530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73bffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 0x5620ffc92a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c5680700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc92fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c4e7f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc92530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60796/0 (socket says 172.21.15.57:60796) 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c4e7f700 1 -- 172.21.15.57:0/2178217305 learned_addr learned my addr 172.21.15.57:0/2178217305 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73bffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 0x5620ffc92a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60184/0 (socket says 172.21.15.57:60184) 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c4e7f700 1 -- 172.21.15.57:0/2178217305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 msgr2=0x5620ffc92a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 0x5620ffc92a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c4e7f700 1 -- 172.21.15.57:0/2178217305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 msgr2=0x5620ffc92fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc92fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c4e7f700 1 -- 172.21.15.57:0/2178217305 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b0017070 con 0x5620ffc890b0 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73bffff700 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 0x5620ffc92a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c5680700 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc92fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc92530 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f73a809b190 tx=0x7f73a809adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b73da61983e58ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f739ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f739ffff700 1 -- 172.21.15.57:0/2178217305 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a800fc80 con 0x5620ffc890b0 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f739ffff700 1 -- 172.21.15.57:0/2178217305 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f73a809b490 con 0x5620ffc890b0 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.706+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2178217305 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620ffba3e50 con 0x5620ffc890b0 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f739ffff700 1 -- 172.21.15.57:0/2178217305 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73a80ce5e0 con 0x5620ffc890b0 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f73d32ccb80 10 client.26594 fetch_fsmap learned FSMap version 45 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2178217305 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5620ff9efc00 con 0x5620ffc890b0 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f739ffff700 1 -- 172.21.15.57:0/2178217305 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620ffba3e50 con 0x5620ffc890b0 2023-10-08T13:59:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f739ffff700 1 -- 172.21.15.57:0/2178217305 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f73a809b6f0 con 0x5620ffc890b0 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f73d32ccb80 10 client.26594 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f73d32ccb80 10 client.26594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f73d32ccb80 20 client.26594 populate_metadata read hostname 'smithi057' 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5620ffca2aa0 0x5620ffca4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f73d32ccb80 4 client.26594 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2178217305 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620ffbd4db0 con 0x5620ffca2aa0 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.707+0000 7f73bffff700 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5620ffca2aa0 0x5620ffca4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f739ffff700 10 client.26594 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73bffff700 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5620ffca2aa0 0x5620ffca4ec0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b85ada1fc91c0d server_cookie=2dcda849fa9135ac in_seq=0 out_seq=0 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f739ffff700 1 -- 172.21.15.57:0/2178217305 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5620ffbd4db0 con 0x5620ffca2aa0 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f739ffff700 10 client.26594 handle_command_reply: tid=0 2023-10-08T13:59:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 client.26594 shutdown 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2178217305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 msgr2=0x5620ffc92530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc92530 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f73a809b190 tx=0x7f73a809adf0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2178217305 shutdown_connections 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5620ffca2aa0 0x5620ffca4ec0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc92fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f739ffff700 0 client.26594 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc92530 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2178217305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 0x5620ffc92a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2178217305 >> 172.21.15.57:0/2178217305 conn(0x5620ffbfd200 msgr2=0x5620ffc83010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2178217305 shutdown_connections 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.708+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2178217305 wait complete. 2023-10-08T13:59:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.709+0000 7f73d32ccb80 20 client.26594 trim_cache size 0 max 16384 2023-10-08T13:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.723+0000 7f73d32ccb80 1 Processor -- start 2023-10-08T13:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.723+0000 7f73d32ccb80 1 -- start start 2023-10-08T13:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.723+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc890b0 0x5620ffc88b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.723+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8e880 0x5620ffc91e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.723+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc8f1a0 0x5620ffc99c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.723+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffa5c8c0 con 0x5620ffc8e880 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.723+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffba3e50 con 0x5620ffc8f1a0 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.723+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffc80830 con 0x5620ffc890b0 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73c4e7f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc890b0 0x5620ffc88b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73c4e7f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc890b0 0x5620ffc88b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60190/0 (socket says 172.21.15.57:60190) 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73c4e7f700 1 -- 172.21.15.57:0/4019288369 learned_addr learned my addr 172.21.15.57:0/4019288369 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73bffff700 1 --2- 172.21.15.57:0/4019288369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8e880 0x5620ffc91e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73c5680700 1 --2- 172.21.15.57:0/4019288369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc8f1a0 0x5620ffc99c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73c4e7f700 1 -- 172.21.15.57:0/4019288369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc8f1a0 msgr2=0x5620ffc99c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/4019288369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc8f1a0 0x5620ffc99c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73c4e7f700 1 -- 172.21.15.57:0/4019288369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8e880 msgr2=0x5620ffc91e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/4019288369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8e880 0x5620ffc91e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73c4e7f700 1 -- 172.21.15.57:0/4019288369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620ffa2d1f0 con 0x5620ffc890b0 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73bffff700 1 --2- 172.21.15.57:0/4019288369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8e880 0x5620ffc91e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/4019288369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc890b0 0x5620ffc88b70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f73a80a3260 tx=0x7f73a8022c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b55bae5e56c683c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73b77fe700 1 -- 172.21.15.57:0/4019288369 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a80ce5e0 con 0x5620ffc890b0 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73b77fe700 1 -- 172.21.15.57:0/4019288369 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a809b490 con 0x5620ffc890b0 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.724+0000 7f73b77fe700 1 -- 172.21.15.57:0/4019288369 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a800fce0 con 0x5620ffc890b0 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 -- 172.21.15.57:0/4019288369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc890b0 msgr2=0x5620ffc88b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/4019288369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc890b0 0x5620ffc88b70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f73a80a3260 tx=0x7f73a8022c80 comp rx=0 tx=0).stop 2023-10-08T13:59:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 -- 172.21.15.57:0/4019288369 shutdown_connections 2023-10-08T13:59:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/4019288369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc8f1a0 0x5620ffc99c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/4019288369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc890b0 0x5620ffc88b70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/4019288369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8e880 0x5620ffc91e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 -- 172.21.15.57:0/4019288369 >> 172.21.15.57:0/4019288369 conn(0x5620ffbfd200 msgr2=0x5620ffc8d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 -- 172.21.15.57:0/4019288369 shutdown_connections 2023-10-08T13:59:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 -- 172.21.15.57:0/4019288369 wait complete. 2023-10-08T13:59:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 -- start start 2023-10-08T13:59:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.725+0000 7f73d32ccb80 1 Processor -- start 2023-10-08T13:59:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc99c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 0x5620ffc97c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73d32ccb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffc98680 con 0x5620ffc8f1a0 2023-10-08T13:59:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffc987f0 con 0x5620ffc890b0 2023-10-08T13:59:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73d32ccb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ffc989b0 con 0x5620ffc8e880 2023-10-08T13:59:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73d32ccb80 10 client.0 _ll_register_callbacks cb 0x7f73a809b2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T13:59:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c4e7f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc99c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c5680700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c4e7f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc99c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60810/0 (socket says 172.21.15.57:60810) 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c5680700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc98140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59388/0 (socket says 172.21.15.57:59388) 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c4e7f700 1 -- 172.21.15.57:0/2068691293 learned_addr learned my addr 172.21.15.57:0/2068691293 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73bffff700 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 0x5620ffc97c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c4e7f700 1 -- 172.21.15.57:0/2068691293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 msgr2=0x5620ffc97c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 0x5620ffc97c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c4e7f700 1 -- 172.21.15.57:0/2068691293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 msgr2=0x5620ffc98140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc98140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c4e7f700 1 -- 172.21.15.57:0/2068691293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620ffa06d80 con 0x5620ffc890b0 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c5680700 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f73c4e7f700 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc99c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f73a8097990 tx=0x7f73a809af20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09cd70b4f640fe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T13:59:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.726+0000 7f739ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f739ffff700 1 -- 172.21.15.57:0/2068691293 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a800fc80 con 0x5620ffc890b0 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2068691293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73b0021070 con 0x5620ffc890b0 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f739ffff700 1 -- 172.21.15.57:0/2068691293 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f73a80a3790 con 0x5620ffc890b0 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f739ffff700 1 -- 172.21.15.57:0/2068691293 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73a80ce5e0 con 0x5620ffc890b0 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f73d32ccb80 10 client.26600 fetch_fsmap learned FSMap version 45 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2068691293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5620ff9efc00 con 0x5620ffc890b0 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f739ffff700 1 -- 172.21.15.57:0/2068691293 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b0021070 con 0x5620ffc890b0 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f739ffff700 1 -- 172.21.15.57:0/2068691293 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f73a8097280 con 0x5620ffc890b0 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f73d32ccb80 10 client.26600 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f73d32ccb80 10 client.26600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T13:59:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f73d32ccb80 20 client.26600 populate_metadata read hostname 'smithi057' 2023-10-08T13:59:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5620ffca2aa0 0x5620ffcf88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T13:59:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f73d32ccb80 4 client.26600 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T13:59:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2068691293 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5620ffbd4ad0 con 0x5620ffca2aa0 2023-10-08T13:59:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.727+0000 7f73bffff700 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5620ffca2aa0 0x5620ffcf88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T13:59:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.728+0000 7f739ffff700 10 client.26600 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.728+0000 7f73bffff700 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5620ffca2aa0 0x5620ffcf88e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2314cb3e48042134 server_cookie=909f03a7acd824ce in_seq=0 out_seq=0 2023-10-08T13:59:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f739ffff700 1 -- 172.21.15.57:0/2068691293 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5620ffbd4ad0 con 0x5620ffca2aa0 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f739ffff700 10 client.26600 handle_command_reply: tid=0 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 client.26600 shutdown 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2068691293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 msgr2=0x5620ffc99c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc99c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f73a8097990 tx=0x7f73a809af20 comp rx=0 tx=0).stop 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2068691293 shutdown_connections 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5620ffca2aa0 0x5620ffcf88e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5620ffc8f1a0 0x5620ffc98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620ffc890b0 0x5620ffc99c20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f739ffff700 0 client.26600 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 --2- 172.21.15.57:0/2068691293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620ffc8e880 0x5620ffc97c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2068691293 >> 172.21.15.57:0/2068691293 conn(0x5620ffbfd200 msgr2=0x5620ffc8fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2068691293 shutdown_connections 2023-10-08T13:59:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 1 -- 172.21.15.57:0/2068691293 wait complete. 2023-10-08T13:59:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.839+0000 7f73d32ccb80 20 client.26600 trim_cache size 0 max 16384 2023-10-08T13:59:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T13:59:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T13:59:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T13:59:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T13:59:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 -- 172.21.15.57:0/3282252629 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73ac0a15f0 msgr2=0x7f73ac0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73ac0a15f0 0x7f73ac0a3ab0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f73c0197af0 tx=0x7f73b0009260 comp rx=0 tx=0).stop 2023-10-08T13:59:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 -- 172.21.15.57:0/3282252629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73c0067b00 msgr2=0x7f73c019c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T13:59:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73c0067b00 0x7f73c019c800 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f73a8000c00 tx=0x7f73a8004730 comp rx=0 tx=0).stop 2023-10-08T13:59:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 -- 172.21.15.57:0/3282252629 shutdown_connections 2023-10-08T13:59:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f73ac0a15f0 0x7f73ac0a3ab0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c01066c0 0x7f73c0196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73c0067b00 0x7f73c019c800 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 --2- 172.21.15.57:0/3282252629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c00684c0 0x7f73c019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T13:59:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 -- 172.21.15.57:0/3282252629 >> 172.21.15.57:0/3282252629 conn(0x7f73c0076790 msgr2=0x7f73c00fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T13:59:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 -- 172.21.15.57:0/3282252629 shutdown_connections 2023-10-08T13:59:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T13:59:51.840+0000 7f73c70e3700 1 -- 172.21.15.57:0/3282252629 wait complete. 2023-10-08T13:59:51.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:51 smithi115 ceph-mon[104408]: pgmap v553: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2023-10-08T13:59:51.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:51 smithi057 ceph-mon[99476]: pgmap v553: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2023-10-08T13:59:52.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:51 smithi099 ceph-mon[103849]: pgmap v553: 129 pgs: 129 active+clean; 2.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2023-10-08T13:59:53.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:53 smithi115 ceph-mon[104408]: pgmap v554: 129 pgs: 129 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 266 op/s 2023-10-08T13:59:53.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:53 smithi057 ceph-mon[99476]: pgmap v554: 129 pgs: 129 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 266 op/s 2023-10-08T13:59:54.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:53 smithi099 ceph-mon[103849]: pgmap v554: 129 pgs: 129 active+clean; 2.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 266 op/s 2023-10-08T13:59:55.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:55 smithi115 ceph-mon[104408]: pgmap v555: 129 pgs: 129 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2023-10-08T13:59:55.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:55 smithi057 ceph-mon[99476]: pgmap v555: 129 pgs: 129 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2023-10-08T13:59:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:55 smithi099 ceph-mon[103849]: pgmap v555: 129 pgs: 129 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2023-10-08T13:59:57.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:57 smithi115 ceph-mon[104408]: pgmap v556: 129 pgs: 129 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 332 op/s 2023-10-08T13:59:57.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:57 smithi057 ceph-mon[99476]: pgmap v556: 129 pgs: 129 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 332 op/s 2023-10-08T13:59:58.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:57 smithi099 ceph-mon[103849]: pgmap v556: 129 pgs: 129 active+clean; 2.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 332 op/s 2023-10-08T13:59:59.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 13:59:59 smithi057 ceph-mon[99476]: pgmap v557: 129 pgs: 129 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 359 op/s 2023-10-08T13:59:59.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 13:59:59 smithi099 ceph-mon[103849]: pgmap v557: 129 pgs: 129 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 359 op/s 2023-10-08T13:59:59.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 13:59:59 smithi115 ceph-mon[104408]: pgmap v557: 129 pgs: 129 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 359 op/s 2023-10-08T14:00:00.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a[99452]: 2023-10-08T13:59:59.998+0000 7f3dce8c4700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-08T14:00:00.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a[99452]: 2023-10-08T13:59:59.998+0000 7f3dce8c4700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-08T14:00:00.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a[99452]: 2023-10-08T13:59:59.998+0000 7f3dce8c4700 -1 log_channel(cluster) log [ERR] : mds.i(mds.1): Metadata damage detected 2023-10-08T14:00:00.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:00 smithi057 ceph-mon[99476]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-08T14:00:00.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:00 smithi057 ceph-mon[99476]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-08T14:00:00.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:00 smithi057 ceph-mon[99476]: mds.i(mds.1): Metadata damage detected 2023-10-08T14:00:00.526 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:00 smithi099 ceph-mon[103849]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-08T14:00:00.526 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:00 smithi099 ceph-mon[103849]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-08T14:00:00.527 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:00 smithi099 ceph-mon[103849]: mds.i(mds.1): Metadata damage detected 2023-10-08T14:00:00.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:00 smithi115 ceph-mon[104408]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-08T14:00:00.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:00 smithi115 ceph-mon[104408]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-08T14:00:00.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:00 smithi115 ceph-mon[104408]: mds.i(mds.1): Metadata damage detected 2023-10-08T14:00:01.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:01 smithi057 ceph-mon[99476]: pgmap v558: 129 pgs: 129 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 444 op/s 2023-10-08T14:00:01.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:01 smithi099 ceph-mon[103849]: pgmap v558: 129 pgs: 129 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 444 op/s 2023-10-08T14:00:01.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:01 smithi115 ceph-mon[104408]: pgmap v558: 129 pgs: 129 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 444 op/s 2023-10-08T14:00:03.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:03 smithi115 ceph-mon[104408]: pgmap v559: 129 pgs: 129 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 438 op/s 2023-10-08T14:00:03.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:03 smithi057 ceph-mon[99476]: pgmap v559: 129 pgs: 129 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 438 op/s 2023-10-08T14:00:04.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:03 smithi099 ceph-mon[103849]: pgmap v559: 129 pgs: 129 active+clean; 2.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 438 op/s 2023-10-08T14:00:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.963+0000 7f0012e9d700 1 -- 172.21.15.57:0/232586641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f000c106760 msgr2=0x7f000c106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.963+0000 7f0012e9d700 1 --2- 172.21.15.57:0/232586641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f000c106760 0x7f000c106b40 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0000009990 tx=0x7f0000009ca0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.963+0000 7f0012e9d700 1 -- 172.21.15.57:0/232586641 shutdown_connections 2023-10-08T14:00:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.963+0000 7f0012e9d700 1 --2- 172.21.15.57:0/232586641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0ff620 0x7f000c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.963+0000 7f0012e9d700 1 --2- 172.21.15.57:0/232586641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f000c106760 0x7f000c106b40 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.963+0000 7f0012e9d700 1 --2- 172.21.15.57:0/232586641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f000c0fec60 0x7f000c0ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.963+0000 7f0012e9d700 1 -- 172.21.15.57:0/232586641 >> 172.21.15.57:0/232586641 conn(0x7f000c075bf0 msgr2=0x7f000c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.963+0000 7f0012e9d700 1 -- 172.21.15.57:0/232586641 shutdown_connections 2023-10-08T14:00:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.963+0000 7f0012e9d700 1 -- 172.21.15.57:0/232586641 wait complete. 2023-10-08T14:00:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0012e9d700 1 Processor -- start 2023-10-08T14:00:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0012e9d700 1 -- start start 2023-10-08T14:00:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0012e9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f000c0fec60 0x7f000c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0012e9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0ff620 0x7f000c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0012e9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f000c106760 0x7f000c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0012e9d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c199170 con 0x7f000c0fec60 2023-10-08T14:00:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0012e9d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c1992e0 con 0x7f000c0ff620 2023-10-08T14:00:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0012e9d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c1994a0 con 0x7f000c106760 2023-10-08T14:00:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0010c39700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f000c0fec60 0x7f000c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0010c39700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f000c0fec60 0x7f000c19e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59398/0 (socket says 172.21.15.57:59398) 2023-10-08T14:00:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0010c39700 1 -- 172.21.15.57:0/2104950086 learned_addr learned my addr 172.21.15.57:0/2104950086 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f001143a700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f000c106760 0x7f000c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f000bfff700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0ff620 0x7f000c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0010c39700 1 -- 172.21.15.57:0/2104950086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f000c106760 msgr2=0x7f000c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0010c39700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f000c106760 0x7f000c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0010c39700 1 -- 172.21.15.57:0/2104950086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0ff620 msgr2=0x7f000c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0010c39700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0ff620 0x7f000c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f0010c39700 1 -- 172.21.15.57:0/2104950086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00000095f0 con 0x7f000c0fec60 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f001143a700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f000c106760 0x7f000c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.964+0000 7f000bfff700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0ff620 0x7f000c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.965+0000 7f0010c39700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f000c0fec60 0x7f000c19e9e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c00 tx=0x7f0000004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.965+0000 7f0009ffb700 1 -- 172.21.15.57:0/2104950086 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000001d070 con 0x7f000c0fec60 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.965+0000 7f0012e9d700 1 -- 172.21.15.57:0/2104950086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c199720 con 0x7f000c0fec60 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.965+0000 7f0009ffb700 1 -- 172.21.15.57:0/2104950086 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0000022470 con 0x7f000c0fec60 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.965+0000 7f0009ffb700 1 -- 172.21.15.57:0/2104950086 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000017650 con 0x7f000c0fec60 2023-10-08T14:00:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.965+0000 7f0012e9d700 1 -- 172.21.15.57:0/2104950086 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c1a59d0 con 0x7f000c0fec60 2023-10-08T14:00:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.967+0000 7f0009ffb700 1 -- 172.21.15.57:0/2104950086 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f00000225e0 con 0x7f000c0fec60 2023-10-08T14:00:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.968+0000 7f0009ffb700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efffc0a15f0 0x7efffc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.968+0000 7f0009ffb700 1 -- 172.21.15.57:0/2104950086 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f00000ce600 con 0x7f000c0fec60 2023-10-08T14:00:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.968+0000 7f000bfff700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efffc0a15f0 0x7efffc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.968+0000 7f001f086b80 1 Processor -- start 2023-10-08T14:00:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.968+0000 7f001f086b80 1 -- start start 2023-10-08T14:00:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f000bfff700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efffc0a15f0 0x7efffc0a3ab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f000c199cd0 tx=0x7efff8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f001f086b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c5ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f001f086b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5f1a0 0x562559c58b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f001f086b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c59510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f001f086b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559bc1400 con 0x562559c5e880 2023-10-08T14:00:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f001f086b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559bb2bc0 con 0x562559c590b0 2023-10-08T14:00:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f001f086b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559bb04e0 con 0x562559c5f1a0 2023-10-08T14:00:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f0010c39700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c5ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f000bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5f1a0 0x562559c58b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f0010c39700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c5ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59406/0 (socket says 172.21.15.57:59406) 2023-10-08T14:00:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f0010c39700 1 -- 172.21.15.57:0/4247534216 learned_addr learned my addr 172.21.15.57:0/4247534216 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f001143a700 1 --2- 172.21.15.57:0/4247534216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c59510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f0010c39700 1 -- 172.21.15.57:0/4247534216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5f1a0 msgr2=0x562559c58b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f0010c39700 1 --2- 172.21.15.57:0/4247534216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5f1a0 0x562559c58b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f0010c39700 1 -- 172.21.15.57:0/4247534216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 msgr2=0x562559c59510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f0010c39700 1 --2- 172.21.15.57:0/4247534216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c59510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f0010c39700 1 -- 172.21.15.57:0/4247534216 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625599d6d80 con 0x562559c5e880 2023-10-08T14:00:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f000bfff700 1 --2- 172.21.15.57:0/4247534216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5f1a0 0x562559c58b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:00:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7f0010c39700 1 --2- 172.21.15.57:0/4247534216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c5ec60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0000097d80 tx=0x7f000009af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c765671b0715fbd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7efff6ffd700 1 -- 172.21.15.57:0/4247534216 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000009b310 con 0x562559c5e880 2023-10-08T14:00:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7efff6ffd700 1 -- 172.21.15.57:0/4247534216 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00000cd070 con 0x562559c5e880 2023-10-08T14:00:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.969+0000 7efff6ffd700 1 -- 172.21.15.57:0/4247534216 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00000ce240 con 0x562559c5e880 2023-10-08T14:00:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.970+0000 7f001f086b80 1 -- 172.21.15.57:0/4247534216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 msgr2=0x562559c5ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.970+0000 7f001f086b80 1 --2- 172.21.15.57:0/4247534216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c5ec60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0000097d80 tx=0x7f000009af80 comp rx=0 tx=0).stop 2023-10-08T14:00:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.970+0000 7f001f086b80 1 -- 172.21.15.57:0/4247534216 shutdown_connections 2023-10-08T14:00:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.970+0000 7f001f086b80 1 --2- 172.21.15.57:0/4247534216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c59510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.970+0000 7f001f086b80 1 --2- 172.21.15.57:0/4247534216 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c5ec60 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.970+0000 7f001f086b80 1 --2- 172.21.15.57:0/4247534216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5f1a0 0x562559c58b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.970+0000 7f001f086b80 1 -- 172.21.15.57:0/4247534216 >> 172.21.15.57:0/4247534216 conn(0x562559bcd200 msgr2=0x562559c53b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.970+0000 7f001f086b80 1 -- 172.21.15.57:0/4247534216 shutdown_connections 2023-10-08T14:00:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.970+0000 7f001f086b80 1 -- 172.21.15.57:0/4247534216 wait complete. 2023-10-08T14:00:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.970+0000 7f001f086b80 1 -- start start 2023-10-08T14:00:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001f086b80 1 Processor -- start 2023-10-08T14:00:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001f086b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c590b0 0x562559c65190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001f086b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001f086b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c5f1a0 0x562559c62be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001f086b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559babdd0 con 0x562559c5e880 2023-10-08T14:00:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001f086b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559b486b0 con 0x562559c5f1a0 2023-10-08T14:00:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001f086b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559b48350 con 0x562559c590b0 2023-10-08T14:00:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001f086b80 10 client.0 _ll_register_callbacks cb 0x5625599a57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:00:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f000bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f0010c39700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c590b0 0x562559c65190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001143a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c5f1a0 0x562559c62be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f000bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c624f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59414/0 (socket says 172.21.15.57:59414) 2023-10-08T14:00:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f000bfff700 1 -- 172.21.15.57:0/1702281451 learned_addr learned my addr 172.21.15.57:0/1702281451 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001143a700 1 -- 172.21.15.57:0/1702281451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c590b0 msgr2=0x562559c65190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001143a700 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c590b0 0x562559c65190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001143a700 1 -- 172.21.15.57:0/1702281451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 msgr2=0x562559c624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001143a700 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f001143a700 1 -- 172.21.15.57:0/1702281451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff8017070 con 0x562559c5f1a0 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.971+0000 7f000bfff700 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7f001143a700 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c5f1a0 0x562559c62be0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0004003fd0 tx=0x7f0004004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5e1cbb5fe86c28c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7effebfff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7effebfff700 1 -- 172.21.15.57:0/1702281451 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004004910 con 0x562559c5f1a0 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7effebfff700 1 -- 172.21.15.57:0/1702281451 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0004004bb0 con 0x562559c5f1a0 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7f001f086b80 1 -- 172.21.15.57:0/1702281451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562559b73e50 con 0x562559c5f1a0 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7effebfff700 1 -- 172.21.15.57:0/1702281451 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0004004d70 con 0x562559c5f1a0 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7f001f086b80 10 client.26612 fetch_fsmap learned FSMap version 45 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7f001f086b80 1 -- 172.21.15.57:0/1702281451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f000c1a59d0 con 0x562559c5f1a0 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7effebfff700 1 -- 172.21.15.57:0/1702281451 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562559b73e50 con 0x562559c5f1a0 2023-10-08T14:00:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7effebfff700 1 -- 172.21.15.57:0/1702281451 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f0004007620 con 0x562559c5f1a0 2023-10-08T14:00:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7f001f086b80 10 client.26612 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T14:00:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7f001f086b80 10 client.26612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:00:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7f001f086b80 20 client.26612 populate_metadata read hostname 'smithi057' 2023-10-08T14:00:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7f001f086b80 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562559c72ae0 0x562559c74f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7f001f086b80 4 client.26612 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:00:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.972+0000 7f001f086b80 1 -- 172.21.15.57:0/1702281451 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562559ba4db0 con 0x562559c72ae0 2023-10-08T14:00:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.973+0000 7f0010c39700 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562559c72ae0 0x562559c74f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.973+0000 7effebfff700 10 client.26612 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.973+0000 7f0010c39700 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562559c72ae0 0x562559c74f00 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e88f004268fae8 server_cookie=19c4c95525e94f4d in_seq=0 out_seq=0 2023-10-08T14:00:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.973+0000 7effebfff700 1 -- 172.21.15.57:0/1702281451 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562559ba4db0 con 0x562559c72ae0 2023-10-08T14:00:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7effebfff700 10 client.26612 handle_command_reply: tid=0 2023-10-08T14:00:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 client.26612 shutdown 2023-10-08T14:00:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 -- 172.21.15.57:0/1702281451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c5f1a0 msgr2=0x562559c62be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c5f1a0 0x562559c62be0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0004003fd0 tx=0x7f0004004040 comp rx=0 tx=0).stop 2023-10-08T14:00:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 -- 172.21.15.57:0/1702281451 shutdown_connections 2023-10-08T14:00:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562559c72ae0 0x562559c74f00 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c5f1a0 0x562559c62be0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c590b0 0x562559c65190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 --2- 172.21.15.57:0/1702281451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5e880 0x562559c624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7effebfff700 0 client.26612 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 -- 172.21.15.57:0/1702281451 >> 172.21.15.57:0/1702281451 conn(0x562559bcd200 msgr2=0x562559c53010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 -- 172.21.15.57:0/1702281451 shutdown_connections 2023-10-08T14:00:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 1 -- 172.21.15.57:0/1702281451 wait complete. 2023-10-08T14:00:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.974+0000 7f001f086b80 20 client.26612 trim_cache size 0 max 16384 2023-10-08T14:00:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001f086b80 1 Processor -- start 2023-10-08T14:00:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001f086b80 1 -- start start 2023-10-08T14:00:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001f086b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c58b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001f086b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5e880 0x562559c69960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001f086b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c53790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001f086b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559a2c8c0 con 0x562559c5f1a0 2023-10-08T14:00:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001f086b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559c50860 con 0x562559c590b0 2023-10-08T14:00:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001f086b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559c69ea0 con 0x562559c5e880 2023-10-08T14:00:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001143a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c53790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001143a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c53790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59424/0 (socket says 172.21.15.57:59424) 2023-10-08T14:00:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001143a700 1 -- 172.21.15.57:0/2615373627 learned_addr learned my addr 172.21.15.57:0/2615373627 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f000bfff700 1 --2- 172.21.15.57:0/2615373627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5e880 0x562559c69960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f0010c39700 1 --2- 172.21.15.57:0/2615373627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c58b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001143a700 1 -- 172.21.15.57:0/2615373627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5e880 msgr2=0x562559c69960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001143a700 1 --2- 172.21.15.57:0/2615373627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5e880 0x562559c69960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001143a700 1 -- 172.21.15.57:0/2615373627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 msgr2=0x562559c58b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001143a700 1 --2- 172.21.15.57:0/2615373627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c58b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001143a700 1 -- 172.21.15.57:0/2615373627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625599bfc00 con 0x562559c5f1a0 2023-10-08T14:00:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f0010c39700 1 --2- 172.21.15.57:0/2615373627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c58b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T14:00:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.989+0000 7f001143a700 1 --2- 172.21.15.57:0/2615373627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c53790 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0004003fa0 tx=0x7f0004007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af95b1d05b45d911 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7efff77fe700 1 -- 172.21.15.57:0/2615373627 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000402e070 con 0x562559c5f1a0 2023-10-08T14:00:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7efff77fe700 1 -- 172.21.15.57:0/2615373627 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000400f040 con 0x562559c5f1a0 2023-10-08T14:00:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7efff77fe700 1 -- 172.21.15.57:0/2615373627 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00040077c0 con 0x562559c5f1a0 2023-10-08T14:00:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7f001f086b80 1 -- 172.21.15.57:0/2615373627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 msgr2=0x562559c53790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7f001f086b80 1 --2- 172.21.15.57:0/2615373627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c53790 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0004003fa0 tx=0x7f0004007bf0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7f001f086b80 1 -- 172.21.15.57:0/2615373627 shutdown_connections 2023-10-08T14:00:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7f001f086b80 1 --2- 172.21.15.57:0/2615373627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c53790 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7f001f086b80 1 --2- 172.21.15.57:0/2615373627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c58b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7f001f086b80 1 --2- 172.21.15.57:0/2615373627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5e880 0x562559c69960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7f001f086b80 1 -- 172.21.15.57:0/2615373627 >> 172.21.15.57:0/2615373627 conn(0x562559bcd200 msgr2=0x562559c60dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7f001f086b80 1 -- 172.21.15.57:0/2615373627 shutdown_connections 2023-10-08T14:00:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.990+0000 7f001f086b80 1 -- 172.21.15.57:0/2615373627 wait complete. 2023-10-08T14:00:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001f086b80 1 -- start start 2023-10-08T14:00:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001f086b80 1 Processor -- start 2023-10-08T14:00:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001f086b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c69960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001f086b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5e880 0x562559c67940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001f086b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c68080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001f086b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559c68790 con 0x562559c5f1a0 2023-10-08T14:00:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001f086b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559cc5910 con 0x562559c590b0 2023-10-08T14:00:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001f086b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562559cc5a50 con 0x562559c5e880 2023-10-08T14:00:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001f086b80 10 client.0 _ll_register_callbacks cb 0x7f0004009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:00:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001143a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c68080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f000bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5e880 0x562559c67940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001143a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c68080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59430/0 (socket says 172.21.15.57:59430) 2023-10-08T14:00:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001143a700 1 -- 172.21.15.57:0/1097907776 learned_addr learned my addr 172.21.15.57:0/1097907776 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f0010c39700 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c69960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.991+0000 7f001143a700 1 -- 172.21.15.57:0/1097907776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5e880 msgr2=0x562559c67940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001143a700 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5e880 0x562559c67940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001143a700 1 -- 172.21.15.57:0/1097907776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 msgr2=0x562559c69960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001143a700 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c69960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001143a700 1 -- 172.21.15.57:0/1097907776 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c1a59d0 con 0x562559c5f1a0 2023-10-08T14:00:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001143a700 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c68080 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0004007980 tx=0x7f0004003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7102c50915dbfeb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7effebfff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:00:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7effebfff700 1 -- 172.21.15.57:0/1097907776 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000402e070 con 0x562559c5f1a0 2023-10-08T14:00:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7effebfff700 1 -- 172.21.15.57:0/1097907776 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f0004004b20 con 0x562559c5f1a0 2023-10-08T14:00:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001f086b80 1 -- 172.21.15.57:0/1097907776 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00000cd070 con 0x562559c5f1a0 2023-10-08T14:00:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7effebfff700 1 -- 172.21.15.57:0/1097907776 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0004004ce0 con 0x562559c5f1a0 2023-10-08T14:00:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001f086b80 10 client.17058 fetch_fsmap learned FSMap version 45 2023-10-08T14:00:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001f086b80 1 -- 172.21.15.57:0/1097907776 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5625599fd1f0 con 0x562559c5f1a0 2023-10-08T14:00:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7effebfff700 1 -- 172.21.15.57:0/1097907776 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00000cd070 con 0x562559c5f1a0 2023-10-08T14:00:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7effebfff700 1 -- 172.21.15.57:0/1097907776 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 45) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f000401f4e0 con 0x562559c5f1a0 2023-10-08T14:00:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001f086b80 10 client.17058 fetch_fsmap finished waiting for FSMap version 45 2023-10-08T14:00:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001f086b80 10 client.17058 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:00:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001f086b80 20 client.17058 populate_metadata read hostname 'smithi057' 2023-10-08T14:00:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001f086b80 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562559cc7030 0x562559cc9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001f086b80 4 client.17058 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:00:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.992+0000 7f001f086b80 1 -- 172.21.15.57:0/1097907776 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562559ba4ad0 con 0x562559cc7030 2023-10-08T14:00:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.993+0000 7f0010c39700 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562559cc7030 0x562559cc9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.993+0000 7effebfff700 10 client.17058 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:04.993+0000 7f0010c39700 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562559cc7030 0x562559cc9450 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd693dd39b22945 server_cookie=f070309ee954ff5 in_seq=0 out_seq=0 2023-10-08T14:00:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7effebfff700 1 -- 172.21.15.57:0/1097907776 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562559ba4ad0 con 0x562559cc7030 2023-10-08T14:00:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7effebfff700 10 client.17058 handle_command_reply: tid=0 2023-10-08T14:00:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 client.17058 shutdown 2023-10-08T14:00:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 -- 172.21.15.57:0/1097907776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 msgr2=0x562559c68080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c68080 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0004007980 tx=0x7f0004003fd0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 -- 172.21.15.57:0/1097907776 shutdown_connections 2023-10-08T14:00:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562559cc7030 0x562559cc9450 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562559c5f1a0 0x562559c68080 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7effebfff700 0 client.17058 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562559c590b0 0x562559c69960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 --2- 172.21.15.57:0/1097907776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562559c5e880 0x562559c67940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 -- 172.21.15.57:0/1097907776 >> 172.21.15.57:0/1097907776 conn(0x562559bcd200 msgr2=0x562559c5d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 -- 172.21.15.57:0/1097907776 shutdown_connections 2023-10-08T14:00:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 1 -- 172.21.15.57:0/1097907776 wait complete. 2023-10-08T14:00:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.007+0000 7f001f086b80 20 client.17058 trim_cache size 0 max 16384 2023-10-08T14:00:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:00:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:00:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:00:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:00:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 -- 172.21.15.57:0/2104950086 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efffc0a15f0 msgr2=0x7efffc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efffc0a15f0 0x7efffc0a3ab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f000c199cd0 tx=0x7efff8009260 comp rx=0 tx=0).stop 2023-10-08T14:00:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 -- 172.21.15.57:0/2104950086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f000c0fec60 msgr2=0x7f000c19e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f000c0fec60 0x7f000c19e9e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c00 tx=0x7f0000004730 comp rx=0 tx=0).stop 2023-10-08T14:00:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 -- 172.21.15.57:0/2104950086 shutdown_connections 2023-10-08T14:00:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f000c106760 0x7f000c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f000c0fec60 0x7f000c19e9e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efffc0a15f0 0x7efffc0a3ab0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 --2- 172.21.15.57:0/2104950086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0ff620 0x7f000c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 -- 172.21.15.57:0/2104950086 >> 172.21.15.57:0/2104950086 conn(0x7f000c075bf0 msgr2=0x7f000c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 -- 172.21.15.57:0/2104950086 shutdown_connections 2023-10-08T14:00:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:05.008+0000 7f0012e9d700 1 -- 172.21.15.57:0/2104950086 wait complete. 2023-10-08T14:00:05.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:05 smithi115 ceph-mon[104408]: pgmap v560: 129 pgs: 129 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 481 op/s 2023-10-08T14:00:05.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:05 smithi057 ceph-mon[99476]: pgmap v560: 129 pgs: 129 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 481 op/s 2023-10-08T14:00:06.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:05 smithi099 ceph-mon[103849]: pgmap v560: 129 pgs: 129 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 481 op/s 2023-10-08T14:00:06.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:06 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:00:07.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:06 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:00:07.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:06 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:00:07.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:07 smithi057 ceph-mon[99476]: pgmap v561: 129 pgs: 129 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 493 op/s 2023-10-08T14:00:08.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:07 smithi099 ceph-mon[103849]: pgmap v561: 129 pgs: 129 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 493 op/s 2023-10-08T14:00:08.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:07 smithi115 ceph-mon[104408]: pgmap v561: 129 pgs: 129 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 493 op/s 2023-10-08T14:00:09.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:09 smithi057 ceph-mon[99476]: pgmap v562: 129 pgs: 129 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 488 op/s 2023-10-08T14:00:09.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:09 smithi099 ceph-mon[103849]: pgmap v562: 129 pgs: 129 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 488 op/s 2023-10-08T14:00:09.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:09 smithi115 ceph-mon[104408]: pgmap v562: 129 pgs: 129 active+clean; 2.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 488 op/s 2023-10-08T14:00:11.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:11 smithi057 ceph-mon[99476]: osdmap e98: 12 total, 12 up, 12 in 2023-10-08T14:00:11.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:11 smithi057 ceph-mon[99476]: pgmap v564: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 491 op/s 2023-10-08T14:00:11.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:11 smithi099 ceph-mon[103849]: osdmap e98: 12 total, 12 up, 12 in 2023-10-08T14:00:11.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:11 smithi099 ceph-mon[103849]: pgmap v564: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 491 op/s 2023-10-08T14:00:11.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:11 smithi115 ceph-mon[104408]: osdmap e98: 12 total, 12 up, 12 in 2023-10-08T14:00:11.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:11 smithi115 ceph-mon[104408]: pgmap v564: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 491 op/s 2023-10-08T14:00:12.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:00:13.007 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:00:13.037 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:00:13.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:13 smithi115 ceph-mon[104408]: pgmap v565: 129 pgs: 10 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 84 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 453 op/s 2023-10-08T14:00:13.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:13 smithi057 ceph-mon[99476]: pgmap v565: 129 pgs: 10 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 84 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 453 op/s 2023-10-08T14:00:14.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:13 smithi099 ceph-mon[103849]: pgmap v565: 129 pgs: 10 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 84 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 453 op/s 2023-10-08T14:00:14.967 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:00:14.967 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:00:14.967 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:14 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:00:14.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:00:14.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:00:14.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:14 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:00:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:00:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:00:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:14 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:00:15.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:15 smithi057 ceph-mon[99476]: pgmap v566: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 399 op/s 2023-10-08T14:00:16.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:15 smithi115 ceph-mon[104408]: pgmap v566: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 399 op/s 2023-10-08T14:00:16.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:15 smithi099 ceph-mon[103849]: pgmap v566: 129 pgs: 18 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 399 op/s 2023-10-08T14:00:17.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:17 smithi115 ceph-mon[104408]: pgmap v567: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 344 op/s 2023-10-08T14:00:17.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:17 smithi057 ceph-mon[99476]: pgmap v567: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 344 op/s 2023-10-08T14:00:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.191+0000 7f8236454700 1 -- 172.21.15.57:0/3170718868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8230108380 msgr2=0x7f823010b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.191+0000 7f8236454700 1 --2- 172.21.15.57:0/3170718868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8230108380 0x7f823010b940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f822800b0a0 tx=0x7f822800b3b0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.192+0000 7f8236454700 1 -- 172.21.15.57:0/3170718868 shutdown_connections 2023-10-08T14:00:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.192+0000 7f8236454700 1 --2- 172.21.15.57:0/3170718868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f823010c020 0x7f8230072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.192+0000 7f8236454700 1 --2- 172.21.15.57:0/3170718868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82301079d0 0x7f8230107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.192+0000 7f8236454700 1 --2- 172.21.15.57:0/3170718868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8230108380 0x7f823010b940 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.192+0000 7f8236454700 1 -- 172.21.15.57:0/3170718868 >> 172.21.15.57:0/3170718868 conn(0x7f823006b7c0 msgr2=0x7f823006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.192+0000 7f8236454700 1 -- 172.21.15.57:0/3170718868 shutdown_connections 2023-10-08T14:00:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.192+0000 7f8236454700 1 -- 172.21.15.57:0/3170718868 wait complete. 2023-10-08T14:00:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.193+0000 7f8236454700 1 Processor -- start 2023-10-08T14:00:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.194+0000 7f8236454700 1 -- start start 2023-10-08T14:00:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.195+0000 7f8236454700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82301079d0 0x7f82300773e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.195+0000 7f8236454700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8230108380 0x7f82300757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.195+0000 7f8236454700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823010c020 0x7f8230075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.195+0000 7f8236454700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230076380 con 0x7f82301079d0 2023-10-08T14:00:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.195+0000 7f8236454700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300764f0 con 0x7f8230108380 2023-10-08T14:00:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.195+0000 7f8236454700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82301c30d0 con 0x7f823010c020 2023-10-08T14:00:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.197+0000 7f822f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8230108380 0x7f82300757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.197+0000 7f822f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8230108380 0x7f82300757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60868/0 (socket says 172.21.15.57:60868) 2023-10-08T14:00:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.197+0000 7f822f7fe700 1 -- 172.21.15.57:0/3340998752 learned_addr learned my addr 172.21.15.57:0/3340998752 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.197+0000 7f822f7fe700 1 -- 172.21.15.57:0/3340998752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823010c020 msgr2=0x7f8230075ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:00:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.197+0000 7f822f7fe700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823010c020 0x7f8230075ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.197+0000 7f822f7fe700 1 -- 172.21.15.57:0/3340998752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82301079d0 msgr2=0x7f82300773e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.197+0000 7f822f7fe700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82301079d0 0x7f82300773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.197+0000 7f822f7fe700 1 -- 172.21.15.57:0/3340998752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8228009d00 con 0x7f8230108380 2023-10-08T14:00:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.198+0000 7f822f7fe700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8230108380 0x7f82300757a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8228009850 tx=0x7f8228004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.198+0000 7f822d7fa700 1 -- 172.21.15.57:0/3340998752 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822800e040 con 0x7f8230108380 2023-10-08T14:00:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.198+0000 7f822d7fa700 1 -- 172.21.15.57:0/3340998752 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f822800be40 con 0x7f8230108380 2023-10-08T14:00:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.198+0000 7f822d7fa700 1 -- 172.21.15.57:0/3340998752 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280045e0 con 0x7f8230108380 2023-10-08T14:00:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.206+0000 7f8236454700 1 -- 172.21.15.57:0/3340998752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82301c3270 con 0x7f8230108380 2023-10-08T14:00:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.206+0000 7f8236454700 1 -- 172.21.15.57:0/3340998752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82301c36a0 con 0x7f8230108380 2023-10-08T14:00:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.212+0000 7f824263db80 1 Processor -- start 2023-10-08T14:00:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.213+0000 7f824263db80 1 -- start start 2023-10-08T14:00:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.213+0000 7f824263db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b3965880 0x55f4b3965c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.213+0000 7f824263db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b39661a0 0x55f4b395fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.213+0000 7f824263db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b39600b0 0x55f4b3960510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.213+0000 7f824263db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b38c8400 con 0x55f4b3965880 2023-10-08T14:00:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.213+0000 7f824263db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b38b9bc0 con 0x55f4b39600b0 2023-10-08T14:00:18.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.213+0000 7f824263db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b38b74e0 con 0x55f4b39661a0 2023-10-08T14:00:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.214+0000 7f82349f1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b3965880 0x55f4b3965c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.215+0000 7f82349f1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b3965880 0x55f4b3965c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59446/0 (socket says 172.21.15.57:59446) 2023-10-08T14:00:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.215+0000 7f82349f1700 1 -- 172.21.15.57:0/2443901446 learned_addr learned my addr 172.21.15.57:0/2443901446 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.218+0000 7f822ffff700 1 --2- 172.21.15.57:0/2443901446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b39661a0 0x55f4b395fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.218+0000 7f822ffff700 1 -- 172.21.15.57:0/2443901446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b39600b0 msgr2=0x55f4b3960510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.218+0000 7f822ffff700 1 --2- 172.21.15.57:0/2443901446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b39600b0 0x55f4b3960510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.218+0000 7f822ffff700 1 -- 172.21.15.57:0/2443901446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b3965880 msgr2=0x55f4b3965c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.218+0000 7f822ffff700 1 --2- 172.21.15.57:0/2443901446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b3965880 0x55f4b3965c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.218+0000 7f822ffff700 1 -- 172.21.15.57:0/2443901446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4b36ddd80 con 0x55f4b39661a0 2023-10-08T14:00:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.219+0000 7f822ffff700 1 --2- 172.21.15.57:0/2443901446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b39661a0 0x55f4b395fb70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8224009bd0 tx=0x7f8224009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e7fa20980d10a5d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.222+0000 7f82167fc700 1 -- 172.21.15.57:0/2443901446 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224004230 con 0x55f4b39661a0 2023-10-08T14:00:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.222+0000 7f82167fc700 1 -- 172.21.15.57:0/2443901446 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f822400ed60 con 0x55f4b39661a0 2023-10-08T14:00:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.222+0000 7f82167fc700 1 -- 172.21.15.57:0/2443901446 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224005020 con 0x55f4b39661a0 2023-10-08T14:00:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.222+0000 7f822d7fa700 1 -- 172.21.15.57:0/3340998752 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8228004740 con 0x7f8230108380 2023-10-08T14:00:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.223+0000 7f822d7fa700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82180a16f0 0x7f82180a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.224+0000 7f822d7fa700 1 -- 172.21.15.57:0/3340998752 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f82280d0060 con 0x7f8230108380 2023-10-08T14:00:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.225+0000 7f824263db80 1 -- 172.21.15.57:0/2443901446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b39661a0 msgr2=0x55f4b395fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.225+0000 7f824263db80 1 --2- 172.21.15.57:0/2443901446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b39661a0 0x55f4b395fb70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8224009bd0 tx=0x7f8224009ee0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.227+0000 7f824263db80 1 -- 172.21.15.57:0/2443901446 shutdown_connections 2023-10-08T14:00:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.227+0000 7f824263db80 1 --2- 172.21.15.57:0/2443901446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b39600b0 0x55f4b3960510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.227+0000 7f824263db80 1 --2- 172.21.15.57:0/2443901446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b3965880 0x55f4b3965c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.227+0000 7f824263db80 1 --2- 172.21.15.57:0/2443901446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b39661a0 0x55f4b395fb70 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8224009bd0 tx=0x7f8224009ee0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.227+0000 7f824263db80 1 -- 172.21.15.57:0/2443901446 >> 172.21.15.57:0/2443901446 conn(0x55f4b38d4200 msgr2=0x55f4b395ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.227+0000 7f82349f1700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82180a16f0 0x7f82180a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.228+0000 7f82349f1700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82180a16f0 0x7f82180a3bb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8220009990 tx=0x7f8220009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.233+0000 7f824263db80 1 -- 172.21.15.57:0/2443901446 shutdown_connections 2023-10-08T14:00:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.234+0000 7f824263db80 1 -- 172.21.15.57:0/2443901446 wait complete. 2023-10-08T14:00:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.234+0000 7f824263db80 1 -- start start 2023-10-08T14:00:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f824263db80 1 Processor -- start 2023-10-08T14:00:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f824263db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b39697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f824263db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b3965880 0x55f4b3969d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f824263db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b396a260 0x55f4b3975d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f824263db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b38b2dd0 con 0x55f4b39600b0 2023-10-08T14:00:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f824263db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b384f6b0 con 0x55f4b396a260 2023-10-08T14:00:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f824263db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b384f350 con 0x55f4b3965880 2023-10-08T14:00:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f824263db80 10 client.0 _ll_register_callbacks cb 0x55f4b36ac7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:00:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f822ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b39697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f822ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b39697e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59452/0 (socket says 172.21.15.57:59452) 2023-10-08T14:00:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.235+0000 7f822ffff700 1 -- 172.21.15.57:0/1571151638 learned_addr learned my addr 172.21.15.57:0/1571151638 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f82349f1700 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b396a260 0x55f4b3975d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f822f7fe700 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b3965880 0x55f4b3969d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f822ffff700 1 -- 172.21.15.57:0/1571151638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b3965880 msgr2=0x55f4b3969d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f822ffff700 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b3965880 0x55f4b3969d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f822ffff700 1 -- 172.21.15.57:0/1571151638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b396a260 msgr2=0x55f4b3975d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f822ffff700 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b396a260 0x55f4b3975d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f822ffff700 1 -- 172.21.15.57:0/1571151638 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8224009880 con 0x55f4b39600b0 2023-10-08T14:00:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f822f7fe700 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b3965880 0x55f4b3969d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T14:00:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f82349f1700 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b396a260 0x55f4b3975d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:00:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f822ffff700 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b39697e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8224009ba0 tx=0x7f822400eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632983936855ef73 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.236+0000 7f820f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:00:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.237+0000 7f820f7fe700 1 -- 172.21.15.57:0/1571151638 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224004450 con 0x55f4b39600b0 2023-10-08T14:00:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.237+0000 7f820f7fe700 1 -- 172.21.15.57:0/1571151638 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f8224014070 con 0x55f4b39600b0 2023-10-08T14:00:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.237+0000 7f824263db80 1 -- 172.21.15.57:0/1571151638 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4b387ae50 con 0x55f4b39600b0 2023-10-08T14:00:18.241 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:17 smithi099 ceph-mon[103849]: pgmap v567: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 344 op/s 2023-10-08T14:00:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.237+0000 7f820f7fe700 1 -- 172.21.15.57:0/1571151638 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f822400eed0 con 0x55f4b39600b0 2023-10-08T14:00:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.237+0000 7f824263db80 10 client.17067 fetch_fsmap learned FSMap version 46 2023-10-08T14:00:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.237+0000 7f824263db80 1 -- 172.21.15.57:0/1571151638 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55f4b36c6c00 con 0x55f4b39600b0 2023-10-08T14:00:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.237+0000 7f820f7fe700 1 -- 172.21.15.57:0/1571151638 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f4b387ae50 con 0x55f4b39600b0 2023-10-08T14:00:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.237+0000 7f820f7fe700 1 -- 172.21.15.57:0/1571151638 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f8224003710 con 0x55f4b39600b0 2023-10-08T14:00:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.237+0000 7f824263db80 10 client.17067 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:00:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.237+0000 7f824263db80 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:00:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.238+0000 7f824263db80 20 client.17067 populate_metadata read hostname 'smithi057' 2023-10-08T14:00:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.238+0000 7f824263db80 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f4b397a210 0x55f4b397c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.238+0000 7f824263db80 4 client.17067 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:00:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.238+0000 7f824263db80 1 -- 172.21.15.57:0/1571151638 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4b38abdb0 con 0x55f4b397a210 2023-10-08T14:00:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.238+0000 7f822f7fe700 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f4b397a210 0x55f4b397c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.242+0000 7f820f7fe700 10 client.17067 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.242+0000 7f822f7fe700 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f4b397a210 0x55f4b397c630 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ba6e944479f9472 server_cookie=dad8f478c7d406f3 in_seq=0 out_seq=0 2023-10-08T14:00:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f820f7fe700 1 -- 172.21.15.57:0/1571151638 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f4b38abdb0 con 0x55f4b397a210 2023-10-08T14:00:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f820f7fe700 10 client.17067 handle_command_reply: tid=0 2023-10-08T14:00:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f824263db80 1 client.17067 shutdown 2023-10-08T14:00:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f824263db80 1 -- 172.21.15.57:0/1571151638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 msgr2=0x55f4b39697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f824263db80 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b39697e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8224009ba0 tx=0x7f822400eab0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f824263db80 1 -- 172.21.15.57:0/1571151638 shutdown_connections 2023-10-08T14:00:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f824263db80 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f4b397a210 0x55f4b397c630 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f824263db80 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b396a260 0x55f4b3975d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f824263db80 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b39697e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f824263db80 1 --2- 172.21.15.57:0/1571151638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b3965880 0x55f4b3969d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f824263db80 1 -- 172.21.15.57:0/1571151638 >> 172.21.15.57:0/1571151638 conn(0x55f4b38d4200 msgr2=0x55f4b395a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.243+0000 7f820f7fe700 0 client.17067 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.247+0000 7f824263db80 1 -- 172.21.15.57:0/1571151638 shutdown_connections 2023-10-08T14:00:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.247+0000 7f824263db80 1 -- 172.21.15.57:0/1571151638 wait complete. 2023-10-08T14:00:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.248+0000 7f824263db80 20 client.17067 trim_cache size 0 max 16384 2023-10-08T14:00:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.274+0000 7f824263db80 1 Processor -- start 2023-10-08T14:00:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.274+0000 7f824263db80 1 -- start start 2023-10-08T14:00:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.274+0000 7f824263db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b395fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.274+0000 7f824263db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b3965880 0x55f4b39690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.274+0000 7f824263db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b396a260 0x55f4b395a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.274+0000 7f824263db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b37338c0 con 0x55f4b39600b0 2023-10-08T14:00:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.274+0000 7f824263db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b3952670 con 0x55f4b3965880 2023-10-08T14:00:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.274+0000 7f824263db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b396c4f0 con 0x55f4b396a260 2023-10-08T14:00:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.275+0000 7f822ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b395fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.275+0000 7f822ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b395fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59460/0 (socket says 172.21.15.57:59460) 2023-10-08T14:00:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.275+0000 7f822ffff700 1 -- 172.21.15.57:0/294277356 learned_addr learned my addr 172.21.15.57:0/294277356 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.275+0000 7f822ffff700 1 -- 172.21.15.57:0/294277356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b396a260 msgr2=0x55f4b395a790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:00:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.275+0000 7f822ffff700 1 --2- 172.21.15.57:0/294277356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b396a260 0x55f4b395a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.275+0000 7f822ffff700 1 -- 172.21.15.57:0/294277356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b3965880 msgr2=0x55f4b39690c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:00:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.275+0000 7f822ffff700 1 --2- 172.21.15.57:0/294277356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b3965880 0x55f4b39690c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.275+0000 7f822ffff700 1 -- 172.21.15.57:0/294277356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4b37041f0 con 0x55f4b39600b0 2023-10-08T14:00:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.276+0000 7f822ffff700 1 --2- 172.21.15.57:0/294277356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b395fb70 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8224003f00 tx=0x7f8224009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b37fffb4656727 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.276+0000 7f8216ffd700 1 -- 172.21.15.57:0/294277356 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224003c40 con 0x55f4b39600b0 2023-10-08T14:00:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.276+0000 7f8216ffd700 1 -- 172.21.15.57:0/294277356 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f822401d070 con 0x55f4b39600b0 2023-10-08T14:00:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.276+0000 7f8216ffd700 1 -- 172.21.15.57:0/294277356 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82240053c0 con 0x55f4b39600b0 2023-10-08T14:00:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.281+0000 7f824263db80 1 -- 172.21.15.57:0/294277356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 msgr2=0x55f4b395fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.281+0000 7f824263db80 1 --2- 172.21.15.57:0/294277356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b395fb70 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8224003f00 tx=0x7f8224009440 comp rx=0 tx=0).stop 2023-10-08T14:00:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.287+0000 7f824263db80 1 -- 172.21.15.57:0/294277356 shutdown_connections 2023-10-08T14:00:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.287+0000 7f824263db80 1 --2- 172.21.15.57:0/294277356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b396a260 0x55f4b395a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.287+0000 7f824263db80 1 --2- 172.21.15.57:0/294277356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b39600b0 0x55f4b395fb70 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.287+0000 7f824263db80 1 --2- 172.21.15.57:0/294277356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b3965880 0x55f4b39690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.287+0000 7f824263db80 1 -- 172.21.15.57:0/294277356 >> 172.21.15.57:0/294277356 conn(0x55f4b38d4200 msgr2=0x55f4b39648b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.296+0000 7f824263db80 1 -- 172.21.15.57:0/294277356 shutdown_connections 2023-10-08T14:00:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.296+0000 7f824263db80 1 -- 172.21.15.57:0/294277356 wait complete. 2023-10-08T14:00:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.296+0000 7f824263db80 1 -- start start 2023-10-08T14:00:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.296+0000 7f824263db80 1 Processor -- start 2023-10-08T14:00:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.297+0000 7f824263db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b39600b0 0x55f4b3970e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.297+0000 7f824263db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b3965880 0x55f4b396ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.297+0000 7f824263db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b396a260 0x55f4b396f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.297+0000 7f824263db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b396f930 con 0x55f4b396a260 2023-10-08T14:00:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.297+0000 7f824263db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b396faa0 con 0x55f4b39600b0 2023-10-08T14:00:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.297+0000 7f824263db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b39de030 con 0x55f4b3965880 2023-10-08T14:00:18.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.297+0000 7f824263db80 10 client.0 _ll_register_callbacks cb 0x55f4b36ac7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:00:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.298+0000 7f82349f1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b396a260 0x55f4b396f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.298+0000 7f82349f1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b396a260 0x55f4b396f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59466/0 (socket says 172.21.15.57:59466) 2023-10-08T14:00:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.298+0000 7f82349f1700 1 -- 172.21.15.57:0/1753652302 learned_addr learned my addr 172.21.15.57:0/1753652302 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.299+0000 7f82349f1700 1 -- 172.21.15.57:0/1753652302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b3965880 msgr2=0x55f4b396ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.299+0000 7f82349f1700 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b3965880 0x55f4b396ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.299+0000 7f82349f1700 1 -- 172.21.15.57:0/1753652302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b39600b0 msgr2=0x55f4b3970e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:00:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.299+0000 7f82349f1700 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b39600b0 0x55f4b3970e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.299+0000 7f82349f1700 1 -- 172.21.15.57:0/1753652302 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4b36c6c00 con 0x55f4b396a260 2023-10-08T14:00:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.300+0000 7f82349f1700 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b396a260 0x55f4b396f220 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8220015730 tx=0x7f8220004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2f79f23d2168a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.301+0000 7f820f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:00:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.305+0000 7f820f7fe700 1 -- 172.21.15.57:0/1753652302 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220015cd0 con 0x55f4b396a260 2023-10-08T14:00:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.305+0000 7f820f7fe700 1 -- 172.21.15.57:0/1753652302 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f8220015e30 con 0x55f4b396a260 2023-10-08T14:00:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.305+0000 7f824263db80 1 -- 172.21.15.57:0/1753652302 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f822400ed60 con 0x55f4b396a260 2023-10-08T14:00:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.306+0000 7f820f7fe700 1 -- 172.21.15.57:0/1753652302 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8220005a20 con 0x55f4b396a260 2023-10-08T14:00:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.306+0000 7f824263db80 10 client.17073 fetch_fsmap learned FSMap version 46 2023-10-08T14:00:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.306+0000 7f824263db80 1 -- 172.21.15.57:0/1753652302 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55f4b36dc250 con 0x55f4b396a260 2023-10-08T14:00:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.306+0000 7f820f7fe700 1 -- 172.21.15.57:0/1753652302 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f822400ed60 con 0x55f4b396a260 2023-10-08T14:00:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.306+0000 7f820f7fe700 1 -- 172.21.15.57:0/1753652302 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f8220021440 con 0x55f4b396a260 2023-10-08T14:00:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.307+0000 7f824263db80 10 client.17073 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:00:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.307+0000 7f824263db80 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:00:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.307+0000 7f824263db80 20 client.17073 populate_metadata read hostname 'smithi057' 2023-10-08T14:00:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.307+0000 7f824263db80 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f4b39c4840 0x55f4b39c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.307+0000 7f824263db80 4 client.17073 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:00:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.307+0000 7f824263db80 1 -- 172.21.15.57:0/1753652302 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4b38abad0 con 0x55f4b39c4840 2023-10-08T14:00:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.311+0000 7f822ffff700 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f4b39c4840 0x55f4b39c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.312+0000 7f820f7fe700 10 client.17073 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.312+0000 7f822ffff700 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f4b39c4840 0x55f4b39c6c60 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=364aac74e7d673cf server_cookie=319aa7e07a9d1e93 in_seq=0 out_seq=0 2023-10-08T14:00:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.432+0000 7f820f7fe700 1 -- 172.21.15.57:0/1753652302 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4b38abad0 con 0x55f4b39c4840 2023-10-08T14:00:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.432+0000 7f820f7fe700 10 client.17073 handle_command_reply: tid=0 2023-10-08T14:00:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.433+0000 7f824263db80 1 client.17073 shutdown 2023-10-08T14:00:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.437+0000 7f824263db80 1 -- 172.21.15.57:0/1753652302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b396a260 msgr2=0x55f4b396f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.437+0000 7f824263db80 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b396a260 0x55f4b396f220 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8220015730 tx=0x7f8220004800 comp rx=0 tx=0).stop 2023-10-08T14:00:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.437+0000 7f824263db80 1 -- 172.21.15.57:0/1753652302 shutdown_connections 2023-10-08T14:00:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.437+0000 7f824263db80 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f4b39c4840 0x55f4b39c6c60 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.437+0000 7f824263db80 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f4b396a260 0x55f4b396f220 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.437+0000 7f824263db80 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4b39600b0 0x55f4b3970e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.437+0000 7f824263db80 1 --2- 172.21.15.57:0/1753652302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b3965880 0x55f4b396ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.437+0000 7f824263db80 1 -- 172.21.15.57:0/1753652302 >> 172.21.15.57:0/1753652302 conn(0x55f4b38d4200 msgr2=0x55f4b39753f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.437+0000 7f820f7fe700 0 client.17073 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.447+0000 7f824263db80 1 -- 172.21.15.57:0/1753652302 shutdown_connections 2023-10-08T14:00:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.447+0000 7f824263db80 1 -- 172.21.15.57:0/1753652302 wait complete. 2023-10-08T14:00:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.453+0000 7f824263db80 20 client.17073 trim_cache size 0 max 16384 2023-10-08T14:00:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:00:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:00:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:00:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:00:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.461+0000 7f8236454700 1 -- 172.21.15.57:0/3340998752 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82180a16f0 msgr2=0x7f82180a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.461+0000 7f8236454700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82180a16f0 0x7f82180a3bb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8220009990 tx=0x7f8220009110 comp rx=0 tx=0).stop 2023-10-08T14:00:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.461+0000 7f8236454700 1 -- 172.21.15.57:0/3340998752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8230108380 msgr2=0x7f82300757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.461+0000 7f8236454700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8230108380 0x7f82300757a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8228009850 tx=0x7f8228004db0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.461+0000 7f8236454700 1 -- 172.21.15.57:0/3340998752 shutdown_connections 2023-10-08T14:00:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.461+0000 7f8236454700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823010c020 0x7f8230075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.461+0000 7f8236454700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82301079d0 0x7f82300773e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.461+0000 7f8236454700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82180a16f0 0x7f82180a3bb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.461+0000 7f8236454700 1 --2- 172.21.15.57:0/3340998752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8230108380 0x7f82300757a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.461+0000 7f8236454700 1 -- 172.21.15.57:0/3340998752 >> 172.21.15.57:0/3340998752 conn(0x7f823006b7c0 msgr2=0x7f823010a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.462+0000 7f8236454700 1 -- 172.21.15.57:0/3340998752 shutdown_connections 2023-10-08T14:00:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:18.462+0000 7f8236454700 1 -- 172.21.15.57:0/3340998752 wait complete. 2023-10-08T14:00:19.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:19 smithi057 ceph-mon[99476]: pgmap v568: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2023-10-08T14:00:19.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:19 smithi099 ceph-mon[103849]: pgmap v568: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2023-10-08T14:00:19.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:19 smithi115 ceph-mon[104408]: pgmap v568: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2023-10-08T14:00:21.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:21 smithi057 ceph-mon[99476]: pgmap v569: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 270 op/s 2023-10-08T14:00:22.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:21 smithi099 ceph-mon[103849]: pgmap v569: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 270 op/s 2023-10-08T14:00:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:21 smithi115 ceph-mon[104408]: pgmap v569: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 270 op/s 2023-10-08T14:00:23.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:22 smithi099 ceph-mon[103849]: pgmap v570: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2023-10-08T14:00:23.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:22 smithi115 ceph-mon[104408]: pgmap v570: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2023-10-08T14:00:23.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:22 smithi057 ceph-mon[99476]: pgmap v570: 129 pgs: 18 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 68 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2023-10-08T14:00:25.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:25 smithi057 ceph-mon[99476]: pgmap v571: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 238 op/s 2023-10-08T14:00:26.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:25 smithi099 ceph-mon[103849]: pgmap v571: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 238 op/s 2023-10-08T14:00:26.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:25 smithi115 ceph-mon[104408]: pgmap v571: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 238 op/s 2023-10-08T14:00:27.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:27 smithi057 ceph-mon[99476]: pgmap v572: 129 pgs: 11 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 90 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2023-10-08T14:00:28.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:27 smithi099 ceph-mon[103849]: pgmap v572: 129 pgs: 11 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 90 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2023-10-08T14:00:28.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:27 smithi115 ceph-mon[104408]: pgmap v572: 129 pgs: 11 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 90 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2023-10-08T14:00:29.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:29 smithi057 ceph-mon[99476]: pgmap v573: 129 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 106 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-10-08T14:00:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:29 smithi099 ceph-mon[103849]: pgmap v573: 129 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 106 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-10-08T14:00:29.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:29 smithi115 ceph-mon[104408]: pgmap v573: 129 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 106 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 245 op/s 2023-10-08T14:00:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.630+0000 7fa57bb43700 1 -- 172.21.15.57:0/3926730778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5741045e0 msgr2=0x7fa5741049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.630+0000 7fa57bb43700 1 --2- 172.21.15.57:0/3926730778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5741045e0 0x7fa5741049c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa570009990 tx=0x7fa570009ca0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.630+0000 7fa57bb43700 1 -- 172.21.15.57:0/3926730778 shutdown_connections 2023-10-08T14:00:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.630+0000 7fa57bb43700 1 --2- 172.21.15.57:0/3926730778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5740fff90 0x7fa57410c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.630+0000 7fa57bb43700 1 --2- 172.21.15.57:0/3926730778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5741045e0 0x7fa5741049c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.630+0000 7fa57bb43700 1 --2- 172.21.15.57:0/3926730778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5740ff5d0 0x7fa5740ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.630+0000 7fa57bb43700 1 -- 172.21.15.57:0/3926730778 >> 172.21.15.57:0/3926730778 conn(0x7fa574076790 msgr2=0x7fa574076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.630+0000 7fa57bb43700 1 -- 172.21.15.57:0/3926730778 shutdown_connections 2023-10-08T14:00:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.630+0000 7fa57bb43700 1 -- 172.21.15.57:0/3926730778 wait complete. 2023-10-08T14:00:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.631+0000 7fa57bb43700 1 Processor -- start 2023-10-08T14:00:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.631+0000 7fa57bb43700 1 -- start start 2023-10-08T14:00:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.631+0000 7fa57bb43700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5740ff5d0 0x7fa574197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.631+0000 7fa57bb43700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5740fff90 0x7fa574197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.631+0000 7fa57bb43700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5741045e0 0x7fa5741a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.631+0000 7fa57bb43700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa574198700 con 0x7fa5740fff90 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.631+0000 7fa57bb43700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5741a2f30 con 0x7fa5740ff5d0 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.631+0000 7fa57bb43700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5741a3070 con 0x7fa5741045e0 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa5790de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5740fff90 0x7fa574197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa5790de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5740fff90 0x7fa574197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59482/0 (socket says 172.21.15.57:59482) 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa5790de700 1 -- 172.21.15.57:0/3519727676 learned_addr learned my addr 172.21.15.57:0/3519727676 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa57a0e0700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5741045e0 0x7fa5741a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa5798df700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5740ff5d0 0x7fa574197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa5790de700 1 -- 172.21.15.57:0/3519727676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5741045e0 msgr2=0x7fa5741a29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa5790de700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5741045e0 0x7fa5741a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa5790de700 1 -- 172.21.15.57:0/3519727676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5740ff5d0 msgr2=0x7fa574197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa5790de700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5740ff5d0 0x7fa574197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa5790de700 1 -- 172.21.15.57:0/3519727676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c009c40 con 0x7fa5740fff90 2023-10-08T14:00:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa5790de700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5740fff90 0x7fa574197eb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa56400e9d0 tx=0x7fa56400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa56affd700 1 -- 172.21.15.57:0/3519727676 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564009790 con 0x7fa5740fff90 2023-10-08T14:00:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa57bb43700 1 -- 172.21.15.57:0/3519727676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5700095f0 con 0x7fa5740fff90 2023-10-08T14:00:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa56affd700 1 -- 172.21.15.57:0/3519727676 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa56400ee60 con 0x7fa5740fff90 2023-10-08T14:00:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa56affd700 1 -- 172.21.15.57:0/3519727676 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564018680 con 0x7fa5740fff90 2023-10-08T14:00:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.632+0000 7fa57bb43700 1 -- 172.21.15.57:0/3519727676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5741a3650 con 0x7fa5740fff90 2023-10-08T14:00:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.636+0000 7fa56affd700 1 -- 172.21.15.57:0/3519727676 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa564010430 con 0x7fa5740fff90 2023-10-08T14:00:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa587d2cb80 1 Processor -- start 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa587d2cb80 1 -- start start 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa56affd700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5600a15f0 0x7fa5600a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 0x55fba8768550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba876c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba876d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba86cc400 con 0x55fba8768170 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba86bdbc0 con 0x55fba876ce60 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba86bb4e0 con 0x55fba8768a90 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa56affd700 1 -- 172.21.15.57:0/3519727676 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fa564014070 con 0x7fa5740fff90 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa57a0e0700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5600a15f0 0x7fa5600a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.637+0000 7fa5798df700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 0x55fba8768550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa5790de700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba876c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa5798df700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 0x55fba8768550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59488/0 (socket says 172.21.15.57:59488) 2023-10-08T14:00:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa5798df700 1 -- 172.21.15.57:0/1368994498 learned_addr learned my addr 172.21.15.57:0/1368994498 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa57a0e0700 1 --2- 172.21.15.57:0/1368994498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba876d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa5790de700 1 -- 172.21.15.57:0/1368994498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 msgr2=0x55fba876d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa5790de700 1 --2- 172.21.15.57:0/1368994498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba876d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa5790de700 1 -- 172.21.15.57:0/1368994498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 msgr2=0x55fba8768550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa5790de700 1 --2- 172.21.15.57:0/1368994498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 0x55fba8768550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa5790de700 1 -- 172.21.15.57:0/1368994498 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fba84e1d80 con 0x55fba8768a90 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa57a0e0700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5600a15f0 0x7fa5600a3ab0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009fd0 tx=0x7fa56c00fdd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.638+0000 7fa5790de700 1 --2- 172.21.15.57:0/1368994498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba876c920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa5640caa30 tx=0x7fa564099f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dac8fc35172bb8e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.639+0000 7fa55ffff700 1 -- 172.21.15.57:0/1368994498 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56409a2c0 con 0x55fba8768a90 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.639+0000 7fa55ffff700 1 -- 172.21.15.57:0/1368994498 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5640b0020 con 0x55fba8768a90 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.639+0000 7fa55ffff700 1 -- 172.21.15.57:0/1368994498 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640cd290 con 0x55fba8768a90 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 -- 172.21.15.57:0/1368994498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 msgr2=0x55fba876c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/1368994498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba876c920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa5640caa30 tx=0x7fa564099f10 comp rx=0 tx=0).stop 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 -- 172.21.15.57:0/1368994498 shutdown_connections 2023-10-08T14:00:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/1368994498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba876d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/1368994498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 0x55fba8768550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/1368994498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba876c920 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 -- 172.21.15.57:0/1368994498 >> 172.21.15.57:0/1368994498 conn(0x55fba86d8200 msgr2=0x55fba875eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 -- 172.21.15.57:0/1368994498 shutdown_connections 2023-10-08T14:00:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 -- 172.21.15.57:0/1368994498 wait complete. 2023-10-08T14:00:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 -- start start 2023-10-08T14:00:31.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.640+0000 7fa587d2cb80 1 Processor -- start 2023-10-08T14:00:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 0x55fba8771a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba8771f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba8772490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba86b6dd0 con 0x55fba8768170 2023-10-08T14:00:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba86536b0 con 0x55fba876ce60 2023-10-08T14:00:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba8653350 con 0x55fba8768a90 2023-10-08T14:00:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa587d2cb80 10 client.0 _ll_register_callbacks cb 0x55fba84b07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:00:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa5790de700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba8771f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa5790de700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba8771f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60308/0 (socket says 172.21.15.57:60308) 2023-10-08T14:00:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa5790de700 1 -- 172.21.15.57:0/929558124 learned_addr learned my addr 172.21.15.57:0/929558124 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa5798df700 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 0x55fba8771a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.641+0000 7fa57a0e0700 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba8772490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa5790de700 1 -- 172.21.15.57:0/929558124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 msgr2=0x55fba8772490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa5790de700 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba8772490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa5790de700 1 -- 172.21.15.57:0/929558124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 msgr2=0x55fba8771a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa5790de700 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 0x55fba8771a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa5790de700 1 -- 172.21.15.57:0/929558124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5741a3650 con 0x55fba8768a90 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa5790de700 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba8771f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa564099fc0 tx=0x7fa564096000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13158d408b832b5f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa55d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa55d7fa700 1 -- 172.21.15.57:0/929558124 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640a2650 con 0x55fba8768a90 2023-10-08T14:00:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa55d7fa700 1 -- 172.21.15.57:0/929558124 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fa5640b1070 con 0x55fba8768a90 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa587d2cb80 1 -- 172.21.15.57:0/929558124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fba867ee50 con 0x55fba8768a90 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa55d7fa700 1 -- 172.21.15.57:0/929558124 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa56409a2c0 con 0x55fba8768a90 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa587d2cb80 10 client.26668 fetch_fsmap learned FSMap version 46 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.642+0000 7fa587d2cb80 1 -- 172.21.15.57:0/929558124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55fba84cac00 con 0x55fba8768a90 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa55d7fa700 1 -- 172.21.15.57:0/929558124 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fba867ee50 con 0x55fba8768a90 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa55d7fa700 1 -- 172.21.15.57:0/929558124 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fa5640a21c0 con 0x55fba8768a90 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa587d2cb80 10 client.26668 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa587d2cb80 10 client.26668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa587d2cb80 20 client.26668 populate_metadata read hostname 'smithi057' 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fba8781e20 0x55fba8784240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa587d2cb80 4 client.26668 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa587d2cb80 1 -- 172.21.15.57:0/929558124 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fba86afdb0 con 0x55fba8781e20 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa5798df700 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fba8781e20 0x55fba8784240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa55d7fa700 10 client.26668 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.643+0000 7fa5798df700 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fba8781e20 0x55fba8784240 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74ab6a21710699e server_cookie=ec917c1f5495c467 in_seq=0 out_seq=0 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.644+0000 7fa55d7fa700 1 -- 172.21.15.57:0/929558124 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fba86afdb0 con 0x55fba8781e20 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.644+0000 7fa55d7fa700 10 client.26668 handle_command_reply: tid=0 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.644+0000 7fa587d2cb80 1 client.26668 shutdown 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.644+0000 7fa587d2cb80 1 -- 172.21.15.57:0/929558124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 msgr2=0x55fba8771f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.644+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba8771f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa564099fc0 tx=0x7fa564096000 comp rx=0 tx=0).stop 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.645+0000 7fa587d2cb80 1 -- 172.21.15.57:0/929558124 shutdown_connections 2023-10-08T14:00:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.645+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fba8781e20 0x55fba8784240 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.645+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba8772490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.645+0000 7fa55d7fa700 0 client.26668 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.645+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768170 0x55fba8771a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.645+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/929558124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768a90 0x55fba8771f50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.645+0000 7fa587d2cb80 1 -- 172.21.15.57:0/929558124 >> 172.21.15.57:0/929558124 conn(0x55fba86d8200 msgr2=0x55fba8762b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.645+0000 7fa587d2cb80 1 -- 172.21.15.57:0/929558124 shutdown_connections 2023-10-08T14:00:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.645+0000 7fa587d2cb80 1 -- 172.21.15.57:0/929558124 wait complete. 2023-10-08T14:00:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.645+0000 7fa587d2cb80 20 client.26668 trim_cache size 0 max 16384 2023-10-08T14:00:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.665+0000 7fa587d2cb80 1 Processor -- start 2023-10-08T14:00:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.665+0000 7fa587d2cb80 1 -- start start 2023-10-08T14:00:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.665+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 0x55fba8768550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.665+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768a90 0x55fba8779010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.665+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba876e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.665+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba85378c0 con 0x55fba8768a90 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.665+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba8769170 con 0x55fba876ce60 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.665+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba8760550 con 0x55fba8768170 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa5790de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768a90 0x55fba8779010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa5790de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768a90 0x55fba8779010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59502/0 (socket says 172.21.15.57:59502) 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa5790de700 1 -- 172.21.15.57:0/4211562172 learned_addr learned my addr 172.21.15.57:0/4211562172 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa5798df700 1 --2- 172.21.15.57:0/4211562172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 0x55fba8768550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa57a0e0700 1 --2- 172.21.15.57:0/4211562172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba876e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa5790de700 1 -- 172.21.15.57:0/4211562172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 msgr2=0x55fba8768550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa5790de700 1 --2- 172.21.15.57:0/4211562172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 0x55fba8768550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa5790de700 1 -- 172.21.15.57:0/4211562172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 msgr2=0x55fba876e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa5790de700 1 --2- 172.21.15.57:0/4211562172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba876e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa5790de700 1 -- 172.21.15.57:0/4211562172 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fba85081f0 con 0x55fba8768a90 2023-10-08T14:00:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa5790de700 1 --2- 172.21.15.57:0/4211562172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768a90 0x55fba8779010 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa56409a250 tx=0x7fa564010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d37147e41a6207 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.666+0000 7fa568d82700 1 -- 172.21.15.57:0/4211562172 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56409a2c0 con 0x55fba8768a90 2023-10-08T14:00:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.667+0000 7fa568d82700 1 -- 172.21.15.57:0/4211562172 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5640a2650 con 0x55fba8768a90 2023-10-08T14:00:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.667+0000 7fa568d82700 1 -- 172.21.15.57:0/4211562172 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640997b0 con 0x55fba8768a90 2023-10-08T14:00:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.668+0000 7fa587d2cb80 1 -- 172.21.15.57:0/4211562172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768a90 msgr2=0x55fba8779010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.668+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/4211562172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768a90 0x55fba8779010 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa56409a250 tx=0x7fa564010a00 comp rx=0 tx=0).stop 2023-10-08T14:00:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.668+0000 7fa587d2cb80 1 -- 172.21.15.57:0/4211562172 shutdown_connections 2023-10-08T14:00:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.668+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/4211562172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba876ce60 0x55fba876e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.668+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/4211562172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 0x55fba8768550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.668+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/4211562172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba8768a90 0x55fba8779010 secure :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa56409a250 tx=0x7fa564010a00 comp rx=0 tx=0).stop 2023-10-08T14:00:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.668+0000 7fa587d2cb80 1 -- 172.21.15.57:0/4211562172 >> 172.21.15.57:0/4211562172 conn(0x55fba86d8200 msgr2=0x55fba87702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.675+0000 7fa587d2cb80 1 -- 172.21.15.57:0/4211562172 shutdown_connections 2023-10-08T14:00:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.676+0000 7fa587d2cb80 1 -- 172.21.15.57:0/4211562172 wait complete. 2023-10-08T14:00:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.676+0000 7fa587d2cb80 1 -- start start 2023-10-08T14:00:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.676+0000 7fa587d2cb80 1 Processor -- start 2023-10-08T14:00:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 0x55fba8763b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba876ce60 0x55fba87640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa587d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba87645f0 0x55fba8778010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba8772930 con 0x55fba876ce60 2023-10-08T14:00:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba8778550 con 0x55fba87645f0 2023-10-08T14:00:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa587d2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba8778690 con 0x55fba8768170 2023-10-08T14:00:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa587d2cb80 10 client.0 _ll_register_callbacks cb 0x7fa564095ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:00:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa57a0e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba87645f0 0x55fba8778010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa5798df700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 0x55fba8763b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa57a0e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba87645f0 0x55fba8778010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60934/0 (socket says 172.21.15.57:60934) 2023-10-08T14:00:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.677+0000 7fa57a0e0700 1 -- 172.21.15.57:0/3498759161 learned_addr learned my addr 172.21.15.57:0/3498759161 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa57a0e0700 1 -- 172.21.15.57:0/3498759161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 msgr2=0x55fba8763b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa57a0e0700 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 0x55fba8763b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa57a0e0700 1 -- 172.21.15.57:0/3498759161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba876ce60 msgr2=0x55fba87640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa57a0e0700 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba876ce60 0x55fba87640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa57a0e0700 1 -- 172.21.15.57:0/3498759161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fba84cac00 con 0x55fba87645f0 2023-10-08T14:00:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa5798df700 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 0x55fba8763b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:00:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa57a0e0700 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba87645f0 0x55fba8778010 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa56c018e00 tx=0x7fa56c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d37802598dc891 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa55d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:00:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa55d7fa700 1 -- 172.21.15.57:0/3498759161 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00e040 con 0x55fba87645f0 2023-10-08T14:00:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa55d7fa700 1 -- 172.21.15.57:0/3498759161 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fa56c030440 con 0x55fba87645f0 2023-10-08T14:00:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa587d2cb80 1 -- 172.21.15.57:0/3498759161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5640b0020 con 0x55fba87645f0 2023-10-08T14:00:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa55d7fa700 1 -- 172.21.15.57:0/3498759161 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa56c030600 con 0x55fba87645f0 2023-10-08T14:00:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.678+0000 7fa587d2cb80 10 client.26642 fetch_fsmap learned FSMap version 46 2023-10-08T14:00:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.679+0000 7fa587d2cb80 1 -- 172.21.15.57:0/3498759161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55fba84e0250 con 0x55fba87645f0 2023-10-08T14:00:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.679+0000 7fa55d7fa700 1 -- 172.21.15.57:0/3498759161 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5640b0020 con 0x55fba87645f0 2023-10-08T14:00:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.679+0000 7fa55d7fa700 1 -- 172.21.15.57:0/3498759161 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fa56c01a850 con 0x55fba87645f0 2023-10-08T14:00:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.679+0000 7fa587d2cb80 10 client.26642 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:00:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.679+0000 7fa587d2cb80 10 client.26642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:00:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.679+0000 7fa587d2cb80 20 client.26642 populate_metadata read hostname 'smithi057' 2023-10-08T14:00:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.679+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fba87ce3e0 0x55fba87d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.679+0000 7fa587d2cb80 4 client.26642 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:00:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.679+0000 7fa587d2cb80 1 -- 172.21.15.57:0/3498759161 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fba86afad0 con 0x55fba87ce3e0 2023-10-08T14:00:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.679+0000 7fa5798df700 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fba87ce3e0 0x55fba87d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.680+0000 7fa5798df700 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fba87ce3e0 0x55fba87d0800 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293b24c1aee85206 server_cookie=97eeabe98cf89562 in_seq=0 out_seq=0 2023-10-08T14:00:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.680+0000 7fa55d7fa700 10 client.26642 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.713+0000 7fa55d7fa700 1 -- 172.21.15.57:0/3498759161 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fba86afad0 con 0x55fba87ce3e0 2023-10-08T14:00:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa55d7fa700 10 client.26642 handle_command_reply: tid=0 2023-10-08T14:00:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa587d2cb80 1 client.26642 shutdown 2023-10-08T14:00:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa587d2cb80 1 -- 172.21.15.57:0/3498759161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba87645f0 msgr2=0x55fba8778010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba87645f0 0x55fba8778010 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa56c018e00 tx=0x7fa56c004b40 comp rx=0 tx=0).stop 2023-10-08T14:00:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa587d2cb80 1 -- 172.21.15.57:0/3498759161 shutdown_connections 2023-10-08T14:00:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fba87ce3e0 0x55fba87d0800 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fba87645f0 0x55fba8778010 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fba8768170 0x55fba8763b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa587d2cb80 1 --2- 172.21.15.57:0/3498759161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fba876ce60 0x55fba87640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa587d2cb80 1 -- 172.21.15.57:0/3498759161 >> 172.21.15.57:0/3498759161 conn(0x55fba86d8200 msgr2=0x55fba876f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.714+0000 7fa55d7fa700 0 client.26642 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.715+0000 7fa587d2cb80 1 -- 172.21.15.57:0/3498759161 shutdown_connections 2023-10-08T14:00:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.715+0000 7fa587d2cb80 1 -- 172.21.15.57:0/3498759161 wait complete. 2023-10-08T14:00:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.715+0000 7fa587d2cb80 20 client.26642 trim_cache size 0 max 16384 2023-10-08T14:00:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:00:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:00:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:00:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:00:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 -- 172.21.15.57:0/3519727676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5600a15f0 msgr2=0x7fa5600a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5600a15f0 0x7fa5600a3ab0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009fd0 tx=0x7fa56c00fdd0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 -- 172.21.15.57:0/3519727676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5740fff90 msgr2=0x7fa574197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5740fff90 0x7fa574197eb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa56400e9d0 tx=0x7fa56400ece0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 -- 172.21.15.57:0/3519727676 shutdown_connections 2023-10-08T14:00:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5741045e0 0x7fa5741a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5600a15f0 0x7fa5600a3ab0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5740ff5d0 0x7fa574197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 --2- 172.21.15.57:0/3519727676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5740fff90 0x7fa574197eb0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 -- 172.21.15.57:0/3519727676 >> 172.21.15.57:0/3519727676 conn(0x7fa574076790 msgr2=0x7fa5740fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 -- 172.21.15.57:0/3519727676 shutdown_connections 2023-10-08T14:00:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:31.717+0000 7fa57bb43700 1 -- 172.21.15.57:0/3519727676 wait complete. 2023-10-08T14:00:31.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:31 smithi057 ceph-mon[99476]: pgmap v574: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2023-10-08T14:00:32.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:31 smithi099 ceph-mon[103849]: pgmap v574: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2023-10-08T14:00:32.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:31 smithi115 ceph-mon[104408]: pgmap v574: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2023-10-08T14:00:33.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:33 smithi057 ceph-mon[99476]: pgmap v575: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 303 op/s 2023-10-08T14:00:34.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:33 smithi099 ceph-mon[103849]: pgmap v575: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 303 op/s 2023-10-08T14:00:34.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:33 smithi115 ceph-mon[104408]: pgmap v575: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 303 op/s 2023-10-08T14:00:35.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:35 smithi057 ceph-mon[99476]: pgmap v576: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 333 op/s 2023-10-08T14:00:36.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:35 smithi099 ceph-mon[103849]: pgmap v576: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 333 op/s 2023-10-08T14:00:36.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:35 smithi115 ceph-mon[104408]: pgmap v576: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 333 op/s 2023-10-08T14:00:37.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:37 smithi057 ceph-mon[99476]: pgmap v577: 129 pgs: 129 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 331 op/s 2023-10-08T14:00:38.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:37 smithi099 ceph-mon[103849]: pgmap v577: 129 pgs: 129 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 331 op/s 2023-10-08T14:00:38.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:37 smithi115 ceph-mon[104408]: pgmap v577: 129 pgs: 129 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 331 op/s 2023-10-08T14:00:39.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:39 smithi057 ceph-mon[99476]: osdmap e99: 12 total, 12 up, 12 in 2023-10-08T14:00:39.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:39 smithi057 ceph-mon[99476]: pgmap v579: 129 pgs: 129 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 330 op/s 2023-10-08T14:00:39.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:39 smithi099 ceph-mon[103849]: osdmap e99: 12 total, 12 up, 12 in 2023-10-08T14:00:39.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:39 smithi099 ceph-mon[103849]: pgmap v579: 129 pgs: 129 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 330 op/s 2023-10-08T14:00:39.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:39 smithi115 ceph-mon[104408]: osdmap e99: 12 total, 12 up, 12 in 2023-10-08T14:00:39.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:39 smithi115 ceph-mon[104408]: pgmap v579: 129 pgs: 129 active+clean; 2.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 330 op/s 2023-10-08T14:00:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:41 smithi057 ceph-mon[99476]: pgmap v580: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 305 op/s 2023-10-08T14:00:42.021 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:41 smithi099 ceph-mon[103849]: pgmap v580: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 305 op/s 2023-10-08T14:00:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:41 smithi115 ceph-mon[104408]: pgmap v580: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 305 op/s 2023-10-08T14:00:43.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:43 smithi057 ceph-mon[99476]: pgmap v581: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 282 op/s 2023-10-08T14:00:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:43 smithi099 ceph-mon[103849]: pgmap v581: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 282 op/s 2023-10-08T14:00:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:43 smithi115 ceph-mon[104408]: pgmap v581: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 282 op/s 2023-10-08T14:00:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.875+0000 7f3c61b38700 1 -- 172.21.15.57:0/3711384979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c101810 msgr2=0x7f3c5c0fec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.875+0000 7f3c61b38700 1 --2- 172.21.15.57:0/3711384979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c101810 0x7f3c5c0fec10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400b0a0 tx=0x7f3c5400b3b0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.881+0000 7f3c61b38700 1 -- 172.21.15.57:0/3711384979 shutdown_connections 2023-10-08T14:00:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.881+0000 7f3c61b38700 1 --2- 172.21.15.57:0/3711384979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5c10d390 0x7f3c5c10f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.881+0000 7f3c61b38700 1 --2- 172.21.15.57:0/3711384979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c101810 0x7f3c5c0fec10 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.881+0000 7f3c61b38700 1 --2- 172.21.15.57:0/3711384979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c0ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.881+0000 7f3c61b38700 1 -- 172.21.15.57:0/3711384979 >> 172.21.15.57:0/3711384979 conn(0x7f3c5c075ba0 msgr2=0x7f3c5c0fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.885+0000 7f3c61b38700 1 -- 172.21.15.57:0/3711384979 shutdown_connections 2023-10-08T14:00:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.890+0000 7f3c61b38700 1 -- 172.21.15.57:0/3711384979 wait complete. 2023-10-08T14:00:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.891+0000 7f3c61b38700 1 Processor -- start 2023-10-08T14:00:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.893+0000 7f3c61b38700 1 -- start start 2023-10-08T14:00:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.894+0000 7f3c61b38700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.894+0000 7f3c61b38700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c101810 0x7f3c5c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.894+0000 7f3c61b38700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5c10d390 0x7f3c5c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.894+0000 7f3c61b38700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c19dff0 con 0x7f3c5c0ff150 2023-10-08T14:00:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.894+0000 7f3c61b38700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c19e160 con 0x7f3c5c101810 2023-10-08T14:00:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.894+0000 7f3c61b38700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c19e320 con 0x7f3c5c10d390 2023-10-08T14:00:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.894+0000 7f3c5b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.894+0000 7f3c5b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59520/0 (socket says 172.21.15.57:59520) 2023-10-08T14:00:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.894+0000 7f3c5b7fe700 1 -- 172.21.15.57:0/1246153969 learned_addr learned my addr 172.21.15.57:0/1246153969 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.897+0000 7f3c5b7fe700 1 -- 172.21.15.57:0/1246153969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5c10d390 msgr2=0x7f3c5c19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.897+0000 7f3c5b7fe700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5c10d390 0x7f3c5c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.897+0000 7f3c5b7fe700 1 -- 172.21.15.57:0/1246153969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c101810 msgr2=0x7f3c5c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.897+0000 7f3c5b7fe700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c101810 0x7f3c5c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.897+0000 7f3c5b7fe700 1 -- 172.21.15.57:0/1246153969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c54009d00 con 0x7f3c5c0ff150 2023-10-08T14:00:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.897+0000 7f3c5b7fe700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1990d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400b070 tx=0x7f3c54007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.897+0000 7f3c58ff9700 1 -- 172.21.15.57:0/1246153969 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5400e040 con 0x7f3c5c0ff150 2023-10-08T14:00:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.898+0000 7f3c58ff9700 1 -- 172.21.15.57:0/1246153969 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c5401d480 con 0x7f3c5c0ff150 2023-10-08T14:00:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.898+0000 7f3c58ff9700 1 -- 172.21.15.57:0/1246153969 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5401b670 con 0x7f3c5c0ff150 2023-10-08T14:00:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.898+0000 7f3c61b38700 1 -- 172.21.15.57:0/1246153969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5c19e5a0 con 0x7f3c5c0ff150 2023-10-08T14:00:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.898+0000 7f3c61b38700 1 -- 172.21.15.57:0/1246153969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5c1a5e40 con 0x7f3c5c0ff150 2023-10-08T14:00:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c6dd21b80 1 Processor -- start 2023-10-08T14:00:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c6dd21b80 1 -- start start 2023-10-08T14:00:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa03880 0x55f9efa03c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa041a0 0x55f9ef9fdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9ef9fe0b0 0x55f9ef9fe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9ef966400 con 0x55f9efa041a0 2023-10-08T14:00:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9ef957bc0 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9ef9554e0 con 0x55f9efa03880 2023-10-08T14:00:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c5bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa03880 0x55f9efa03c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c5affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9ef9fe0b0 0x55f9ef9fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c5bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa03880 0x55f9efa03c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60338/0 (socket says 172.21.15.57:60338) 2023-10-08T14:00:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.903+0000 7f3c5bfff700 1 -- 172.21.15.57:0/3532286185 learned_addr learned my addr 172.21.15.57:0/3532286185 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.904+0000 7f3c5affd700 1 -- 172.21.15.57:0/3532286185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa03880 msgr2=0x55f9efa03c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.904+0000 7f3c5affd700 1 --2- 172.21.15.57:0/3532286185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa03880 0x55f9efa03c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.904+0000 7f3c5affd700 1 -- 172.21.15.57:0/3532286185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa041a0 msgr2=0x55f9ef9fdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.904+0000 7f3c5affd700 1 --2- 172.21.15.57:0/3532286185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa041a0 0x55f9ef9fdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.904+0000 7f3c5affd700 1 -- 172.21.15.57:0/3532286185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9ef77bd80 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.904+0000 7f3c5affd700 1 --2- 172.21.15.57:0/3532286185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9ef9fe0b0 0x55f9ef9fe510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009780 tx=0x7f3c4c009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=734c43b8dab4fcfd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.904+0000 7f3c49ffb700 1 -- 172.21.15.57:0/3532286185 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c011830 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.904+0000 7f3c49ffb700 1 -- 172.21.15.57:0/3532286185 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c4c011e50 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.905+0000 7f3c49ffb700 1 -- 172.21.15.57:0/3532286185 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00b360 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.907+0000 7f3c58ff9700 1 -- 172.21.15.57:0/1246153969 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3c54012a60 con 0x7f3c5c0ff150 2023-10-08T14:00:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.908+0000 7f3c58ff9700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3c440a1740 0x7f3c440a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.908+0000 7f3c58ff9700 1 -- 172.21.15.57:0/1246153969 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f3c540d1380 con 0x7f3c5c0ff150 2023-10-08T14:00:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.908+0000 7f3c5bfff700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3c440a1740 0x7f3c440a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.909+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3532286185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9ef9fe0b0 msgr2=0x55f9ef9fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.909+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/3532286185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9ef9fe0b0 0x55f9ef9fe510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009780 tx=0x7f3c4c009a90 comp rx=0 tx=0).stop 2023-10-08T14:00:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.909+0000 7f3c5bfff700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3c440a1740 0x7f3c440a3c00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3c50006fd0 tx=0x7f3c50008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.909+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3532286185 shutdown_connections 2023-10-08T14:00:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.909+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/3532286185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9ef9fe0b0 0x55f9ef9fe510 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.909+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/3532286185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa03880 0x55f9efa03c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.909+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/3532286185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa041a0 0x55f9ef9fdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.909+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3532286185 >> 172.21.15.57:0/3532286185 conn(0x55f9ef972200 msgr2=0x55f9ef9f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.909+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3532286185 shutdown_connections 2023-10-08T14:00:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.909+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3532286185 wait complete. 2023-10-08T14:00:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.910+0000 7f3c6dd21b80 1 -- start start 2023-10-08T14:00:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.910+0000 7f3c6dd21b80 1 Processor -- start 2023-10-08T14:00:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.910+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 0x55f9efa0a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.910+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.910+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 0x55f9efa07b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.910+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9ef950dd0 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.910+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9ef8ed6b0 con 0x55f9efa03880 2023-10-08T14:00:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.910+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9ef8ed350 con 0x55f9efa041a0 2023-10-08T14:00:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c6dd21b80 10 client.0 _ll_register_callbacks cb 0x55f9ef74a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:00:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c5affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c5bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 0x55f9efa07b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c5affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa074c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:60964/0 (socket says 172.21.15.57:60964) 2023-10-08T14:00:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c5affd700 1 -- 172.21.15.57:0/3511508963 learned_addr learned my addr 172.21.15.57:0/3511508963 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c5b7fe700 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 0x55f9efa0a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c5affd700 1 -- 172.21.15.57:0/3511508963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 msgr2=0x55f9efa07b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c5affd700 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 0x55f9efa07b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c5affd700 1 -- 172.21.15.57:0/3511508963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 msgr2=0x55f9efa0a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c5affd700 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 0x55f9efa0a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.911+0000 7f3c5affd700 1 -- 172.21.15.57:0/3511508963 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c0093e0 con 0x55f9efa03880 2023-10-08T14:00:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.912+0000 7f3c5affd700 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa074c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009f10 tx=0x7f3c4c0105b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760bd8792f3480f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c3effd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:00:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c3effd700 1 -- 172.21.15.57:0/3511508963 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c010f10 con 0x55f9efa03880 2023-10-08T14:00:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c3effd700 1 -- 172.21.15.57:0/3511508963 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f3c4c011630 con 0x55f9efa03880 2023-10-08T14:00:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3511508963 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9ef918e50 con 0x55f9efa03880 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c3effd700 1 -- 172.21.15.57:0/3511508963 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c4c0117f0 con 0x55f9efa03880 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c6dd21b80 10 client.26648 fetch_fsmap learned FSMap version 46 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3511508963 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55f9ef764c00 con 0x55f9efa03880 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c3effd700 1 -- 172.21.15.57:0/3511508963 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9ef918e50 con 0x55f9efa03880 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c3effd700 1 -- 172.21.15.57:0/3511508963 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f3c4c04fcf0 con 0x55f9efa03880 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c6dd21b80 10 client.26648 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.914+0000 7f3c6dd21b80 10 client.26648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.915+0000 7f3c6dd21b80 20 client.26648 populate_metadata read hostname 'smithi057' 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.915+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9efa26bb0 0x55f9efa28fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.915+0000 7f3c6dd21b80 4 client.26648 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.915+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3511508963 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9ef949db0 con 0x55f9efa26bb0 2023-10-08T14:00:44.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.915+0000 7f3c5b7fe700 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9efa26bb0 0x55f9efa28fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.916+0000 7f3c3effd700 10 client.26648 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.916+0000 7f3c5b7fe700 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9efa26bb0 0x55f9efa28fd0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1c80dfea309359 server_cookie=b16887da45bed7d4 in_seq=0 out_seq=0 2023-10-08T14:00:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.917+0000 7f3c3effd700 1 -- 172.21.15.57:0/3511508963 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f9ef949db0 con 0x55f9efa26bb0 2023-10-08T14:00:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.917+0000 7f3c3effd700 10 client.26648 handle_command_reply: tid=0 2023-10-08T14:00:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.917+0000 7f3c6dd21b80 1 client.26648 shutdown 2023-10-08T14:00:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.917+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3511508963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 msgr2=0x55f9efa074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.917+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa074c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009f10 tx=0x7f3c4c0105b0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.923+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3511508963 shutdown_connections 2023-10-08T14:00:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.923+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9efa26bb0 0x55f9efa28fd0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.923+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 0x55f9efa07b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.923+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 0x55f9efa0a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.923+0000 7f3c3effd700 0 client.26648 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.923+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/3511508963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa074c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.923+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3511508963 >> 172.21.15.57:0/3511508963 conn(0x55f9ef972200 msgr2=0x55f9ef9f8010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.924+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3511508963 shutdown_connections 2023-10-08T14:00:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.924+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/3511508963 wait complete. 2023-10-08T14:00:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.928+0000 7f3c6dd21b80 20 client.26648 trim_cache size 0 max 16384 2023-10-08T14:00:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.951+0000 7f3c6dd21b80 1 Processor -- start 2023-10-08T14:00:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.951+0000 7f3c6dd21b80 1 -- start start 2023-10-08T14:00:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9ef9fe0b0 0x55f9ef9fdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa03880 0x55f9efa1fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa041a0 0x55f9ef9f8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9ef7d18c0 con 0x55f9efa03880 2023-10-08T14:00:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9efa072d0 con 0x55f9efa041a0 2023-10-08T14:00:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9efa08140 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c5affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa03880 0x55f9efa1fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c5affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa03880 0x55f9efa1fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59546/0 (socket says 172.21.15.57:59546) 2023-10-08T14:00:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c5affd700 1 -- 172.21.15.57:0/2767412489 learned_addr learned my addr 172.21.15.57:0/2767412489 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c5affd700 1 -- 172.21.15.57:0/2767412489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9ef9fe0b0 msgr2=0x55f9ef9fdb70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:00:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c5affd700 1 --2- 172.21.15.57:0/2767412489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9ef9fe0b0 0x55f9ef9fdb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c5affd700 1 -- 172.21.15.57:0/2767412489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa041a0 msgr2=0x55f9ef9f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c5affd700 1 --2- 172.21.15.57:0/2767412489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa041a0 0x55f9ef9f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c5affd700 1 -- 172.21.15.57:0/2767412489 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9ef7a21f0 con 0x55f9efa03880 2023-10-08T14:00:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.952+0000 7f3c5affd700 1 --2- 172.21.15.57:0/2767412489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa03880 0x55f9efa1fc60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009fd0 tx=0x7f3c4c010750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ba28d76abb3bb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.955+0000 7f3c4a7fc700 1 -- 172.21.15.57:0/2767412489 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c006e10 con 0x55f9efa03880 2023-10-08T14:00:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.955+0000 7f3c4a7fc700 1 -- 172.21.15.57:0/2767412489 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c4c010d10 con 0x55f9efa03880 2023-10-08T14:00:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.955+0000 7f3c4a7fc700 1 -- 172.21.15.57:0/2767412489 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c04f470 con 0x55f9efa03880 2023-10-08T14:00:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.955+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/2767412489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa03880 msgr2=0x55f9efa1fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.955+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/2767412489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa03880 0x55f9efa1fc60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009fd0 tx=0x7f3c4c010750 comp rx=0 tx=0).stop 2023-10-08T14:00:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.956+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/2767412489 shutdown_connections 2023-10-08T14:00:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.956+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/2767412489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa041a0 0x55f9ef9f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.956+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/2767412489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9ef9fe0b0 0x55f9ef9fdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.956+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/2767412489 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9efa03880 0x55f9efa1fc60 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.956+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/2767412489 >> 172.21.15.57:0/2767412489 conn(0x55f9ef972200 msgr2=0x55f9efa016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.956+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/2767412489 shutdown_connections 2023-10-08T14:00:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.956+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/2767412489 wait complete. 2023-10-08T14:00:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.956+0000 7f3c6dd21b80 1 -- start start 2023-10-08T14:00:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.957+0000 7f3c6dd21b80 1 Processor -- start 2023-10-08T14:00:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.957+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 0x55f9efa75d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.957+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.957+0000 7f3c6dd21b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 0x55f9efa1dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.957+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9efa1e350 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.957+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9efa1e4c0 con 0x55f9efa03880 2023-10-08T14:00:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.957+0000 7f3c6dd21b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9efa1e680 con 0x55f9efa041a0 2023-10-08T14:00:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.957+0000 7f3c6dd21b80 10 client.0 _ll_register_callbacks cb 0x7f3c4c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:00:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.957+0000 7f3c5b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 0x55f9efa75d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 0x55f9efa75d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59550/0 (socket says 172.21.15.57:59550) 2023-10-08T14:00:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5b7fe700 1 -- 172.21.15.57:0/1758637371 learned_addr learned my addr 172.21.15.57:0/1758637371 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5affd700 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5bfff700 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 0x55f9efa1dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5b7fe700 1 -- 172.21.15.57:0/1758637371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 msgr2=0x55f9efa1dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5b7fe700 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 0x55f9efa1dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5b7fe700 1 -- 172.21.15.57:0/1758637371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 msgr2=0x55f9efa762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5b7fe700 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5b7fe700 1 -- 172.21.15.57:0/1758637371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9ef764c00 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5bfff700 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 0x55f9efa1dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:00:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.958+0000 7f3c5affd700 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:00:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.959+0000 7f3c5b7fe700 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 0x55f9efa75d80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3c5409dd10 tx=0x7f3c5409da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7762fe1c2aa7cc72 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.959+0000 7f3c3effd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:00:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.959+0000 7f3c3effd700 1 -- 172.21.15.57:0/1758637371 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c540a2ea0 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.959+0000 7f3c3effd700 1 -- 172.21.15.57:0/1758637371 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f3c540cee40 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.959+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/1758637371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c4c011e50 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.959+0000 7f3c3effd700 1 -- 172.21.15.57:0/1758637371 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c5409a720 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.959+0000 7f3c6dd21b80 10 client.17103 fetch_fsmap learned FSMap version 46 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.959+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/1758637371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55f9ef77a250 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.960+0000 7f3c3effd700 1 -- 172.21.15.57:0/1758637371 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c4c011e50 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.960+0000 7f3c3effd700 1 -- 172.21.15.57:0/1758637371 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f3c5409e270 con 0x55f9ef9fe0b0 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.960+0000 7f3c6dd21b80 10 client.17103 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.960+0000 7f3c6dd21b80 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.960+0000 7f3c6dd21b80 20 client.17103 populate_metadata read hostname 'smithi057' 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.960+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9efa7d9a0 0x55f9efa7fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.960+0000 7f3c6dd21b80 4 client.17103 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:00:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.960+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/1758637371 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f9ef949ad0 con 0x55f9efa7d9a0 2023-10-08T14:00:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.960+0000 7f3c5affd700 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9efa7d9a0 0x55f9efa7fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.974+0000 7f3c3effd700 10 client.17103 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:44.974+0000 7f3c5affd700 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9efa7d9a0 0x55f9efa7fdc0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0c9fe55d7b01fd server_cookie=32ff22844c6a7f69 in_seq=0 out_seq=0 2023-10-08T14:00:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.016+0000 7f3c3effd700 1 -- 172.21.15.57:0/1758637371 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f9ef949ad0 con 0x55f9efa7d9a0 2023-10-08T14:00:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.016+0000 7f3c3effd700 10 client.17103 handle_command_reply: tid=0 2023-10-08T14:00:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.016+0000 7f3c6dd21b80 1 client.17103 shutdown 2023-10-08T14:00:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.018+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/1758637371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 msgr2=0x55f9efa75d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.018+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 0x55f9efa75d80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3c5409dd10 tx=0x7f3c5409da90 comp rx=0 tx=0).stop 2023-10-08T14:00:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.018+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/1758637371 shutdown_connections 2023-10-08T14:00:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.018+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55f9efa7d9a0 0x55f9efa7fdc0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.018+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9efa041a0 0x55f9efa1dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.018+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55f9ef9fe0b0 0x55f9efa75d80 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.018+0000 7f3c6dd21b80 1 --2- 172.21.15.57:0/1758637371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9efa03880 0x55f9efa762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.018+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/1758637371 >> 172.21.15.57:0/1758637371 conn(0x55f9ef972200 msgr2=0x55f9efa05d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.019+0000 7f3c3effd700 0 client.17103 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.019+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/1758637371 shutdown_connections 2023-10-08T14:00:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.021+0000 7f3c6dd21b80 1 -- 172.21.15.57:0/1758637371 wait complete. 2023-10-08T14:00:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c6dd21b80 20 client.17103 trim_cache size 0 max 16384 2023-10-08T14:00:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:00:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:00:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:00:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:00:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c61b38700 1 -- 172.21.15.57:0/1246153969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3c440a1740 msgr2=0x7f3c440a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c61b38700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3c440a1740 0x7f3c440a3c00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3c50006fd0 tx=0x7f3c50008040 comp rx=0 tx=0).stop 2023-10-08T14:00:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c61b38700 1 -- 172.21.15.57:0/1246153969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 msgr2=0x7f3c5c1990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c61b38700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1990d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400b070 tx=0x7f3c54007be0 comp rx=0 tx=0).stop 2023-10-08T14:00:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c61b38700 1 -- 172.21.15.57:0/1246153969 shutdown_connections 2023-10-08T14:00:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c61b38700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5c10d390 0x7f3c5c19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c61b38700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c5c0ff150 0x7f3c5c1990d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c61b38700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3c440a1740 0x7f3c440a3c00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c61b38700 1 --2- 172.21.15.57:0/1246153969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c101810 0x7f3c5c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.023+0000 7f3c61b38700 1 -- 172.21.15.57:0/1246153969 >> 172.21.15.57:0/1246153969 conn(0x7f3c5c075ba0 msgr2=0x7f3c5c10e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.024+0000 7f3c61b38700 1 -- 172.21.15.57:0/1246153969 shutdown_connections 2023-10-08T14:00:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:45.025+0000 7f3c61b38700 1 -- 172.21.15.57:0/1246153969 wait complete. 2023-10-08T14:00:45.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:45 smithi057 ceph-mon[99476]: pgmap v582: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2023-10-08T14:00:46.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:45 smithi099 ceph-mon[103849]: pgmap v582: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2023-10-08T14:00:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:45 smithi115 ceph-mon[104408]: pgmap v582: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2023-10-08T14:00:47.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:47 smithi057 ceph-mon[99476]: pgmap v583: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 269 op/s 2023-10-08T14:00:48.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:47 smithi099 ceph-mon[103849]: pgmap v583: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 269 op/s 2023-10-08T14:00:48.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:47 smithi115 ceph-mon[104408]: pgmap v583: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 269 op/s 2023-10-08T14:00:49.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:49 smithi057 ceph-mon[99476]: pgmap v584: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2023-10-08T14:00:49.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:49 smithi099 ceph-mon[103849]: pgmap v584: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2023-10-08T14:00:49.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:49 smithi115 ceph-mon[104408]: pgmap v584: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2023-10-08T14:00:51.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:51 smithi057 ceph-mon[99476]: pgmap v585: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2023-10-08T14:00:52.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:51 smithi099 ceph-mon[103849]: pgmap v585: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2023-10-08T14:00:52.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:51 smithi115 ceph-mon[104408]: pgmap v585: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2023-10-08T14:00:53.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:53 smithi057 ceph-mon[99476]: pgmap v586: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-10-08T14:00:54.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:53 smithi099 ceph-mon[103849]: pgmap v586: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-10-08T14:00:54.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:53 smithi115 ceph-mon[104408]: pgmap v586: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2023-10-08T14:00:55.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:55 smithi057 ceph-mon[99476]: pgmap v587: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2023-10-08T14:00:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:55 smithi099 ceph-mon[103849]: pgmap v587: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2023-10-08T14:00:56.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:55 smithi115 ceph-mon[104408]: pgmap v587: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 276 op/s 2023-10-08T14:00:57.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:57 smithi057 ceph-mon[99476]: pgmap v588: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2023-10-08T14:00:58.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:57 smithi099 ceph-mon[103849]: pgmap v588: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2023-10-08T14:00:58.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:57 smithi115 ceph-mon[104408]: pgmap v588: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2023-10-08T14:00:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.199+0000 7f3ac35a4700 1 -- 172.21.15.57:0/1961934113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3abc108380 msgr2=0x7f3abc10b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.199+0000 7f3ac35a4700 1 --2- 172.21.15.57:0/1961934113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3abc108380 0x7f3abc10b940 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009990 tx=0x7f3ab8009ca0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.200+0000 7f3ac35a4700 1 -- 172.21.15.57:0/1961934113 shutdown_connections 2023-10-08T14:00:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.200+0000 7f3ac35a4700 1 --2- 172.21.15.57:0/1961934113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3abc10c070 0x7f3abc078b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.200+0000 7f3ac35a4700 1 --2- 172.21.15.57:0/1961934113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3abc107a60 0x7f3abc107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.200+0000 7f3ac35a4700 1 --2- 172.21.15.57:0/1961934113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3abc108380 0x7f3abc10b940 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.200+0000 7f3ac35a4700 1 -- 172.21.15.57:0/1961934113 >> 172.21.15.57:0/1961934113 conn(0x7f3abc06b7c0 msgr2=0x7f3abc06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.200+0000 7f3ac35a4700 1 -- 172.21.15.57:0/1961934113 shutdown_connections 2023-10-08T14:00:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.200+0000 7f3ac35a4700 1 -- 172.21.15.57:0/1961934113 wait complete. 2023-10-08T14:00:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.200+0000 7f3ac35a4700 1 Processor -- start 2023-10-08T14:00:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac35a4700 1 -- start start 2023-10-08T14:00:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac35a4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3abc107a60 0x7f3abc078950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac35a4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3abc108380 0x7f3abc074a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac35a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3abc10c070 0x7f3abc074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac35a4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc075720 con 0x7f3abc108380 2023-10-08T14:00:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac35a4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc075890 con 0x7f3abc10c070 2023-10-08T14:00:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac35a4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc07afb0 con 0x7f3abc107a60 2023-10-08T14:00:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac0b3f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3abc108380 0x7f3abc074a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac1b41700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3abc10c070 0x7f3abc074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac0b3f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3abc108380 0x7f3abc074a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59576/0 (socket says 172.21.15.57:59576) 2023-10-08T14:00:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.201+0000 7f3ac0b3f700 1 -- 172.21.15.57:0/2514920709 learned_addr learned my addr 172.21.15.57:0/2514920709 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ac1b41700 1 -- 172.21.15.57:0/2514920709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3abc107a60 msgr2=0x7f3abc078950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ac1b41700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3abc107a60 0x7f3abc078950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ac1b41700 1 -- 172.21.15.57:0/2514920709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3abc108380 msgr2=0x7f3abc074a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ac1b41700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3abc108380 0x7f3abc074a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ac1b41700 1 -- 172.21.15.57:0/2514920709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80095f0 con 0x7f3abc10c070 2023-10-08T14:00:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ac1b41700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3abc10c070 0x7f3abc074f50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400bb00 tx=0x7f3ab400be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ab27fc700 1 -- 172.21.15.57:0/2514920709 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab400ca30 con 0x7f3abc10c070 2023-10-08T14:00:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ab27fc700 1 -- 172.21.15.57:0/2514920709 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab400cb90 con 0x7f3abc10c070 2023-10-08T14:00:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ac35a4700 1 -- 172.21.15.57:0/2514920709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc07b290 con 0x7f3abc10c070 2023-10-08T14:00:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ab27fc700 1 -- 172.21.15.57:0/2514920709 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40125a0 con 0x7f3abc10c070 2023-10-08T14:00:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.202+0000 7f3ac35a4700 1 -- 172.21.15.57:0/2514920709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc07b7a0 con 0x7f3abc10c070 2023-10-08T14:00:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.205+0000 7f3ab27fc700 1 -- 172.21.15.57:0/2514920709 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3ab400cd00 con 0x7f3abc10c070 2023-10-08T14:00:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.206+0000 7f3ab27fc700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa80a15d0 0x7f3aa80a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.206+0000 7f3ab27fc700 1 -- 172.21.15.57:0/2514920709 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f3ab40ccbe0 con 0x7f3abc10c070 2023-10-08T14:00:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.206+0000 7f3ac1340700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa80a15d0 0x7f3aa80a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.207+0000 7f3ac1340700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa80a15d0 0x7f3aa80a3a90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005950 tx=0x7f3aac0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.207+0000 7f3acf78db80 1 Processor -- start 2023-10-08T14:00:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.207+0000 7f3acf78db80 1 -- start start 2023-10-08T14:00:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.207+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a0880 0x5570ae5a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.207+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a11a0 0x5570ae59ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.208+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 0x5570ae59b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.208+0000 7f3acf78db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae503400 con 0x5570ae5a0880 2023-10-08T14:00:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.208+0000 7f3acf78db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae4f4bc0 con 0x5570ae59b0b0 2023-10-08T14:00:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.208+0000 7f3acf78db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae4f24e0 con 0x5570ae5a11a0 2023-10-08T14:00:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.208+0000 7f3ac1340700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a0880 0x5570ae5a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.208+0000 7f3ac1340700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a0880 0x5570ae5a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59580/0 (socket says 172.21.15.57:59580) 2023-10-08T14:00:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.208+0000 7f3ac1340700 1 -- 172.21.15.57:0/1297398374 learned_addr learned my addr 172.21.15.57:0/1297398374 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.212+0000 7f3ac1340700 1 -- 172.21.15.57:0/1297398374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a11a0 msgr2=0x5570ae59ab70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:00:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.212+0000 7f3ac1340700 1 --2- 172.21.15.57:0/1297398374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a11a0 0x5570ae59ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.212+0000 7f3ac1340700 1 -- 172.21.15.57:0/1297398374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 msgr2=0x5570ae59b510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:00:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.212+0000 7f3ac1340700 1 --2- 172.21.15.57:0/1297398374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 0x5570ae59b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.212+0000 7f3ac1340700 1 -- 172.21.15.57:0/1297398374 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570ae318d80 con 0x5570ae5a0880 2023-10-08T14:00:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.212+0000 7f3ac1340700 1 --2- 172.21.15.57:0/1297398374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a0880 0x5570ae5a0c60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009270 tx=0x7f3aac010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1938a647477702b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.213+0000 7f3aa77fe700 1 -- 172.21.15.57:0/1297398374 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac010ac0 con 0x5570ae5a0880 2023-10-08T14:00:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.213+0000 7f3aa77fe700 1 -- 172.21.15.57:0/1297398374 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aac010c20 con 0x5570ae5a0880 2023-10-08T14:00:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.213+0000 7f3aa77fe700 1 -- 172.21.15.57:0/1297398374 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00f620 con 0x5570ae5a0880 2023-10-08T14:00:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.214+0000 7f3acf78db80 1 -- 172.21.15.57:0/1297398374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a0880 msgr2=0x5570ae5a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.214+0000 7f3acf78db80 1 --2- 172.21.15.57:0/1297398374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a0880 0x5570ae5a0c60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009270 tx=0x7f3aac010000 comp rx=0 tx=0).stop 2023-10-08T14:00:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.218+0000 7f3acf78db80 1 -- 172.21.15.57:0/1297398374 shutdown_connections 2023-10-08T14:00:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.218+0000 7f3acf78db80 1 --2- 172.21.15.57:0/1297398374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 0x5570ae59b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.218+0000 7f3acf78db80 1 --2- 172.21.15.57:0/1297398374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a0880 0x5570ae5a0c60 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.218+0000 7f3acf78db80 1 --2- 172.21.15.57:0/1297398374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a11a0 0x5570ae59ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.218+0000 7f3acf78db80 1 -- 172.21.15.57:0/1297398374 >> 172.21.15.57:0/1297398374 conn(0x5570ae50f200 msgr2=0x5570ae595b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.221+0000 7f3acf78db80 1 -- 172.21.15.57:0/1297398374 shutdown_connections 2023-10-08T14:00:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.221+0000 7f3acf78db80 1 -- 172.21.15.57:0/1297398374 wait complete. 2023-10-08T14:00:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.222+0000 7f3acf78db80 1 -- start start 2023-10-08T14:00:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.222+0000 7f3acf78db80 1 Processor -- start 2023-10-08T14:00:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae5a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a0880 0x5570ae5a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a11a0 0x5570ae5a5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3acf78db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae4eddd0 con 0x5570ae59b0b0 2023-10-08T14:00:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3acf78db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae48a6b0 con 0x5570ae5a11a0 2023-10-08T14:00:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3acf78db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae48a350 con 0x5570ae5a0880 2023-10-08T14:00:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3acf78db80 10 client.0 _ll_register_callbacks cb 0x5570ae2e77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:00:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3ac1b41700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a11a0 0x5570ae5a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3ac1b41700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a11a0 0x5570ae5a5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:32778/0 (socket says 172.21.15.57:32778) 2023-10-08T14:00:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3ac1b41700 1 -- 172.21.15.57:0/1863909053 learned_addr learned my addr 172.21.15.57:0/1863909053 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3ac0b3f700 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a0880 0x5570ae5a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.223+0000 7f3ac1340700 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae5a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.224+0000 7f3ac1b41700 1 -- 172.21.15.57:0/1863909053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a0880 msgr2=0x5570ae5a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.224+0000 7f3ac1b41700 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a0880 0x5570ae5a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.224+0000 7f3ac1b41700 1 -- 172.21.15.57:0/1863909053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 msgr2=0x5570ae5a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.224+0000 7f3ac1b41700 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae5a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.224+0000 7f3ac1b41700 1 -- 172.21.15.57:0/1863909053 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac00d960 con 0x5570ae5a11a0 2023-10-08T14:00:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.224+0000 7f3ac1340700 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae5a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T14:00:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.224+0000 7f3ac1b41700 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a11a0 0x5570ae5a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40ca5d0 tx=0x7f3ab4099350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e0199337e0f4d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.224+0000 7f3aa4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:00:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.224+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/1863909053 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab409dac0 con 0x5570ae5a11a0 2023-10-08T14:00:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.225+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/1863909053 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f3ab40a1d00 con 0x5570ae5a11a0 2023-10-08T14:00:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.225+0000 7f3acf78db80 1 -- 172.21.15.57:0/1863909053 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570ae4b5e50 con 0x5570ae5a11a0 2023-10-08T14:00:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.233+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/1863909053 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ab4099bd0 con 0x5570ae5a11a0 2023-10-08T14:00:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.233+0000 7f3acf78db80 10 client.26657 fetch_fsmap learned FSMap version 46 2023-10-08T14:00:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.233+0000 7f3acf78db80 1 -- 172.21.15.57:0/1863909053 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5570ae301c00 con 0x5570ae5a11a0 2023-10-08T14:00:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.237+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/1863909053 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5570ae4b5e50 con 0x5570ae5a11a0 2023-10-08T14:00:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.237+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/1863909053 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f3ab40a1870 con 0x5570ae5a11a0 2023-10-08T14:00:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.237+0000 7f3acf78db80 10 client.26657 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:00:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.237+0000 7f3acf78db80 10 client.26657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:00:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.237+0000 7f3acf78db80 20 client.26657 populate_metadata read hostname 'smithi057' 2023-10-08T14:00:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.237+0000 7f3acf78db80 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5570ae5b4d50 0x5570ae5b7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.238+0000 7f3acf78db80 4 client.26657 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:00:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.238+0000 7f3acf78db80 1 -- 172.21.15.57:0/1863909053 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570ae4e6db0 con 0x5570ae5b4d50 2023-10-08T14:00:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.240+0000 7f3ac1340700 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5570ae5b4d50 0x5570ae5b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.241+0000 7f3aa4ff9700 10 client.26657 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.241+0000 7f3ac1340700 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5570ae5b4d50 0x5570ae5b7170 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614400c7ea0ffc2e server_cookie=96cfcadfece5f9e in_seq=0 out_seq=0 2023-10-08T14:00:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.241+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/1863909053 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5570ae4e6db0 con 0x5570ae5b4d50 2023-10-08T14:00:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.241+0000 7f3aa4ff9700 10 client.26657 handle_command_reply: tid=0 2023-10-08T14:00:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.241+0000 7f3acf78db80 1 client.26657 shutdown 2023-10-08T14:00:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.253+0000 7f3acf78db80 1 -- 172.21.15.57:0/1863909053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a11a0 msgr2=0x5570ae5a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.253+0000 7f3acf78db80 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a11a0 0x5570ae5a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40ca5d0 tx=0x7f3ab4099350 comp rx=0 tx=0).stop 2023-10-08T14:00:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.253+0000 7f3acf78db80 1 -- 172.21.15.57:0/1863909053 shutdown_connections 2023-10-08T14:00:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.253+0000 7f3acf78db80 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5570ae5b4d50 0x5570ae5b7170 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.253+0000 7f3acf78db80 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a11a0 0x5570ae5a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.253+0000 7f3acf78db80 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae5a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.253+0000 7f3acf78db80 1 --2- 172.21.15.57:0/1863909053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a0880 0x5570ae5a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.253+0000 7f3acf78db80 1 -- 172.21.15.57:0/1863909053 >> 172.21.15.57:0/1863909053 conn(0x5570ae50f200 msgr2=0x5570ae595010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.255+0000 7f3aa4ff9700 0 client.26657 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.258+0000 7f3acf78db80 1 -- 172.21.15.57:0/1863909053 shutdown_connections 2023-10-08T14:00:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.258+0000 7f3acf78db80 1 -- 172.21.15.57:0/1863909053 wait complete. 2023-10-08T14:00:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.258+0000 7f3acf78db80 20 client.26657 trim_cache size 0 max 16384 2023-10-08T14:00:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.283+0000 7f3acf78db80 1 Processor -- start 2023-10-08T14:00:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.283+0000 7f3acf78db80 1 -- start start 2023-10-08T14:00:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.283+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae595790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.283+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a0880 0x5570ae595cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.283+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a11a0 0x5570ae5a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.283+0000 7f3acf78db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae36e8c0 con 0x5570ae59b0b0 2023-10-08T14:00:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.283+0000 7f3acf78db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae592830 con 0x5570ae5a0880 2023-10-08T14:00:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.283+0000 7f3acf78db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae596610 con 0x5570ae5a11a0 2023-10-08T14:00:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac0b3f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a0880 0x5570ae595cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.283+0000 7f3ac1340700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae595790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac1340700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae595790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59592/0 (socket says 172.21.15.57:59592) 2023-10-08T14:00:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac0b3f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a0880 0x5570ae595cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:32788/0 (socket says 172.21.15.57:32788) 2023-10-08T14:00:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac1340700 1 -- 172.21.15.57:0/2148927468 learned_addr learned my addr 172.21.15.57:0/2148927468 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac1b41700 1 --2- 172.21.15.57:0/2148927468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a11a0 0x5570ae5a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac1340700 1 -- 172.21.15.57:0/2148927468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a11a0 msgr2=0x5570ae5a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac1340700 1 --2- 172.21.15.57:0/2148927468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a11a0 0x5570ae5a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac1340700 1 -- 172.21.15.57:0/2148927468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a0880 msgr2=0x5570ae595cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac1340700 1 --2- 172.21.15.57:0/2148927468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a0880 0x5570ae595cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac1340700 1 -- 172.21.15.57:0/2148927468 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570ae5bbd60 con 0x5570ae59b0b0 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac0b3f700 1 --2- 172.21.15.57:0/2148927468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a0880 0x5570ae595cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.284+0000 7f3ac1340700 1 --2- 172.21.15.57:0/2148927468 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae595790 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3aac021000 tx=0x7f3aac005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d4b76b3d5961ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.285+0000 7f3aa7fff700 1 -- 172.21.15.57:0/2148927468 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac02b040 con 0x5570ae59b0b0 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.285+0000 7f3aa7fff700 1 -- 172.21.15.57:0/2148927468 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aac014040 con 0x5570ae59b0b0 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.285+0000 7f3aa7fff700 1 -- 172.21.15.57:0/2148927468 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac032910 con 0x5570ae59b0b0 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.285+0000 7f3acf78db80 1 -- 172.21.15.57:0/2148927468 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 msgr2=0x5570ae595790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.285+0000 7f3acf78db80 1 --2- 172.21.15.57:0/2148927468 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae595790 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3aac021000 tx=0x7f3aac005d20 comp rx=0 tx=0).stop 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.285+0000 7f3acf78db80 1 -- 172.21.15.57:0/2148927468 shutdown_connections 2023-10-08T14:00:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.285+0000 7f3acf78db80 1 --2- 172.21.15.57:0/2148927468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a11a0 0x5570ae5a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.285+0000 7f3acf78db80 1 --2- 172.21.15.57:0/2148927468 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae59b0b0 0x5570ae595790 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.285+0000 7f3acf78db80 1 --2- 172.21.15.57:0/2148927468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae5a0880 0x5570ae595cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.285+0000 7f3acf78db80 1 -- 172.21.15.57:0/2148927468 >> 172.21.15.57:0/2148927468 conn(0x5570ae50f200 msgr2=0x5570ae59e740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.287+0000 7f3acf78db80 1 -- 172.21.15.57:0/2148927468 shutdown_connections 2023-10-08T14:00:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.288+0000 7f3acf78db80 1 -- 172.21.15.57:0/2148927468 wait complete. 2023-10-08T14:00:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.288+0000 7f3acf78db80 1 -- start start 2023-10-08T14:00:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.288+0000 7f3acf78db80 1 Processor -- start 2023-10-08T14:00:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.291+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 0x5570ae5a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.291+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a0880 0x5570ae5aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.291+0000 7f3acf78db80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a11a0 0x5570ae5aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.291+0000 7f3acf78db80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae607c70 con 0x5570ae5a11a0 2023-10-08T14:00:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.291+0000 7f3acf78db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae607de0 con 0x5570ae59b0b0 2023-10-08T14:00:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3acf78db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ae607fa0 con 0x5570ae5a0880 2023-10-08T14:00:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3acf78db80 10 client.0 _ll_register_callbacks cb 0x7f3ab4099740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:00:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac1340700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 0x5570ae5a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac1340700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 0x5570ae5a9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:32792/0 (socket says 172.21.15.57:32792) 2023-10-08T14:00:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac1340700 1 -- 172.21.15.57:0/2836152261 learned_addr learned my addr 172.21.15.57:0/2836152261 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:00:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac1b41700 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a11a0 0x5570ae5aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac0b3f700 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a0880 0x5570ae5aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac1340700 1 -- 172.21.15.57:0/2836152261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a0880 msgr2=0x5570ae5aa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac1340700 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a0880 0x5570ae5aa0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac1340700 1 -- 172.21.15.57:0/2836152261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a11a0 msgr2=0x5570ae5aa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:00:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac1340700 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a11a0 0x5570ae5aa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac1340700 1 -- 172.21.15.57:0/2836152261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc0793c0 con 0x5570ae59b0b0 2023-10-08T14:00:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3ac1340700 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 0x5570ae5a9b70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00d5c0 tx=0x7f3aac005fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c14f06b6d14aab07 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:00:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.292+0000 7f3aa4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:00:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/2836152261 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac02b050 con 0x5570ae59b0b0 2023-10-08T14:00:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/2836152261 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f3aac0108e0 con 0x5570ae59b0b0 2023-10-08T14:00:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3acf78db80 1 -- 172.21.15.57:0/2836152261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3aac010c20 con 0x5570ae59b0b0 2023-10-08T14:00:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/2836152261 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aac01dc30 con 0x5570ae59b0b0 2023-10-08T14:00:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3acf78db80 10 client.26663 fetch_fsmap learned FSMap version 46 2023-10-08T14:00:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3acf78db80 1 -- 172.21.15.57:0/2836152261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5570ae318d80 con 0x5570ae59b0b0 2023-10-08T14:00:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/2836152261 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aac010c20 con 0x5570ae59b0b0 2023-10-08T14:00:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/2836152261 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f3aac03a030 con 0x5570ae59b0b0 2023-10-08T14:00:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3acf78db80 10 client.26663 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:00:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3acf78db80 10 client.26663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:00:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3acf78db80 20 client.26663 populate_metadata read hostname 'smithi057' 2023-10-08T14:00:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3acf78db80 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5570ae6093f0 0x5570ae60b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:00:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3acf78db80 4 client.26663 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:00:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.293+0000 7f3acf78db80 1 -- 172.21.15.57:0/2836152261 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5570ae4e6ad0 con 0x5570ae6093f0 2023-10-08T14:00:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.294+0000 7f3ac0b3f700 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5570ae6093f0 0x5570ae60b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:00:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.294+0000 7f3aa4ff9700 10 client.26663 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.294+0000 7f3ac0b3f700 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5570ae6093f0 0x5570ae60b810 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01b040dfae5bc00 server_cookie=ae651d90a75e3d58 in_seq=0 out_seq=0 2023-10-08T14:00:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.316+0000 7f3aa4ff9700 1 -- 172.21.15.57:0/2836152261 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5570ae4e6ad0 con 0x5570ae6093f0 2023-10-08T14:00:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.316+0000 7f3aa4ff9700 10 client.26663 handle_command_reply: tid=0 2023-10-08T14:00:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.316+0000 7f3acf78db80 1 client.26663 shutdown 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.316+0000 7f3acf78db80 1 -- 172.21.15.57:0/2836152261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 msgr2=0x5570ae5a9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.316+0000 7f3acf78db80 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 0x5570ae5a9b70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00d5c0 tx=0x7f3aac005fc0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3acf78db80 1 -- 172.21.15.57:0/2836152261 shutdown_connections 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3acf78db80 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5570ae6093f0 0x5570ae60b810 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3acf78db80 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5570ae5a11a0 0x5570ae5aa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3aa4ff9700 0 client.26663 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3acf78db80 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ae59b0b0 0x5570ae5a9b70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3acf78db80 1 --2- 172.21.15.57:0/2836152261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5570ae5a0880 0x5570ae5aa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3acf78db80 1 -- 172.21.15.57:0/2836152261 >> 172.21.15.57:0/2836152261 conn(0x5570ae50f200 msgr2=0x5570ae5a2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3acf78db80 1 -- 172.21.15.57:0/2836152261 shutdown_connections 2023-10-08T14:00:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3acf78db80 1 -- 172.21.15.57:0/2836152261 wait complete. 2023-10-08T14:00:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3acf78db80 20 client.26663 trim_cache size 0 max 16384 2023-10-08T14:00:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:00:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:00:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:00:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:00:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3ac35a4700 1 -- 172.21.15.57:0/2514920709 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa80a15d0 msgr2=0x7f3aa80a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3ac35a4700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa80a15d0 0x7f3aa80a3a90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005950 tx=0x7f3aac0058e0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3ac35a4700 1 -- 172.21.15.57:0/2514920709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3abc10c070 msgr2=0x7f3abc074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:00:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3ac35a4700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3abc10c070 0x7f3abc074f50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400bb00 tx=0x7f3ab400be10 comp rx=0 tx=0).stop 2023-10-08T14:00:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3ac35a4700 1 -- 172.21.15.57:0/2514920709 shutdown_connections 2023-10-08T14:00:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3ac35a4700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f3aa80a15d0 0x7f3aa80a3a90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3ac35a4700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3abc10c070 0x7f3abc074f50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3ac35a4700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3abc107a60 0x7f3abc078950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3ac35a4700 1 --2- 172.21.15.57:0/2514920709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3abc108380 0x7f3abc074a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:00:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.317+0000 7f3ac35a4700 1 -- 172.21.15.57:0/2514920709 >> 172.21.15.57:0/2514920709 conn(0x7f3abc06b7c0 msgr2=0x7f3abc070550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:00:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.318+0000 7f3ac35a4700 1 -- 172.21.15.57:0/2514920709 shutdown_connections 2023-10-08T14:00:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:00:58.318+0000 7f3ac35a4700 1 -- 172.21.15.57:0/2514920709 wait complete. 2023-10-08T14:00:59.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:00:59 smithi099 ceph-mon[103849]: pgmap v589: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2023-10-08T14:00:59.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:00:59 smithi115 ceph-mon[104408]: pgmap v589: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2023-10-08T14:00:59.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:00:59 smithi057 ceph-mon[99476]: pgmap v589: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2023-10-08T14:01:01.977 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:01 smithi057 ceph-mon[99476]: pgmap v590: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 267 op/s 2023-10-08T14:01:01.984 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:01 smithi099 ceph-mon[103849]: pgmap v590: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 267 op/s 2023-10-08T14:01:02.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:01 smithi115 ceph-mon[104408]: pgmap v590: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 267 op/s 2023-10-08T14:01:03.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:03 smithi115 ceph-mon[104408]: pgmap v591: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2023-10-08T14:01:03.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:03 smithi057 ceph-mon[99476]: pgmap v591: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2023-10-08T14:01:04.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:03 smithi099 ceph-mon[103849]: pgmap v591: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2023-10-08T14:01:05.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:05 smithi057 ceph-mon[99476]: pgmap v592: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2023-10-08T14:01:06.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:05 smithi099 ceph-mon[103849]: pgmap v592: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2023-10-08T14:01:06.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:05 smithi115 ceph-mon[104408]: pgmap v592: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2023-10-08T14:01:07.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:07 smithi057 ceph-mon[99476]: pgmap v593: 129 pgs: 129 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 242 op/s 2023-10-08T14:01:08.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:07 smithi099 ceph-mon[103849]: pgmap v593: 129 pgs: 129 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 242 op/s 2023-10-08T14:01:08.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:07 smithi115 ceph-mon[104408]: pgmap v593: 129 pgs: 129 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 242 op/s 2023-10-08T14:01:09.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:09 smithi099 ceph-mon[103849]: pgmap v594: 129 pgs: 129 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2023-10-08T14:01:09.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:09 smithi115 ceph-mon[104408]: pgmap v594: 129 pgs: 129 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2023-10-08T14:01:09.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:09 smithi057 ceph-mon[99476]: pgmap v594: 129 pgs: 129 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2023-10-08T14:01:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.514+0000 7fe89128f700 1 -- 172.21.15.57:0/2755030799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c107a60 msgr2=0x7fe88c107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.514+0000 7fe89128f700 1 --2- 172.21.15.57:0/2755030799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c107a60 0x7fe88c107e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe87c007570 tx=0x7fe87c007880 comp rx=0 tx=0).stop 2023-10-08T14:01:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.514+0000 7fe89128f700 1 -- 172.21.15.57:0/2755030799 shutdown_connections 2023-10-08T14:01:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.514+0000 7fe89128f700 1 --2- 172.21.15.57:0/2755030799 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe88c10c070 0x7fe88c078b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.514+0000 7fe89128f700 1 --2- 172.21.15.57:0/2755030799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c107a60 0x7fe88c107e40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.514+0000 7fe89128f700 1 --2- 172.21.15.57:0/2755030799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe88c108380 0x7fe88c10b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.514+0000 7fe89128f700 1 -- 172.21.15.57:0/2755030799 >> 172.21.15.57:0/2755030799 conn(0x7fe88c06b7c0 msgr2=0x7fe88c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.515+0000 7fe89128f700 1 -- 172.21.15.57:0/2755030799 shutdown_connections 2023-10-08T14:01:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.515+0000 7fe89128f700 1 -- 172.21.15.57:0/2755030799 wait complete. 2023-10-08T14:01:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.516+0000 7fe89128f700 1 Processor -- start 2023-10-08T14:01:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.516+0000 7fe89128f700 1 -- start start 2023-10-08T14:01:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.516+0000 7fe89128f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c107a60 0x7fe88c083360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.516+0000 7fe89128f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe88c108380 0x7fe88c07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.516+0000 7fe89128f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe88c10c070 0x7fe88c07d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.516+0000 7fe89128f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c07e1b0 con 0x7fe88c10c070 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.516+0000 7fe89128f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c1c30d0 con 0x7fe88c107a60 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.516+0000 7fe89128f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c1c3210 con 0x7fe88c108380 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.517+0000 7fe88affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c107a60 0x7fe88c083360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.517+0000 7fe88affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c107a60 0x7fe88c083360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:32806/0 (socket says 172.21.15.57:32806) 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.517+0000 7fe88affd700 1 -- 172.21.15.57:0/524835263 learned_addr learned my addr 172.21.15.57:0/524835263 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.517+0000 7fe88b7fe700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe88c10c070 0x7fe88c07d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe88b7fe700 1 -- 172.21.15.57:0/524835263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe88c108380 msgr2=0x7fe88c07d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe88b7fe700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe88c108380 0x7fe88c07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe88b7fe700 1 -- 172.21.15.57:0/524835263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c107a60 msgr2=0x7fe88c083360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe88b7fe700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c107a60 0x7fe88c083360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe88b7fe700 1 -- 172.21.15.57:0/524835263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c007220 con 0x7fe88c10c070 2023-10-08T14:01:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe88b7fe700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe88c10c070 0x7fe88c07d9e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe88400be20 tx=0x7fe88400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe873fff700 1 -- 172.21.15.57:0/524835263 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88400cd10 con 0x7fe88c10c070 2023-10-08T14:01:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe89128f700 1 -- 172.21.15.57:0/524835263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c1c34a0 con 0x7fe88c10c070 2023-10-08T14:01:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe873fff700 1 -- 172.21.15.57:0/524835263 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe88400ce70 con 0x7fe88c10c070 2023-10-08T14:01:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe873fff700 1 -- 172.21.15.57:0/524835263 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe884012800 con 0x7fe88c10c070 2023-10-08T14:01:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.518+0000 7fe89128f700 1 -- 172.21.15.57:0/524835263 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c1c39b0 con 0x7fe88c10c070 2023-10-08T14:01:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.522+0000 7fe873fff700 1 -- 172.21.15.57:0/524835263 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe8840129a0 con 0x7fe88c10c070 2023-10-08T14:01:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.523+0000 7fe873fff700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8740a1620 0x7fe8740a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.523+0000 7fe873fff700 1 -- 172.21.15.57:0/524835263 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fe884006e40 con 0x7fe88c10c070 2023-10-08T14:01:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.524+0000 7fe89d478b80 1 Processor -- start 2023-10-08T14:01:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.524+0000 7fe89d478b80 1 -- start start 2023-10-08T14:01:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.524+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9a880 0x558d5bd9ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.524+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bd9b1a0 0x558d5bd94b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.524+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd95510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.524+0000 7fe89d478b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bcfd400 con 0x558d5bd950b0 2023-10-08T14:01:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.524+0000 7fe89d478b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bceebc0 con 0x558d5bd9b1a0 2023-10-08T14:01:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.524+0000 7fe89d478b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bcec4e0 con 0x558d5bd9a880 2023-10-08T14:01:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.525+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bd9b1a0 0x558d5bd94b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.525+0000 7fe88b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd95510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.525+0000 7fe88b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd95510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59620/0 (socket says 172.21.15.57:59620) 2023-10-08T14:01:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.525+0000 7fe88b7fe700 1 -- 172.21.15.57:0/3221363734 learned_addr learned my addr 172.21.15.57:0/3221363734 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.525+0000 7fe88b7fe700 1 -- 172.21.15.57:0/3221363734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9a880 msgr2=0x558d5bd9ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.525+0000 7fe88b7fe700 1 --2- 172.21.15.57:0/3221363734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9a880 0x558d5bd9ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.525+0000 7fe88b7fe700 1 -- 172.21.15.57:0/3221363734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bd9b1a0 msgr2=0x558d5bd94b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.525+0000 7fe88b7fe700 1 --2- 172.21.15.57:0/3221363734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bd9b1a0 0x558d5bd94b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.525+0000 7fe88b7fe700 1 -- 172.21.15.57:0/3221363734 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d5bb12d80 con 0x558d5bd950b0 2023-10-08T14:01:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.525+0000 7fe88b7fe700 1 --2- 172.21.15.57:0/3221363734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd95510 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe8840a1930 tx=0x7fe884099740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f625f5389509ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.526+0000 7fe8717fa700 1 -- 172.21.15.57:0/3221363734 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8840a1e60 con 0x558d5bd950b0 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.526+0000 7fe8717fa700 1 -- 172.21.15.57:0/3221363734 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe884097e70 con 0x558d5bd950b0 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.526+0000 7fe8717fa700 1 -- 172.21.15.57:0/3221363734 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe884012e70 con 0x558d5bd950b0 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.528+0000 7fe89d478b80 1 -- 172.21.15.57:0/3221363734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 msgr2=0x558d5bd95510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.528+0000 7fe89d478b80 1 --2- 172.21.15.57:0/3221363734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd95510 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe8840a1930 tx=0x7fe884099740 comp rx=0 tx=0).stop 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.529+0000 7fe89d478b80 1 -- 172.21.15.57:0/3221363734 shutdown_connections 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.529+0000 7fe89d478b80 1 --2- 172.21.15.57:0/3221363734 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd95510 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.529+0000 7fe89d478b80 1 --2- 172.21.15.57:0/3221363734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9a880 0x558d5bd9ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.529+0000 7fe89d478b80 1 --2- 172.21.15.57:0/3221363734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bd9b1a0 0x558d5bd94b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.529+0000 7fe89d478b80 1 -- 172.21.15.57:0/3221363734 >> 172.21.15.57:0/3221363734 conn(0x558d5bd09200 msgr2=0x558d5bd8fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.536+0000 7fe88affd700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8740a1620 0x7fe8740a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.541+0000 7fe89d478b80 1 -- 172.21.15.57:0/3221363734 shutdown_connections 2023-10-08T14:01:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.542+0000 7fe89d478b80 1 -- 172.21.15.57:0/3221363734 wait complete. 2023-10-08T14:01:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.543+0000 7fe88affd700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8740a1620 0x7fe8740a3ae0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x558d5bd91770 tx=0x7fe87c007880 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.543+0000 7fe89d478b80 1 -- start start 2023-10-08T14:01:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.543+0000 7fe89d478b80 1 Processor -- start 2023-10-08T14:01:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.543+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd9e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.543+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bd9f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.544+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bda8cb0 0x558d5bdab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.544+0000 7fe89d478b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bce7dd0 con 0x558d5bd950b0 2023-10-08T14:01:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.544+0000 7fe89d478b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bc846b0 con 0x558d5bda8cb0 2023-10-08T14:01:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.544+0000 7fe89d478b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bc84350 con 0x558d5bd9ed00 2023-10-08T14:01:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.544+0000 7fe89d478b80 10 client.0 _ll_register_callbacks cb 0x558d5bae17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:01:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.544+0000 7fe88affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd9e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.544+0000 7fe88affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd9e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59630/0 (socket says 172.21.15.57:59630) 2023-10-08T14:01:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.544+0000 7fe88affd700 1 -- 172.21.15.57:0/1771259054 learned_addr learned my addr 172.21.15.57:0/1771259054 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.544+0000 7fe88a7fc700 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bd9f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.545+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1771259054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bda8cb0 msgr2=0x558d5bdab0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:01:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.545+0000 7fe88a7fc700 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bda8cb0 0x558d5bdab0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.545+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1771259054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 msgr2=0x558d5bd9e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.545+0000 7fe88a7fc700 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd9e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.545+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1771259054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c1c39b0 con 0x558d5bd9ed00 2023-10-08T14:01:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.545+0000 7fe88a7fc700 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bd9f180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x558d5bd917c0 tx=0x7fe88000c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7792bbbf267f06c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.545+0000 7fe86eff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:01:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.545+0000 7fe86eff5700 1 -- 172.21.15.57:0/1771259054 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800113f0 con 0x558d5bd9ed00 2023-10-08T14:01:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.545+0000 7fe89d478b80 1 -- 172.21.15.57:0/1771259054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d5bcafe50 con 0x558d5bd9ed00 2023-10-08T14:01:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.545+0000 7fe86eff5700 1 -- 172.21.15.57:0/1771259054 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fe880011550 con 0x558d5bd9ed00 2023-10-08T14:01:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.546+0000 7fe86eff5700 1 -- 172.21.15.57:0/1771259054 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe88001c950 con 0x558d5bd9ed00 2023-10-08T14:01:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.549+0000 7fe89d478b80 10 client.26698 fetch_fsmap learned FSMap version 46 2023-10-08T14:01:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.549+0000 7fe89d478b80 1 -- 172.21.15.57:0/1771259054 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x558d5bafbc00 con 0x558d5bd9ed00 2023-10-08T14:01:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.551+0000 7fe86eff5700 1 -- 172.21.15.57:0/1771259054 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d5bcafe50 con 0x558d5bd9ed00 2023-10-08T14:01:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.551+0000 7fe86eff5700 1 -- 172.21.15.57:0/1771259054 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fe880011710 con 0x558d5bd9ed00 2023-10-08T14:01:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.554+0000 7fe89d478b80 10 client.26698 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:01:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.554+0000 7fe89d478b80 10 client.26698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:01:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.554+0000 7fe89d478b80 20 client.26698 populate_metadata read hostname 'smithi057' 2023-10-08T14:01:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.554+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558d5bdaf830 0x558d5bdb1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.554+0000 7fe89d478b80 4 client.26698 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:01:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.554+0000 7fe89d478b80 1 -- 172.21.15.57:0/1771259054 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d5bce0db0 con 0x558d5bdaf830 2023-10-08T14:01:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.557+0000 7fe88affd700 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558d5bdaf830 0x558d5bdb1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.566+0000 7fe86eff5700 10 client.26698 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.566+0000 7fe88affd700 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558d5bdaf830 0x558d5bdb1c50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e587ce26d7c5157 server_cookie=dc836f19ffdbde32 in_seq=0 out_seq=0 2023-10-08T14:01:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.567+0000 7fe86eff5700 1 -- 172.21.15.57:0/1771259054 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558d5bce0db0 con 0x558d5bdaf830 2023-10-08T14:01:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.567+0000 7fe86eff5700 10 client.26698 handle_command_reply: tid=0 2023-10-08T14:01:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.567+0000 7fe89d478b80 1 client.26698 shutdown 2023-10-08T14:01:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.567+0000 7fe89d478b80 1 -- 172.21.15.57:0/1771259054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 msgr2=0x558d5bd9f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.567+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bd9f180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x558d5bd917c0 tx=0x7fe88000c430 comp rx=0 tx=0).stop 2023-10-08T14:01:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.570+0000 7fe89d478b80 1 -- 172.21.15.57:0/1771259054 shutdown_connections 2023-10-08T14:01:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.570+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558d5bdaf830 0x558d5bdb1c50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.570+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bda8cb0 0x558d5bdab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.570+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd9e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.570+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1771259054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bd9f180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.570+0000 7fe89d478b80 1 -- 172.21.15.57:0/1771259054 >> 172.21.15.57:0/1771259054 conn(0x558d5bd09200 msgr2=0x558d5bd8f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.570+0000 7fe86eff5700 0 client.26698 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.579+0000 7fe89d478b80 1 -- 172.21.15.57:0/1771259054 shutdown_connections 2023-10-08T14:01:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.581+0000 7fe89d478b80 1 -- 172.21.15.57:0/1771259054 wait complete. 2023-10-08T14:01:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.582+0000 7fe89d478b80 20 client.26698 trim_cache size 0 max 16384 2023-10-08T14:01:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.606+0000 7fe89d478b80 1 Processor -- start 2023-10-08T14:01:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.607+0000 7fe89d478b80 1 -- start start 2023-10-08T14:01:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.607+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd94b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.607+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bdb1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.607+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bda8cb0 0x558d5bdab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.607+0000 7fe89d478b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bb688c0 con 0x558d5bd950b0 2023-10-08T14:01:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.607+0000 7fe89d478b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bd91170 con 0x558d5bda8cb0 2023-10-08T14:01:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.607+0000 7fe89d478b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bd99450 con 0x558d5bd9ed00 2023-10-08T14:01:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.608+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bdb1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.608+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bdb1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60448/0 (socket says 172.21.15.57:60448) 2023-10-08T14:01:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.608+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1406403846 learned_addr learned my addr 172.21.15.57:0/1406403846 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.608+0000 7fe88affd700 1 --2- 172.21.15.57:0/1406403846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd94b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.608+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1406403846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bda8cb0 msgr2=0x558d5bdab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.608+0000 7fe88a7fc700 1 --2- 172.21.15.57:0/1406403846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bda8cb0 0x558d5bdab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.608+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1406403846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 msgr2=0x558d5bd94b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.608+0000 7fe88a7fc700 1 --2- 172.21.15.57:0/1406403846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd94b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.608+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1406403846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d5bb391f0 con 0x558d5bd9ed00 2023-10-08T14:01:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.609+0000 7fe88a7fc700 1 --2- 172.21.15.57:0/1406403846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bdb1c50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe880011b20 tx=0x7fe88000cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3de6a87aece46759 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.610+0000 7fe871ffb700 1 -- 172.21.15.57:0/1406403846 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800115f0 con 0x558d5bd9ed00 2023-10-08T14:01:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.610+0000 7fe871ffb700 1 -- 172.21.15.57:0/1406403846 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe88000a260 con 0x558d5bd9ed00 2023-10-08T14:01:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.610+0000 7fe871ffb700 1 -- 172.21.15.57:0/1406403846 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88001c6f0 con 0x558d5bd9ed00 2023-10-08T14:01:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.611+0000 7fe89d478b80 1 -- 172.21.15.57:0/1406403846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 msgr2=0x558d5bdb1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.611+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1406403846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bdb1c50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe880011b20 tx=0x7fe88000cd20 comp rx=0 tx=0).stop 2023-10-08T14:01:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.611+0000 7fe89d478b80 1 -- 172.21.15.57:0/1406403846 shutdown_connections 2023-10-08T14:01:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.611+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1406403846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bda8cb0 0x558d5bdab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.611+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1406403846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bd950b0 0x558d5bd94b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.611+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1406403846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bdb1c50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.611+0000 7fe89d478b80 1 -- 172.21.15.57:0/1406403846 >> 172.21.15.57:0/1406403846 conn(0x558d5bd09200 msgr2=0x558d5bdb65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.613+0000 7fe89d478b80 1 -- 172.21.15.57:0/1406403846 shutdown_connections 2023-10-08T14:01:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.613+0000 7fe89d478b80 1 -- 172.21.15.57:0/1406403846 wait complete. 2023-10-08T14:01:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.613+0000 7fe89d478b80 1 -- start start 2023-10-08T14:01:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.613+0000 7fe89d478b80 1 Processor -- start 2023-10-08T14:01:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bd950b0 0x558d5bda3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bda4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe89d478b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bda8cb0 0x558d5bdfe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe89d478b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bda4a90 con 0x558d5bda8cb0 2023-10-08T14:01:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe89d478b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bdfeb80 con 0x558d5bd950b0 2023-10-08T14:01:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe89d478b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5bdfed40 con 0x558d5bd9ed00 2023-10-08T14:01:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe89d478b80 10 client.0 _ll_register_callbacks cb 0x558d5bae17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:01:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bda4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe88a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bda4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60452/0 (socket says 172.21.15.57:60452) 2023-10-08T14:01:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1989317025 learned_addr learned my addr 172.21.15.57:0/1989317025 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1989317025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bd950b0 msgr2=0x558d5bda3de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:01:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe88a7fc700 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bd950b0 0x558d5bda3de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1989317025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bda8cb0 msgr2=0x558d5bdfe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe88a7fc700 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bda8cb0 0x558d5bdfe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.614+0000 7fe88a7fc700 1 -- 172.21.15.57:0/1989317025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d5bafbc00 con 0x558d5bd9ed00 2023-10-08T14:01:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.615+0000 7fe88a7fc700 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bda4320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe88000ae50 tx=0x7fe880011740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9822fe53e244e7e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.615+0000 7fe86eff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:01:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.615+0000 7fe86eff5700 1 -- 172.21.15.57:0/1989317025 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880030870 con 0x558d5bd9ed00 2023-10-08T14:01:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.615+0000 7fe86eff5700 1 -- 172.21.15.57:0/1989317025 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fe8800309d0 con 0x558d5bd9ed00 2023-10-08T14:01:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.615+0000 7fe89d478b80 1 -- 172.21.15.57:0/1989317025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe884097e70 con 0x558d5bd9ed00 2023-10-08T14:01:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.620+0000 7fe86eff5700 1 -- 172.21.15.57:0/1989317025 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe880030b90 con 0x558d5bd9ed00 2023-10-08T14:01:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.624+0000 7fe89d478b80 10 client.26704 fetch_fsmap learned FSMap version 46 2023-10-08T14:01:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.624+0000 7fe89d478b80 1 -- 172.21.15.57:0/1989317025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x558d5bb11250 con 0x558d5bd9ed00 2023-10-08T14:01:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.624+0000 7fe86eff5700 1 -- 172.21.15.57:0/1989317025 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe884097e70 con 0x558d5bd9ed00 2023-10-08T14:01:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.625+0000 7fe86eff5700 1 -- 172.21.15.57:0/1989317025 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fe880025430 con 0x558d5bd9ed00 2023-10-08T14:01:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.625+0000 7fe89d478b80 10 client.26704 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:01:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.625+0000 7fe89d478b80 10 client.26704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:01:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.625+0000 7fe89d478b80 20 client.26704 populate_metadata read hostname 'smithi057' 2023-10-08T14:01:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.625+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558d5bdff820 0x558d5be01c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.625+0000 7fe89d478b80 4 client.26704 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:01:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.625+0000 7fe89d478b80 1 -- 172.21.15.57:0/1989317025 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d5bce0ad0 con 0x558d5bdff820 2023-10-08T14:01:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.625+0000 7fe88affd700 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558d5bdff820 0x558d5be01c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.630+0000 7fe88affd700 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558d5bdff820 0x558d5be01c60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13f0ff5416522063 server_cookie=6fda064b66402f4d in_seq=0 out_seq=0 2023-10-08T14:01:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.630+0000 7fe86eff5700 10 client.26704 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:11.688 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:11 smithi115 ceph-mon[104408]: osdmap e100: 12 total, 12 up, 12 in 2023-10-08T14:01:11.688 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:11 smithi115 ceph-mon[104408]: pgmap v596: 129 pgs: 8 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 119 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2023-10-08T14:01:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.724+0000 7fe86eff5700 1 -- 172.21.15.57:0/1989317025 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d5bce0ad0 con 0x558d5bdff820 2023-10-08T14:01:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.724+0000 7fe86eff5700 10 client.26704 handle_command_reply: tid=0 2023-10-08T14:01:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.724+0000 7fe89d478b80 1 client.26704 shutdown 2023-10-08T14:01:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe89d478b80 1 -- 172.21.15.57:0/1989317025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 msgr2=0x558d5bda4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bda4320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe88000ae50 tx=0x7fe880011740 comp rx=0 tx=0).stop 2023-10-08T14:01:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe89d478b80 1 -- 172.21.15.57:0/1989317025 shutdown_connections 2023-10-08T14:01:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x558d5bdff820 0x558d5be01c60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558d5bda8cb0 0x558d5bdfe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d5bd950b0 0x558d5bda3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe86eff5700 0 client.26704 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe89d478b80 1 --2- 172.21.15.57:0/1989317025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d5bd9ed00 0x558d5bda4320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe89d478b80 1 -- 172.21.15.57:0/1989317025 >> 172.21.15.57:0/1989317025 conn(0x558d5bd09200 msgr2=0x558d5bd9d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe89d478b80 1 -- 172.21.15.57:0/1989317025 shutdown_connections 2023-10-08T14:01:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.725+0000 7fe89d478b80 1 -- 172.21.15.57:0/1989317025 wait complete. 2023-10-08T14:01:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.728+0000 7fe89d478b80 20 client.26704 trim_cache size 0 max 16384 2023-10-08T14:01:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:01:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:01:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:01:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:01:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.734+0000 7fe89128f700 1 -- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8740a1620 msgr2=0x7fe8740a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.734+0000 7fe89128f700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8740a1620 0x7fe8740a3ae0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x558d5bd91770 tx=0x7fe87c007880 comp rx=0 tx=0).stop 2023-10-08T14:01:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.734+0000 7fe89128f700 1 -- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe88c10c070 msgr2=0x7fe88c07d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.734+0000 7fe89128f700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe88c10c070 0x7fe88c07d9e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe88400be20 tx=0x7fe88400bf00 comp rx=0 tx=0).stop 2023-10-08T14:01:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.744+0000 7fe89128f700 1 -- 172.21.15.57:0/524835263 shutdown_connections 2023-10-08T14:01:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.744+0000 7fe89128f700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe88c10c070 0x7fe88c07d9e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.744+0000 7fe89128f700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8740a1620 0x7fe8740a3ae0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.744+0000 7fe89128f700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe88c107a60 0x7fe88c083360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.744+0000 7fe89128f700 1 --2- 172.21.15.57:0/524835263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe88c108380 0x7fe88c07d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.744+0000 7fe89128f700 1 -- 172.21.15.57:0/524835263 >> 172.21.15.57:0/524835263 conn(0x7fe88c06b7c0 msgr2=0x7fe88c0709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.744+0000 7fe89128f700 1 -- 172.21.15.57:0/524835263 shutdown_connections 2023-10-08T14:01:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:11.744+0000 7fe89128f700 1 -- 172.21.15.57:0/524835263 wait complete. 2023-10-08T14:01:11.746 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:11 smithi057 ceph-mon[99476]: osdmap e100: 12 total, 12 up, 12 in 2023-10-08T14:01:11.747 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:11 smithi057 ceph-mon[99476]: pgmap v596: 129 pgs: 8 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 119 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2023-10-08T14:01:11.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:11 smithi099 ceph-mon[103849]: osdmap e100: 12 total, 12 up, 12 in 2023-10-08T14:01:11.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:11 smithi099 ceph-mon[103849]: pgmap v596: 129 pgs: 8 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 119 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2023-10-08T14:01:13.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:13 smithi057 ceph-mon[99476]: pgmap v597: 129 pgs: 29 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 91 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2023-10-08T14:01:14.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:13 smithi099 ceph-mon[103849]: pgmap v597: 129 pgs: 29 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 91 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2023-10-08T14:01:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:13 smithi115 ceph-mon[104408]: pgmap v597: 129 pgs: 29 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 91 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2023-10-08T14:01:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:14 smithi099 ceph-mon[103849]: pgmap v598: 129 pgs: 66 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 241 op/s 2023-10-08T14:01:15.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:14 smithi115 ceph-mon[104408]: pgmap v598: 129 pgs: 66 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 241 op/s 2023-10-08T14:01:15.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:14 smithi057 ceph-mon[99476]: pgmap v598: 129 pgs: 66 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 241 op/s 2023-10-08T14:01:17.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:17 smithi057 ceph-mon[99476]: pgmap v599: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 207 op/s 2023-10-08T14:01:17.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:17 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:01:18.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:17 smithi099 ceph-mon[103849]: pgmap v599: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 207 op/s 2023-10-08T14:01:18.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:17 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:01:18.191 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:17 smithi115 ceph-mon[104408]: pgmap v599: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 207 op/s 2023-10-08T14:01:18.191 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:17 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:01:19.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:19 smithi115 ceph-mon[104408]: pgmap v600: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2023-10-08T14:01:19.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:01:19.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:01:19.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:01:19.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:19 smithi057 ceph-mon[99476]: pgmap v600: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2023-10-08T14:01:19.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:01:19.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:01:19.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:01:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:19 smithi099 ceph-mon[103849]: pgmap v600: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2023-10-08T14:01:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:01:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:01:19.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:01:21.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:21 smithi057 ceph-mon[99476]: pgmap v601: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 134 op/s 2023-10-08T14:01:22.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:21 smithi099 ceph-mon[103849]: pgmap v601: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 134 op/s 2023-10-08T14:01:22.190 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:21 smithi115 ceph-mon[104408]: pgmap v601: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 134 op/s 2023-10-08T14:01:23.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:22 smithi115 ceph-mon[104408]: pgmap v602: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2023-10-08T14:01:23.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:22 smithi057 ceph-mon[99476]: pgmap v602: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2023-10-08T14:01:23.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:22 smithi099 ceph-mon[103849]: pgmap v602: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2023-10-08T14:01:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.932+0000 7ff1b710e700 1 -- 172.21.15.57:0/1678692258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b0119fd0 msgr2=0x7ff1b011c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.932+0000 7ff1b710e700 1 --2- 172.21.15.57:0/1678692258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b0119fd0 0x7ff1b011c3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff1a801c0d0 tx=0x7ff1a801c3e0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.934+0000 7ff1b710e700 1 -- 172.21.15.57:0/1678692258 shutdown_connections 2023-10-08T14:01:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.934+0000 7ff1b710e700 1 --2- 172.21.15.57:0/1678692258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b0119fd0 0x7ff1b011c3c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.934+0000 7ff1b710e700 1 --2- 172.21.15.57:0/1678692258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1b010ef10 0x7ff1b01077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.934+0000 7ff1b710e700 1 --2- 172.21.15.57:0/1678692258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b010f380 0x7ff1b0107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.934+0000 7ff1b710e700 1 -- 172.21.15.57:0/1678692258 >> 172.21.15.57:0/1678692258 conn(0x7ff1b006b7c0 msgr2=0x7ff1b006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.941+0000 7ff1b710e700 1 -- 172.21.15.57:0/1678692258 shutdown_connections 2023-10-08T14:01:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.951+0000 7ff1b710e700 1 -- 172.21.15.57:0/1678692258 wait complete. 2023-10-08T14:01:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.952+0000 7ff1b710e700 1 Processor -- start 2023-10-08T14:01:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.959+0000 7ff1b710e700 1 -- start start 2023-10-08T14:01:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.959+0000 7ff1b710e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1b010ef10 0x7ff1b01156e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.959+0000 7ff1b710e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b010f380 0x7ff1b0115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.959+0000 7ff1b710e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b0119fd0 0x7ff1b0118fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.959+0000 7ff1b710e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b01195f0 con 0x7ff1b010ef10 2023-10-08T14:01:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.959+0000 7ff1b710e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0119760 con 0x7ff1b0119fd0 2023-10-08T14:01:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.959+0000 7ff1b710e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0119920 con 0x7ff1b010f380 2023-10-08T14:01:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.960+0000 7ff1b4eaa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1b010ef10 0x7ff1b01156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.960+0000 7ff1b4eaa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1b010ef10 0x7ff1b01156e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59652/0 (socket says 172.21.15.57:59652) 2023-10-08T14:01:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.960+0000 7ff1b4eaa700 1 -- 172.21.15.57:0/393334371 learned_addr learned my addr 172.21.15.57:0/393334371 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.960+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b0119fd0 0x7ff1b0118fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.960+0000 7ff1b4eaa700 1 -- 172.21.15.57:0/393334371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b010f380 msgr2=0x7ff1b0115c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:01:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.960+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b010f380 0x7ff1b0115c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.960+0000 7ff1b4eaa700 1 -- 172.21.15.57:0/393334371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b0119fd0 msgr2=0x7ff1b0118fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.960+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b0119fd0 0x7ff1b0118fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.960+0000 7ff1b4eaa700 1 -- 172.21.15.57:0/393334371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a801ad10 con 0x7ff1b010ef10 2023-10-08T14:01:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.960+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1b010ef10 0x7ff1b01156e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000b550 tx=0x7ff1a000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.961+0000 7ff1adffb700 1 -- 172.21.15.57:0/393334371 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0011830 con 0x7ff1b010ef10 2023-10-08T14:01:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.961+0000 7ff1adffb700 1 -- 172.21.15.57:0/393334371 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a0011e50 con 0x7ff1b010ef10 2023-10-08T14:01:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.961+0000 7ff1adffb700 1 -- 172.21.15.57:0/393334371 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a000f2e0 con 0x7ff1b010ef10 2023-10-08T14:01:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.961+0000 7ff1b710e700 1 -- 172.21.15.57:0/393334371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b0119c00 con 0x7ff1b010ef10 2023-10-08T14:01:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.961+0000 7ff1b710e700 1 -- 172.21.15.57:0/393334371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b01aa040 con 0x7ff1b010ef10 2023-10-08T14:01:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.964+0000 7ff1adffb700 1 -- 172.21.15.57:0/393334371 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff1a0011990 con 0x7ff1b010ef10 2023-10-08T14:01:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.965+0000 7ff1adffb700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1980a1740 0x7ff1980a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.965+0000 7ff1adffb700 1 -- 172.21.15.57:0/393334371 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7ff1a00ccde0 con 0x7ff1b010ef10 2023-10-08T14:01:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.965+0000 7ff1affff700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1980a1740 0x7ff1980a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.966+0000 7ff1affff700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1980a1740 0x7ff1980a3c00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0116c30 tx=0x7ff1a4006c60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.966+0000 7ff1c32f7b80 1 Processor -- start 2023-10-08T14:01:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.967+0000 7ff1c32f7b80 1 -- start start 2023-10-08T14:01:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.967+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624eb880 0x5652624ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.967+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624ec1a0 0x5652624e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.967+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 0x5652624e6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.967+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526244e400 con 0x5652624e60b0 2023-10-08T14:01:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.967+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526243fbc0 con 0x5652624eb880 2023-10-08T14:01:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.967+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526243d4e0 con 0x5652624ec1a0 2023-10-08T14:01:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.968+0000 7ff1affff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624ec1a0 0x5652624e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.968+0000 7ff1affff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624ec1a0 0x5652624e5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60472/0 (socket says 172.21.15.57:60472) 2023-10-08T14:01:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.968+0000 7ff1affff700 1 -- 172.21.15.57:0/4217174090 learned_addr learned my addr 172.21.15.57:0/4217174090 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.972+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/4217174090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 0x5652624e6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.972+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/4217174090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624eb880 0x5652624ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.972+0000 7ff1b56ab700 1 -- 172.21.15.57:0/4217174090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624ec1a0 msgr2=0x5652624e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.972+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/4217174090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624ec1a0 0x5652624e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.972+0000 7ff1b56ab700 1 -- 172.21.15.57:0/4217174090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624eb880 msgr2=0x5652624ebc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.972+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/4217174090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624eb880 0x5652624ebc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.972+0000 7ff1b56ab700 1 -- 172.21.15.57:0/4217174090 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565262263d80 con 0x5652624e60b0 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.973+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/4217174090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 0x5652624e6510 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff1a801cf50 tx=0x7ff1a801a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db37d2f0ef2f55b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.973+0000 7ff196ffd700 1 -- 172.21.15.57:0/4217174090 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a801f040 con 0x5652624e60b0 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.973+0000 7ff196ffd700 1 -- 172.21.15.57:0/4217174090 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a8003c00 con 0x5652624e60b0 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.973+0000 7ff196ffd700 1 -- 172.21.15.57:0/4217174090 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80052d0 con 0x5652624e60b0 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.974+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/4217174090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 msgr2=0x5652624e6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.974+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/4217174090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 0x5652624e6510 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff1a801cf50 tx=0x7ff1a801a280 comp rx=0 tx=0).stop 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.974+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/4217174090 shutdown_connections 2023-10-08T14:01:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.974+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/4217174090 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 0x5652624e6510 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.974+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/4217174090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624eb880 0x5652624ebc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.974+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/4217174090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624ec1a0 0x5652624e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.974+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/4217174090 >> 172.21.15.57:0/4217174090 conn(0x56526245a200 msgr2=0x5652624e0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.977+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/4217174090 shutdown_connections 2023-10-08T14:01:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.977+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/4217174090 wait complete. 2023-10-08T14:01:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.978+0000 7ff1c32f7b80 1 -- start start 2023-10-08T14:01:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.978+0000 7ff1c32f7b80 1 Processor -- start 2023-10-08T14:01:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624e60b0 0x5652624ef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624efcb0 0x5652624f0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624f9c50 0x5652624fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x565262438dd0 con 0x5652624efcb0 2023-10-08T14:01:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652623d56b0 con 0x5652624f9c50 2023-10-08T14:01:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652623d5350 con 0x5652624e60b0 2023-10-08T14:01:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1c32f7b80 10 client.0 _ll_register_callbacks cb 0x5652622327a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:01:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1b4eaa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624e60b0 0x5652624ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1b4eaa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624e60b0 0x5652624ef770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60474/0 (socket says 172.21.15.57:60474) 2023-10-08T14:01:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1b4eaa700 1 -- 172.21.15.57:0/1040262005 learned_addr learned my addr 172.21.15.57:0/1040262005 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1b4eaa700 1 -- 172.21.15.57:0/1040262005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624f9c50 msgr2=0x5652624fc040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:01:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624f9c50 0x5652624fc040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1b4eaa700 1 -- 172.21.15.57:0/1040262005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624efcb0 msgr2=0x5652624f0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624efcb0 0x5652624f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.979+0000 7ff1b4eaa700 1 -- 172.21.15.57:0/1040262005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b011cc50 con 0x5652624e60b0 2023-10-08T14:01:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.980+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624e60b0 0x5652624ef770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000bfd0 tx=0x7ff1a009a220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abcaf9f28696da0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.980+0000 7ff18ffff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:01:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.980+0000 7ff18ffff700 1 -- 172.21.15.57:0/1040262005 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00b1070 con 0x5652624e60b0 2023-10-08T14:01:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.980+0000 7ff18ffff700 1 -- 172.21.15.57:0/1040262005 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7ff1a00a1eb0 con 0x5652624e60b0 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.980+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/1040262005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565262400e50 con 0x5652624e60b0 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff18ffff700 1 -- 172.21.15.57:0/1040262005 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1a0010560 con 0x5652624e60b0 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff1c32f7b80 10 client.26710 fetch_fsmap learned FSMap version 46 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/1040262005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56526224cc00 con 0x5652624e60b0 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff18ffff700 1 -- 172.21.15.57:0/1040262005 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565262400e50 con 0x5652624e60b0 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff18ffff700 1 -- 172.21.15.57:0/1040262005 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff1a0099ea0 con 0x5652624e60b0 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff1c32f7b80 10 client.26710 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff1c32f7b80 10 client.26710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff1c32f7b80 20 client.26710 populate_metadata read hostname 'smithi057' 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x565262500720 0x565262502b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff1c32f7b80 4 client.26710 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:01:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/1040262005 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565262431db0 con 0x565262500720 2023-10-08T14:01:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.981+0000 7ff1affff700 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x565262500720 0x565262502b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.982+0000 7ff1affff700 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x565262500720 0x565262502b60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c4487323d3de39 server_cookie=a251c593c3c04ad3 in_seq=0 out_seq=0 2023-10-08T14:01:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.982+0000 7ff18ffff700 10 client.26710 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.990+0000 7ff18ffff700 1 -- 172.21.15.57:0/1040262005 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565262431db0 con 0x565262500720 2023-10-08T14:01:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.990+0000 7ff18ffff700 10 client.26710 handle_command_reply: tid=0 2023-10-08T14:01:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.990+0000 7ff1c32f7b80 1 client.26710 shutdown 2023-10-08T14:01:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.993+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/1040262005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624e60b0 msgr2=0x5652624ef770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.993+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624e60b0 0x5652624ef770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000bfd0 tx=0x7ff1a009a220 comp rx=0 tx=0).stop 2023-10-08T14:01:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.999+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/1040262005 shutdown_connections 2023-10-08T14:01:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.999+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x565262500720 0x565262502b60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.999+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624f9c50 0x5652624fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.999+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624e60b0 0x5652624ef770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.999+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/1040262005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624efcb0 0x5652624f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:24.999+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/1040262005 >> 172.21.15.57:0/1040262005 conn(0x56526245a200 msgr2=0x5652624e0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.001+0000 7ff18ffff700 0 client.26710 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.012+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/1040262005 shutdown_connections 2023-10-08T14:01:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.018+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/1040262005 wait complete. 2023-10-08T14:01:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.028+0000 7ff1c32f7b80 20 client.26710 trim_cache size 0 max 16384 2023-10-08T14:01:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.062+0000 7ff1c32f7b80 1 Processor -- start 2023-10-08T14:01:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.062+0000 7ff1c32f7b80 1 -- start start 2023-10-08T14:01:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.063+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 0x5652624e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.063+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624efcb0 0x5652624ef000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.063+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624f9c50 0x5652624f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.063+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652622b98c0 con 0x5652624e60b0 2023-10-08T14:01:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.063+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652624dd860 con 0x5652624efcb0 2023-10-08T14:01:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.063+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652624e1610 con 0x5652624f9c50 2023-10-08T14:01:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.070+0000 7ff1b56ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624f9c50 0x5652624f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.070+0000 7ff1b56ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624f9c50 0x5652624f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60484/0 (socket says 172.21.15.57:60484) 2023-10-08T14:01:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.070+0000 7ff1b56ab700 1 -- 172.21.15.57:0/4030401089 learned_addr learned my addr 172.21.15.57:0/4030401089 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.070+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/4030401089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 0x5652624e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.071+0000 7ff1b56ab700 1 -- 172.21.15.57:0/4030401089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624efcb0 msgr2=0x5652624ef000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:01:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.071+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/4030401089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624efcb0 0x5652624ef000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.071+0000 7ff1b56ab700 1 -- 172.21.15.57:0/4030401089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 msgr2=0x5652624e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.071+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/4030401089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 0x5652624e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.071+0000 7ff1b56ab700 1 -- 172.21.15.57:0/4030401089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56526228a1f0 con 0x5652624f9c50 2023-10-08T14:01:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.071+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/4030401089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624f9c50 0x5652624f6bb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff1a801a150 tx=0x7ff1a801a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b0dde90eb22232 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.071+0000 7ff1977fe700 1 -- 172.21.15.57:0/4030401089 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8018370 con 0x5652624f9c50 2023-10-08T14:01:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.072+0000 7ff1977fe700 1 -- 172.21.15.57:0/4030401089 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a8018990 con 0x5652624f9c50 2023-10-08T14:01:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.072+0000 7ff1977fe700 1 -- 172.21.15.57:0/4030401089 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8005950 con 0x5652624f9c50 2023-10-08T14:01:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.073+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/4030401089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624f9c50 msgr2=0x5652624f6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.073+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/4030401089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624f9c50 0x5652624f6bb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff1a801a150 tx=0x7ff1a801a530 comp rx=0 tx=0).stop 2023-10-08T14:01:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.080+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/4030401089 shutdown_connections 2023-10-08T14:01:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.080+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/4030401089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624f9c50 0x5652624f6bb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.080+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/4030401089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624e60b0 0x5652624e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.080+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/4030401089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624efcb0 0x5652624ef000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.080+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/4030401089 >> 172.21.15.57:0/4030401089 conn(0x56526245a200 msgr2=0x5652625074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.087+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/4030401089 shutdown_connections 2023-10-08T14:01:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.088+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/4030401089 wait complete. 2023-10-08T14:01:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.088+0000 7ff1c32f7b80 1 -- start start 2023-10-08T14:01:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.088+0000 7ff1c32f7b80 1 Processor -- start 2023-10-08T14:01:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624e60b0 0x5652624f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624efcb0 0x5652624f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1c32f7b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624f9c50 0x5652624f5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652624f5990 con 0x5652624f9c50 2023-10-08T14:01:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565262553570 con 0x5652624e60b0 2023-10-08T14:01:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1c32f7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652625536b0 con 0x5652624efcb0 2023-10-08T14:01:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1c32f7b80 10 client.0 _ll_register_callbacks cb 0x7ff1a0099500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:01:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1b56ab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624f9c50 0x5652624f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1b56ab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624f9c50 0x5652624f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59678/0 (socket says 172.21.15.57:59678) 2023-10-08T14:01:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1b56ab700 1 -- 172.21.15.57:0/2399596613 learned_addr learned my addr 172.21.15.57:0/2399596613 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624e60b0 0x5652624f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1b56ab700 1 -- 172.21.15.57:0/2399596613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624efcb0 msgr2=0x5652624f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624efcb0 0x5652624f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1b56ab700 1 -- 172.21.15.57:0/2399596613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624e60b0 msgr2=0x5652624f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624e60b0 0x5652624f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.089+0000 7ff1b56ab700 1 -- 172.21.15.57:0/2399596613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565262263d80 con 0x5652624f9c50 2023-10-08T14:01:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.090+0000 7ff1b56ab700 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624f9c50 0x5652624f5280 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff1a801cf50 tx=0x7ff1a80185a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebecaf9250463a56 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.091+0000 7ff18ffff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:01:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.091+0000 7ff18ffff700 1 -- 172.21.15.57:0/2399596613 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a801f030 con 0x5652624f9c50 2023-10-08T14:01:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.091+0000 7ff18ffff700 1 -- 172.21.15.57:0/2399596613 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7ff1a8005740 con 0x5652624f9c50 2023-10-08T14:01:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.091+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/2399596613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1a8003c00 con 0x5652624f9c50 2023-10-08T14:01:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.102+0000 7ff18ffff700 1 -- 172.21.15.57:0/2399596613 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1a8018800 con 0x5652624f9c50 2023-10-08T14:01:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.102+0000 7ff1c32f7b80 10 client.17145 fetch_fsmap learned FSMap version 46 2023-10-08T14:01:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.102+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/2399596613 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x565262262250 con 0x5652624f9c50 2023-10-08T14:01:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.102+0000 7ff18ffff700 1 -- 172.21.15.57:0/2399596613 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a8003c00 con 0x5652624f9c50 2023-10-08T14:01:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.102+0000 7ff18ffff700 1 -- 172.21.15.57:0/2399596613 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff1a8038070 con 0x5652624f9c50 2023-10-08T14:01:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.104+0000 7ff1c32f7b80 10 client.17145 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:01:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.104+0000 7ff1c32f7b80 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:01:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.104+0000 7ff1c32f7b80 20 client.17145 populate_metadata read hostname 'smithi057' 2023-10-08T14:01:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.104+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5652625549b0 0x565262556dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.104+0000 7ff1c32f7b80 4 client.17145 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:01:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.104+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/2399596613 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565262431ad0 con 0x5652625549b0 2023-10-08T14:01:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.104+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5652625549b0 0x565262556dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.104+0000 7ff18ffff700 10 client.17145 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.104+0000 7ff1b4eaa700 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5652625549b0 0x565262556dd0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=630de1363c436975 server_cookie=520507b4c867340d in_seq=0 out_seq=0 2023-10-08T14:01:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.175+0000 7ff18ffff700 1 -- 172.21.15.57:0/2399596613 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565262431ad0 con 0x5652625549b0 2023-10-08T14:01:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.175+0000 7ff18ffff700 10 client.17145 handle_command_reply: tid=0 2023-10-08T14:01:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.175+0000 7ff1c32f7b80 1 client.17145 shutdown 2023-10-08T14:01:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.185+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/2399596613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624f9c50 msgr2=0x5652624f5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.185+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624f9c50 0x5652624f5280 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff1a801cf50 tx=0x7ff1a80185a0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.190+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/2399596613 shutdown_connections 2023-10-08T14:01:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.190+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5652625549b0 0x565262556dd0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.190+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5652624f9c50 0x5652624f5280 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.190+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652624e60b0 0x5652624f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.190+0000 7ff1c32f7b80 1 --2- 172.21.15.57:0/2399596613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5652624efcb0 0x5652624f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.190+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/2399596613 >> 172.21.15.57:0/2399596613 conn(0x56526245a200 msgr2=0x5652624eefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.192+0000 7ff18ffff700 0 client.17145 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.194+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/2399596613 shutdown_connections 2023-10-08T14:01:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.196+0000 7ff1c32f7b80 1 -- 172.21.15.57:0/2399596613 wait complete. 2023-10-08T14:01:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.196+0000 7ff1c32f7b80 20 client.17145 trim_cache size 0 max 16384 2023-10-08T14:01:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:01:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:01:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:01:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:01:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.201+0000 7ff1b710e700 1 -- 172.21.15.57:0/393334371 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1980a1740 msgr2=0x7ff1980a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.201+0000 7ff1b710e700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1980a1740 0x7ff1980a3c00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0116c30 tx=0x7ff1a4006c60 comp rx=0 tx=0).stop 2023-10-08T14:01:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.201+0000 7ff1b710e700 1 -- 172.21.15.57:0/393334371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1b010ef10 msgr2=0x7ff1b01156e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.201+0000 7ff1b710e700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1b010ef10 0x7ff1b01156e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000b550 tx=0x7ff1a000b860 comp rx=0 tx=0).stop 2023-10-08T14:01:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.201+0000 7ff1b710e700 1 -- 172.21.15.57:0/393334371 shutdown_connections 2023-10-08T14:01:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.201+0000 7ff1b710e700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff1980a1740 0x7ff1980a3c00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.202+0000 7ff1b710e700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b0119fd0 0x7ff1b0118fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.202+0000 7ff1b710e700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1b010ef10 0x7ff1b01156e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.202+0000 7ff1b710e700 1 --2- 172.21.15.57:0/393334371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b010f380 0x7ff1b0115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.202+0000 7ff1b710e700 1 -- 172.21.15.57:0/393334371 >> 172.21.15.57:0/393334371 conn(0x7ff1b006b7c0 msgr2=0x7ff1b011a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.202+0000 7ff1b710e700 1 -- 172.21.15.57:0/393334371 shutdown_connections 2023-10-08T14:01:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:25.202+0000 7ff1b710e700 1 -- 172.21.15.57:0/393334371 wait complete. 2023-10-08T14:01:25.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:25 smithi057 ceph-mon[99476]: pgmap v603: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 965 KiB/s wr, 119 op/s 2023-10-08T14:01:26.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:25 smithi099 ceph-mon[103849]: pgmap v603: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 965 KiB/s wr, 119 op/s 2023-10-08T14:01:26.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:25 smithi115 ceph-mon[104408]: pgmap v603: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 965 KiB/s wr, 119 op/s 2023-10-08T14:01:28.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:27 smithi099 ceph-mon[103849]: pgmap v604: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 946 KiB/s wr, 90 op/s 2023-10-08T14:01:28.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:27 smithi115 ceph-mon[104408]: pgmap v604: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 946 KiB/s wr, 90 op/s 2023-10-08T14:01:28.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:27 smithi057 ceph-mon[99476]: pgmap v604: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 946 KiB/s wr, 90 op/s 2023-10-08T14:01:29.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:29 smithi115 ceph-mon[104408]: pgmap v605: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 753 KiB/s wr, 97 op/s 2023-10-08T14:01:29.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:29 smithi057 ceph-mon[99476]: pgmap v605: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 753 KiB/s wr, 97 op/s 2023-10-08T14:01:29.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:29 smithi099 ceph-mon[103849]: pgmap v605: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 753 KiB/s wr, 97 op/s 2023-10-08T14:01:31.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:31 smithi057 ceph-mon[99476]: pgmap v606: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 683 KiB/s wr, 106 op/s 2023-10-08T14:01:32.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:31 smithi099 ceph-mon[103849]: pgmap v606: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 683 KiB/s wr, 106 op/s 2023-10-08T14:01:32.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:31 smithi115 ceph-mon[104408]: pgmap v606: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 683 KiB/s wr, 106 op/s 2023-10-08T14:01:34.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:33 smithi099 ceph-mon[103849]: pgmap v607: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 964 KiB/s wr, 95 op/s 2023-10-08T14:01:34.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:33 smithi115 ceph-mon[104408]: pgmap v607: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 964 KiB/s wr, 95 op/s 2023-10-08T14:01:34.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:33 smithi057 ceph-mon[99476]: pgmap v607: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 964 KiB/s wr, 95 op/s 2023-10-08T14:01:35.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:34 smithi115 ceph-mon[104408]: pgmap v608: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 613 KiB/s wr, 115 op/s 2023-10-08T14:01:35.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:34 smithi057 ceph-mon[99476]: pgmap v608: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 613 KiB/s wr, 115 op/s 2023-10-08T14:01:35.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:34 smithi099 ceph-mon[103849]: pgmap v608: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 613 KiB/s wr, 115 op/s 2023-10-08T14:01:38.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:37 smithi099 ceph-mon[103849]: pgmap v609: 129 pgs: 54 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 59 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 892 KiB/s wr, 102 op/s 2023-10-08T14:01:38.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:37 smithi115 ceph-mon[104408]: pgmap v609: 129 pgs: 54 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 59 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 892 KiB/s wr, 102 op/s 2023-10-08T14:01:38.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:37 smithi057 ceph-mon[99476]: pgmap v609: 129 pgs: 54 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 59 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 892 KiB/s wr, 102 op/s 2023-10-08T14:01:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.394+0000 7f710e8ec700 1 -- 172.21.15.57:0/762468512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7108067b20 msgr2=0x7f7108067fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.394+0000 7f710e8ec700 1 --2- 172.21.15.57:0/762468512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7108067b20 0x7f7108067fa0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f70f801a9a0 tx=0x7f70f801acb0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.395+0000 7f710e8ec700 1 -- 172.21.15.57:0/762468512 shutdown_connections 2023-10-08T14:01:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.395+0000 7f710e8ec700 1 --2- 172.21.15.57:0/762468512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7108067b20 0x7f7108067fa0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.395+0000 7f710e8ec700 1 --2- 172.21.15.57:0/762468512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7108105680 0x7f7108103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.395+0000 7f710e8ec700 1 --2- 172.21.15.57:0/762468512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7108103640 0x7f7108103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.395+0000 7f710e8ec700 1 -- 172.21.15.57:0/762468512 >> 172.21.15.57:0/762468512 conn(0x7f7108076790 msgr2=0x7f7108076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.403+0000 7f710e8ec700 1 -- 172.21.15.57:0/762468512 shutdown_connections 2023-10-08T14:01:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.403+0000 7f710e8ec700 1 -- 172.21.15.57:0/762468512 wait complete. 2023-10-08T14:01:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.404+0000 7f710e8ec700 1 Processor -- start 2023-10-08T14:01:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.407+0000 7f710e8ec700 1 -- start start 2023-10-08T14:01:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.408+0000 7f710e8ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7108067b20 0x7f7108113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.408+0000 7f710e8ec700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7108105680 0x7f710810fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.408+0000 7f710e8ec700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7108110000 0x7f71081104b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.408+0000 7f710e8ec700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71081141b0 con 0x7f7108110000 2023-10-08T14:01:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.408+0000 7f710e8ec700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71081c3430 con 0x7f7108105680 2023-10-08T14:01:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.408+0000 7f710e8ec700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71081c3570 con 0x7f7108067b20 2023-10-08T14:01:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.408+0000 7f71077fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7108105680 0x7f710810fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.408+0000 7f71077fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7108105680 0x7f710810fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:32884/0 (socket says 172.21.15.57:32884) 2023-10-08T14:01:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.408+0000 7f71077fe700 1 -- 172.21.15.57:0/2495627079 learned_addr learned my addr 172.21.15.57:0/2495627079 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.409+0000 7f710ce89700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7108110000 0x7f71081104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.409+0000 7f7107fff700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7108067b20 0x7f7108113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.409+0000 7f71077fe700 1 -- 172.21.15.57:0/2495627079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7108067b20 msgr2=0x7f7108113a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.409+0000 7f71077fe700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7108067b20 0x7f7108113a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.409+0000 7f71077fe700 1 -- 172.21.15.57:0/2495627079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7108110000 msgr2=0x7f71081104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.409+0000 7f71077fe700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7108110000 0x7f71081104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.409+0000 7f71077fe700 1 -- 172.21.15.57:0/2495627079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f801a600 con 0x7f7108105680 2023-10-08T14:01:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.409+0000 7f71077fe700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7108105680 0x7f710810fac0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00b450 tx=0x7f70fc00b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.411+0000 7f71057fa700 1 -- 172.21.15.57:0/2495627079 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00be60 con 0x7f7108105680 2023-10-08T14:01:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.411+0000 7f71057fa700 1 -- 172.21.15.57:0/2495627079 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70fc011af0 con 0x7f7108105680 2023-10-08T14:01:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.411+0000 7f710e8ec700 1 -- 172.21.15.57:0/2495627079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71081c3800 con 0x7f7108105680 2023-10-08T14:01:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.411+0000 7f710e8ec700 1 -- 172.21.15.57:0/2495627079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71081c3d10 con 0x7f7108105680 2023-10-08T14:01:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.411+0000 7f71057fa700 1 -- 172.21.15.57:0/2495627079 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc010c10 con 0x7f7108105680 2023-10-08T14:01:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.416+0000 7f71057fa700 1 -- 172.21.15.57:0/2495627079 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f70fc01a410 con 0x7f7108105680 2023-10-08T14:01:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.417+0000 7f71057fa700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70f00a1800 0x7f70f00a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.417+0000 7f71057fa700 1 -- 172.21.15.57:0/2495627079 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f70fc0cd8c0 con 0x7f7108105680 2023-10-08T14:01:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.418+0000 7f7107fff700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70f00a1800 0x7f70f00a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.419+0000 7f7107fff700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70f00a1800 0x7f70f00a3cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7100009fd0 tx=0x7f710000d040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.419+0000 7f711aad5b80 1 Processor -- start 2023-10-08T14:01:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.419+0000 7f711aad5b80 1 -- start start 2023-10-08T14:01:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 0x562ef22a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef22a11a0 0x562ef229ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 0x562ef229b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f711aad5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef2203400 con 0x562ef229b0b0 2023-10-08T14:01:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f711aad5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef21f4bc0 con 0x562ef22a0880 2023-10-08T14:01:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f711aad5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef21f24e0 con 0x562ef22a11a0 2023-10-08T14:01:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f710ce89700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 0x562ef229b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f710ce89700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 0x562ef229b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59700/0 (socket says 172.21.15.57:59700) 2023-10-08T14:01:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f710ce89700 1 -- 172.21.15.57:0/4039295609 learned_addr learned my addr 172.21.15.57:0/4039295609 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f7107fff700 1 --2- 172.21.15.57:0/4039295609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 0x562ef22a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f710ce89700 1 -- 172.21.15.57:0/4039295609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef22a11a0 msgr2=0x562ef229ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f710ce89700 1 --2- 172.21.15.57:0/4039295609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef22a11a0 0x562ef229ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f710ce89700 1 -- 172.21.15.57:0/4039295609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 msgr2=0x562ef22a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f710ce89700 1 --2- 172.21.15.57:0/4039295609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 0x562ef22a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.420+0000 7f710ce89700 1 -- 172.21.15.57:0/4039295609 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ef2018d80 con 0x562ef229b0b0 2023-10-08T14:01:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.421+0000 7f710ce89700 1 --2- 172.21.15.57:0/4039295609 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 0x562ef229b510 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005cd0 tx=0x7f70f8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d729fc44ae3da824 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.421+0000 7f70f67fc700 1 -- 172.21.15.57:0/4039295609 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f801c9f0 con 0x562ef229b0b0 2023-10-08T14:01:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.421+0000 7f70f67fc700 1 -- 172.21.15.57:0/4039295609 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f8018460 con 0x562ef229b0b0 2023-10-08T14:01:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.421+0000 7f70f67fc700 1 -- 172.21.15.57:0/4039295609 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f800f660 con 0x562ef229b0b0 2023-10-08T14:01:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.422+0000 7f711aad5b80 1 -- 172.21.15.57:0/4039295609 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 msgr2=0x562ef229b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.422+0000 7f711aad5b80 1 --2- 172.21.15.57:0/4039295609 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 0x562ef229b510 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005cd0 tx=0x7f70f8005750 comp rx=0 tx=0).stop 2023-10-08T14:01:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.426+0000 7f711aad5b80 1 -- 172.21.15.57:0/4039295609 shutdown_connections 2023-10-08T14:01:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.426+0000 7f711aad5b80 1 --2- 172.21.15.57:0/4039295609 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 0x562ef229b510 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.426+0000 7f711aad5b80 1 --2- 172.21.15.57:0/4039295609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 0x562ef22a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.426+0000 7f711aad5b80 1 --2- 172.21.15.57:0/4039295609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef22a11a0 0x562ef229ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.426+0000 7f711aad5b80 1 -- 172.21.15.57:0/4039295609 >> 172.21.15.57:0/4039295609 conn(0x562ef220f200 msgr2=0x562ef2295b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.426+0000 7f711aad5b80 1 -- 172.21.15.57:0/4039295609 shutdown_connections 2023-10-08T14:01:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.426+0000 7f711aad5b80 1 -- 172.21.15.57:0/4039295609 wait complete. 2023-10-08T14:01:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.427+0000 7f711aad5b80 1 -- start start 2023-10-08T14:01:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.427+0000 7f711aad5b80 1 Processor -- start 2023-10-08T14:01:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.428+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef229b0b0 0x562ef22a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.428+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef22a0880 0x562ef22a4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.428+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef22a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.428+0000 7f711aad5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef21eddd0 con 0x562ef22a0880 2023-10-08T14:01:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.428+0000 7f711aad5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef218a6b0 con 0x562ef22a11a0 2023-10-08T14:01:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.428+0000 7f711aad5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef218a350 con 0x562ef229b0b0 2023-10-08T14:01:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.428+0000 7f711aad5b80 10 client.0 _ll_register_callbacks cb 0x562ef1fe77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:01:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.428+0000 7f710ce89700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef22a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.428+0000 7f710ce89700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef22a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:32896/0 (socket says 172.21.15.57:32896) 2023-10-08T14:01:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.428+0000 7f710ce89700 1 -- 172.21.15.57:0/74271548 learned_addr learned my addr 172.21.15.57:0/74271548 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.429+0000 7f710ce89700 1 -- 172.21.15.57:0/74271548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef229b0b0 msgr2=0x562ef22a7260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:01:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.429+0000 7f710ce89700 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef229b0b0 0x562ef22a7260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.429+0000 7f710ce89700 1 -- 172.21.15.57:0/74271548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef22a0880 msgr2=0x562ef22a4550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:01:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.429+0000 7f710ce89700 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef22a0880 0x562ef22a4550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.429+0000 7f710ce89700 1 -- 172.21.15.57:0/74271548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7108114530 con 0x562ef22a11a0 2023-10-08T14:01:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.429+0000 7f710ce89700 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef22a4c90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f70f801acf0 tx=0x7f70f80188b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11ca22d79442acb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.429+0000 7f70eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:01:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.429+0000 7f70eb7fe700 1 -- 172.21.15.57:0/74271548 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8018ea0 con 0x562ef22a11a0 2023-10-08T14:01:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.430+0000 7f70eb7fe700 1 -- 172.21.15.57:0/74271548 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f70f801ce00 con 0x562ef22a11a0 2023-10-08T14:01:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.430+0000 7f711aad5b80 1 -- 172.21.15.57:0/74271548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ef21b5e50 con 0x562ef22a11a0 2023-10-08T14:01:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.430+0000 7f70eb7fe700 1 -- 172.21.15.57:0/74271548 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f8005a00 con 0x562ef22a11a0 2023-10-08T14:01:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.430+0000 7f711aad5b80 10 client.26690 fetch_fsmap learned FSMap version 46 2023-10-08T14:01:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.430+0000 7f711aad5b80 1 -- 172.21.15.57:0/74271548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x562ef2001c00 con 0x562ef22a11a0 2023-10-08T14:01:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.430+0000 7f70eb7fe700 1 -- 172.21.15.57:0/74271548 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ef21b5e50 con 0x562ef22a11a0 2023-10-08T14:01:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.430+0000 7f70eb7fe700 1 -- 172.21.15.57:0/74271548 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f70f800f420 con 0x562ef22a11a0 2023-10-08T14:01:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.431+0000 7f711aad5b80 10 client.26690 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:01:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.431+0000 7f711aad5b80 10 client.26690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:01:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.431+0000 7f711aad5b80 20 client.26690 populate_metadata read hostname 'smithi057' 2023-10-08T14:01:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.431+0000 7f711aad5b80 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562ef22c5940 0x562ef22c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.431+0000 7f711aad5b80 4 client.26690 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:01:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.431+0000 7f711aad5b80 1 -- 172.21.15.57:0/74271548 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ef21e6db0 con 0x562ef22c5940 2023-10-08T14:01:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.431+0000 7f7107fff700 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562ef22c5940 0x562ef22c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.431+0000 7f70eb7fe700 10 client.26690 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.431+0000 7f7107fff700 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562ef22c5940 0x562ef22c7d60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487704f159f01197 server_cookie=93cf5dadd6e6d6f in_seq=0 out_seq=0 2023-10-08T14:01:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.438+0000 7f70eb7fe700 1 -- 172.21.15.57:0/74271548 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562ef21e6db0 con 0x562ef22c5940 2023-10-08T14:01:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.438+0000 7f70eb7fe700 10 client.26690 handle_command_reply: tid=0 2023-10-08T14:01:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.438+0000 7f711aad5b80 1 client.26690 shutdown 2023-10-08T14:01:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.440+0000 7f711aad5b80 1 -- 172.21.15.57:0/74271548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 msgr2=0x562ef22a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.440+0000 7f711aad5b80 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef22a4c90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f70f801acf0 tx=0x7f70f80188b0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.440+0000 7f711aad5b80 1 -- 172.21.15.57:0/74271548 shutdown_connections 2023-10-08T14:01:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.440+0000 7f711aad5b80 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562ef22c5940 0x562ef22c7d60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.440+0000 7f711aad5b80 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef22a4c90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.440+0000 7f711aad5b80 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef229b0b0 0x562ef22a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.440+0000 7f711aad5b80 1 --2- 172.21.15.57:0/74271548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef22a0880 0x562ef22a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.440+0000 7f711aad5b80 1 -- 172.21.15.57:0/74271548 >> 172.21.15.57:0/74271548 conn(0x562ef220f200 msgr2=0x562ef2295010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.440+0000 7f70eb7fe700 0 client.26690 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.441+0000 7f711aad5b80 1 -- 172.21.15.57:0/74271548 shutdown_connections 2023-10-08T14:01:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.441+0000 7f711aad5b80 1 -- 172.21.15.57:0/74271548 wait complete. 2023-10-08T14:01:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.443+0000 7f711aad5b80 20 client.26690 trim_cache size 0 max 16384 2023-10-08T14:01:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.469+0000 7f711aad5b80 1 Processor -- start 2023-10-08T14:01:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.470+0000 7f711aad5b80 1 -- start start 2023-10-08T14:01:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.470+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 0x562ef229ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.470+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef22a0880 0x562ef2295790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.470+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef2295cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.470+0000 7f711aad5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef206e8c0 con 0x562ef229b0b0 2023-10-08T14:01:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.470+0000 7f711aad5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef21b5e50 con 0x562ef22a11a0 2023-10-08T14:01:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.470+0000 7f711aad5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef22a5a70 con 0x562ef22a0880 2023-10-08T14:01:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f710ce89700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef2295cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f710ce89700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef2295cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:32900/0 (socket says 172.21.15.57:32900) 2023-10-08T14:01:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f710ce89700 1 -- 172.21.15.57:0/286892415 learned_addr learned my addr 172.21.15.57:0/286892415 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f71077fe700 1 --2- 172.21.15.57:0/286892415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef22a0880 0x562ef2295790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f710ce89700 1 -- 172.21.15.57:0/286892415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef22a0880 msgr2=0x562ef2295790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f710ce89700 1 --2- 172.21.15.57:0/286892415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef22a0880 0x562ef2295790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f710ce89700 1 -- 172.21.15.57:0/286892415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 msgr2=0x562ef229ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f710ce89700 1 --2- 172.21.15.57:0/286892415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 0x562ef229ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f710ce89700 1 -- 172.21.15.57:0/286892415 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ef203f1f0 con 0x562ef22a11a0 2023-10-08T14:01:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f710ce89700 1 --2- 172.21.15.57:0/286892415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef2295cd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000c00 tx=0x7f70f801aef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc8c543b4fad9a4b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.474+0000 7f70f6ffd700 1 -- 172.21.15.57:0/286892415 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f801c5d0 con 0x562ef22a11a0 2023-10-08T14:01:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.475+0000 7f70f6ffd700 1 -- 172.21.15.57:0/286892415 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f800f9a0 con 0x562ef22a11a0 2023-10-08T14:01:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.475+0000 7f70f6ffd700 1 -- 172.21.15.57:0/286892415 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f803b930 con 0x562ef22a11a0 2023-10-08T14:01:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.476+0000 7f711aad5b80 1 -- 172.21.15.57:0/286892415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 msgr2=0x562ef2295cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.476+0000 7f711aad5b80 1 --2- 172.21.15.57:0/286892415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef2295cd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000c00 tx=0x7f70f801aef0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.479+0000 7f711aad5b80 1 -- 172.21.15.57:0/286892415 shutdown_connections 2023-10-08T14:01:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.479+0000 7f711aad5b80 1 --2- 172.21.15.57:0/286892415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a11a0 0x562ef2295cd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.479+0000 7f711aad5b80 1 --2- 172.21.15.57:0/286892415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef229b0b0 0x562ef229ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.479+0000 7f711aad5b80 1 --2- 172.21.15.57:0/286892415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef22a0880 0x562ef2295790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.479+0000 7f711aad5b80 1 -- 172.21.15.57:0/286892415 >> 172.21.15.57:0/286892415 conn(0x562ef220f200 msgr2=0x562ef22a3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.482+0000 7f711aad5b80 1 -- 172.21.15.57:0/286892415 shutdown_connections 2023-10-08T14:01:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.486+0000 7f711aad5b80 1 -- 172.21.15.57:0/286892415 wait complete. 2023-10-08T14:01:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.486+0000 7f711aad5b80 1 -- start start 2023-10-08T14:01:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.487+0000 7f711aad5b80 1 Processor -- start 2023-10-08T14:01:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.487+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef229b0b0 0x562ef22a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.487+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 0x562ef22a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.487+0000 7f711aad5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef22a11a0 0x562ef22a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.487+0000 7f711aad5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef2318760 con 0x562ef22a11a0 2023-10-08T14:01:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.487+0000 7f711aad5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef23188a0 con 0x562ef22a0880 2023-10-08T14:01:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.487+0000 7f711aad5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef2318a60 con 0x562ef229b0b0 2023-10-08T14:01:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.487+0000 7f711aad5b80 10 client.0 _ll_register_callbacks cb 0x7f7100009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:01:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.490+0000 7f7107fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef229b0b0 0x562ef22a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.490+0000 7f7107fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef229b0b0 0x562ef22a7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60528/0 (socket says 172.21.15.57:60528) 2023-10-08T14:01:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.490+0000 7f7107fff700 1 -- 172.21.15.57:0/1870325736 learned_addr learned my addr 172.21.15.57:0/1870325736 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.490+0000 7f71077fe700 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 0x562ef22a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.490+0000 7f71077fe700 1 -- 172.21.15.57:0/1870325736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef229b0b0 msgr2=0x562ef22a7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.490+0000 7f71077fe700 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef229b0b0 0x562ef22a7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.490+0000 7f71077fe700 1 -- 172.21.15.57:0/1870325736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef22a11a0 msgr2=0x562ef22a8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.490+0000 7f71077fe700 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef22a11a0 0x562ef22a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.490+0000 7f71077fe700 1 -- 172.21.15.57:0/1870325736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ef2001c00 con 0x562ef22a0880 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.491+0000 7f71077fe700 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 0x562ef22a8130 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0cace0 tx=0x7f70fc09a020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7df4dec8b6cb0a8c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.492+0000 7f70eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.492+0000 7f70eb7fe700 1 -- 172.21.15.57:0/1870325736 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc09a5a0 con 0x562ef22a0880 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.492+0000 7f70eb7fe700 1 -- 172.21.15.57:0/1870325736 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f70fc09e6f0 con 0x562ef22a0880 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.492+0000 7f711aad5b80 1 -- 172.21.15.57:0/1870325736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70f8018460 con 0x562ef22a0880 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.492+0000 7f70eb7fe700 1 -- 172.21.15.57:0/1870325736 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70fc0a2930 con 0x562ef22a0880 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.492+0000 7f711aad5b80 10 client.26696 fetch_fsmap learned FSMap version 46 2023-10-08T14:01:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.492+0000 7f711aad5b80 1 -- 172.21.15.57:0/1870325736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x562ef2017250 con 0x562ef22a0880 2023-10-08T14:01:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.499+0000 7f70eb7fe700 1 -- 172.21.15.57:0/1870325736 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70fc0b6030 con 0x562ef22a0880 2023-10-08T14:01:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.499+0000 7f70eb7fe700 1 -- 172.21.15.57:0/1870325736 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f70fc099a90 con 0x562ef22a0880 2023-10-08T14:01:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.499+0000 7f711aad5b80 10 client.26696 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:01:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.499+0000 7f711aad5b80 10 client.26696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:01:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.499+0000 7f711aad5b80 20 client.26696 populate_metadata read hostname 'smithi057' 2023-10-08T14:01:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.499+0000 7f711aad5b80 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562ef231a360 0x562ef231c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.499+0000 7f711aad5b80 4 client.26696 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:01:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.499+0000 7f711aad5b80 1 -- 172.21.15.57:0/1870325736 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ef21e6ad0 con 0x562ef231a360 2023-10-08T14:01:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.499+0000 7f7107fff700 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562ef231a360 0x562ef231c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.500+0000 7f7107fff700 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562ef231a360 0x562ef231c780 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4acb5b53ae86bc4 server_cookie=20bf5b1462ec34ab in_seq=0 out_seq=0 2023-10-08T14:01:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.500+0000 7f70eb7fe700 10 client.26696 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.535+0000 7f70eb7fe700 1 -- 172.21.15.57:0/1870325736 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ef21e6ad0 con 0x562ef231a360 2023-10-08T14:01:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.535+0000 7f70eb7fe700 10 client.26696 handle_command_reply: tid=0 2023-10-08T14:01:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.538+0000 7f711aad5b80 1 client.26696 shutdown 2023-10-08T14:01:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.539+0000 7f711aad5b80 1 -- 172.21.15.57:0/1870325736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 msgr2=0x562ef22a8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.539+0000 7f711aad5b80 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 0x562ef22a8130 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0cace0 tx=0x7f70fc09a020 comp rx=0 tx=0).stop 2023-10-08T14:01:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.544+0000 7f711aad5b80 1 -- 172.21.15.57:0/1870325736 shutdown_connections 2023-10-08T14:01:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.544+0000 7f711aad5b80 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x562ef231a360 0x562ef231c780 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.544+0000 7f711aad5b80 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562ef22a11a0 0x562ef22a8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.544+0000 7f711aad5b80 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562ef229b0b0 0x562ef22a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.544+0000 7f711aad5b80 1 --2- 172.21.15.57:0/1870325736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef22a0880 0x562ef22a8130 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.544+0000 7f711aad5b80 1 -- 172.21.15.57:0/1870325736 >> 172.21.15.57:0/1870325736 conn(0x562ef220f200 msgr2=0x562ef229f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.544+0000 7f70eb7fe700 0 client.26696 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.545+0000 7f711aad5b80 1 -- 172.21.15.57:0/1870325736 shutdown_connections 2023-10-08T14:01:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.545+0000 7f711aad5b80 1 -- 172.21.15.57:0/1870325736 wait complete. 2023-10-08T14:01:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.550+0000 7f711aad5b80 20 client.26696 trim_cache size 0 max 16384 2023-10-08T14:01:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:01:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:01:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:01:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:01:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.554+0000 7f710e8ec700 1 -- 172.21.15.57:0/2495627079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70f00a1800 msgr2=0x7f70f00a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.554+0000 7f710e8ec700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70f00a1800 0x7f70f00a3cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7100009fd0 tx=0x7f710000d040 comp rx=0 tx=0).stop 2023-10-08T14:01:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.554+0000 7f710e8ec700 1 -- 172.21.15.57:0/2495627079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7108105680 msgr2=0x7f710810fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.554+0000 7f710e8ec700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7108105680 0x7f710810fac0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00b450 tx=0x7f70fc00b760 comp rx=0 tx=0).stop 2023-10-08T14:01:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.557+0000 7f710e8ec700 1 -- 172.21.15.57:0/2495627079 shutdown_connections 2023-10-08T14:01:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.557+0000 7f710e8ec700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7108110000 0x7f71081104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.557+0000 7f710e8ec700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7108067b20 0x7f7108113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.557+0000 7f710e8ec700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f70f00a1800 0x7f70f00a3cc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.557+0000 7f710e8ec700 1 --2- 172.21.15.57:0/2495627079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7108105680 0x7f710810fac0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.557+0000 7f710e8ec700 1 -- 172.21.15.57:0/2495627079 >> 172.21.15.57:0/2495627079 conn(0x7f7108076790 msgr2=0x7f71080fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.559+0000 7f710e8ec700 1 -- 172.21.15.57:0/2495627079 shutdown_connections 2023-10-08T14:01:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:38.559+0000 7f710e8ec700 1 -- 172.21.15.57:0/2495627079 wait complete. 2023-10-08T14:01:39.688 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:39 smithi115 ceph-mon[104408]: pgmap v610: 129 pgs: 32 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 86 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 883 KiB/s wr, 129 op/s 2023-10-08T14:01:39.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:39 smithi057 ceph-mon[99476]: pgmap v610: 129 pgs: 32 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 86 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 883 KiB/s wr, 129 op/s 2023-10-08T14:01:39.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:39 smithi099 ceph-mon[103849]: pgmap v610: 129 pgs: 32 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 86 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 883 KiB/s wr, 129 op/s 2023-10-08T14:01:41.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:41 smithi057 ceph-mon[99476]: pgmap v611: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 139 op/s 2023-10-08T14:01:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:41 smithi099 ceph-mon[103849]: pgmap v611: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 139 op/s 2023-10-08T14:01:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:41 smithi115 ceph-mon[104408]: pgmap v611: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 139 op/s 2023-10-08T14:01:44.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:43 smithi099 ceph-mon[103849]: pgmap v612: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1017 KiB/s wr, 132 op/s 2023-10-08T14:01:44.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:43 smithi115 ceph-mon[104408]: pgmap v612: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1017 KiB/s wr, 132 op/s 2023-10-08T14:01:44.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:43 smithi057 ceph-mon[99476]: pgmap v612: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1017 KiB/s wr, 132 op/s 2023-10-08T14:01:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:45 smithi115 ceph-mon[104408]: pgmap v613: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 147 op/s 2023-10-08T14:01:46.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:45 smithi057 ceph-mon[99476]: pgmap v613: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 147 op/s 2023-10-08T14:01:46.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:45 smithi099 ceph-mon[103849]: pgmap v613: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 147 op/s 2023-10-08T14:01:47.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:46 smithi115 ceph-mon[104408]: pgmap v614: 129 pgs: 7 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 116 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-10-08T14:01:47.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:46 smithi057 ceph-mon[99476]: pgmap v614: 129 pgs: 7 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 116 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-10-08T14:01:47.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:46 smithi099 ceph-mon[103849]: pgmap v614: 129 pgs: 7 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 116 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-10-08T14:01:49.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:49 smithi115 ceph-mon[104408]: pgmap v615: 129 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 118 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-10-08T14:01:49.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:49 smithi057 ceph-mon[99476]: pgmap v615: 129 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 118 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-10-08T14:01:49.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:49 smithi099 ceph-mon[103849]: pgmap v615: 129 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 118 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 151 op/s 2023-10-08T14:01:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.732+0000 7f9f07f47700 1 -- 172.21.15.57:0/1573638586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f000ff5f0 msgr2=0x7f9f000ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.732+0000 7f9f07f47700 1 --2- 172.21.15.57:0/1573638586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f000ff5f0 0x7f9f000ffa70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009990 tx=0x7f9efc009ca0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.741+0000 7f9f07f47700 1 -- 172.21.15.57:0/1573638586 shutdown_connections 2023-10-08T14:01:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.741+0000 7f9f07f47700 1 --2- 172.21.15.57:0/1573638586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f000fffb0 0x7f9f0010c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.741+0000 7f9f07f47700 1 --2- 172.21.15.57:0/1573638586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f00105690 0x7f9f00105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.741+0000 7f9f07f47700 1 --2- 172.21.15.57:0/1573638586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f000ff5f0 0x7f9f000ffa70 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.741+0000 7f9f07f47700 1 -- 172.21.15.57:0/1573638586 >> 172.21.15.57:0/1573638586 conn(0x7f9f00076790 msgr2=0x7f9f00076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.743+0000 7f9f07f47700 1 -- 172.21.15.57:0/1573638586 shutdown_connections 2023-10-08T14:01:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.743+0000 7f9f07f47700 1 -- 172.21.15.57:0/1573638586 wait complete. 2023-10-08T14:01:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.745+0000 7f9f07f47700 1 Processor -- start 2023-10-08T14:01:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.745+0000 7f9f07f47700 1 -- start start 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f07f47700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f000ff5f0 0x7f9f0019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f07f47700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f000fffb0 0x7f9f0019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f07f47700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f00105690 0x7f9f001969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f07f47700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f00197010 con 0x7f9f00105690 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f07f47700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f00197180 con 0x7f9f000ff5f0 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f07f47700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f00197340 con 0x7f9f000fffb0 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f054e2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f000fffb0 0x7f9f0019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f054e2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f000fffb0 0x7f9f0019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60540/0 (socket says 172.21.15.57:60540) 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f054e2700 1 -- 172.21.15.57:0/3229448722 learned_addr learned my addr 172.21.15.57:0/3229448722 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f054e2700 1 -- 172.21.15.57:0/3229448722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f000ff5f0 msgr2=0x7f9f0019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f054e2700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f000ff5f0 0x7f9f0019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f054e2700 1 -- 172.21.15.57:0/3229448722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f00105690 msgr2=0x7f9f001969b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:01:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.746+0000 7f9f054e2700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f00105690 0x7f9f001969b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.747+0000 7f9f054e2700 1 -- 172.21.15.57:0/3229448722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef0009520 con 0x7f9f000fffb0 2023-10-08T14:01:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.747+0000 7f9f054e2700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f000fffb0 0x7f9f0019cdc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9efc005720 tx=0x7f9efc005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.747+0000 7f9ef6ffd700 1 -- 172.21.15.57:0/3229448722 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc01d070 con 0x7f9f000fffb0 2023-10-08T14:01:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.747+0000 7f9ef6ffd700 1 -- 172.21.15.57:0/3229448722 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9efc022470 con 0x7f9f000fffb0 2023-10-08T14:01:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.747+0000 7f9ef6ffd700 1 -- 172.21.15.57:0/3229448722 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc017650 con 0x7f9f000fffb0 2023-10-08T14:01:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.747+0000 7f9f07f47700 1 -- 172.21.15.57:0/3229448722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc0095f0 con 0x7f9f000fffb0 2023-10-08T14:01:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.747+0000 7f9f07f47700 1 -- 172.21.15.57:0/3229448722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f001a3690 con 0x7f9f000fffb0 2023-10-08T14:01:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.752+0000 7f9f14130b80 1 Processor -- start 2023-10-08T14:01:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.752+0000 7f9f14130b80 1 -- start start 2023-10-08T14:01:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 0x556c78af5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 0x556c78b01d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78b02710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f14130b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c78a5d400 con 0x556c78af5680 2023-10-08T14:01:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f14130b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c78a4ebc0 con 0x556c78b022b0 2023-10-08T14:01:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f14130b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c78a4c4e0 con 0x556c78af4d60 2023-10-08T14:01:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9ef6ffd700 1 -- 172.21.15.57:0/3229448722 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9efc0177b0 con 0x7f9f000fffb0 2023-10-08T14:01:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f064e4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 0x556c78af5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f064e4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 0x556c78af5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60542/0 (socket says 172.21.15.57:60542) 2023-10-08T14:01:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f064e4700 1 -- 172.21.15.57:0/443652638 learned_addr learned my addr 172.21.15.57:0/443652638 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f054e2700 1 --2- 172.21.15.57:0/443652638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78b02710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f064e4700 1 -- 172.21.15.57:0/443652638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 msgr2=0x556c78b02710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f064e4700 1 --2- 172.21.15.57:0/443652638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78b02710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f064e4700 1 -- 172.21.15.57:0/443652638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 msgr2=0x556c78b01d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f064e4700 1 --2- 172.21.15.57:0/443652638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 0x556c78b01d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.753+0000 7f9f064e4700 1 -- 172.21.15.57:0/443652638 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c78872d80 con 0x556c78af4d60 2023-10-08T14:01:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.754+0000 7f9ef6ffd700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9eec0a15f0 0x7f9eec0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.754+0000 7f9ef6ffd700 1 -- 172.21.15.57:0/3229448722 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f9efc0ce8c0 con 0x7f9f000fffb0 2023-10-08T14:01:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.754+0000 7f9f05ce3700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9eec0a15f0 0x7f9eec0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.754+0000 7f9f064e4700 1 --2- 172.21.15.57:0/443652638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 0x556c78af5140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009d00 tx=0x7f9ef800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2e84e3dd75518 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.755+0000 7f9f05ce3700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9eec0a15f0 0x7f9eec0a3ab0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9f00100730 tx=0x7f9ef0005d50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.755+0000 7f9eebfff700 1 -- 172.21.15.57:0/443652638 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800e070 con 0x556c78af4d60 2023-10-08T14:01:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.755+0000 7f9eebfff700 1 -- 172.21.15.57:0/443652638 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ef8009830 con 0x556c78af4d60 2023-10-08T14:01:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.755+0000 7f9eebfff700 1 -- 172.21.15.57:0/443652638 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8011390 con 0x556c78af4d60 2023-10-08T14:01:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.756+0000 7f9f14130b80 1 -- 172.21.15.57:0/443652638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 msgr2=0x556c78af5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.756+0000 7f9f14130b80 1 --2- 172.21.15.57:0/443652638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 0x556c78af5140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009d00 tx=0x7f9ef800b330 comp rx=0 tx=0).stop 2023-10-08T14:01:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.766+0000 7f9f14130b80 1 -- 172.21.15.57:0/443652638 shutdown_connections 2023-10-08T14:01:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.766+0000 7f9f14130b80 1 --2- 172.21.15.57:0/443652638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78b02710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.766+0000 7f9f14130b80 1 --2- 172.21.15.57:0/443652638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 0x556c78af5140 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.766+0000 7f9f14130b80 1 --2- 172.21.15.57:0/443652638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 0x556c78b01d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.766+0000 7f9f14130b80 1 -- 172.21.15.57:0/443652638 >> 172.21.15.57:0/443652638 conn(0x556c78a69200 msgr2=0x556c78aefd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.766+0000 7f9f14130b80 1 -- 172.21.15.57:0/443652638 shutdown_connections 2023-10-08T14:01:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.769+0000 7f9f14130b80 1 -- 172.21.15.57:0/443652638 wait complete. 2023-10-08T14:01:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.769+0000 7f9f14130b80 1 -- start start 2023-10-08T14:01:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.769+0000 7f9f14130b80 1 Processor -- start 2023-10-08T14:01:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 0x556c78b09a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 0x556c78b06d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78b07470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f14130b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c78a47dd0 con 0x556c78af5680 2023-10-08T14:01:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f14130b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c789e46b0 con 0x556c78b022b0 2023-10-08T14:01:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f14130b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c789e4350 con 0x556c78af4d60 2023-10-08T14:01:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f14130b80 10 client.0 _ll_register_callbacks cb 0x556c788417a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:01:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f054e2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 0x556c78b06d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f054e2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 0x556c78b06d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59738/0 (socket says 172.21.15.57:59738) 2023-10-08T14:01:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f054e2700 1 -- 172.21.15.57:0/2077595519 learned_addr learned my addr 172.21.15.57:0/2077595519 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f064e4700 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78b07470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f054e2700 1 -- 172.21.15.57:0/2077595519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 msgr2=0x556c78b09a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:01:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f054e2700 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 0x556c78b09a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f054e2700 1 -- 172.21.15.57:0/2077595519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 msgr2=0x556c78b07470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f054e2700 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78b07470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.771+0000 7f9f054e2700 1 -- 172.21.15.57:0/2077595519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef80099b0 con 0x556c78af5680 2023-10-08T14:01:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.772+0000 7f9f054e2700 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 0x556c78b06d30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0cbcc0 tx=0x7f9efc0972b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43be4acb75490d69 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.773+0000 7f9ee97fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:01:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.773+0000 7f9ee97fa700 1 -- 172.21.15.57:0/2077595519 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc09b350 con 0x556c78af5680 2023-10-08T14:01:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.773+0000 7f9ee97fa700 1 -- 172.21.15.57:0/2077595519 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f9efc097040 con 0x556c78af5680 2023-10-08T14:01:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.773+0000 7f9f14130b80 1 -- 172.21.15.57:0/2077595519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c78a40db0 con 0x556c78af5680 2023-10-08T14:01:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.773+0000 7f9ee97fa700 1 -- 172.21.15.57:0/2077595519 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9efc09b0c0 con 0x556c78af5680 2023-10-08T14:01:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.776+0000 7f9f14130b80 10 client.17160 fetch_fsmap learned FSMap version 46 2023-10-08T14:01:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.776+0000 7f9f14130b80 1 -- 172.21.15.57:0/2077595519 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556c7885bc00 con 0x556c78af5680 2023-10-08T14:01:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.777+0000 7f9ee97fa700 1 -- 172.21.15.57:0/2077595519 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c78a40db0 con 0x556c78af5680 2023-10-08T14:01:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.778+0000 7f9ee97fa700 1 -- 172.21.15.57:0/2077595519 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f9efc0a33f0 con 0x556c78af5680 2023-10-08T14:01:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.778+0000 7f9f14130b80 10 client.17160 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:01:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.778+0000 7f9f14130b80 10 client.17160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:01:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.778+0000 7f9f14130b80 20 client.17160 populate_metadata read hostname 'smithi057' 2023-10-08T14:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.778+0000 7f9f14130b80 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556c78af9be0 0x556c78afc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.778+0000 7f9f14130b80 4 client.17160 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.778+0000 7f9f14130b80 1 -- 172.21.15.57:0/2077595519 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c78a40ad0 con 0x556c78af9be0 2023-10-08T14:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.780+0000 7f9f05ce3700 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556c78af9be0 0x556c78afc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.783+0000 7f9ee97fa700 10 client.17160 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.783+0000 7f9f05ce3700 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556c78af9be0 0x556c78afc000 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa1efea0294bb63 server_cookie=e827d2e27c709d3e in_seq=0 out_seq=0 2023-10-08T14:01:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.784+0000 7f9ee97fa700 1 -- 172.21.15.57:0/2077595519 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556c78a40ad0 con 0x556c78af9be0 2023-10-08T14:01:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.784+0000 7f9ee97fa700 10 client.17160 handle_command_reply: tid=0 2023-10-08T14:01:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.787+0000 7f9f14130b80 1 client.17160 shutdown 2023-10-08T14:01:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.787+0000 7f9f14130b80 1 -- 172.21.15.57:0/2077595519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 msgr2=0x556c78b06d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.787+0000 7f9f14130b80 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 0x556c78b06d30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0cbcc0 tx=0x7f9efc0972b0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.790+0000 7f9f14130b80 1 -- 172.21.15.57:0/2077595519 shutdown_connections 2023-10-08T14:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.790+0000 7f9f14130b80 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556c78af9be0 0x556c78afc000 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.790+0000 7f9f14130b80 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78b07470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.790+0000 7f9f14130b80 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af4d60 0x556c78b09a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.790+0000 7f9f14130b80 1 --2- 172.21.15.57:0/2077595519 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af5680 0x556c78b06d30 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.790+0000 7f9ee97fa700 0 client.17160 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.790+0000 7f9f14130b80 1 -- 172.21.15.57:0/2077595519 >> 172.21.15.57:0/2077595519 conn(0x556c78a69200 msgr2=0x556c78b036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.790+0000 7f9f14130b80 1 -- 172.21.15.57:0/2077595519 shutdown_connections 2023-10-08T14:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.790+0000 7f9f14130b80 1 -- 172.21.15.57:0/2077595519 wait complete. 2023-10-08T14:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.790+0000 7f9f14130b80 20 client.17160 trim_cache size 0 max 16384 2023-10-08T14:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.807+0000 7f9f14130b80 1 Processor -- start 2023-10-08T14:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.807+0000 7f9f14130b80 1 -- start start 2023-10-08T14:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.807+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78af4d60 0x556c78b0e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.807+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 0x556c78aee780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.807+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78b022b0 0x556c78aef000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.807+0000 7f9f14130b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c788c88c0 con 0x556c78b022b0 2023-10-08T14:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.807+0000 7f9f14130b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c78aea2b0 con 0x556c78af4d60 2023-10-08T14:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.807+0000 7f9f14130b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c78b07c80 con 0x556c78af5680 2023-10-08T14:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f064e4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78b022b0 0x556c78aef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f054e2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 0x556c78aee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f064e4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78b022b0 0x556c78aef000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59750/0 (socket says 172.21.15.57:59750) 2023-10-08T14:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f064e4700 1 -- 172.21.15.57:0/2557379106 learned_addr learned my addr 172.21.15.57:0/2557379106 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f05ce3700 1 --2- 172.21.15.57:0/2557379106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78af4d60 0x556c78b0e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f064e4700 1 -- 172.21.15.57:0/2557379106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 msgr2=0x556c78aee780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f064e4700 1 --2- 172.21.15.57:0/2557379106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 0x556c78aee780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f064e4700 1 -- 172.21.15.57:0/2557379106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78af4d60 msgr2=0x556c78b0e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f064e4700 1 --2- 172.21.15.57:0/2557379106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78af4d60 0x556c78b0e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f064e4700 1 -- 172.21.15.57:0/2557379106 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c788991f0 con 0x556c78b022b0 2023-10-08T14:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9f064e4700 1 --2- 172.21.15.57:0/2557379106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78b022b0 0x556c78aef000 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x556c78af5c40 tx=0x7f9ef8009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170d9fdb22bd9f9a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9ef4ff9700 1 -- 172.21.15.57:0/2557379106 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800e040 con 0x556c78b022b0 2023-10-08T14:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9ef4ff9700 1 -- 172.21.15.57:0/2557379106 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ef801f070 con 0x556c78b022b0 2023-10-08T14:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.808+0000 7f9ef4ff9700 1 -- 172.21.15.57:0/2557379106 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef801b650 con 0x556c78b022b0 2023-10-08T14:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.809+0000 7f9f14130b80 1 -- 172.21.15.57:0/2557379106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78b022b0 msgr2=0x556c78aef000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.809+0000 7f9f14130b80 1 --2- 172.21.15.57:0/2557379106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78b022b0 0x556c78aef000 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x556c78af5c40 tx=0x7f9ef8009e50 comp rx=0 tx=0).stop 2023-10-08T14:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.809+0000 7f9f14130b80 1 -- 172.21.15.57:0/2557379106 shutdown_connections 2023-10-08T14:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.809+0000 7f9f14130b80 1 --2- 172.21.15.57:0/2557379106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78b022b0 0x556c78aef000 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.809+0000 7f9f14130b80 1 --2- 172.21.15.57:0/2557379106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78af4d60 0x556c78b0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.809+0000 7f9f14130b80 1 --2- 172.21.15.57:0/2557379106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 0x556c78aee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.809+0000 7f9f14130b80 1 -- 172.21.15.57:0/2557379106 >> 172.21.15.57:0/2557379106 conn(0x556c78a69200 msgr2=0x556c78aebc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.809+0000 7f9f14130b80 1 -- 172.21.15.57:0/2557379106 shutdown_connections 2023-10-08T14:01:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.809+0000 7f9f14130b80 1 -- 172.21.15.57:0/2557379106 wait complete. 2023-10-08T14:01:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.809+0000 7f9f14130b80 1 -- start start 2023-10-08T14:01:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f14130b80 1 Processor -- start 2023-10-08T14:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af4d60 0x556c78afdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 0x556c78afe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f14130b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78afc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f14130b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c78afce80 con 0x556c78af4d60 2023-10-08T14:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f14130b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c78afcfc0 con 0x556c78b022b0 2023-10-08T14:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f14130b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c78afd180 con 0x556c78af5680 2023-10-08T14:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f14130b80 10 client.0 _ll_register_callbacks cb 0x556c788417a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f05ce3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af4d60 0x556c78afdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f05ce3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af4d60 0x556c78afdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59752/0 (socket says 172.21.15.57:59752) 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f064e4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78afc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f054e2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 0x556c78afe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f05ce3700 1 -- 172.21.15.57:0/160500315 learned_addr learned my addr 172.21.15.57:0/160500315 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f054e2700 1 -- 172.21.15.57:0/160500315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 msgr2=0x556c78afc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f054e2700 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78afc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f054e2700 1 -- 172.21.15.57:0/160500315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af4d60 msgr2=0x556c78afdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f054e2700 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af4d60 0x556c78afdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.810+0000 7f9f054e2700 1 -- 172.21.15.57:0/160500315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c78872d80 con 0x556c78af5680 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9f054e2700 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 0x556c78afe050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00f740 tx=0x7f9efc009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=128a3c1d98b8cdb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:01:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9ee97fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9ee97fa700 1 -- 172.21.15.57:0/160500315 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc097570 con 0x556c78af5680 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9f14130b80 1 -- 172.21.15.57:0/160500315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c78a0fe50 con 0x556c78af5680 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9ee97fa700 1 -- 172.21.15.57:0/160500315 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f9efc0976d0 con 0x556c78af5680 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9ee97fa700 1 -- 172.21.15.57:0/160500315 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9efc09b0c0 con 0x556c78af5680 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9f14130b80 10 client.26734 fetch_fsmap learned FSMap version 46 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9f14130b80 1 -- 172.21.15.57:0/160500315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556c78871250 con 0x556c78af5680 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9ee97fa700 1 -- 172.21.15.57:0/160500315 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c78a0fe50 con 0x556c78af5680 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9ee97fa700 1 -- 172.21.15.57:0/160500315 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f9efc09a4e0 con 0x556c78af5680 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9f14130b80 10 client.26734 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9f14130b80 10 client.26734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:01:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9f14130b80 20 client.26734 populate_metadata read hostname 'smithi057' 2023-10-08T14:01:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9f14130b80 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556c78b74110 0x556c78b76530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:01:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9f14130b80 4 client.26734 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:01:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.811+0000 7f9f14130b80 1 -- 172.21.15.57:0/160500315 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556c78893f10 con 0x556c78b74110 2023-10-08T14:01:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.812+0000 7f9f05ce3700 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556c78b74110 0x556c78b76530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:01:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.812+0000 7f9ee97fa700 10 client.26734 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.812+0000 7f9f05ce3700 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556c78b74110 0x556c78b76530 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f2ace0c89d0a8cb server_cookie=561445ecc1a5dc67 in_seq=0 out_seq=0 2023-10-08T14:01:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9ee97fa700 1 -- 172.21.15.57:0/160500315 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556c78893f10 con 0x556c78b74110 2023-10-08T14:01:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9ee97fa700 10 client.26734 handle_command_reply: tid=0 2023-10-08T14:01:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 client.26734 shutdown 2023-10-08T14:01:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 -- 172.21.15.57:0/160500315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 msgr2=0x556c78afe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 0x556c78afe050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00f740 tx=0x7f9efc009910 comp rx=0 tx=0).stop 2023-10-08T14:01:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 -- 172.21.15.57:0/160500315 shutdown_connections 2023-10-08T14:01:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556c78b74110 0x556c78b76530 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c78b022b0 0x556c78afc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9ee97fa700 0 client.26734 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:01:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556c78af4d60 0x556c78afdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 --2- 172.21.15.57:0/160500315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c78af5680 0x556c78afe050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 -- 172.21.15.57:0/160500315 >> 172.21.15.57:0/160500315 conn(0x556c78a69200 msgr2=0x556c78b01370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 -- 172.21.15.57:0/160500315 shutdown_connections 2023-10-08T14:01:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 1 -- 172.21.15.57:0/160500315 wait complete. 2023-10-08T14:01:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.821+0000 7f9f14130b80 20 client.26734 trim_cache size 0 max 16384 2023-10-08T14:01:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:01:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:01:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:01:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:01:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 -- 172.21.15.57:0/3229448722 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9eec0a15f0 msgr2=0x7f9eec0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9eec0a15f0 0x7f9eec0a3ab0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9f00100730 tx=0x7f9ef0005d50 comp rx=0 tx=0).stop 2023-10-08T14:01:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 -- 172.21.15.57:0/3229448722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f000fffb0 msgr2=0x7f9f0019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:01:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f000fffb0 0x7f9f0019cdc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9efc005720 tx=0x7f9efc005800 comp rx=0 tx=0).stop 2023-10-08T14:01:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 -- 172.21.15.57:0/3229448722 shutdown_connections 2023-10-08T14:01:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f00105690 0x7f9f001969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9eec0a15f0 0x7f9eec0a3ab0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f000ff5f0 0x7f9f0019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 --2- 172.21.15.57:0/3229448722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f000fffb0 0x7f9f0019cdc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:01:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 -- 172.21.15.57:0/3229448722 >> 172.21.15.57:0/3229448722 conn(0x7f9f00076790 msgr2=0x7f9f000fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:01:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 -- 172.21.15.57:0/3229448722 shutdown_connections 2023-10-08T14:01:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:01:51.822+0000 7f9f07f47700 1 -- 172.21.15.57:0/3229448722 wait complete. 2023-10-08T14:01:51.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:51 smithi057 ceph-mon[99476]: pgmap v616: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2023-10-08T14:01:52.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:51 smithi099 ceph-mon[103849]: pgmap v616: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2023-10-08T14:01:52.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:51 smithi115 ceph-mon[104408]: pgmap v616: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2023-10-08T14:01:54.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:53 smithi099 ceph-mon[103849]: pgmap v617: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2023-10-08T14:01:54.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:53 smithi115 ceph-mon[104408]: pgmap v617: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2023-10-08T14:01:54.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:53 smithi057 ceph-mon[99476]: pgmap v617: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2023-10-08T14:01:56.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:55 smithi099 ceph-mon[103849]: pgmap v618: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2023-10-08T14:01:56.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:55 smithi115 ceph-mon[104408]: pgmap v618: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2023-10-08T14:01:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:55 smithi057 ceph-mon[99476]: pgmap v618: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2023-10-08T14:01:58.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:57 smithi099 ceph-mon[103849]: pgmap v619: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-10-08T14:01:58.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:57 smithi115 ceph-mon[104408]: pgmap v619: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-10-08T14:01:58.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:57 smithi057 ceph-mon[99476]: pgmap v619: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-10-08T14:01:59.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:59 smithi115 ceph-mon[104408]: osdmap e101: 12 total, 12 up, 12 in 2023-10-08T14:01:59.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:01:59 smithi115 ceph-mon[104408]: pgmap v621: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2023-10-08T14:01:59.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:59 smithi057 ceph-mon[99476]: osdmap e101: 12 total, 12 up, 12 in 2023-10-08T14:01:59.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:01:59 smithi057 ceph-mon[99476]: pgmap v621: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2023-10-08T14:01:59.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:59 smithi099 ceph-mon[103849]: osdmap e101: 12 total, 12 up, 12 in 2023-10-08T14:01:59.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:01:59 smithi099 ceph-mon[103849]: pgmap v621: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 208 op/s 2023-10-08T14:02:01.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:01 smithi057 ceph-mon[99476]: pgmap v622: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2023-10-08T14:02:02.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:01 smithi099 ceph-mon[103849]: pgmap v622: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2023-10-08T14:02:02.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:01 smithi115 ceph-mon[104408]: pgmap v622: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2023-10-08T14:02:04.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:03 smithi099 ceph-mon[103849]: pgmap v623: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-10-08T14:02:04.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:03 smithi115 ceph-mon[104408]: pgmap v623: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-10-08T14:02:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:03 smithi057 ceph-mon[99476]: pgmap v623: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2023-10-08T14:02:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.946+0000 7f66b48ca700 1 -- 172.21.15.57:0/1860256377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66b00fff90 msgr2=0x7f66b010c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.946+0000 7f66b48ca700 1 --2- 172.21.15.57:0/1860256377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66b00fff90 0x7f66b010c490 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f66a800b0a0 tx=0x7f66a800b3b0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.946+0000 7f66b48ca700 1 -- 172.21.15.57:0/1860256377 shutdown_connections 2023-10-08T14:02:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.946+0000 7f66b48ca700 1 --2- 172.21.15.57:0/1860256377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66b00fff90 0x7f66b010c490 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.946+0000 7f66b48ca700 1 --2- 172.21.15.57:0/1860256377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b01045e0 0x7f66b01049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.946+0000 7f66b48ca700 1 --2- 172.21.15.57:0/1860256377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b00ff5d0 0x7f66b00ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.946+0000 7f66b48ca700 1 -- 172.21.15.57:0/1860256377 >> 172.21.15.57:0/1860256377 conn(0x7f66b0076790 msgr2=0x7f66b0076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.946+0000 7f66b48ca700 1 -- 172.21.15.57:0/1860256377 shutdown_connections 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.946+0000 7f66b48ca700 1 -- 172.21.15.57:0/1860256377 wait complete. 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66b48ca700 1 Processor -- start 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66b48ca700 1 -- start start 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66b48ca700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66b00ff5d0 0x7f66b019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66b48ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b00fff90 0x7f66b019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66b48ca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b01045e0 0x7f66b0196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66b48ca700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b0197090 con 0x7f66b00ff5d0 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66b48ca700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b0197200 con 0x7f66b01045e0 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66b48ca700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b01973c0 con 0x7f66b00fff90 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66b00ff5d0 0x7f66b019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66b00ff5d0 0x7f66b019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59766/0 (socket says 172.21.15.57:59766) 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.947+0000 7f66ae59c700 1 -- 172.21.15.57:0/2271512936 learned_addr learned my addr 172.21.15.57:0/2271512936 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66add9b700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b00fff90 0x7f66b019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66aed9d700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b01045e0 0x7f66b0196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66ae59c700 1 -- 172.21.15.57:0/2271512936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b00fff90 msgr2=0x7f66b019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66ae59c700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b00fff90 0x7f66b019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66ae59c700 1 -- 172.21.15.57:0/2271512936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b01045e0 msgr2=0x7f66b0196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66ae59c700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b01045e0 0x7f66b0196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66ae59c700 1 -- 172.21.15.57:0/2271512936 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a8009d00 con 0x7f66b00ff5d0 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66add9b700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b00fff90 0x7f66b019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66aed9d700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b01045e0 0x7f66b0196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T14:02:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66ae59c700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66b00ff5d0 0x7f66b019c900 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f66a400e970 tx=0x7f66a400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f669f7fe700 1 -- 172.21.15.57:0/2271512936 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4009790 con 0x7f66b00ff5d0 2023-10-08T14:02:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f669f7fe700 1 -- 172.21.15.57:0/2271512936 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a40044e0 con 0x7f66b00ff5d0 2023-10-08T14:02:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66b48ca700 1 -- 172.21.15.57:0/2271512936 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b01976a0 con 0x7f66b00ff5d0 2023-10-08T14:02:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f669f7fe700 1 -- 172.21.15.57:0/2271512936 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4003e30 con 0x7f66b00ff5d0 2023-10-08T14:02:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.948+0000 7f66b48ca700 1 -- 172.21.15.57:0/2271512936 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b01a38f0 con 0x7f66b00ff5d0 2023-10-08T14:02:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.950+0000 7f669f7fe700 1 -- 172.21.15.57:0/2271512936 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f66a4003f90 con 0x7f66b00ff5d0 2023-10-08T14:02:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.951+0000 7f669f7fe700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f66980a15f0 0x7f66980a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.951+0000 7f669f7fe700 1 -- 172.21.15.57:0/2271512936 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f66a4014070 con 0x7f66b00ff5d0 2023-10-08T14:02:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.951+0000 7f66add9b700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f66980a15f0 0x7f66980a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.951+0000 7f66add9b700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f66980a15f0 0x7f66980a3ab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f66a0005fd0 tx=0x7f66a0005ee0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66c0ab3b80 1 Processor -- start 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66c0ab3b80 1 -- start start 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91170 0x56242ed91550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 0x56242ed95920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 0x56242ed962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242ecf5400 con 0x56242ed91170 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242ece6bc0 con 0x56242ed91a90 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242ece44e0 con 0x56242ed95e60 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91170 0x56242ed91550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66ae59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91170 0x56242ed91550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59774/0 (socket says 172.21.15.57:59774) 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.952+0000 7f66ae59c700 1 -- 172.21.15.57:0/2925002404 learned_addr learned my addr 172.21.15.57:0/2925002404 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f66add9b700 1 --2- 172.21.15.57:0/2925002404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 0x56242ed95920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f66aed9d700 1 --2- 172.21.15.57:0/2925002404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 0x56242ed962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f66ae59c700 1 -- 172.21.15.57:0/2925002404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 msgr2=0x56242ed962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f66ae59c700 1 --2- 172.21.15.57:0/2925002404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 0x56242ed962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f66ae59c700 1 -- 172.21.15.57:0/2925002404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 msgr2=0x56242ed95920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f66ae59c700 1 --2- 172.21.15.57:0/2925002404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 0x56242ed95920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f66ae59c700 1 -- 172.21.15.57:0/2925002404 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56242eb0ad80 con 0x56242ed91170 2023-10-08T14:02:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f66add9b700 1 --2- 172.21.15.57:0/2925002404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 0x56242ed95920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T14:02:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f66ae59c700 1 --2- 172.21.15.57:0/2925002404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91170 0x56242ed91550 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f66a400c5b0 tx=0x7f66a409a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1a472b90079c78 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f669cff9700 1 -- 172.21.15.57:0/2925002404 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a409a440 con 0x56242ed91170 2023-10-08T14:02:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f669cff9700 1 -- 172.21.15.57:0/2925002404 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a409e590 con 0x56242ed91170 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.953+0000 7f669cff9700 1 -- 172.21.15.57:0/2925002404 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a40988e0 con 0x56242ed91170 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.954+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2925002404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91170 msgr2=0x56242ed91550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.954+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/2925002404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91170 0x56242ed91550 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f66a400c5b0 tx=0x7f66a409a000 comp rx=0 tx=0).stop 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.954+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2925002404 shutdown_connections 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.954+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/2925002404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 0x56242ed962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.954+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/2925002404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91170 0x56242ed91550 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.954+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/2925002404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 0x56242ed95920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.954+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2925002404 >> 172.21.15.57:0/2925002404 conn(0x56242ed01200 msgr2=0x56242ed87b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.954+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2925002404 shutdown_connections 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.954+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2925002404 wait complete. 2023-10-08T14:02:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.954+0000 7f66c0ab3b80 1 -- start start 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66c0ab3b80 1 Processor -- start 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed91170 0x56242ed9d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 0x56242ed9a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed95e60 0x56242ed9b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242ecdfdd0 con 0x56242ed95e60 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242ec7c6b0 con 0x56242ed91a90 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242ec7c350 con 0x56242ed91170 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66c0ab3b80 10 client.0 _ll_register_callbacks cb 0x56242ead97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66aed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed95e60 0x56242ed9b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66aed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed95e60 0x56242ed9b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59784/0 (socket says 172.21.15.57:59784) 2023-10-08T14:02:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66aed9d700 1 -- 172.21.15.57:0/1875280346 learned_addr learned my addr 172.21.15.57:0/1875280346 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66ae59c700 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed91170 0x56242ed9d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.955+0000 7f66add9b700 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 0x56242ed9a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f66aed9d700 1 -- 172.21.15.57:0/1875280346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed91170 msgr2=0x56242ed9d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f66aed9d700 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed91170 0x56242ed9d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f66aed9d700 1 -- 172.21.15.57:0/1875280346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 msgr2=0x56242ed9a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f66aed9d700 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 0x56242ed9a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f66aed9d700 1 -- 172.21.15.57:0/1875280346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b01a38f0 con 0x56242ed95e60 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f66add9b700 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 0x56242ed9a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f66aed9d700 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed95e60 0x56242ed9b100 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f66a8015000 tx=0x7f66a8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8afb4d54c5650d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f6695ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:02:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f6695ffb700 1 -- 172.21.15.57:0/1875280346 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a800e040 con 0x56242ed95e60 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f6695ffb700 1 -- 172.21.15.57:0/1875280346 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f66a8004c50 con 0x56242ed95e60 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/1875280346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56242eca7e50 con 0x56242ed95e60 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f6695ffb700 1 -- 172.21.15.57:0/1875280346 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a800bea0 con 0x56242ed95e60 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f66c0ab3b80 10 client.17178 fetch_fsmap learned FSMap version 46 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/1875280346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56242eaf3c00 con 0x56242ed95e60 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.956+0000 7f6695ffb700 1 -- 172.21.15.57:0/1875280346 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56242eca7e50 con 0x56242ed95e60 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.957+0000 7f6695ffb700 1 -- 172.21.15.57:0/1875280346 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f66a8019050 con 0x56242ed95e60 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.957+0000 7f66c0ab3b80 10 client.17178 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.957+0000 7f66c0ab3b80 10 client.17178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.957+0000 7f66c0ab3b80 20 client.17178 populate_metadata read hostname 'smithi057' 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.957+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56242edab200 0x56242edad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.957+0000 7f66c0ab3b80 4 client.17178 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:02:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.957+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/1875280346 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56242ecd8db0 con 0x56242edab200 2023-10-08T14:02:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.957+0000 7f66ae59c700 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56242edab200 0x56242edad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.957+0000 7f66ae59c700 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56242edab200 0x56242edad620 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5800028bf14fc155 server_cookie=750f95404fe2e677 in_seq=0 out_seq=0 2023-10-08T14:02:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.957+0000 7f6695ffb700 10 client.17178 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.958+0000 7f6695ffb700 1 -- 172.21.15.57:0/1875280346 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56242ecd8db0 con 0x56242edab200 2023-10-08T14:02:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.958+0000 7f6695ffb700 10 client.17178 handle_command_reply: tid=0 2023-10-08T14:02:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.958+0000 7f66c0ab3b80 1 client.17178 shutdown 2023-10-08T14:02:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.958+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/1875280346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed95e60 msgr2=0x56242ed9b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.958+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed95e60 0x56242ed9b100 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f66a8015000 tx=0x7f66a8007f50 comp rx=0 tx=0).stop 2023-10-08T14:02:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.958+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/1875280346 shutdown_connections 2023-10-08T14:02:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.959+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56242edab200 0x56242edad620 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.959+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed95e60 0x56242ed9b100 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.959+0000 7f6695ffb700 0 client.17178 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.959+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed91170 0x56242ed9d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.959+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/1875280346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91a90 0x56242ed9a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.959+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/1875280346 >> 172.21.15.57:0/1875280346 conn(0x56242ed01200 msgr2=0x56242ed972c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.959+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/1875280346 shutdown_connections 2023-10-08T14:02:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.959+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/1875280346 wait complete. 2023-10-08T14:02:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.959+0000 7f66c0ab3b80 20 client.17178 trim_cache size 0 max 16384 2023-10-08T14:02:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66c0ab3b80 1 Processor -- start 2023-10-08T14:02:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66c0ab3b80 1 -- start start 2023-10-08T14:02:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91170 0x56242ed83c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242ed9d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 0x56242ed94020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242eb608c0 con 0x56242ed91a90 2023-10-08T14:02:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242ed841c0 con 0x56242ed91170 2023-10-08T14:02:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242eda8750 con 0x56242ed95e60 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66add9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242ed9d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66add9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242ed9d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59790/0 (socket says 172.21.15.57:59790) 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66add9b700 1 -- 172.21.15.57:0/3246375634 learned_addr learned my addr 172.21.15.57:0/3246375634 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66ae59c700 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91170 0x56242ed83c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66add9b700 1 -- 172.21.15.57:0/3246375634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 msgr2=0x56242ed94020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66aed9d700 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 0x56242ed94020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66add9b700 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 0x56242ed94020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66add9b700 1 -- 172.21.15.57:0/3246375634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91170 msgr2=0x56242ed83c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66add9b700 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91170 0x56242ed83c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66add9b700 1 -- 172.21.15.57:0/3246375634 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56242eb311f0 con 0x56242ed91a90 2023-10-08T14:02:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.974+0000 7f66ae59c700 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91170 0x56242ed83c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f66aed9d700 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 0x56242ed94020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f66add9b700 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242ed9d300 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f66a000ecf0 tx=0x7f66a0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0198fcffc20a43 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f669d7fa700 1 -- 172.21.15.57:0/3246375634 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0004220 con 0x56242ed91a90 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f669d7fa700 1 -- 172.21.15.57:0/3246375634 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a000ee90 con 0x56242ed91a90 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f669d7fa700 1 -- 172.21.15.57:0/3246375634 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0003ce0 con 0x56242ed91a90 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/3246375634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 msgr2=0x56242ed9d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242ed9d300 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f66a000ecf0 tx=0x7f66a0009f90 comp rx=0 tx=0).stop 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/3246375634 shutdown_connections 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed95e60 0x56242ed94020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed91170 0x56242ed83c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/3246375634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242ed9d300 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/3246375634 >> 172.21.15.57:0/3246375634 conn(0x56242ed01200 msgr2=0x56242edb1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.975+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/3246375634 shutdown_connections 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/3246375634 wait complete. 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66c0ab3b80 1 -- start start 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66c0ab3b80 1 Processor -- start 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed91170 0x56242eda0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242eda0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66c0ab3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed95e60 0x56242eda0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242edfe160 con 0x56242ed91a90 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242edfe2d0 con 0x56242ed95e60 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66c0ab3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242edfe490 con 0x56242ed91170 2023-10-08T14:02:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66c0ab3b80 10 client.0 _ll_register_callbacks cb 0x7f66a4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66add9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242eda0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66ae59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed91170 0x56242eda0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66add9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242eda0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59796/0 (socket says 172.21.15.57:59796) 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66add9b700 1 -- 172.21.15.57:0/2970967650 learned_addr learned my addr 172.21.15.57:0/2970967650 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.976+0000 7f66aed9d700 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed95e60 0x56242eda0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f66add9b700 1 -- 172.21.15.57:0/2970967650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed91170 msgr2=0x56242eda0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f66add9b700 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed91170 0x56242eda0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f66add9b700 1 -- 172.21.15.57:0/2970967650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed95e60 msgr2=0x56242eda0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f66add9b700 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed95e60 0x56242eda0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f66add9b700 1 -- 172.21.15.57:0/2970967650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a000e950 con 0x56242ed91a90 2023-10-08T14:02:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f66add9b700 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242eda0540 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009410 tx=0x7f66a0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c7217adfb5159c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f6695ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f6695ffb700 1 -- 172.21.15.57:0/2970967650 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a002d070 con 0x56242ed91a90 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f6695ffb700 1 -- 172.21.15.57:0/2970967650 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9381+0+0 (secure 0 0 0) 0x7f66a0020640 con 0x56242ed91a90 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2970967650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66a409e590 con 0x56242ed91a90 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f6695ffb700 1 -- 172.21.15.57:0/2970967650 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a0020800 con 0x56242ed91a90 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f66c0ab3b80 10 client.17184 fetch_fsmap learned FSMap version 46 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2970967650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56242eb09250 con 0x56242ed91a90 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.977+0000 7f6695ffb700 1 -- 172.21.15.57:0/2970967650 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a409e590 con 0x56242ed91a90 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.978+0000 7f6695ffb700 1 -- 172.21.15.57:0/2970967650 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f66a00040a0 con 0x56242ed91a90 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.978+0000 7f66c0ab3b80 10 client.17184 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.978+0000 7f66c0ab3b80 10 client.17184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:02:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.978+0000 7f66c0ab3b80 20 client.17184 populate_metadata read hostname 'smithi057' 2023-10-08T14:02:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.978+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56242edffaf0 0x56242ee01f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.978+0000 7f66c0ab3b80 4 client.17184 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:02:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.978+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2970967650 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56242ecd8ad0 con 0x56242edffaf0 2023-10-08T14:02:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.978+0000 7f66ae59c700 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56242edffaf0 0x56242ee01f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.978+0000 7f66ae59c700 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56242edffaf0 0x56242ee01f10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cc7a1e5998ef4aa server_cookie=1dd73417bfddb733 in_seq=0 out_seq=0 2023-10-08T14:02:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.978+0000 7f6695ffb700 10 client.17184 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f6695ffb700 1 -- 172.21.15.57:0/2970967650 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56242ecd8ad0 con 0x56242edffaf0 2023-10-08T14:02:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f6695ffb700 10 client.17184 handle_command_reply: tid=0 2023-10-08T14:02:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 client.17184 shutdown 2023-10-08T14:02:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2970967650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 msgr2=0x56242eda0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242eda0540 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009410 tx=0x7f66a0005760 comp rx=0 tx=0).stop 2023-10-08T14:02:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2970967650 shutdown_connections 2023-10-08T14:02:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56242edffaf0 0x56242ee01f10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56242ed95e60 0x56242eda0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f6695ffb700 0 client.17184 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56242ed91170 0x56242eda0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 --2- 172.21.15.57:0/2970967650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56242ed91a90 0x56242eda0540 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2970967650 >> 172.21.15.57:0/2970967650 conn(0x56242ed01200 msgr2=0x56242ed94020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2970967650 shutdown_connections 2023-10-08T14:02:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.989+0000 7f66c0ab3b80 1 -- 172.21.15.57:0/2970967650 wait complete. 2023-10-08T14:02:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66c0ab3b80 20 client.17184 trim_cache size 0 max 16384 2023-10-08T14:02:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:02:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:02:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:02:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:02:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 -- 172.21.15.57:0/2271512936 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f66980a15f0 msgr2=0x7f66980a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f66980a15f0 0x7f66980a3ab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f66a0005fd0 tx=0x7f66a0005ee0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 -- 172.21.15.57:0/2271512936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66b00ff5d0 msgr2=0x7f66b019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66b00ff5d0 0x7f66b019c900 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f66a400e970 tx=0x7f66a400ed30 comp rx=0 tx=0).stop 2023-10-08T14:02:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 -- 172.21.15.57:0/2271512936 shutdown_connections 2023-10-08T14:02:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f66980a15f0 0x7f66980a3ab0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b01045e0 0x7f66b0196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66b00ff5d0 0x7f66b019c900 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 --2- 172.21.15.57:0/2271512936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b00fff90 0x7f66b019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 -- 172.21.15.57:0/2271512936 >> 172.21.15.57:0/2271512936 conn(0x7f66b0076790 msgr2=0x7f66b0067d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 -- 172.21.15.57:0/2271512936 shutdown_connections 2023-10-08T14:02:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:04.990+0000 7f66b48ca700 1 -- 172.21.15.57:0/2271512936 wait complete. 2023-10-08T14:02:06.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:05 smithi099 ceph-mon[103849]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2023-10-08T14:02:06.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:05 smithi115 ceph-mon[104408]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2023-10-08T14:02:06.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:05 smithi057 ceph-mon[99476]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2023-10-08T14:02:08.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:07 smithi099 ceph-mon[103849]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2023-10-08T14:02:08.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:07 smithi115 ceph-mon[104408]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2023-10-08T14:02:08.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:07 smithi057 ceph-mon[99476]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2023-10-08T14:02:09.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:09 smithi115 ceph-mon[104408]: pgmap v626: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2023-10-08T14:02:09.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:09 smithi057 ceph-mon[99476]: pgmap v626: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2023-10-08T14:02:09.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:09 smithi099 ceph-mon[103849]: pgmap v626: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2023-10-08T14:02:11.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:11 smithi115 ceph-mon[104408]: osdmap e102: 12 total, 12 up, 12 in 2023-10-08T14:02:11.688 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:11 smithi115 ceph-mon[104408]: pgmap v628: 129 pgs: 8 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 119 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 150 op/s 2023-10-08T14:02:11.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:11 smithi057 ceph-mon[99476]: osdmap e102: 12 total, 12 up, 12 in 2023-10-08T14:02:11.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:11 smithi057 ceph-mon[99476]: pgmap v628: 129 pgs: 8 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 119 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 150 op/s 2023-10-08T14:02:11.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:11 smithi099 ceph-mon[103849]: osdmap e102: 12 total, 12 up, 12 in 2023-10-08T14:02:11.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:11 smithi099 ceph-mon[103849]: pgmap v628: 129 pgs: 8 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 119 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 150 op/s 2023-10-08T14:02:14.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:13 smithi099 ceph-mon[103849]: pgmap v629: 129 pgs: 29 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2023-10-08T14:02:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:13 smithi115 ceph-mon[104408]: pgmap v629: 129 pgs: 29 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2023-10-08T14:02:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:13 smithi057 ceph-mon[99476]: pgmap v629: 129 pgs: 29 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2023-10-08T14:02:16.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:15 smithi115 ceph-mon[104408]: pgmap v630: 129 pgs: 66 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 130 op/s 2023-10-08T14:02:16.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:15 smithi057 ceph-mon[99476]: pgmap v630: 129 pgs: 66 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 130 op/s 2023-10-08T14:02:16.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:15 smithi099 ceph-mon[103849]: pgmap v630: 129 pgs: 66 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 130 op/s 2023-10-08T14:02:17.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:16 smithi115 ceph-mon[104408]: pgmap v631: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 132 op/s 2023-10-08T14:02:17.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:16 smithi057 ceph-mon[99476]: pgmap v631: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 132 op/s 2023-10-08T14:02:17.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:16 smithi099 ceph-mon[103849]: pgmap v631: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 132 op/s 2023-10-08T14:02:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.180+0000 7f1b6e80c700 1 -- 172.21.15.57:0/2119815279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b6810f380 msgr2=0x7f1b68107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.180+0000 7f1b6e80c700 1 --2- 172.21.15.57:0/2119815279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b6810f380 0x7f1b68107d30 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0099c0 tx=0x7f1b5c009cd0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.182+0000 7f1b6e80c700 1 -- 172.21.15.57:0/2119815279 shutdown_connections 2023-10-08T14:02:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.182+0000 7f1b6e80c700 1 --2- 172.21.15.57:0/2119815279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b68119fd0 0x7f1b6811c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.182+0000 7f1b6e80c700 1 --2- 172.21.15.57:0/2119815279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6810ef10 0x7f1b681077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.182+0000 7f1b6e80c700 1 --2- 172.21.15.57:0/2119815279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b6810f380 0x7f1b68107d30 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.183+0000 7f1b6e80c700 1 -- 172.21.15.57:0/2119815279 >> 172.21.15.57:0/2119815279 conn(0x7f1b6806b7c0 msgr2=0x7f1b6806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.186+0000 7f1b6e80c700 1 -- 172.21.15.57:0/2119815279 shutdown_connections 2023-10-08T14:02:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.187+0000 7f1b6e80c700 1 -- 172.21.15.57:0/2119815279 wait complete. 2023-10-08T14:02:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.187+0000 7f1b6e80c700 1 Processor -- start 2023-10-08T14:02:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.187+0000 7f1b6e80c700 1 -- start start 2023-10-08T14:02:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.188+0000 7f1b6e80c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6810ef10 0x7f1b681184a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.188+0000 7f1b6e80c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6810f380 0x7f1b68118a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.188+0000 7f1b6e80c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b68119fd0 0x7f1b681116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.188+0000 7f1b6e80c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b68111eb0 con 0x7f1b68119fd0 2023-10-08T14:02:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.188+0000 7f1b6e80c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b68112020 con 0x7f1b6810f380 2023-10-08T14:02:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.188+0000 7f1b6e80c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681121e0 con 0x7f1b6810ef10 2023-10-08T14:02:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.188+0000 7f1b6cda9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b68119fd0 0x7f1b681116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.188+0000 7f1b6cda9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b68119fd0 0x7f1b681116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59812/0 (socket says 172.21.15.57:59812) 2023-10-08T14:02:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.188+0000 7f1b6cda9700 1 -- 172.21.15.57:0/3422040964 learned_addr learned my addr 172.21.15.57:0/3422040964 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.188+0000 7f1b677fe700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6810f380 0x7f1b68118a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.189+0000 7f1b6cda9700 1 -- 172.21.15.57:0/3422040964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6810ef10 msgr2=0x7f1b681184a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.189+0000 7f1b6cda9700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6810ef10 0x7f1b681184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.189+0000 7f1b6cda9700 1 -- 172.21.15.57:0/3422040964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6810f380 msgr2=0x7f1b68118a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.189+0000 7f1b6cda9700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6810f380 0x7f1b68118a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.189+0000 7f1b6cda9700 1 -- 172.21.15.57:0/3422040964 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c0095f0 con 0x7f1b68119fd0 2023-10-08T14:02:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.189+0000 7f1b6cda9700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b68119fd0 0x7f1b681116e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800b480 tx=0x7f1b5800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.189+0000 7f1b657fa700 1 -- 172.21.15.57:0/3422040964 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58011830 con 0x7f1b68119fd0 2023-10-08T14:02:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.190+0000 7f1b6e80c700 1 -- 172.21.15.57:0/3422040964 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b68116030 con 0x7f1b68119fd0 2023-10-08T14:02:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.190+0000 7f1b657fa700 1 -- 172.21.15.57:0/3422040964 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b58011e50 con 0x7f1b68119fd0 2023-10-08T14:02:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.190+0000 7f1b6e80c700 1 -- 172.21.15.57:0/3422040964 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b68116560 con 0x7f1b68119fd0 2023-10-08T14:02:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.192+0000 7f1b657fa700 1 -- 172.21.15.57:0/3422040964 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58010dc0 con 0x7f1b68119fd0 2023-10-08T14:02:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.195+0000 7f1b7a9f5b80 1 Processor -- start 2023-10-08T14:02:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.195+0000 7f1b657fa700 1 -- 172.21.15.57:0/3422040964 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1b5801a400 con 0x7f1b68119fd0 2023-10-08T14:02:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.195+0000 7f1b7a9f5b80 1 -- start start 2023-10-08T14:02:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 0x55fcc2671c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc26721a0 0x55fcc266bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc266c0b0 0x55fcc266c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc25d4400 con 0x55fcc266c0b0 2023-10-08T14:02:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc25c5bc0 con 0x55fcc26721a0 2023-10-08T14:02:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc25c34e0 con 0x55fcc2671880 2023-10-08T14:02:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 0x55fcc2671c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 0x55fcc2671c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60628/0 (socket says 172.21.15.57:60628) 2023-10-08T14:02:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b677fe700 1 -- 172.21.15.57:0/594754232 learned_addr learned my addr 172.21.15.57:0/594754232 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b67fff700 1 --2- 172.21.15.57:0/594754232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc26721a0 0x55fcc266bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b657fa700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b500a15f0 0x7f1b500a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b677fe700 1 -- 172.21.15.57:0/594754232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc26721a0 msgr2=0x55fcc266bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b677fe700 1 --2- 172.21.15.57:0/594754232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc26721a0 0x55fcc266bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b677fe700 1 -- 172.21.15.57:0/594754232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc266c0b0 msgr2=0x55fcc266c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b677fe700 1 --2- 172.21.15.57:0/594754232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc266c0b0 0x55fcc266c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b677fe700 1 -- 172.21.15.57:0/594754232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcc23e9d80 con 0x55fcc2671880 2023-10-08T14:02:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b657fa700 1 -- 172.21.15.57:0/3422040964 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f1b580cd1a0 con 0x7f1b68119fd0 2023-10-08T14:02:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.196+0000 7f1b6cda9700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b500a15f0 0x7f1b500a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.197+0000 7f1b677fe700 1 --2- 172.21.15.57:0/594754232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 0x55fcc2671c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1b6806e060 tx=0x7f1b5c005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278d8aebdd015671 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.197+0000 7f1b6cda9700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b500a15f0 0x7f1b500a3ab0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1b58096ff0 tx=0x7f1b580662f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.199+0000 7f1b4e7fc700 1 -- 172.21.15.57:0/594754232 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c005610 con 0x55fcc2671880 2023-10-08T14:02:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.199+0000 7f1b4e7fc700 1 -- 172.21.15.57:0/594754232 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b5c00bc50 con 0x55fcc2671880 2023-10-08T14:02:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.199+0000 7f1b4e7fc700 1 -- 172.21.15.57:0/594754232 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c003fc0 con 0x55fcc2671880 2023-10-08T14:02:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.200+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/594754232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 msgr2=0x55fcc2671c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.200+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/594754232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 0x55fcc2671c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1b6806e060 tx=0x7f1b5c005ea0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.203+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/594754232 shutdown_connections 2023-10-08T14:02:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.203+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/594754232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc266c0b0 0x55fcc266c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.203+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/594754232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 0x55fcc2671c60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.203+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/594754232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc26721a0 0x55fcc266bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.203+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/594754232 >> 172.21.15.57:0/594754232 conn(0x55fcc25e0200 msgr2=0x55fcc2666b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.204+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/594754232 shutdown_connections 2023-10-08T14:02:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.204+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/594754232 wait complete. 2023-10-08T14:02:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.205+0000 7f1b7a9f5b80 1 -- start start 2023-10-08T14:02:18.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.205+0000 7f1b7a9f5b80 1 Processor -- start 2023-10-08T14:02:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.206+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc266c0b0 0x55fcc26755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.206+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc2671880 0x55fcc2675b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.206+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc26721a0 0x55fcc2676060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.206+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc25bedd0 con 0x55fcc26721a0 2023-10-08T14:02:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.206+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc255b6b0 con 0x55fcc2671880 2023-10-08T14:02:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.206+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc255b350 con 0x55fcc266c0b0 2023-10-08T14:02:18.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.206+0000 7f1b7a9f5b80 10 client.0 _ll_register_callbacks cb 0x55fcc23b87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:02:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.206+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc266c0b0 0x55fcc26755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.206+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc266c0b0 0x55fcc26755e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60636/0 (socket says 172.21.15.57:60636) 2023-10-08T14:02:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.206+0000 7f1b677fe700 1 -- 172.21.15.57:0/1113996472 learned_addr learned my addr 172.21.15.57:0/1113996472 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.207+0000 7f1b677fe700 1 -- 172.21.15.57:0/1113996472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc2671880 msgr2=0x55fcc2675b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:02:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.207+0000 7f1b677fe700 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc2671880 0x55fcc2675b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.207+0000 7f1b677fe700 1 -- 172.21.15.57:0/1113996472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc26721a0 msgr2=0x55fcc2676060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.207+0000 7f1b677fe700 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc26721a0 0x55fcc2676060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.207+0000 7f1b677fe700 1 -- 172.21.15.57:0/1113996472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c020040 con 0x55fcc266c0b0 2023-10-08T14:02:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.207+0000 7f1b677fe700 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc266c0b0 0x55fcc26755e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0094e0 tx=0x7f1b5c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bba86ca56e6f0e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.207+0000 7f1b477fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:02:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.207+0000 7f1b477fe700 1 -- 172.21.15.57:0/1113996472 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c005590 con 0x55fcc266c0b0 2023-10-08T14:02:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.208+0000 7f1b477fe700 1 -- 172.21.15.57:0/1113996472 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f1b5c003860 con 0x55fcc266c0b0 2023-10-08T14:02:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.210+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1113996472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcc2586e50 con 0x55fcc266c0b0 2023-10-08T14:02:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.212+0000 7f1b477fe700 1 -- 172.21.15.57:0/1113996472 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b5c003a20 con 0x55fcc266c0b0 2023-10-08T14:02:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.212+0000 7f1b7a9f5b80 10 client.26752 fetch_fsmap learned FSMap version 46 2023-10-08T14:02:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.212+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1113996472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55fcc23d2c00 con 0x55fcc266c0b0 2023-10-08T14:02:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.219+0000 7f1b477fe700 1 -- 172.21.15.57:0/1113996472 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fcc2586e50 con 0x55fcc266c0b0 2023-10-08T14:02:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.219+0000 7f1b477fe700 1 -- 172.21.15.57:0/1113996472 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f1b5c037070 con 0x55fcc266c0b0 2023-10-08T14:02:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.219+0000 7f1b7a9f5b80 10 client.26752 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:02:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.219+0000 7f1b7a9f5b80 10 client.26752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:02:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.219+0000 7f1b7a9f5b80 20 client.26752 populate_metadata read hostname 'smithi057' 2023-10-08T14:02:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.219+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fcc2685910 0x55fcc2687d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.219+0000 7f1b7a9f5b80 4 client.26752 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:02:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.219+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1113996472 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcc25b7db0 con 0x55fcc2685910 2023-10-08T14:02:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.220+0000 7f1b67fff700 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fcc2685910 0x55fcc2687d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.220+0000 7f1b477fe700 10 client.26752 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.220+0000 7f1b67fff700 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fcc2685910 0x55fcc2687d30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca69d206a915b86 server_cookie=e67b57c83d96765d in_seq=0 out_seq=0 2023-10-08T14:02:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.221+0000 7f1b477fe700 1 -- 172.21.15.57:0/1113996472 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fcc25b7db0 con 0x55fcc2685910 2023-10-08T14:02:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.221+0000 7f1b477fe700 10 client.26752 handle_command_reply: tid=0 2023-10-08T14:02:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.224+0000 7f1b7a9f5b80 1 client.26752 shutdown 2023-10-08T14:02:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.233+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1113996472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc266c0b0 msgr2=0x55fcc26755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.233+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc266c0b0 0x55fcc26755e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0094e0 tx=0x7f1b5c000bc0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.233+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1113996472 shutdown_connections 2023-10-08T14:02:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.233+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fcc2685910 0x55fcc2687d30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.233+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc26721a0 0x55fcc2676060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.233+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc266c0b0 0x55fcc26755e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.233+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1113996472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc2671880 0x55fcc2675b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.233+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1113996472 >> 172.21.15.57:0/1113996472 conn(0x55fcc25e0200 msgr2=0x55fcc2666010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.235+0000 7f1b477fe700 0 client.26752 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.245+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1113996472 shutdown_connections 2023-10-08T14:02:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.252+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1113996472 wait complete. 2023-10-08T14:02:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.252+0000 7f1b7a9f5b80 20 client.26752 trim_cache size 0 max 16384 2023-10-08T14:02:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b7a9f5b80 1 Processor -- start 2023-10-08T14:02:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b7a9f5b80 1 -- start start 2023-10-08T14:02:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc266c0b0 0x55fcc2666790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 0x55fcc2666cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc26721a0 0x55fcc2673eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc243f8c0 con 0x55fcc26721a0 2023-10-08T14:02:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc2663830 con 0x55fcc266c0b0 2023-10-08T14:02:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc26612b0 con 0x55fcc2671880 2023-10-08T14:02:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc266c0b0 0x55fcc2666790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc266c0b0 0x55fcc2666790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33024/0 (socket says 172.21.15.57:33024) 2023-10-08T14:02:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.283+0000 7f1b677fe700 1 -- 172.21.15.57:0/1371124653 learned_addr learned my addr 172.21.15.57:0/1371124653 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.284+0000 7f1b677fe700 1 -- 172.21.15.57:0/1371124653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 msgr2=0x55fcc2666cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.284+0000 7f1b677fe700 1 --2- 172.21.15.57:0/1371124653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 0x55fcc2666cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.284+0000 7f1b677fe700 1 -- 172.21.15.57:0/1371124653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc26721a0 msgr2=0x55fcc2673eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.284+0000 7f1b677fe700 1 --2- 172.21.15.57:0/1371124653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc26721a0 0x55fcc2673eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.284+0000 7f1b677fe700 1 -- 172.21.15.57:0/1371124653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcc24101f0 con 0x55fcc266c0b0 2023-10-08T14:02:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.284+0000 7f1b677fe700 1 --2- 172.21.15.57:0/1371124653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc266c0b0 0x55fcc2666790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c006010 tx=0x7f1b5c00fca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff105faf1bd8abf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.287+0000 7f1b4effd700 1 -- 172.21.15.57:0/1371124653 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c005590 con 0x55fcc266c0b0 2023-10-08T14:02:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.287+0000 7f1b4effd700 1 -- 172.21.15.57:0/1371124653 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b5c0056f0 con 0x55fcc266c0b0 2023-10-08T14:02:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.287+0000 7f1b4effd700 1 -- 172.21.15.57:0/1371124653 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c00f6c0 con 0x55fcc266c0b0 2023-10-08T14:02:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.288+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1371124653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc266c0b0 msgr2=0x55fcc2666790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.288+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1371124653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc266c0b0 0x55fcc2666790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c006010 tx=0x7f1b5c00fca0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.288+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1371124653 shutdown_connections 2023-10-08T14:02:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.288+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1371124653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc26721a0 0x55fcc2673eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.288+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1371124653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc266c0b0 0x55fcc2666790 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c006010 tx=0x7f1b5c00fca0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.288+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1371124653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc2671880 0x55fcc2666cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.288+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1371124653 >> 172.21.15.57:0/1371124653 conn(0x55fcc25e0200 msgr2=0x55fcc266f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.301+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1371124653 shutdown_connections 2023-10-08T14:02:18.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.308+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1371124653 wait complete. 2023-10-08T14:02:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.308+0000 7f1b7a9f5b80 1 -- start start 2023-10-08T14:02:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.308+0000 7f1b7a9f5b80 1 Processor -- start 2023-10-08T14:02:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.309+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc2671880 0x55fcc267cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.309+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc26721a0 0x55fcc267ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.309+0000 7f1b7a9f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc267b140 0x55fcc267b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.309+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc26d8700 con 0x55fcc2671880 2023-10-08T14:02:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.309+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc26d8840 con 0x55fcc267b140 2023-10-08T14:02:18.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.309+0000 7f1b7a9f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc26d89d0 con 0x55fcc26721a0 2023-10-08T14:02:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.309+0000 7f1b7a9f5b80 10 client.0 _ll_register_callbacks cb 0x55fcc23b87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:02:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.309+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc2671880 0x55fcc267cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.309+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc2671880 0x55fcc267cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59838/0 (socket says 172.21.15.57:59838) 2023-10-08T14:02:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.309+0000 7f1b677fe700 1 -- 172.21.15.57:0/1382045133 learned_addr learned my addr 172.21.15.57:0/1382045133 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.310+0000 7f1b677fe700 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc267b140 0x55fcc267b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.310+0000 7f1b677fe700 1 -- 172.21.15.57:0/1382045133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc26721a0 msgr2=0x55fcc267ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.310+0000 7f1b677fe700 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc26721a0 0x55fcc267ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.310+0000 7f1b677fe700 1 -- 172.21.15.57:0/1382045133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc267b140 msgr2=0x55fcc267b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.310+0000 7f1b677fe700 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc267b140 0x55fcc267b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.310+0000 7f1b677fe700 1 -- 172.21.15.57:0/1382045133 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcc23d2c00 con 0x55fcc2671880 2023-10-08T14:02:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.325+0000 7f1b677fe700 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc2671880 0x55fcc267cc20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c027000 tx=0x7f1b5c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce2d670cd513251 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:18.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.325+0000 7f1b477fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:02:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.330+0000 7f1b477fe700 1 -- 172.21.15.57:0/1382045133 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c00f6f0 con 0x55fcc2671880 2023-10-08T14:02:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.330+0000 7f1b477fe700 1 -- 172.21.15.57:0/1382045133 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f1b5c00b790 con 0x55fcc2671880 2023-10-08T14:02:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.331+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1382045133 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b5c00bc50 con 0x55fcc2671880 2023-10-08T14:02:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.331+0000 7f1b477fe700 1 -- 172.21.15.57:0/1382045133 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b5c00b950 con 0x55fcc2671880 2023-10-08T14:02:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.335+0000 7f1b7a9f5b80 10 client.17193 fetch_fsmap learned FSMap version 46 2023-10-08T14:02:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.335+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1382045133 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55fcc23e8250 con 0x55fcc2671880 2023-10-08T14:02:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.337+0000 7f1b477fe700 1 -- 172.21.15.57:0/1382045133 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b5c00bc50 con 0x55fcc2671880 2023-10-08T14:02:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.337+0000 7f1b477fe700 1 -- 172.21.15.57:0/1382045133 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f1b5c0099f0 con 0x55fcc2671880 2023-10-08T14:02:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.337+0000 7f1b7a9f5b80 10 client.17193 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:02:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.337+0000 7f1b7a9f5b80 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:02:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.337+0000 7f1b7a9f5b80 20 client.17193 populate_metadata read hostname 'smithi057' 2023-10-08T14:02:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.337+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fcc26d9cb0 0x55fcc26dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.337+0000 7f1b7a9f5b80 4 client.17193 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:02:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.337+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1382045133 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fcc25b7ad0 con 0x55fcc26d9cb0 2023-10-08T14:02:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.338+0000 7f1b67fff700 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fcc26d9cb0 0x55fcc26dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.347+0000 7f1b477fe700 10 client.17193 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.347+0000 7f1b67fff700 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fcc26d9cb0 0x55fcc26dc0a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f37a18d9dd866d server_cookie=3f993c9f4c51e924 in_seq=0 out_seq=0 2023-10-08T14:02:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.461+0000 7f1b477fe700 1 -- 172.21.15.57:0/1382045133 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fcc25b7ad0 con 0x55fcc26d9cb0 2023-10-08T14:02:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.461+0000 7f1b477fe700 10 client.17193 handle_command_reply: tid=0 2023-10-08T14:02:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.462+0000 7f1b7a9f5b80 1 client.17193 shutdown 2023-10-08T14:02:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.467+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1382045133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc2671880 msgr2=0x55fcc267cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.467+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc2671880 0x55fcc267cc20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c027000 tx=0x7f1b5c00bfd0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.475+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1382045133 shutdown_connections 2023-10-08T14:02:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.475+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fcc26d9cb0 0x55fcc26dc0a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.475+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fcc267b140 0x55fcc267b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.475+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fcc2671880 0x55fcc267cc20 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.475+0000 7f1b7a9f5b80 1 --2- 172.21.15.57:0/1382045133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fcc26721a0 0x55fcc267ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.475+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1382045133 >> 172.21.15.57:0/1382045133 conn(0x55fcc25e0200 msgr2=0x55fcc2673eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.478+0000 7f1b477fe700 0 client.17193 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.478+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1382045133 shutdown_connections 2023-10-08T14:02:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.478+0000 7f1b7a9f5b80 1 -- 172.21.15.57:0/1382045133 wait complete. 2023-10-08T14:02:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.478+0000 7f1b7a9f5b80 20 client.17193 trim_cache size 0 max 16384 2023-10-08T14:02:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:02:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:02:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:02:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:02:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.489+0000 7f1b6e80c700 1 -- 172.21.15.57:0/3422040964 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b500a15f0 msgr2=0x7f1b500a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.489+0000 7f1b6e80c700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b500a15f0 0x7f1b500a3ab0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1b58096ff0 tx=0x7f1b580662f0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.489+0000 7f1b6e80c700 1 -- 172.21.15.57:0/3422040964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b68119fd0 msgr2=0x7f1b681116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.489+0000 7f1b6e80c700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b68119fd0 0x7f1b681116e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800b480 tx=0x7f1b5800b790 comp rx=0 tx=0).stop 2023-10-08T14:02:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.491+0000 7f1b6e80c700 1 -- 172.21.15.57:0/3422040964 shutdown_connections 2023-10-08T14:02:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.491+0000 7f1b6e80c700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1b68119fd0 0x7f1b681116e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.491+0000 7f1b6e80c700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b6810ef10 0x7f1b681184a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.491+0000 7f1b6e80c700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1b500a15f0 0x7f1b500a3ab0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.491+0000 7f1b6e80c700 1 --2- 172.21.15.57:0/3422040964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6810f380 0x7f1b68118a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.491+0000 7f1b6e80c700 1 -- 172.21.15.57:0/3422040964 >> 172.21.15.57:0/3422040964 conn(0x7f1b6806b7c0 msgr2=0x7f1b6806f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.497+0000 7f1b6e80c700 1 -- 172.21.15.57:0/3422040964 shutdown_connections 2023-10-08T14:02:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:18.497+0000 7f1b6e80c700 1 -- 172.21.15.57:0/3422040964 wait complete. 2023-10-08T14:02:19.690 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:19 smithi115 ceph-mon[104408]: pgmap v632: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2023-10-08T14:02:19.691 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:19 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:02:19.743 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:19 smithi057 ceph-mon[99476]: pgmap v632: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2023-10-08T14:02:19.744 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:19 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:02:19.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:19 smithi099 ceph-mon[103849]: pgmap v632: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2023-10-08T14:02:19.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:19 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:02:22.040 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:21 smithi115 ceph-mon[104408]: pgmap v633: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 130 op/s 2023-10-08T14:02:22.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:21 smithi057 ceph-mon[99476]: pgmap v633: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 130 op/s 2023-10-08T14:02:22.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:21 smithi099 ceph-mon[103849]: pgmap v633: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 130 op/s 2023-10-08T14:02:23.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:22 smithi115 ceph-mon[104408]: pgmap v634: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2023-10-08T14:02:23.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:22 smithi057 ceph-mon[99476]: pgmap v634: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2023-10-08T14:02:23.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:22 smithi099 ceph-mon[103849]: pgmap v634: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2023-10-08T14:02:24.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:24 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:24 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:24 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:24 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.743 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:24 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.744 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:24 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.744 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:24 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.744 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:24 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:24 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:24 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:24 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:24.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:24 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:25.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:25.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:25.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:25 smithi115 ceph-mon[104408]: pgmap v635: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 121 op/s 2023-10-08T14:02:25.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:02:25.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:02:25.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:25.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:25.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:25.995 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:25 smithi057 ceph-mon[99476]: pgmap v635: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 121 op/s 2023-10-08T14:02:25.996 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:02:25.996 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:02:25.996 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:26.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:26.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:26.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:25 smithi099 ceph-mon[103849]: pgmap v635: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 121 op/s 2023-10-08T14:02:26.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:02:26.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:02:26.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:02:28.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:27 smithi115 ceph-mon[104408]: pgmap v636: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-10-08T14:02:28.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:27 smithi057 ceph-mon[99476]: pgmap v636: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-10-08T14:02:28.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:27 smithi099 ceph-mon[103849]: pgmap v636: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-10-08T14:02:29.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:29 smithi115 ceph-mon[104408]: pgmap v637: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 737 KiB/s wr, 88 op/s 2023-10-08T14:02:29.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:29 smithi057 ceph-mon[99476]: pgmap v637: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 737 KiB/s wr, 88 op/s 2023-10-08T14:02:29.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:29 smithi099 ceph-mon[103849]: pgmap v637: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 737 KiB/s wr, 88 op/s 2023-10-08T14:02:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.698+0000 7f52c6ce2700 1 -- 172.21.15.57:0/2645970555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c00ff5d0 msgr2=0x7f52c00ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.698+0000 7f52c6ce2700 1 --2- 172.21.15.57:0/2645970555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c00ff5d0 0x7f52c00ffa50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009990 tx=0x7f52b0009ca0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.698+0000 7f52c6ce2700 1 -- 172.21.15.57:0/2645970555 shutdown_connections 2023-10-08T14:02:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.698+0000 7f52c6ce2700 1 --2- 172.21.15.57:0/2645970555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52c00fff90 0x7f52c010c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.698+0000 7f52c6ce2700 1 --2- 172.21.15.57:0/2645970555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c01045e0 0x7f52c01049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.698+0000 7f52c6ce2700 1 --2- 172.21.15.57:0/2645970555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c00ff5d0 0x7f52c00ffa50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.698+0000 7f52c6ce2700 1 -- 172.21.15.57:0/2645970555 >> 172.21.15.57:0/2645970555 conn(0x7f52c0076790 msgr2=0x7f52c0076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.698+0000 7f52c6ce2700 1 -- 172.21.15.57:0/2645970555 shutdown_connections 2023-10-08T14:02:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.698+0000 7f52c6ce2700 1 -- 172.21.15.57:0/2645970555 wait complete. 2023-10-08T14:02:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.699+0000 7f52c6ce2700 1 Processor -- start 2023-10-08T14:02:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.699+0000 7f52c6ce2700 1 -- start start 2023-10-08T14:02:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.700+0000 7f52c6ce2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c00ff5d0 0x7f52c0115040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.700+0000 7f52c6ce2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c00fff90 0x7f52c010f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.700+0000 7f52c6ce2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52c01045e0 0x7f52c010f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.700+0000 7f52c6ce2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c010fe60 con 0x7f52c01045e0 2023-10-08T14:02:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.700+0000 7f52c6ce2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c010ffd0 con 0x7f52c00fff90 2023-10-08T14:02:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.700+0000 7f52c6ce2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c006a260 con 0x7f52c00ff5d0 2023-10-08T14:02:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.700+0000 7f52c527f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52c01045e0 0x7f52c010f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.700+0000 7f52c527f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52c01045e0 0x7f52c010f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59850/0 (socket says 172.21.15.57:59850) 2023-10-08T14:02:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.700+0000 7f52c527f700 1 -- 172.21.15.57:0/3140759088 learned_addr learned my addr 172.21.15.57:0/3140759088 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.701+0000 7f52c527f700 1 -- 172.21.15.57:0/3140759088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c00ff5d0 msgr2=0x7f52c0115040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:02:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.701+0000 7f52c527f700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c00ff5d0 0x7f52c0115040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.701+0000 7f52c527f700 1 -- 172.21.15.57:0/3140759088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c00fff90 msgr2=0x7f52c010f130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:02:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.701+0000 7f52c527f700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c00fff90 0x7f52c010f130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.701+0000 7f52c527f700 1 -- 172.21.15.57:0/3140759088 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b00095f0 con 0x7f52c01045e0 2023-10-08T14:02:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.701+0000 7f52c527f700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52c01045e0 0x7f52c010f690 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f52b800b970 tx=0x7f52b800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.705+0000 7f52bdffb700 1 -- 172.21.15.57:0/3140759088 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b800c9b0 con 0x7f52c01045e0 2023-10-08T14:02:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.705+0000 7f52bdffb700 1 -- 172.21.15.57:0/3140759088 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b8014440 con 0x7f52c01045e0 2023-10-08T14:02:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.705+0000 7f52bdffb700 1 -- 172.21.15.57:0/3140759088 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b80075d0 con 0x7f52c01045e0 2023-10-08T14:02:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.705+0000 7f52c6ce2700 1 -- 172.21.15.57:0/3140759088 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c006a540 con 0x7f52c01045e0 2023-10-08T14:02:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.706+0000 7f52c6ce2700 1 -- 172.21.15.57:0/3140759088 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c006aa00 con 0x7f52c01045e0 2023-10-08T14:02:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.711+0000 7f52bdffb700 1 -- 172.21.15.57:0/3140759088 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f52b800cb10 con 0x7f52c01045e0 2023-10-08T14:02:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.712+0000 7f52bdffb700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a80a1790 0x7f52a80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.712+0000 7f52bdffb700 1 -- 172.21.15.57:0/3140759088 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f52b8006e40 con 0x7f52c01045e0 2023-10-08T14:02:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.717+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a80a1790 0x7f52a80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.717+0000 7f52d2ecbb80 1 Processor -- start 2023-10-08T14:02:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.717+0000 7f52d2ecbb80 1 -- start start 2023-10-08T14:02:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.718+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c957fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.718+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9579b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.718+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a80a1790 0x7f52a80a3c50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f52c0069cf0 tx=0x7f52b4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.718+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957a0b0 0x55b3c957a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.718+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c94e2400 con 0x55b3c95801a0 2023-10-08T14:02:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.718+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c94d3bc0 con 0x55b3c957f880 2023-10-08T14:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.718+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c94d14e0 con 0x55b3c957a0b0 2023-10-08T14:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.718+0000 7f52c4a7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c957fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.718+0000 7f52c4a7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c957fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33048/0 (socket says 172.21.15.57:33048) 2023-10-08T14:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.718+0000 7f52c4a7e700 1 -- 172.21.15.57:0/3104340570 learned_addr learned my addr 172.21.15.57:0/3104340570 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.719+0000 7f52c4a7e700 1 -- 172.21.15.57:0/3104340570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957a0b0 msgr2=0x55b3c957a510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.719+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/3104340570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957a0b0 0x55b3c957a510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.719+0000 7f52c4a7e700 1 -- 172.21.15.57:0/3104340570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 msgr2=0x55b3c9579b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.719+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/3104340570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9579b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.719+0000 7f52c4a7e700 1 -- 172.21.15.57:0/3104340570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c92f7d80 con 0x55b3c957f880 2023-10-08T14:02:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.719+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/3104340570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c957fc60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f52b400eb30 tx=0x7f52b4011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d878e19e6e8cea5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.719+0000 7f52a6ffd700 1 -- 172.21.15.57:0/3104340570 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4010ac0 con 0x55b3c957f880 2023-10-08T14:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.719+0000 7f52a6ffd700 1 -- 172.21.15.57:0/3104340570 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b4010c20 con 0x55b3c957f880 2023-10-08T14:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.719+0000 7f52a6ffd700 1 -- 172.21.15.57:0/3104340570 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b403d3c0 con 0x55b3c957f880 2023-10-08T14:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.721+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/3104340570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 msgr2=0x55b3c957fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.721+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/3104340570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c957fc60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f52b400eb30 tx=0x7f52b4011000 comp rx=0 tx=0).stop 2023-10-08T14:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.721+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/3104340570 shutdown_connections 2023-10-08T14:02:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.721+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/3104340570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957a0b0 0x55b3c957a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.721+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/3104340570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c957fc60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.721+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/3104340570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9579b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.721+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/3104340570 >> 172.21.15.57:0/3104340570 conn(0x55b3c94ee200 msgr2=0x55b3c9574b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.722+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/3104340570 shutdown_connections 2023-10-08T14:02:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.723+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/3104340570 wait complete. 2023-10-08T14:02:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.724+0000 7f52d2ecbb80 1 -- start start 2023-10-08T14:02:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.724+0000 7f52d2ecbb80 1 Processor -- start 2023-10-08T14:02:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957a0b0 0x55b3c9583540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c9583a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9583fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c94ccdd0 con 0x55b3c95801a0 2023-10-08T14:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c94696b0 con 0x55b3c957f880 2023-10-08T14:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c9469350 con 0x55b3c957a0b0 2023-10-08T14:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52d2ecbb80 10 client.0 _ll_register_callbacks cb 0x55b3c92c67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52c527f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9583fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52c527f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9583fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59862/0 (socket says 172.21.15.57:59862) 2023-10-08T14:02:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52c527f700 1 -- 172.21.15.57:0/2237484435 learned_addr learned my addr 172.21.15.57:0/2237484435 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.725+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957a0b0 0x55b3c9583540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.726+0000 7f52c4a7e700 1 -- 172.21.15.57:0/2237484435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 msgr2=0x55b3c9583a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.726+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c9583a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.726+0000 7f52c4a7e700 1 -- 172.21.15.57:0/2237484435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 msgr2=0x55b3c9583fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.726+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9583fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.726+0000 7f52c4a7e700 1 -- 172.21.15.57:0/2237484435 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b400e7e0 con 0x55b3c957a0b0 2023-10-08T14:02:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.738+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957a0b0 0x55b3c9583540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f52b400e630 tx=0x7f52b4009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d27f0d9bdef630 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.741+0000 7f529ffff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:02:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.741+0000 7f529ffff700 1 -- 172.21.15.57:0/2237484435 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b404d070 con 0x55b3c957a0b0 2023-10-08T14:02:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.741+0000 7f529ffff700 1 -- 172.21.15.57:0/2237484435 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f52b403e640 con 0x55b3c957a0b0 2023-10-08T14:02:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.743+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/2237484435 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3c9494e50 con 0x55b3c957a0b0 2023-10-08T14:02:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.746+0000 7f529ffff700 1 -- 172.21.15.57:0/2237484435 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52b403e800 con 0x55b3c957a0b0 2023-10-08T14:02:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.748+0000 7f52d2ecbb80 10 client.26758 fetch_fsmap learned FSMap version 46 2023-10-08T14:02:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.748+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/2237484435 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55b3c92e0c00 con 0x55b3c957a0b0 2023-10-08T14:02:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.752+0000 7f529ffff700 1 -- 172.21.15.57:0/2237484435 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3c9494e50 con 0x55b3c957a0b0 2023-10-08T14:02:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.752+0000 7f529ffff700 1 -- 172.21.15.57:0/2237484435 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f52b405a070 con 0x55b3c957a0b0 2023-10-08T14:02:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.753+0000 7f52d2ecbb80 10 client.26758 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:02:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.753+0000 7f52d2ecbb80 10 client.26758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:02:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.753+0000 7f52d2ecbb80 20 client.26758 populate_metadata read hostname 'smithi057' 2023-10-08T14:02:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.753+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b3c9593cc0 0x55b3c95960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.753+0000 7f52d2ecbb80 4 client.26758 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:02:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.753+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/2237484435 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3c94c5db0 con 0x55b3c9593cc0 2023-10-08T14:02:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.768+0000 7f52bffff700 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b3c9593cc0 0x55b3c95960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.772+0000 7f52bffff700 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b3c9593cc0 0x55b3c95960e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e6d9c8742cab1a9 server_cookie=5469ea79199b0f29 in_seq=0 out_seq=0 2023-10-08T14:02:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.774+0000 7f529ffff700 10 client.26758 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.774+0000 7f529ffff700 1 -- 172.21.15.57:0/2237484435 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b3c94c5db0 con 0x55b3c9593cc0 2023-10-08T14:02:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.774+0000 7f529ffff700 10 client.26758 handle_command_reply: tid=0 2023-10-08T14:02:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.774+0000 7f52d2ecbb80 1 client.26758 shutdown 2023-10-08T14:02:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.781+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/2237484435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957a0b0 msgr2=0x55b3c9583540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.781+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957a0b0 0x55b3c9583540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f52b400e630 tx=0x7f52b4009820 comp rx=0 tx=0).stop 2023-10-08T14:02:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.784+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/2237484435 shutdown_connections 2023-10-08T14:02:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.784+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b3c9593cc0 0x55b3c95960e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.784+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9583fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.784+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957a0b0 0x55b3c9583540 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.784+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/2237484435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c9583a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.784+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/2237484435 >> 172.21.15.57:0/2237484435 conn(0x55b3c94ee200 msgr2=0x55b3c9574010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.792+0000 7f529ffff700 0 client.26758 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.796+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/2237484435 shutdown_connections 2023-10-08T14:02:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.796+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/2237484435 wait complete. 2023-10-08T14:02:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.798+0000 7f52d2ecbb80 20 client.26758 trim_cache size 0 max 16384 2023-10-08T14:02:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.835+0000 7f52d2ecbb80 1 Processor -- start 2023-10-08T14:02:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.852+0000 7f52d2ecbb80 1 -- start start 2023-10-08T14:02:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.852+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957a0b0 0x55b3c9579b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.852+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957f880 0x55b3c957ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.852+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9583de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.852+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c934d8c0 con 0x55b3c95801a0 2023-10-08T14:02:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.852+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c956f2b0 con 0x55b3c957a0b0 2023-10-08T14:02:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.852+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c9571830 con 0x55b3c957f880 2023-10-08T14:02:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.853+0000 7f52c4a7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957a0b0 0x55b3c9579b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.853+0000 7f52bffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957f880 0x55b3c957ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.853+0000 7f52bffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957f880 0x55b3c957ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60682/0 (socket says 172.21.15.57:60682) 2023-10-08T14:02:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.853+0000 7f52bffff700 1 -- 172.21.15.57:0/220951028 learned_addr learned my addr 172.21.15.57:0/220951028 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.853+0000 7f52c4a7e700 1 -- 172.21.15.57:0/220951028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957f880 msgr2=0x55b3c957ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.853+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/220951028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957f880 0x55b3c957ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.853+0000 7f52c4a7e700 1 -- 172.21.15.57:0/220951028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 msgr2=0x55b3c9583de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:02:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.853+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/220951028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9583de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.853+0000 7f52c4a7e700 1 -- 172.21.15.57:0/220951028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c959ad10 con 0x55b3c957a0b0 2023-10-08T14:02:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.856+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/220951028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957a0b0 0x55b3c9579b70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52b400e630 tx=0x7f52b400eab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3822e7f696caeda2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.856+0000 7f52a77fe700 1 -- 172.21.15.57:0/220951028 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b404d020 con 0x55b3c957a0b0 2023-10-08T14:02:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.856+0000 7f52a77fe700 1 -- 172.21.15.57:0/220951028 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b403ebe0 con 0x55b3c957a0b0 2023-10-08T14:02:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.856+0000 7f52a77fe700 1 -- 172.21.15.57:0/220951028 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4052580 con 0x55b3c957a0b0 2023-10-08T14:02:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.857+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/220951028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957a0b0 msgr2=0x55b3c9579b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.857+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/220951028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957a0b0 0x55b3c9579b70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52b400e630 tx=0x7f52b400eab0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.857+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/220951028 shutdown_connections 2023-10-08T14:02:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.857+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/220951028 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c95801a0 0x55b3c9583de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.857+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/220951028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957a0b0 0x55b3c9579b70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.857+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/220951028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c957f880 0x55b3c957ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.857+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/220951028 >> 172.21.15.57:0/220951028 conn(0x55b3c94ee200 msgr2=0x55b3c957e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.857+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/220951028 shutdown_connections 2023-10-08T14:02:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.857+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/220951028 wait complete. 2023-10-08T14:02:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.858+0000 7f52d2ecbb80 1 -- start start 2023-10-08T14:02:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.858+0000 7f52d2ecbb80 1 Processor -- start 2023-10-08T14:02:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.858+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c957a0b0 0x55b3c958ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.858+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c9588c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.858+0000 7f52d2ecbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c95801a0 0x55b3c9589290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.858+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c95899a0 con 0x55b3c957a0b0 2023-10-08T14:02:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.858+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c9589b10 con 0x55b3c957f880 2023-10-08T14:02:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.858+0000 7f52d2ecbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c95e6b70 con 0x55b3c95801a0 2023-10-08T14:02:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.859+0000 7f52d2ecbb80 10 client.0 _ll_register_callbacks cb 0x55b3c92c67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:02:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.860+0000 7f52c527f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c95801a0 0x55b3c9589290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.861+0000 7f52bffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c9588c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.861+0000 7f52c4a7e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c957a0b0 0x55b3c958ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.861+0000 7f52c4a7e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c957a0b0 0x55b3c958ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59872/0 (socket says 172.21.15.57:59872) 2023-10-08T14:02:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.861+0000 7f52c4a7e700 1 -- 172.21.15.57:0/4081971082 learned_addr learned my addr 172.21.15.57:0/4081971082 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.862+0000 7f52c527f700 1 -- 172.21.15.57:0/4081971082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 msgr2=0x55b3c9588c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.862+0000 7f52c527f700 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c9588c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.862+0000 7f52c527f700 1 -- 172.21.15.57:0/4081971082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c957a0b0 msgr2=0x55b3c958ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.862+0000 7f52c527f700 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c957a0b0 0x55b3c958ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.862+0000 7f52c527f700 1 -- 172.21.15.57:0/4081971082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3c92e0c00 con 0x55b3c95801a0 2023-10-08T14:02:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.862+0000 7f52c527f700 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c95801a0 0x55b3c9589290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52b8099460 tx=0x7f52b8099bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7601ac7977bfc8e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.865+0000 7f529ffff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:02:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.865+0000 7f529ffff700 1 -- 172.21.15.57:0/4081971082 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8098bc0 con 0x55b3c95801a0 2023-10-08T14:02:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f529ffff700 1 -- 172.21.15.57:0/4081971082 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f52b809d620 con 0x55b3c95801a0 2023-10-08T14:02:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/4081971082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52b4010c20 con 0x55b3c95801a0 2023-10-08T14:02:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f529ffff700 1 -- 172.21.15.57:0/4081971082 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52b80a1860 con 0x55b3c95801a0 2023-10-08T14:02:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f52d2ecbb80 10 client.26764 fetch_fsmap learned FSMap version 46 2023-10-08T14:02:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/4081971082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55b3c931e1f0 con 0x55b3c95801a0 2023-10-08T14:02:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f529ffff700 1 -- 172.21.15.57:0/4081971082 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b4010c20 con 0x55b3c95801a0 2023-10-08T14:02:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f529ffff700 1 -- 172.21.15.57:0/4081971082 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 46) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f52b80b8020 con 0x55b3c95801a0 2023-10-08T14:02:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f52d2ecbb80 10 client.26764 fetch_fsmap finished waiting for FSMap version 46 2023-10-08T14:02:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f52d2ecbb80 10 client.26764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:02:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f52d2ecbb80 20 client.26764 populate_metadata read hostname 'smithi057' 2023-10-08T14:02:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.866+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b3c95e8400 0x55b3c95ea820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.867+0000 7f52d2ecbb80 4 client.26764 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:02:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.867+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/4081971082 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b3c94c5ad0 con 0x55b3c95e8400 2023-10-08T14:02:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.871+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b3c95e8400 0x55b3c95ea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.878+0000 7f52c4a7e700 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b3c95e8400 0x55b3c95ea820 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784b41bcc8667b45 server_cookie=8eeb0d9417e706ec in_seq=0 out_seq=0 2023-10-08T14:02:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.878+0000 7f529ffff700 10 client.26764 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.951+0000 7f529ffff700 1 -- 172.21.15.57:0/4081971082 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b3c94c5ad0 con 0x55b3c95e8400 2023-10-08T14:02:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.951+0000 7f529ffff700 10 client.26764 handle_command_reply: tid=0 2023-10-08T14:02:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.951+0000 7f52d2ecbb80 1 client.26764 shutdown 2023-10-08T14:02:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.954+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/4081971082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c95801a0 msgr2=0x55b3c9589290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.954+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c95801a0 0x55b3c9589290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52b8099460 tx=0x7f52b8099bd0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.959+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/4081971082 shutdown_connections 2023-10-08T14:02:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.959+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55b3c95e8400 0x55b3c95ea820 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.959+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b3c95801a0 0x55b3c9589290 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.959+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b3c957a0b0 0x55b3c958ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.959+0000 7f52d2ecbb80 1 --2- 172.21.15.57:0/4081971082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3c957f880 0x55b3c9588c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.959+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/4081971082 >> 172.21.15.57:0/4081971082 conn(0x55b3c94ee200 msgr2=0x55b3c9581e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.963+0000 7f529ffff700 0 client.26764 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.963+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/4081971082 shutdown_connections 2023-10-08T14:02:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.969+0000 7f52d2ecbb80 1 -- 172.21.15.57:0/4081971082 wait complete. 2023-10-08T14:02:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.976+0000 7f52d2ecbb80 20 client.26764 trim_cache size 0 max 16384 2023-10-08T14:02:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:02:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:02:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:02:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:02:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.978+0000 7f52c6ce2700 1 -- 172.21.15.57:0/3140759088 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a80a1790 msgr2=0x7f52a80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.978+0000 7f52c6ce2700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a80a1790 0x7f52a80a3c50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f52c0069cf0 tx=0x7f52b4009260 comp rx=0 tx=0).stop 2023-10-08T14:02:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.978+0000 7f52c6ce2700 1 -- 172.21.15.57:0/3140759088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52c01045e0 msgr2=0x7f52c010f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.978+0000 7f52c6ce2700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52c01045e0 0x7f52c010f690 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f52b800b970 tx=0x7f52b800bd30 comp rx=0 tx=0).stop 2023-10-08T14:02:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.988+0000 7f52c6ce2700 1 -- 172.21.15.57:0/3140759088 shutdown_connections 2023-10-08T14:02:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.988+0000 7f52c6ce2700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52c01045e0 0x7f52c010f690 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.988+0000 7f52c6ce2700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c00ff5d0 0x7f52c0115040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.988+0000 7f52c6ce2700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52a80a1790 0x7f52a80a3c50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.988+0000 7f52c6ce2700 1 --2- 172.21.15.57:0/3140759088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c00fff90 0x7f52c010f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.988+0000 7f52c6ce2700 1 -- 172.21.15.57:0/3140759088 >> 172.21.15.57:0/3140759088 conn(0x7f52c0076790 msgr2=0x7f52c00fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.994+0000 7f52c6ce2700 1 -- 172.21.15.57:0/3140759088 shutdown_connections 2023-10-08T14:02:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:31.994+0000 7f52c6ce2700 1 -- 172.21.15.57:0/3140759088 wait complete. 2023-10-08T14:02:32.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:31 smithi115 ceph-mon[104408]: pgmap v638: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 96 op/s 2023-10-08T14:02:32.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:31 smithi057 ceph-mon[99476]: pgmap v638: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 96 op/s 2023-10-08T14:02:32.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:31 smithi099 ceph-mon[103849]: pgmap v638: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 96 op/s 2023-10-08T14:02:34.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:33 smithi115 ceph-mon[104408]: pgmap v639: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 711 KiB/s wr, 86 op/s 2023-10-08T14:02:34.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:33 smithi057 ceph-mon[99476]: pgmap v639: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 711 KiB/s wr, 86 op/s 2023-10-08T14:02:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:33 smithi099 ceph-mon[103849]: pgmap v639: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 711 KiB/s wr, 86 op/s 2023-10-08T14:02:35.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:34 smithi115 ceph-mon[104408]: pgmap v640: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 99 op/s 2023-10-08T14:02:35.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:34 smithi057 ceph-mon[99476]: pgmap v640: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 99 op/s 2023-10-08T14:02:35.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:34 smithi099 ceph-mon[103849]: pgmap v640: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 99 op/s 2023-10-08T14:02:38.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:37 smithi099 ceph-mon[103849]: pgmap v641: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 770 KiB/s wr, 93 op/s 2023-10-08T14:02:38.197 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:37 smithi115 ceph-mon[104408]: pgmap v641: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 770 KiB/s wr, 93 op/s 2023-10-08T14:02:38.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:37 smithi057 ceph-mon[99476]: pgmap v641: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 770 KiB/s wr, 93 op/s 2023-10-08T14:02:39.687 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:39 smithi115 ceph-mon[104408]: pgmap v642: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 94 op/s 2023-10-08T14:02:39.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:39 smithi057 ceph-mon[99476]: pgmap v642: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 94 op/s 2023-10-08T14:02:39.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:39 smithi099 ceph-mon[103849]: pgmap v642: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 94 op/s 2023-10-08T14:02:41.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:41 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:02:41.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:41 smithi115 ceph-mon[104408]: pgmap v643: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 110 op/s 2023-10-08T14:02:41.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:41 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:02:41.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:41 smithi057 ceph-mon[99476]: pgmap v643: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 110 op/s 2023-10-08T14:02:41.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:41 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:02:41.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:41 smithi099 ceph-mon[103849]: pgmap v643: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 110 op/s 2023-10-08T14:02:44.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:43 smithi115 ceph-mon[104408]: pgmap v644: 129 pgs: 52 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 58 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 759 KiB/s wr, 98 op/s 2023-10-08T14:02:44.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:43 smithi057 ceph-mon[99476]: pgmap v644: 129 pgs: 52 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 58 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 759 KiB/s wr, 98 op/s 2023-10-08T14:02:44.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:43 smithi099 ceph-mon[103849]: pgmap v644: 129 pgs: 52 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 58 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 759 KiB/s wr, 98 op/s 2023-10-08T14:02:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.186+0000 7faa7d8ac700 1 -- 172.21.15.57:0/3675103846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa78107a60 msgr2=0x7faa78107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.186+0000 7faa7d8ac700 1 --2- 172.21.15.57:0/3675103846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa78107a60 0x7faa78107e40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faa68009990 tx=0x7faa68009ca0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.188+0000 7faa7d8ac700 1 -- 172.21.15.57:0/3675103846 shutdown_connections 2023-10-08T14:02:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.188+0000 7faa7d8ac700 1 --2- 172.21.15.57:0/3675103846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7810c070 0x7faa78078b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.188+0000 7faa7d8ac700 1 --2- 172.21.15.57:0/3675103846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa78107a60 0x7faa78107e40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.188+0000 7faa7d8ac700 1 --2- 172.21.15.57:0/3675103846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa78108380 0x7faa7810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.188+0000 7faa7d8ac700 1 -- 172.21.15.57:0/3675103846 >> 172.21.15.57:0/3675103846 conn(0x7faa7806b7c0 msgr2=0x7faa7806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.188+0000 7faa7d8ac700 1 -- 172.21.15.57:0/3675103846 shutdown_connections 2023-10-08T14:02:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.191+0000 7faa7d8ac700 1 -- 172.21.15.57:0/3675103846 wait complete. 2023-10-08T14:02:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.191+0000 7faa7d8ac700 1 Processor -- start 2023-10-08T14:02:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.196+0000 7faa7d8ac700 1 -- start start 2023-10-08T14:02:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.196+0000 7faa7d8ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa78107a60 0x7faa78083100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.196+0000 7faa7d8ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa78108380 0x7faa7807d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.196+0000 7faa7d8ac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7810c070 0x7faa7807d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.196+0000 7faa7d8ac700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7807dcf0 con 0x7faa7810c070 2023-10-08T14:02:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.196+0000 7faa7d8ac700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7807de60 con 0x7faa78108380 2023-10-08T14:02:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.196+0000 7faa7d8ac700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7807e020 con 0x7faa78107a60 2023-10-08T14:02:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.197+0000 7faa777fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7810c070 0x7faa7807d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.197+0000 7faa777fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7810c070 0x7faa7807d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59886/0 (socket says 172.21.15.57:59886) 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.197+0000 7faa777fe700 1 -- 172.21.15.57:0/2889175614 learned_addr learned my addr 172.21.15.57:0/2889175614 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.197+0000 7faa777fe700 1 -- 172.21.15.57:0/2889175614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa78107a60 msgr2=0x7faa78083100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.197+0000 7faa777fe700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa78107a60 0x7faa78083100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.197+0000 7faa777fe700 1 -- 172.21.15.57:0/2889175614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa78108380 msgr2=0x7faa7807d1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.197+0000 7faa777fe700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa78108380 0x7faa7807d1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.197+0000 7faa777fe700 1 -- 172.21.15.57:0/2889175614 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680095f0 con 0x7faa7810c070 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.198+0000 7faa777fe700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7810c070 0x7faa7807d6e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7faa70007ee0 tx=0x7faa7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.198+0000 7faa7c8aa700 1 -- 172.21.15.57:0/2889175614 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70017070 con 0x7faa7810c070 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.198+0000 7faa7c8aa700 1 -- 172.21.15.57:0/2889175614 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa7000f040 con 0x7faa7810c070 2023-10-08T14:02:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.198+0000 7faa7d8ac700 1 -- 172.21.15.57:0/2889175614 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa781b49e0 con 0x7faa7810c070 2023-10-08T14:02:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.198+0000 7faa7c8aa700 1 -- 172.21.15.57:0/2889175614 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70005740 con 0x7faa7810c070 2023-10-08T14:02:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.198+0000 7faa7d8ac700 1 -- 172.21.15.57:0/2889175614 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa781b4ef0 con 0x7faa7810c070 2023-10-08T14:02:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.203+0000 7faa89a95b80 1 Processor -- start 2023-10-08T14:02:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.208+0000 7faa7c8aa700 1 -- 172.21.15.57:0/2889175614 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7faa700058a0 con 0x7faa7810c070 2023-10-08T14:02:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.208+0000 7faa89a95b80 1 -- start start 2023-10-08T14:02:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.208+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a0880 0x559ab54a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.208+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab54a11a0 0x559ab549ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.208+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 0x559ab549b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.208+0000 7faa89a95b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab5403400 con 0x559ab54a11a0 2023-10-08T14:02:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.208+0000 7faa89a95b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab53f4bc0 con 0x559ab549b0b0 2023-10-08T14:02:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.208+0000 7faa89a95b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab53f24e0 con 0x559ab54a0880 2023-10-08T14:02:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.209+0000 7faa7c8aa700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa640c3660 0x7faa640c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.209+0000 7faa7c8aa700 1 -- 172.21.15.57:0/2889175614 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7faa700ccb60 con 0x7faa7810c070 2023-10-08T14:02:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.210+0000 7faa6ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a0880 0x559ab54a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.210+0000 7faa6ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a0880 0x559ab54a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60702/0 (socket says 172.21.15.57:60702) 2023-10-08T14:02:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.210+0000 7faa6ffff700 1 -- 172.21.15.57:0/2584406391 learned_addr learned my addr 172.21.15.57:0/2584406391 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.211+0000 7faa6ffff700 1 -- 172.21.15.57:0/2584406391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 msgr2=0x559ab549b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.211+0000 7faa6ffff700 1 --2- 172.21.15.57:0/2584406391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 0x559ab549b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.211+0000 7faa6ffff700 1 -- 172.21.15.57:0/2584406391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab54a11a0 msgr2=0x559ab549ab70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:02:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.211+0000 7faa6ffff700 1 --2- 172.21.15.57:0/2584406391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab54a11a0 0x559ab549ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.211+0000 7faa6ffff700 1 -- 172.21.15.57:0/2584406391 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ab5218d80 con 0x559ab54a0880 2023-10-08T14:02:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.211+0000 7faa6ffff700 1 --2- 172.21.15.57:0/2584406391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a0880 0x559ab54a0c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa60009cf0 tx=0x7faa600103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0721276a071aef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.211+0000 7faa6dffb700 1 -- 172.21.15.57:0/2584406391 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60010cc0 con 0x559ab54a0880 2023-10-08T14:02:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.211+0000 7faa6dffb700 1 -- 172.21.15.57:0/2584406391 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa60010e20 con 0x559ab54a0880 2023-10-08T14:02:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.211+0000 7faa6dffb700 1 -- 172.21.15.57:0/2584406391 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60005280 con 0x559ab54a0880 2023-10-08T14:02:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.212+0000 7faa777fe700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa640c3660 0x7faa640c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.212+0000 7faa89a95b80 1 -- 172.21.15.57:0/2584406391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a0880 msgr2=0x559ab54a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.212+0000 7faa89a95b80 1 --2- 172.21.15.57:0/2584406391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a0880 0x559ab54a0c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa60009cf0 tx=0x7faa600103d0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.213+0000 7faa89a95b80 1 -- 172.21.15.57:0/2584406391 shutdown_connections 2023-10-08T14:02:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.213+0000 7faa89a95b80 1 --2- 172.21.15.57:0/2584406391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 0x559ab549b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.213+0000 7faa89a95b80 1 --2- 172.21.15.57:0/2584406391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a0880 0x559ab54a0c60 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa60009cf0 tx=0x7faa600103d0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.213+0000 7faa89a95b80 1 --2- 172.21.15.57:0/2584406391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab54a11a0 0x559ab549ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.213+0000 7faa89a95b80 1 -- 172.21.15.57:0/2584406391 >> 172.21.15.57:0/2584406391 conn(0x559ab540f200 msgr2=0x559ab5495b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.213+0000 7faa777fe700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa640c3660 0x7faa640c5b20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa70095c30 tx=0x7faa7000b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.214+0000 7faa89a95b80 1 -- 172.21.15.57:0/2584406391 shutdown_connections 2023-10-08T14:02:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.214+0000 7faa89a95b80 1 -- 172.21.15.57:0/2584406391 wait complete. 2023-10-08T14:02:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.214+0000 7faa89a95b80 1 -- start start 2023-10-08T14:02:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.214+0000 7faa89a95b80 1 Processor -- start 2023-10-08T14:02:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.215+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 0x559ab54a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.215+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a4dd0 0x559ab54a5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.215+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab54aed20 0x559ab54b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.215+0000 7faa89a95b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab53eddd0 con 0x559ab549b0b0 2023-10-08T14:02:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.215+0000 7faa89a95b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab538a6b0 con 0x559ab54aed20 2023-10-08T14:02:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.215+0000 7faa89a95b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab538a350 con 0x559ab54a4dd0 2023-10-08T14:02:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.215+0000 7faa89a95b80 10 client.0 _ll_register_callbacks cb 0x559ab51e77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.215+0000 7faa76ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a4dd0 0x559ab54a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.215+0000 7faa76ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a4dd0 0x559ab54a5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60712/0 (socket says 172.21.15.57:60712) 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.215+0000 7faa76ffd700 1 -- 172.21.15.57:0/3055836648 learned_addr learned my addr 172.21.15.57:0/3055836648 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.216+0000 7faa76ffd700 1 -- 172.21.15.57:0/3055836648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab54aed20 msgr2=0x559ab54b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.216+0000 7faa76ffd700 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab54aed20 0x559ab54b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.216+0000 7faa76ffd700 1 -- 172.21.15.57:0/3055836648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 msgr2=0x559ab54a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.216+0000 7faa76ffd700 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 0x559ab54a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.216+0000 7faa76ffd700 1 -- 172.21.15.57:0/3055836648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa600099d0 con 0x559ab54a4dd0 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.216+0000 7faa76ffd700 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a4dd0 0x559ab54a5250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faa68005f50 tx=0x7faa6800ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=102f5ed4eb3a032 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.216+0000 7faa5effd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:02:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.216+0000 7faa5effd700 1 -- 172.21.15.57:0/3055836648 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6801d070 con 0x559ab54a4dd0 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.216+0000 7faa5effd700 1 -- 172.21.15.57:0/3055836648 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7faa6800f670 con 0x559ab54a4dd0 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.216+0000 7faa89a95b80 1 -- 172.21.15.57:0/3055836648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ab53b5e50 con 0x559ab54a4dd0 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa5effd700 1 -- 172.21.15.57:0/3055836648 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa6800bee0 con 0x559ab54a4dd0 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa89a95b80 10 client.26773 fetch_fsmap learned FSMap version 47 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa89a95b80 1 -- 172.21.15.57:0/3055836648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x559ab5201c00 con 0x559ab54a4dd0 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa5effd700 1 -- 172.21.15.57:0/3055836648 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ab53b5e50 con 0x559ab54a4dd0 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa5effd700 1 -- 172.21.15.57:0/3055836648 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) v1 ==== 7966+0+0 (secure 0 0 0) 0x7faa680178e0 con 0x559ab54a4dd0 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa89a95b80 10 client.26773 fetch_fsmap finished waiting for FSMap version 47 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa89a95b80 10 client.26773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa89a95b80 20 client.26773 populate_metadata read hostname 'smithi057' 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa89a95b80 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559ab54b5960 0x559ab54b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa89a95b80 4 client.26773 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:02:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa89a95b80 1 -- 172.21.15.57:0/3055836648 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ab53e6db0 con 0x559ab54b5960 2023-10-08T14:02:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.217+0000 7faa6ffff700 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559ab54b5960 0x559ab54b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.218+0000 7faa5effd700 10 client.26773 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.218+0000 7faa6ffff700 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559ab54b5960 0x559ab54b7d80 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c2058ab4eddb9d server_cookie=fe381580af3b1131 in_seq=0 out_seq=0 2023-10-08T14:02:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.219+0000 7faa5effd700 1 -- 172.21.15.57:0/3055836648 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559ab53e6db0 con 0x559ab54b5960 2023-10-08T14:02:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.219+0000 7faa5effd700 10 client.26773 handle_command_reply: tid=0 2023-10-08T14:02:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.219+0000 7faa89a95b80 1 client.26773 shutdown 2023-10-08T14:02:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.225+0000 7faa89a95b80 1 -- 172.21.15.57:0/3055836648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a4dd0 msgr2=0x559ab54a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.225+0000 7faa89a95b80 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a4dd0 0x559ab54a5250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faa68005f50 tx=0x7faa6800ba30 comp rx=0 tx=0).stop 2023-10-08T14:02:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.225+0000 7faa89a95b80 1 -- 172.21.15.57:0/3055836648 shutdown_connections 2023-10-08T14:02:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.225+0000 7faa89a95b80 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559ab54b5960 0x559ab54b7d80 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.225+0000 7faa89a95b80 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab54aed20 0x559ab54b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.225+0000 7faa89a95b80 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 0x559ab54a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.225+0000 7faa5effd700 0 client.26773 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.225+0000 7faa89a95b80 1 --2- 172.21.15.57:0/3055836648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a4dd0 0x559ab54a5250 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faa68005f50 tx=0x7faa6800ba30 comp rx=0 tx=0).stop 2023-10-08T14:02:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.225+0000 7faa89a95b80 1 -- 172.21.15.57:0/3055836648 >> 172.21.15.57:0/3055836648 conn(0x559ab540f200 msgr2=0x559ab5495010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.226+0000 7faa89a95b80 1 -- 172.21.15.57:0/3055836648 shutdown_connections 2023-10-08T14:02:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.226+0000 7faa89a95b80 1 -- 172.21.15.57:0/3055836648 wait complete. 2023-10-08T14:02:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.230+0000 7faa89a95b80 20 client.26773 trim_cache size 0 max 16384 2023-10-08T14:02:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.255+0000 7faa89a95b80 1 Processor -- start 2023-10-08T14:02:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.259+0000 7faa89a95b80 1 -- start start 2023-10-08T14:02:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.259+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 0x559ab54b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.259+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab54aed20 0x559ab54abc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.259+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a2590 0x559ab549b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.259+0000 7faa89a95b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab526e8c0 con 0x559ab54aed20 2023-10-08T14:02:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.259+0000 7faa89a95b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab54902b0 con 0x559ab549b0b0 2023-10-08T14:02:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.259+0000 7faa89a95b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab54a7400 con 0x559ab54a2590 2023-10-08T14:02:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.266+0000 7faa6ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 0x559ab54b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.266+0000 7faa6ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 0x559ab54b7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33098/0 (socket says 172.21.15.57:33098) 2023-10-08T14:02:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.266+0000 7faa6ffff700 1 -- 172.21.15.57:0/3073858967 learned_addr learned my addr 172.21.15.57:0/3073858967 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.266+0000 7faa6ffff700 1 -- 172.21.15.57:0/3073858967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a2590 msgr2=0x559ab549b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.266+0000 7faa6ffff700 1 --2- 172.21.15.57:0/3073858967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a2590 0x559ab549b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.266+0000 7faa6ffff700 1 -- 172.21.15.57:0/3073858967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab54aed20 msgr2=0x559ab54abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.266+0000 7faa6ffff700 1 --2- 172.21.15.57:0/3073858967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab54aed20 0x559ab54abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.266+0000 7faa6ffff700 1 -- 172.21.15.57:0/3073858967 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ab523f1f0 con 0x559ab549b0b0 2023-10-08T14:02:45.273 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.272+0000 7faa6ffff700 1 --2- 172.21.15.57:0/3073858967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 0x559ab54b7d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa60009cc0 tx=0x7faa60004a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe5a772b51258c1d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.273+0000 7faa6e7fc700 1 -- 172.21.15.57:0/3073858967 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60009850 con 0x559ab549b0b0 2023-10-08T14:02:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.273+0000 7faa6e7fc700 1 -- 172.21.15.57:0/3073858967 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa60025de0 con 0x559ab549b0b0 2023-10-08T14:02:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.273+0000 7faa6e7fc700 1 -- 172.21.15.57:0/3073858967 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600038e0 con 0x559ab549b0b0 2023-10-08T14:02:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.274+0000 7faa89a95b80 1 -- 172.21.15.57:0/3073858967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 msgr2=0x559ab54b7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.274+0000 7faa89a95b80 1 --2- 172.21.15.57:0/3073858967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 0x559ab54b7d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa60009cc0 tx=0x7faa60004a10 comp rx=0 tx=0).stop 2023-10-08T14:02:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.278+0000 7faa89a95b80 1 -- 172.21.15.57:0/3073858967 shutdown_connections 2023-10-08T14:02:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.278+0000 7faa89a95b80 1 --2- 172.21.15.57:0/3073858967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54a2590 0x559ab549b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.278+0000 7faa89a95b80 1 --2- 172.21.15.57:0/3073858967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab549b0b0 0x559ab54b7d50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.278+0000 7faa89a95b80 1 --2- 172.21.15.57:0/3073858967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab54aed20 0x559ab54abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.278+0000 7faa89a95b80 1 -- 172.21.15.57:0/3073858967 >> 172.21.15.57:0/3073858967 conn(0x559ab540f200 msgr2=0x559ab54bc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.282+0000 7faa89a95b80 1 -- 172.21.15.57:0/3073858967 shutdown_connections 2023-10-08T14:02:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.290+0000 7faa89a95b80 1 -- 172.21.15.57:0/3073858967 wait complete. 2023-10-08T14:02:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.290+0000 7faa89a95b80 1 -- start start 2023-10-08T14:02:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.290+0000 7faa89a95b80 1 Processor -- start 2023-10-08T14:02:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 0x559ab54a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab54a2590 0x559ab54aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa89a95b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54aed20 0x559ab55087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa89a95b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab54aaac0 con 0x559ab549b0b0 2023-10-08T14:02:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa89a95b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab5508cf0 con 0x559ab54a2590 2023-10-08T14:02:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa89a95b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab5508e80 con 0x559ab54aed20 2023-10-08T14:02:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa89a95b80 10 client.0 _ll_register_callbacks cb 0x7faa60005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:02:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa6ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 0x559ab54a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa6ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 0x559ab54a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59910/0 (socket says 172.21.15.57:59910) 2023-10-08T14:02:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa6ffff700 1 -- 172.21.15.57:0/1491003344 learned_addr learned my addr 172.21.15.57:0/1491003344 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.297+0000 7faa6ffff700 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54aed20 0x559ab55087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.299+0000 7faa6ffff700 1 -- 172.21.15.57:0/1491003344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54aed20 msgr2=0x559ab55087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.299+0000 7faa6ffff700 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54aed20 0x559ab55087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.299+0000 7faa6ffff700 1 -- 172.21.15.57:0/1491003344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab54a2590 msgr2=0x559ab54aa390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:02:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.299+0000 7faa6ffff700 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab54a2590 0x559ab54aa390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.299+0000 7faa6ffff700 1 -- 172.21.15.57:0/1491003344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ab5218d80 con 0x559ab549b0b0 2023-10-08T14:02:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.299+0000 7faa6ffff700 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 0x559ab54a9e50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7faa60028040 tx=0x7faa600103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73da368ea7006fc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.299+0000 7faa5effd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:02:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.299+0000 7faa5effd700 1 -- 172.21.15.57:0/1491003344 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60025920 con 0x559ab549b0b0 2023-10-08T14:02:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.299+0000 7faa5effd700 1 -- 172.21.15.57:0/1491003344 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7faa60000e40 con 0x559ab549b0b0 2023-10-08T14:02:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.302+0000 7faa89a95b80 1 -- 172.21.15.57:0/1491003344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa60010e20 con 0x559ab549b0b0 2023-10-08T14:02:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.302+0000 7faa5effd700 1 -- 172.21.15.57:0/1491003344 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa60025a80 con 0x559ab549b0b0 2023-10-08T14:02:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.306+0000 7faa89a95b80 10 client.17211 fetch_fsmap learned FSMap version 47 2023-10-08T14:02:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.306+0000 7faa89a95b80 1 -- 172.21.15.57:0/1491003344 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x559ab5217250 con 0x559ab549b0b0 2023-10-08T14:02:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.307+0000 7faa5effd700 1 -- 172.21.15.57:0/1491003344 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa60010e20 con 0x559ab549b0b0 2023-10-08T14:02:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.308+0000 7faa5effd700 1 -- 172.21.15.57:0/1491003344 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 47) v1 ==== 7966+0+0 (secure 0 0 0) 0x7faa60017070 con 0x559ab549b0b0 2023-10-08T14:02:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.308+0000 7faa89a95b80 10 client.17211 fetch_fsmap finished waiting for FSMap version 47 2023-10-08T14:02:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.308+0000 7faa89a95b80 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:02:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.308+0000 7faa89a95b80 20 client.17211 populate_metadata read hostname 'smithi057' 2023-10-08T14:02:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.308+0000 7faa89a95b80 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559ab550a180 0x559ab550c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.308+0000 7faa89a95b80 4 client.17211 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:02:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.308+0000 7faa89a95b80 1 -- 172.21.15.57:0/1491003344 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559ab53e6ad0 con 0x559ab550a180 2023-10-08T14:02:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.317+0000 7faa76ffd700 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559ab550a180 0x559ab550c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.324+0000 7faa5effd700 10 client.17211 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.324+0000 7faa76ffd700 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559ab550a180 0x559ab550c570 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3648345ba3aaafed server_cookie=7f3cf89e7adac413 in_seq=0 out_seq=0 2023-10-08T14:02:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.429+0000 7faa5effd700 1 -- 172.21.15.57:0/1491003344 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559ab53e6ad0 con 0x559ab550a180 2023-10-08T14:02:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.429+0000 7faa5effd700 10 client.17211 handle_command_reply: tid=0 2023-10-08T14:02:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.429+0000 7faa89a95b80 1 client.17211 shutdown 2023-10-08T14:02:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.430+0000 7faa89a95b80 1 -- 172.21.15.57:0/1491003344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 msgr2=0x559ab54a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.430+0000 7faa89a95b80 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 0x559ab54a9e50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7faa60028040 tx=0x7faa600103d0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.435+0000 7faa89a95b80 1 -- 172.21.15.57:0/1491003344 shutdown_connections 2023-10-08T14:02:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.435+0000 7faa89a95b80 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x559ab550a180 0x559ab550c570 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.435+0000 7faa89a95b80 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ab54aed20 0x559ab55087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.435+0000 7faa89a95b80 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x559ab549b0b0 0x559ab54a9e50 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.435+0000 7faa89a95b80 1 --2- 172.21.15.57:0/1491003344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ab54a2590 0x559ab54aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.435+0000 7faa89a95b80 1 -- 172.21.15.57:0/1491003344 >> 172.21.15.57:0/1491003344 conn(0x559ab540f200 msgr2=0x559ab549b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.435+0000 7faa5effd700 0 client.17211 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.446+0000 7faa89a95b80 1 -- 172.21.15.57:0/1491003344 shutdown_connections 2023-10-08T14:02:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.447+0000 7faa89a95b80 1 -- 172.21.15.57:0/1491003344 wait complete. 2023-10-08T14:02:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.454+0000 7faa89a95b80 20 client.17211 trim_cache size 0 max 16384 2023-10-08T14:02:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:02:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:02:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:02:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:02:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.465+0000 7faa7d8ac700 1 -- 172.21.15.57:0/2889175614 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa640c3660 msgr2=0x7faa640c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.465+0000 7faa7d8ac700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa640c3660 0x7faa640c5b20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa70095c30 tx=0x7faa7000b040 comp rx=0 tx=0).stop 2023-10-08T14:02:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.465+0000 7faa7d8ac700 1 -- 172.21.15.57:0/2889175614 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7810c070 msgr2=0x7faa7807d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.465+0000 7faa7d8ac700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7810c070 0x7faa7807d6e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7faa70007ee0 tx=0x7faa7000d3b0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.467+0000 7faa7d8ac700 1 -- 172.21.15.57:0/2889175614 shutdown_connections 2023-10-08T14:02:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.467+0000 7faa7d8ac700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7810c070 0x7faa7807d6e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.467+0000 7faa7d8ac700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa78107a60 0x7faa78083100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.467+0000 7faa7d8ac700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7faa640c3660 0x7faa640c5b20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.467+0000 7faa7d8ac700 1 --2- 172.21.15.57:0/2889175614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa78108380 0x7faa7807d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.467+0000 7faa7d8ac700 1 -- 172.21.15.57:0/2889175614 >> 172.21.15.57:0/2889175614 conn(0x7faa7806b7c0 msgr2=0x7faa78070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.472+0000 7faa7d8ac700 1 -- 172.21.15.57:0/2889175614 shutdown_connections 2023-10-08T14:02:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:45.477+0000 7faa7d8ac700 1 -- 172.21.15.57:0/2889175614 wait complete. 2023-10-08T14:02:46.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:45 smithi115 ceph-mon[104408]: pgmap v645: 129 pgs: 52 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 60 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 816 KiB/s wr, 112 op/s 2023-10-08T14:02:46.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:45 smithi057 ceph-mon[99476]: pgmap v645: 129 pgs: 52 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 60 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 816 KiB/s wr, 112 op/s 2023-10-08T14:02:46.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:45 smithi099 ceph-mon[103849]: pgmap v645: 129 pgs: 52 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 60 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 816 KiB/s wr, 112 op/s 2023-10-08T14:02:47.191 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:46 smithi115 ceph-mon[104408]: pgmap v646: 129 pgs: 52 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 60 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 776 KiB/s wr, 102 op/s 2023-10-08T14:02:47.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:46 smithi057 ceph-mon[99476]: pgmap v646: 129 pgs: 52 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 60 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 776 KiB/s wr, 102 op/s 2023-10-08T14:02:47.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:46 smithi099 ceph-mon[103849]: pgmap v646: 129 pgs: 52 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 60 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 776 KiB/s wr, 102 op/s 2023-10-08T14:02:49.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:49 smithi115 ceph-mon[104408]: pgmap v647: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 782 KiB/s wr, 100 op/s 2023-10-08T14:02:49.746 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:49 smithi057 ceph-mon[99476]: pgmap v647: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 782 KiB/s wr, 100 op/s 2023-10-08T14:02:49.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:49 smithi099 ceph-mon[103849]: pgmap v647: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 782 KiB/s wr, 100 op/s 2023-10-08T14:02:51.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:51 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:02:51.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:51 smithi115 ceph-mon[104408]: pgmap v648: 129 pgs: 37 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 77 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 870 KiB/s wr, 129 op/s 2023-10-08T14:02:51.742 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:51 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:02:51.742 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:51 smithi057 ceph-mon[99476]: pgmap v648: 129 pgs: 37 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 77 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 870 KiB/s wr, 129 op/s 2023-10-08T14:02:51.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:51 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:02:51.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:51 smithi099 ceph-mon[103849]: pgmap v648: 129 pgs: 37 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 77 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 870 KiB/s wr, 129 op/s 2023-10-08T14:02:54.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:53 smithi115 ceph-mon[104408]: pgmap v649: 129 pgs: 33 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 83 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 902 KiB/s wr, 119 op/s 2023-10-08T14:02:54.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:53 smithi057 ceph-mon[99476]: pgmap v649: 129 pgs: 33 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 83 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 902 KiB/s wr, 119 op/s 2023-10-08T14:02:54.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:53 smithi099 ceph-mon[103849]: pgmap v649: 129 pgs: 33 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 83 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 902 KiB/s wr, 119 op/s 2023-10-08T14:02:56.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:55 smithi115 ceph-mon[104408]: pgmap v650: 129 pgs: 20 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 98 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 932 KiB/s wr, 141 op/s 2023-10-08T14:02:56.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:55 smithi057 ceph-mon[99476]: pgmap v650: 129 pgs: 20 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 98 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 932 KiB/s wr, 141 op/s 2023-10-08T14:02:56.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:55 smithi099 ceph-mon[103849]: pgmap v650: 129 pgs: 20 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 98 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 932 KiB/s wr, 141 op/s 2023-10-08T14:02:58.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:57 smithi115 ceph-mon[104408]: pgmap v651: 129 pgs: 14 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 105 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2023-10-08T14:02:58.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:57 smithi057 ceph-mon[99476]: pgmap v651: 129 pgs: 14 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 105 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2023-10-08T14:02:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:57 smithi099 ceph-mon[103849]: pgmap v651: 129 pgs: 14 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 105 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2023-10-08T14:02:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.667+0000 7fb56e817700 1 -- 172.21.15.57:0/3364636749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5681040e0 msgr2=0x7fb56810bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.667+0000 7fb56e817700 1 --2- 172.21.15.57:0/3364636749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5681040e0 0x7fb56810bc60 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb56001c0d0 tx=0x7fb56001c3e0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.667+0000 7fb56e817700 1 -- 172.21.15.57:0/3364636749 shutdown_connections 2023-10-08T14:02:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.667+0000 7fb56e817700 1 --2- 172.21.15.57:0/3364636749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5681040e0 0x7fb56810bc60 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.667+0000 7fb56e817700 1 --2- 172.21.15.57:0/3364636749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5680fefc0 0x7fb5680ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.667+0000 7fb56e817700 1 --2- 172.21.15.57:0/3364636749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5680ff970 0x7fb5681039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.667+0000 7fb56e817700 1 -- 172.21.15.57:0/3364636749 >> 172.21.15.57:0/3364636749 conn(0x7fb568075bf0 msgr2=0x7fb5680fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.681+0000 7fb56e817700 1 -- 172.21.15.57:0/3364636749 shutdown_connections 2023-10-08T14:02:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.682+0000 7fb56e817700 1 -- 172.21.15.57:0/3364636749 wait complete. 2023-10-08T14:02:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.683+0000 7fb56e817700 1 Processor -- start 2023-10-08T14:02:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.683+0000 7fb56e817700 1 -- start start 2023-10-08T14:02:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.684+0000 7fb56e817700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5680fefc0 0x7fb568106de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.684+0000 7fb56e817700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5681040e0 0x7fb568107320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.684+0000 7fb56e817700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb568107860 0x7fb5681c2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.684+0000 7fb56e817700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56810bf90 con 0x7fb5680fefc0 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.684+0000 7fb56e817700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5681c31d0 con 0x7fb5681040e0 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.684+0000 7fb56e817700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5681c3310 con 0x7fb568107860 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.686+0000 7fb567fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5680fefc0 0x7fb568106de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.686+0000 7fb567fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5680fefc0 0x7fb568106de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59928/0 (socket says 172.21.15.57:59928) 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.686+0000 7fb567fff700 1 -- 172.21.15.57:0/3286891607 learned_addr learned my addr 172.21.15.57:0/3286891607 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.686+0000 7fb567fff700 1 -- 172.21.15.57:0/3286891607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb568107860 msgr2=0x7fb5681c2c90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.686+0000 7fb567fff700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb568107860 0x7fb5681c2c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.686+0000 7fb567fff700 1 -- 172.21.15.57:0/3286891607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5681040e0 msgr2=0x7fb568107320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.686+0000 7fb567fff700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5681040e0 0x7fb568107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.686+0000 7fb567fff700 1 -- 172.21.15.57:0/3286891607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56001ad10 con 0x7fb5680fefc0 2023-10-08T14:02:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.687+0000 7fb567fff700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5680fefc0 0x7fb568106de0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00ddc0 tx=0x7fb55c00a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.688+0000 7fb5657fa700 1 -- 172.21.15.57:0/3286891607 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c00dea0 con 0x7fb5680fefc0 2023-10-08T14:02:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.688+0000 7fb5657fa700 1 -- 172.21.15.57:0/3286891607 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb55c00e910 con 0x7fb5680fefc0 2023-10-08T14:02:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.688+0000 7fb5657fa700 1 -- 172.21.15.57:0/3286891607 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0169a0 con 0x7fb5680fefc0 2023-10-08T14:02:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.688+0000 7fb56e817700 1 -- 172.21.15.57:0/3286891607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5681c35f0 con 0x7fb5680fefc0 2023-10-08T14:02:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.688+0000 7fb56e817700 1 -- 172.21.15.57:0/3286891607 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5681c3ab0 con 0x7fb5680fefc0 2023-10-08T14:02:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.693+0000 7fb57aa00b80 1 Processor -- start 2023-10-08T14:02:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.694+0000 7fb57aa00b80 1 -- start start 2023-10-08T14:02:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.695+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf138c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.695+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1391a0 0x561daf132b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.695+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1330b0 0x561daf133510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.695+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf09b400 con 0x561daf1391a0 2023-10-08T14:02:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.695+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf08cbc0 con 0x561daf1330b0 2023-10-08T14:02:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.695+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf08a4e0 con 0x561daf138880 2023-10-08T14:02:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.695+0000 7fb5657fa700 1 -- 172.21.15.57:0/3286891607 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb55c00ea80 con 0x7fb5680fefc0 2023-10-08T14:02:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.696+0000 7fb5657fa700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb5500a1800 0x7fb5500a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.696+0000 7fb5657fa700 1 -- 172.21.15.57:0/3286891607 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fb55c0cd290 con 0x7fb5680fefc0 2023-10-08T14:02:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.698+0000 7fb567fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1391a0 0x561daf132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.698+0000 7fb567fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1391a0 0x561daf132b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59932/0 (socket says 172.21.15.57:59932) 2023-10-08T14:02:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.698+0000 7fb567fff700 1 -- 172.21.15.57:0/3115901861 learned_addr learned my addr 172.21.15.57:0/3115901861 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.700+0000 7fb56cdb4700 1 --2- 172.21.15.57:0/3115901861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf138c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.700+0000 7fb56cdb4700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb5500a1800 0x7fb5500a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.700+0000 7fb56cdb4700 1 -- 172.21.15.57:0/3115901861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1330b0 msgr2=0x561daf133510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.700+0000 7fb56cdb4700 1 --2- 172.21.15.57:0/3115901861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1330b0 0x561daf133510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.700+0000 7fb56cdb4700 1 -- 172.21.15.57:0/3115901861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1391a0 msgr2=0x561daf132b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.700+0000 7fb56cdb4700 1 --2- 172.21.15.57:0/3115901861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1391a0 0x561daf132b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.700+0000 7fb56cdb4700 1 -- 172.21.15.57:0/3115901861 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561daeeb0d80 con 0x561daf138880 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.700+0000 7fb56cdb4700 1 --2- 172.21.15.57:0/3115901861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf138c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb56001c0d0 tx=0x7fb560019d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b2d81f6a4c482b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.701+0000 7fb54e7fc700 1 -- 172.21.15.57:0/3115901861 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56001f040 con 0x561daf138880 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.701+0000 7fb54e7fc700 1 -- 172.21.15.57:0/3115901861 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb560018640 con 0x561daf138880 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.701+0000 7fb54e7fc700 1 -- 172.21.15.57:0/3115901861 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5600078c0 con 0x561daf138880 2023-10-08T14:02:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.701+0000 7fb56cdb4700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb5500a1800 0x7fb5500a3cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb560018000 tx=0x7fb560016000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.705+0000 7fb57aa00b80 1 -- 172.21.15.57:0/3115901861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 msgr2=0x561daf138c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.705+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/3115901861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf138c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb56001c0d0 tx=0x7fb560019d20 comp rx=0 tx=0).stop 2023-10-08T14:02:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.713+0000 7fb57aa00b80 1 -- 172.21.15.57:0/3115901861 shutdown_connections 2023-10-08T14:02:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.713+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/3115901861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1330b0 0x561daf133510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.713+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/3115901861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf138c60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.713+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/3115901861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1391a0 0x561daf132b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.713+0000 7fb57aa00b80 1 -- 172.21.15.57:0/3115901861 >> 172.21.15.57:0/3115901861 conn(0x561daf0a7200 msgr2=0x561daf12db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.718+0000 7fb57aa00b80 1 -- 172.21.15.57:0/3115901861 shutdown_connections 2023-10-08T14:02:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.721+0000 7fb57aa00b80 1 -- 172.21.15.57:0/3115901861 wait complete. 2023-10-08T14:02:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.722+0000 7fb57aa00b80 1 -- start start 2023-10-08T14:02:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.722+0000 7fb57aa00b80 1 Processor -- start 2023-10-08T14:02:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.722+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1330b0 0x561daf13f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.722+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf13c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.722+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1391a0 0x561daf13cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.722+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf085dd0 con 0x561daf1330b0 2023-10-08T14:02:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.722+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf0226b0 con 0x561daf1391a0 2023-10-08T14:02:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.722+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf022350 con 0x561daf138880 2023-10-08T14:02:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.723+0000 7fb57aa00b80 10 client.0 _ll_register_callbacks cb 0x561daee7f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:02:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.723+0000 7fb5677fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf13c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.723+0000 7fb5677fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf13c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60752/0 (socket says 172.21.15.57:60752) 2023-10-08T14:02:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.723+0000 7fb5677fe700 1 -- 172.21.15.57:0/113720891 learned_addr learned my addr 172.21.15.57:0/113720891 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.724+0000 7fb5677fe700 1 -- 172.21.15.57:0/113720891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1391a0 msgr2=0x561daf13cc00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:02:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.724+0000 7fb5677fe700 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1391a0 0x561daf13cc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.724+0000 7fb5677fe700 1 -- 172.21.15.57:0/113720891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1330b0 msgr2=0x561daf13f1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:02:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.724+0000 7fb5677fe700 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1330b0 0x561daf13f1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.724+0000 7fb5677fe700 1 -- 172.21.15.57:0/113720891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56810c4d0 con 0x561daf138880 2023-10-08T14:02:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.731+0000 7fb5677fe700 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf13c4c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb558009ed0 tx=0x7fb55800b3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d9f9e049e015105 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.731+0000 7fb5477fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:02:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.732+0000 7fb5477fe700 1 -- 172.21.15.57:0/113720891 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558011420 con 0x561daf138880 2023-10-08T14:02:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.732+0000 7fb5477fe700 1 -- 172.21.15.57:0/113720891 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fb558011580 con 0x561daf138880 2023-10-08T14:02:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.732+0000 7fb57aa00b80 1 -- 172.21.15.57:0/113720891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561daf04de50 con 0x561daf138880 2023-10-08T14:02:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.735+0000 7fb5477fe700 1 -- 172.21.15.57:0/113720891 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb558011740 con 0x561daf138880 2023-10-08T14:02:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.735+0000 7fb57aa00b80 10 client.26785 fetch_fsmap learned FSMap version 48 2023-10-08T14:02:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.735+0000 7fb57aa00b80 1 -- 172.21.15.57:0/113720891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x561daee99c00 con 0x561daf138880 2023-10-08T14:02:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.738+0000 7fb5477fe700 1 -- 172.21.15.57:0/113720891 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561daf04de50 con 0x561daf138880 2023-10-08T14:02:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.738+0000 7fb5477fe700 1 -- 172.21.15.57:0/113720891 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fb55801aa10 con 0x561daf138880 2023-10-08T14:02:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.738+0000 7fb57aa00b80 10 client.26785 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:02:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.738+0000 7fb57aa00b80 10 client.26785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:02:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.738+0000 7fb57aa00b80 20 client.26785 populate_metadata read hostname 'smithi057' 2023-10-08T14:02:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.738+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561daf142850 0x561daf144c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.738+0000 7fb57aa00b80 4 client.26785 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:02:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.738+0000 7fb57aa00b80 1 -- 172.21.15.57:0/113720891 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561daf07edb0 con 0x561daf142850 2023-10-08T14:02:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.738+0000 7fb567fff700 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561daf142850 0x561daf144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.738+0000 7fb5477fe700 10 client.26785 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.739+0000 7fb567fff700 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561daf142850 0x561daf144c70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d136c1e77f10ea82 server_cookie=4cf63bed3fd8ef0c in_seq=0 out_seq=0 2023-10-08T14:02:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.740+0000 7fb5477fe700 1 -- 172.21.15.57:0/113720891 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561daf07edb0 con 0x561daf142850 2023-10-08T14:02:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.740+0000 7fb5477fe700 10 client.26785 handle_command_reply: tid=0 2023-10-08T14:02:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 client.26785 shutdown 2023-10-08T14:02:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 -- 172.21.15.57:0/113720891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 msgr2=0x561daf13c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf13c4c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb558009ed0 tx=0x7fb55800b3c0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 -- 172.21.15.57:0/113720891 shutdown_connections 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561daf142850 0x561daf144c70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1391a0 0x561daf13cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1330b0 0x561daf13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/113720891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf138880 0x561daf13c4c0 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb558009ed0 tx=0x7fb55800b3c0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 -- 172.21.15.57:0/113720891 >> 172.21.15.57:0/113720891 conn(0x561daf0a7200 msgr2=0x561daf12d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb5477fe700 0 client.26785 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 -- 172.21.15.57:0/113720891 shutdown_connections 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.743+0000 7fb57aa00b80 1 -- 172.21.15.57:0/113720891 wait complete. 2023-10-08T14:02:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.744+0000 7fb57aa00b80 20 client.26785 trim_cache size 0 max 16384 2023-10-08T14:02:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.772+0000 7fb57aa00b80 1 Processor -- start 2023-10-08T14:02:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.772+0000 7fb57aa00b80 1 -- start start 2023-10-08T14:02:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.772+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf1330b0 0x561daf13c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.772+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1391a0 0x561daf154c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.772+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1495d0 0x561daf13ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.772+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daef068c0 con 0x561daf1391a0 2023-10-08T14:02:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.772+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf12a830 con 0x561daf1495d0 2023-10-08T14:02:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.772+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf13dac0 con 0x561daf1330b0 2023-10-08T14:02:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.773+0000 7fb56cdb4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1495d0 0x561daf13ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.773+0000 7fb56cdb4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1495d0 0x561daf13ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33140/0 (socket says 172.21.15.57:33140) 2023-10-08T14:02:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.773+0000 7fb56cdb4700 1 -- 172.21.15.57:0/4237308684 learned_addr learned my addr 172.21.15.57:0/4237308684 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.773+0000 7fb56cdb4700 1 -- 172.21.15.57:0/4237308684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf1330b0 msgr2=0x561daf13c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:02:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.773+0000 7fb56cdb4700 1 --2- 172.21.15.57:0/4237308684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf1330b0 0x561daf13c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.773+0000 7fb56cdb4700 1 -- 172.21.15.57:0/4237308684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1391a0 msgr2=0x561daf154c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:02:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.773+0000 7fb56cdb4700 1 --2- 172.21.15.57:0/4237308684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1391a0 0x561daf154c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.773+0000 7fb56cdb4700 1 -- 172.21.15.57:0/4237308684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561daeed71f0 con 0x561daf1495d0 2023-10-08T14:02:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.773+0000 7fb56cdb4700 1 --2- 172.21.15.57:0/4237308684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1495d0 0x561daf13ed00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb56001a5f0 tx=0x7fb56001a790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed660081d1e8a6e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.774+0000 7fb54effd700 1 -- 172.21.15.57:0/4237308684 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5600189b0 con 0x561daf1495d0 2023-10-08T14:02:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.774+0000 7fb54effd700 1 -- 172.21.15.57:0/4237308684 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb560007b60 con 0x561daf1495d0 2023-10-08T14:02:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.774+0000 7fb54effd700 1 -- 172.21.15.57:0/4237308684 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560007700 con 0x561daf1495d0 2023-10-08T14:02:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.774+0000 7fb57aa00b80 1 -- 172.21.15.57:0/4237308684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1495d0 msgr2=0x561daf13ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.775+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/4237308684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1495d0 0x561daf13ed00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb56001a5f0 tx=0x7fb56001a790 comp rx=0 tx=0).stop 2023-10-08T14:02:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.777+0000 7fb57aa00b80 1 -- 172.21.15.57:0/4237308684 shutdown_connections 2023-10-08T14:02:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.777+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/4237308684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf1495d0 0x561daf13ed00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.777+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/4237308684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf1330b0 0x561daf13c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.777+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/4237308684 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1391a0 0x561daf154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.777+0000 7fb57aa00b80 1 -- 172.21.15.57:0/4237308684 >> 172.21.15.57:0/4237308684 conn(0x561daf0a7200 msgr2=0x561daf13bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.786+0000 7fb57aa00b80 1 -- 172.21.15.57:0/4237308684 shutdown_connections 2023-10-08T14:02:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.788+0000 7fb57aa00b80 1 -- 172.21.15.57:0/4237308684 wait complete. 2023-10-08T14:02:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.788+0000 7fb57aa00b80 1 -- start start 2023-10-08T14:02:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.788+0000 7fb57aa00b80 1 Processor -- start 2023-10-08T14:02:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.789+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf1391a0 0x561daf154510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.789+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1495d0 0x561daf154a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.789+0000 7fb57aa00b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf152c10 0x561daf153090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.789+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf1535d0 con 0x561daf1495d0 2023-10-08T14:02:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.789+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf153710 con 0x561daf152c10 2023-10-08T14:02:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.789+0000 7fb57aa00b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daf1538d0 con 0x561daf1391a0 2023-10-08T14:02:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.789+0000 7fb57aa00b80 10 client.0 _ll_register_callbacks cb 0x7fb558009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:02:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.790+0000 7fb5677fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1495d0 0x561daf154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.790+0000 7fb5677fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1495d0 0x561daf154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59952/0 (socket says 172.21.15.57:59952) 2023-10-08T14:02:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.790+0000 7fb5677fe700 1 -- 172.21.15.57:0/695671060 learned_addr learned my addr 172.21.15.57:0/695671060 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:02:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.791+0000 7fb5677fe700 1 -- 172.21.15.57:0/695671060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf1391a0 msgr2=0x561daf154510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:02:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.791+0000 7fb5677fe700 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf1391a0 0x561daf154510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.791+0000 7fb5677fe700 1 -- 172.21.15.57:0/695671060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf152c10 msgr2=0x561daf153090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:02:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.791+0000 7fb5677fe700 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf152c10 0x561daf153090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.791+0000 7fb5677fe700 1 -- 172.21.15.57:0/695671060 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561daeeb0d80 con 0x561daf1495d0 2023-10-08T14:02:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.807+0000 7fb5677fe700 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1495d0 0x561daf154a50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb55801ae70 tx=0x7fb55800bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6b10c7608c5257 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:02:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.807+0000 7fb5477fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:02:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.808+0000 7fb5477fe700 1 -- 172.21.15.57:0/695671060 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55801a9b0 con 0x561daf1495d0 2023-10-08T14:02:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.808+0000 7fb5477fe700 1 -- 172.21.15.57:0/695671060 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7fb55801ab10 con 0x561daf1495d0 2023-10-08T14:02:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.808+0000 7fb57aa00b80 1 -- 172.21.15.57:0/695671060 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb560018640 con 0x561daf1495d0 2023-10-08T14:02:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.808+0000 7fb5477fe700 1 -- 172.21.15.57:0/695671060 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb558011420 con 0x561daf1495d0 2023-10-08T14:02:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.810+0000 7fb57aa00b80 10 client.17223 fetch_fsmap learned FSMap version 48 2023-10-08T14:02:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.810+0000 7fb57aa00b80 1 -- 172.21.15.57:0/695671060 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x561daeeaf250 con 0x561daf1495d0 2023-10-08T14:02:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.818+0000 7fb5477fe700 1 -- 172.21.15.57:0/695671060 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb560018640 con 0x561daf1495d0 2023-10-08T14:02:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.818+0000 7fb5477fe700 1 -- 172.21.15.57:0/695671060 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fb5580376c0 con 0x561daf1495d0 2023-10-08T14:02:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.818+0000 7fb57aa00b80 10 client.17223 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:02:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.818+0000 7fb57aa00b80 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:02:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.818+0000 7fb57aa00b80 20 client.17223 populate_metadata read hostname 'smithi057' 2023-10-08T14:02:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.818+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561daf1c45a0 0x561daf1c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:02:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.818+0000 7fb57aa00b80 4 client.17223 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:02:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.818+0000 7fb57aa00b80 1 -- 172.21.15.57:0/695671060 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561daf07ead0 con 0x561daf1c45a0 2023-10-08T14:02:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.821+0000 7fb567fff700 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561daf1c45a0 0x561daf1c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:02:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.829+0000 7fb5477fe700 10 client.17223 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.829+0000 7fb567fff700 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561daf1c45a0 0x561daf1c69c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fda0fb42b140521 server_cookie=5efd7e7ead02f5bf in_seq=0 out_seq=0 2023-10-08T14:02:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.873+0000 7fb5477fe700 1 -- 172.21.15.57:0/695671060 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561daf07ead0 con 0x561daf1c45a0 2023-10-08T14:02:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.873+0000 7fb5477fe700 10 client.17223 handle_command_reply: tid=0 2023-10-08T14:02:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.874+0000 7fb57aa00b80 1 client.17223 shutdown 2023-10-08T14:02:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.874+0000 7fb57aa00b80 1 -- 172.21.15.57:0/695671060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1495d0 msgr2=0x561daf154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.874+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1495d0 0x561daf154a50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb55801ae70 tx=0x7fb55800bfb0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.882+0000 7fb57aa00b80 1 -- 172.21.15.57:0/695671060 shutdown_connections 2023-10-08T14:02:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.882+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561daf1c45a0 0x561daf1c69c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.882+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561daf152c10 0x561daf153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.882+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561daf1391a0 0x561daf154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.882+0000 7fb57aa00b80 1 --2- 172.21.15.57:0/695671060 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561daf1495d0 0x561daf154a50 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.882+0000 7fb57aa00b80 1 -- 172.21.15.57:0/695671060 >> 172.21.15.57:0/695671060 conn(0x561daf0a7200 msgr2=0x561daf13ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.882+0000 7fb5477fe700 0 client.17223 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:02:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.883+0000 7fb57aa00b80 1 -- 172.21.15.57:0/695671060 shutdown_connections 2023-10-08T14:02:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.885+0000 7fb57aa00b80 1 -- 172.21.15.57:0/695671060 wait complete. 2023-10-08T14:02:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.885+0000 7fb57aa00b80 20 client.17223 trim_cache size 0 max 16384 2023-10-08T14:02:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:02:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:02:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:02:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:02:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.887+0000 7fb56e817700 1 -- 172.21.15.57:0/3286891607 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb5500a1800 msgr2=0x7fb5500a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.888+0000 7fb56e817700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb5500a1800 0x7fb5500a3cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb560018000 tx=0x7fb560016000 comp rx=0 tx=0).stop 2023-10-08T14:02:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.888+0000 7fb56e817700 1 -- 172.21.15.57:0/3286891607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5680fefc0 msgr2=0x7fb568106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:02:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.888+0000 7fb56e817700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5680fefc0 0x7fb568106de0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00ddc0 tx=0x7fb55c00a530 comp rx=0 tx=0).stop 2023-10-08T14:02:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.889+0000 7fb56e817700 1 -- 172.21.15.57:0/3286891607 shutdown_connections 2023-10-08T14:02:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.889+0000 7fb56e817700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb568107860 0x7fb5681c2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.889+0000 7fb56e817700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5680fefc0 0x7fb568106de0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.889+0000 7fb56e817700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb5500a1800 0x7fb5500a3cc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.889+0000 7fb56e817700 1 --2- 172.21.15.57:0/3286891607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5681040e0 0x7fb568107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:02:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.889+0000 7fb56e817700 1 -- 172.21.15.57:0/3286891607 >> 172.21.15.57:0/3286891607 conn(0x7fb568075bf0 msgr2=0x7fb5681022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:02:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.892+0000 7fb56e817700 1 -- 172.21.15.57:0/3286891607 shutdown_connections 2023-10-08T14:02:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:02:58.901+0000 7fb56e817700 1 -- 172.21.15.57:0/3286891607 wait complete. 2023-10-08T14:02:59.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:02:59 smithi115 ceph-mon[104408]: pgmap v652: 129 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 107 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 146 op/s 2023-10-08T14:02:59.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:02:59 smithi057 ceph-mon[99476]: pgmap v652: 129 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 107 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 146 op/s 2023-10-08T14:02:59.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:02:59 smithi099 ceph-mon[103849]: pgmap v652: 129 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 107 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 146 op/s 2023-10-08T14:03:02.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:01 smithi115 ceph-mon[104408]: pgmap v653: 129 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 107 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 172 op/s 2023-10-08T14:03:02.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:01 smithi057 ceph-mon[99476]: pgmap v653: 129 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 107 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 172 op/s 2023-10-08T14:03:02.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:01 smithi099 ceph-mon[103849]: pgmap v653: 129 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 107 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 172 op/s 2023-10-08T14:03:04.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:03 smithi115 ceph-mon[104408]: pgmap v654: 129 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 107 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2023-10-08T14:03:04.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:03 smithi057 ceph-mon[99476]: pgmap v654: 129 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 107 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2023-10-08T14:03:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:03 smithi099 ceph-mon[103849]: pgmap v654: 129 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 107 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2023-10-08T14:03:06.190 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:05 smithi115 ceph-mon[104408]: pgmap v655: 129 pgs: 14 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 108 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-10-08T14:03:06.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:05 smithi057 ceph-mon[99476]: pgmap v655: 129 pgs: 14 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 108 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-10-08T14:03:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:05 smithi099 ceph-mon[103849]: pgmap v655: 129 pgs: 14 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 108 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-10-08T14:03:08.190 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:07 smithi115 ceph-mon[104408]: pgmap v656: 129 pgs: 7 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 116 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2023-10-08T14:03:08.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:07 smithi057 ceph-mon[99476]: pgmap v656: 129 pgs: 7 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 116 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2023-10-08T14:03:08.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:07 smithi099 ceph-mon[103849]: pgmap v656: 129 pgs: 7 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 116 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2023-10-08T14:03:09.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:09 smithi115 ceph-mon[104408]: pgmap v657: 129 pgs: 7 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 116 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 731 KiB/s wr, 161 op/s 2023-10-08T14:03:09.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:09 smithi057 ceph-mon[99476]: pgmap v657: 129 pgs: 7 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 116 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 731 KiB/s wr, 161 op/s 2023-10-08T14:03:09.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:09 smithi099 ceph-mon[103849]: pgmap v657: 129 pgs: 7 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 116 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 731 KiB/s wr, 161 op/s 2023-10-08T14:03:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.062+0000 7f2557314700 1 -- 172.21.15.57:0/4179150054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2550106760 msgr2=0x7f2550106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.062+0000 7f2557314700 1 --2- 172.21.15.57:0/4179150054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2550106760 0x7f2550106b40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f254c009990 tx=0x7f254c009ca0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.062+0000 7f2557314700 1 -- 172.21.15.57:0/4179150054 shutdown_connections 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.062+0000 7f2557314700 1 --2- 172.21.15.57:0/4179150054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25500ff620 0x7f255010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.062+0000 7f2557314700 1 --2- 172.21.15.57:0/4179150054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2550106760 0x7f2550106b40 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.062+0000 7f2557314700 1 --2- 172.21.15.57:0/4179150054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25500fec60 0x7f25500ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.062+0000 7f2557314700 1 -- 172.21.15.57:0/4179150054 >> 172.21.15.57:0/4179150054 conn(0x7f2550075bf0 msgr2=0x7f25500fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.062+0000 7f2557314700 1 -- 172.21.15.57:0/4179150054 shutdown_connections 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.062+0000 7f2557314700 1 -- 172.21.15.57:0/4179150054 wait complete. 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f2557314700 1 Processor -- start 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f2557314700 1 -- start start 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f2557314700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500fec60 0x7f255019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f2557314700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25500ff620 0x7f255019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f2557314700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2550106760 0x7f2550196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f2557314700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2550196fb0 con 0x7f25500fec60 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f2557314700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2550197120 con 0x7f2550106760 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f2557314700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501972e0 con 0x7f25500ff620 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f25550b0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500fec60 0x7f255019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f25550b0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500fec60 0x7f255019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59962/0 (socket says 172.21.15.57:59962) 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f25550b0700 1 -- 172.21.15.57:0/4218463959 learned_addr learned my addr 172.21.15.57:0/4218463959 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f25558b1700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2550106760 0x7f2550196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.063+0000 7f25548af700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25500ff620 0x7f255019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f25550b0700 1 -- 172.21.15.57:0/4218463959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25500ff620 msgr2=0x7f255019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f25550b0700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25500ff620 0x7f255019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f25550b0700 1 -- 172.21.15.57:0/4218463959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2550106760 msgr2=0x7f2550196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f25550b0700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2550106760 0x7f2550196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f25550b0700 1 -- 172.21.15.57:0/4218463959 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c0095f0 con 0x7f25500fec60 2023-10-08T14:03:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f25550b0700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500fec60 0x7f255019c820 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f254c000c00 tx=0x7f254c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f25467fc700 1 -- 172.21.15.57:0/4218463959 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c01d070 con 0x7f25500fec60 2023-10-08T14:03:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f25467fc700 1 -- 172.21.15.57:0/4218463959 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254c022470 con 0x7f25500fec60 2023-10-08T14:03:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f2557314700 1 -- 172.21.15.57:0/4218463959 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2550197560 con 0x7f25500fec60 2023-10-08T14:03:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f25467fc700 1 -- 172.21.15.57:0/4218463959 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c017650 con 0x7f25500fec60 2023-10-08T14:03:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.064+0000 7f2557314700 1 -- 172.21.15.57:0/4218463959 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501a3810 con 0x7f25500fec60 2023-10-08T14:03:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.067+0000 7f25467fc700 1 -- 172.21.15.57:0/4218463959 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f254c0225e0 con 0x7f25500fec60 2023-10-08T14:03:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.068+0000 7f25467fc700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f253c0a15f0 0x7f253c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.068+0000 7f25467fc700 1 -- 172.21.15.57:0/4218463959 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f254c0ce710 con 0x7f25500fec60 2023-10-08T14:03:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.068+0000 7f25548af700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f253c0a15f0 0x7f253c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.068+0000 7f25634fdb80 1 Processor -- start 2023-10-08T14:03:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.068+0000 7f25634fdb80 1 -- start start 2023-10-08T14:03:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec23c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec1db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec1e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25634fdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258eb86400 con 0x56258ec23880 2023-10-08T14:03:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25634fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258eb77bc0 con 0x56258ec1e0b0 2023-10-08T14:03:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25634fdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258eb754e0 con 0x56258ec241a0 2023-10-08T14:03:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25548af700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f253c0a15f0 0x7f253c0a3ab0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2550197b10 tx=0x7f2540009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25548af700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec1db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25550b0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec23c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25548af700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec1db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60784/0 (socket says 172.21.15.57:60784) 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25548af700 1 -- 172.21.15.57:0/3223808036 learned_addr learned my addr 172.21.15.57:0/3223808036 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25558b1700 1 --2- 172.21.15.57:0/3223808036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec1e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25550b0700 1 -- 172.21.15.57:0/3223808036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 msgr2=0x56258ec1db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25550b0700 1 --2- 172.21.15.57:0/3223808036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec1db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25550b0700 1 -- 172.21.15.57:0/3223808036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 msgr2=0x56258ec1e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25550b0700 1 --2- 172.21.15.57:0/3223808036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec1e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25550b0700 1 -- 172.21.15.57:0/3223808036 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56258e99bd80 con 0x56258ec23880 2023-10-08T14:03:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25548af700 1 --2- 172.21.15.57:0/3223808036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec1db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.069+0000 7f25550b0700 1 --2- 172.21.15.57:0/3223808036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec23c60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f254c0cba50 tx=0x7f254c0977c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c313db6b8d18f01 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f253b7fe700 1 -- 172.21.15.57:0/3223808036 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c09b370 con 0x56258ec23880 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f253b7fe700 1 -- 172.21.15.57:0/3223808036 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254c09f4c0 con 0x56258ec23880 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f253b7fe700 1 -- 172.21.15.57:0/3223808036 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c0ce2e0 con 0x56258ec23880 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f25634fdb80 1 -- 172.21.15.57:0/3223808036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 msgr2=0x56258ec23c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f25634fdb80 1 --2- 172.21.15.57:0/3223808036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec23c60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f254c0cba50 tx=0x7f254c0977c0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f25634fdb80 1 -- 172.21.15.57:0/3223808036 shutdown_connections 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f25634fdb80 1 --2- 172.21.15.57:0/3223808036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec1e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f25634fdb80 1 --2- 172.21.15.57:0/3223808036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec23c60 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f25634fdb80 1 --2- 172.21.15.57:0/3223808036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec1db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f25634fdb80 1 -- 172.21.15.57:0/3223808036 >> 172.21.15.57:0/3223808036 conn(0x56258eb92200 msgr2=0x56258ec18b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f25634fdb80 1 -- 172.21.15.57:0/3223808036 shutdown_connections 2023-10-08T14:03:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.070+0000 7f25634fdb80 1 -- 172.21.15.57:0/3223808036 wait complete. 2023-10-08T14:03:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.071+0000 7f25634fdb80 1 -- start start 2023-10-08T14:03:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.071+0000 7f25634fdb80 1 Processor -- start 2023-10-08T14:03:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.071+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec2a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.071+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec27490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.071+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec27bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.071+0000 7f25634fdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258eb70dd0 con 0x56258ec23880 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.071+0000 7f25634fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258eb0d6b0 con 0x56258ec1e0b0 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.071+0000 7f25634fdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258eb0d350 con 0x56258ec241a0 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.071+0000 7f25634fdb80 10 client.0 _ll_register_callbacks cb 0x56258e96a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25548af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec27490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25548af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec27490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59978/0 (socket says 172.21.15.57:59978) 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25550b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec2a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25548af700 1 -- 172.21.15.57:0/1009105381 learned_addr learned my addr 172.21.15.57:0/1009105381 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25558b1700 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec27bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25548af700 1 -- 172.21.15.57:0/1009105381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 msgr2=0x56258ec27bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25548af700 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec27bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25548af700 1 -- 172.21.15.57:0/1009105381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 msgr2=0x56258ec2a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25548af700 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec2a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25548af700 1 -- 172.21.15.57:0/1009105381 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2540017070 con 0x56258ec23880 2023-10-08T14:03:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25550b0700 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec2a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:03:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25548af700 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec27490 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2540004520 tx=0x7f25400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86eec575301248c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f2538ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:03:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f2538ff9700 1 -- 172.21.15.57:0/1009105381 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540022070 con 0x56258ec23880 2023-10-08T14:03:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f2538ff9700 1 -- 172.21.15.57:0/1009105381 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f2540003d70 con 0x56258ec23880 2023-10-08T14:03:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f25634fdb80 1 -- 172.21.15.57:0/1009105381 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56258eb38e50 con 0x56258ec23880 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.072+0000 7f2538ff9700 1 -- 172.21.15.57:0/1009105381 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f254001c430 con 0x56258ec23880 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f25634fdb80 10 client.17235 fetch_fsmap learned FSMap version 48 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f25634fdb80 1 -- 172.21.15.57:0/1009105381 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f25501a3810 con 0x56258ec23880 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f2538ff9700 1 -- 172.21.15.57:0/1009105381 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56258eb38e50 con 0x56258ec23880 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f2538ff9700 1 -- 172.21.15.57:0/1009105381 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f2540027560 con 0x56258ec23880 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f25634fdb80 10 client.17235 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f25634fdb80 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f25634fdb80 20 client.17235 populate_metadata read hostname 'smithi057' 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56258ec37a10 0x56258ec39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f25634fdb80 4 client.17235 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:03:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f25634fdb80 1 -- 172.21.15.57:0/1009105381 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56258eb69db0 con 0x56258ec37a10 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.073+0000 7f25550b0700 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56258ec37a10 0x56258ec39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.074+0000 7f2538ff9700 10 client.17235 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.074+0000 7f25550b0700 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56258ec37a10 0x56258ec39e30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6092f2931cc545a server_cookie=5f7504ed795a5daa in_seq=0 out_seq=0 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.074+0000 7f2538ff9700 1 -- 172.21.15.57:0/1009105381 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56258eb69db0 con 0x56258ec37a10 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.074+0000 7f2538ff9700 10 client.17235 handle_command_reply: tid=0 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.074+0000 7f25634fdb80 1 client.17235 shutdown 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 1 -- 172.21.15.57:0/1009105381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 msgr2=0x56258ec27490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec27490 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2540004520 tx=0x7f25400047c0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 1 -- 172.21.15.57:0/1009105381 shutdown_connections 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56258ec37a10 0x56258ec39e30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec27bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f2538ff9700 0 client.17235 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec2a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1009105381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec27490 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 1 -- 172.21.15.57:0/1009105381 >> 172.21.15.57:0/1009105381 conn(0x56258eb92200 msgr2=0x56258ec18010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 1 -- 172.21.15.57:0/1009105381 shutdown_connections 2023-10-08T14:03:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 1 -- 172.21.15.57:0/1009105381 wait complete. 2023-10-08T14:03:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.075+0000 7f25634fdb80 20 client.17235 trim_cache size 0 max 16384 2023-10-08T14:03:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25634fdb80 1 Processor -- start 2023-10-08T14:03:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25634fdb80 1 -- start start 2023-10-08T14:03:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec29cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec18790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec18cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25634fdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258e9f18c0 con 0x56258ec23880 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25634fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258ec281e0 con 0x56258ec1e0b0 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25634fdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258ec19ac0 con 0x56258ec241a0 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25548af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec18790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25548af700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec18790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59986/0 (socket says 172.21.15.57:59986) 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25558b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec18cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.094+0000 7f25548af700 1 -- 172.21.15.57:0/1682857181 learned_addr learned my addr 172.21.15.57:0/1682857181 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f25548af700 1 -- 172.21.15.57:0/1682857181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 msgr2=0x56258ec18cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f25548af700 1 --2- 172.21.15.57:0/1682857181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec18cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f25548af700 1 -- 172.21.15.57:0/1682857181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 msgr2=0x56258ec29cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f25550b0700 1 --2- 172.21.15.57:0/1682857181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec29cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f25548af700 1 --2- 172.21.15.57:0/1682857181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec29cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f25548af700 1 -- 172.21.15.57:0/1682857181 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56258e984c00 con 0x56258ec23880 2023-10-08T14:03:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f25550b0700 1 --2- 172.21.15.57:0/1682857181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec29cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T14:03:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f25548af700 1 --2- 172.21.15.57:0/1682857181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec18790 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f25400279c0 tx=0x7f2540011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1158285320a309 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f253bfff700 1 -- 172.21.15.57:0/1682857181 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540022040 con 0x56258ec23880 2023-10-08T14:03:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f253bfff700 1 -- 172.21.15.57:0/1682857181 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2540027e90 con 0x56258ec23880 2023-10-08T14:03:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.095+0000 7f253bfff700 1 -- 172.21.15.57:0/1682857181 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540003e70 con 0x56258ec23880 2023-10-08T14:03:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 -- 172.21.15.57:0/1682857181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 msgr2=0x56258ec18790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1682857181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec18790 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f25400279c0 tx=0x7f2540011b70 comp rx=0 tx=0).stop 2023-10-08T14:03:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 -- 172.21.15.57:0/1682857181 shutdown_connections 2023-10-08T14:03:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1682857181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec241a0 0x56258ec18cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1682857181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec29cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1682857181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec23880 0x56258ec18790 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 -- 172.21.15.57:0/1682857181 >> 172.21.15.57:0/1682857181 conn(0x56258eb92200 msgr2=0x56258ec1e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 -- 172.21.15.57:0/1682857181 shutdown_connections 2023-10-08T14:03:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 -- 172.21.15.57:0/1682857181 wait complete. 2023-10-08T14:03:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 -- start start 2023-10-08T14:03:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.096+0000 7f25634fdb80 1 Processor -- start 2023-10-08T14:03:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec1f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec23880 0x56258ec1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25634fdb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec241a0 0x56258ec1fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25634fdb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258ec2eee0 con 0x56258ec241a0 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25634fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258ec20280 con 0x56258ec1e0b0 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25634fdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258ec20440 con 0x56258ec23880 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25634fdb80 10 client.0 _ll_register_callbacks cb 0x56258e96a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25548af700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec23880 0x56258ec1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25548af700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec23880 0x56258ec1f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60804/0 (socket says 172.21.15.57:60804) 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25550b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec1f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25548af700 1 -- 172.21.15.57:0/1984486246 learned_addr learned my addr 172.21.15.57:0/1984486246 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25558b1700 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec241a0 0x56258ec1fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25548af700 1 -- 172.21.15.57:0/1984486246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 msgr2=0x56258ec1f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25548af700 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec1f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25548af700 1 -- 172.21.15.57:0/1984486246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec241a0 msgr2=0x56258ec1fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25548af700 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec241a0 0x56258ec1fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25548af700 1 -- 172.21.15.57:0/1984486246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501a3810 con 0x56258ec23880 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.097+0000 7f25558b1700 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec241a0 0x56258ec1fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f25548af700 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec23880 0x56258ec1f800 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2540027990 tx=0x7f2540027ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8103e3b9c8aa1000 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f2538ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f2538ff9700 1 -- 172.21.15.57:0/1984486246 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540005450 con 0x56258ec23880 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f25634fdb80 1 -- 172.21.15.57:0/1984486246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f254c09f4c0 con 0x56258ec23880 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f2538ff9700 1 -- 172.21.15.57:0/1984486246 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9469+0+0 (secure 0 0 0) 0x7f2540027cd0 con 0x56258ec23880 2023-10-08T14:03:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f2538ff9700 1 -- 172.21.15.57:0/1984486246 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25400055b0 con 0x56258ec23880 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f25634fdb80 10 client.26803 fetch_fsmap learned FSMap version 48 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f25634fdb80 1 -- 172.21.15.57:0/1984486246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56258e9c21f0 con 0x56258ec23880 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f2538ff9700 1 -- 172.21.15.57:0/1984486246 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254c09f4c0 con 0x56258ec23880 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f2538ff9700 1 -- 172.21.15.57:0/1984486246 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f2540034780 con 0x56258ec23880 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f25634fdb80 10 client.26803 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f25634fdb80 10 client.26803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f25634fdb80 20 client.26803 populate_metadata read hostname 'smithi057' 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56258ec8bcb0 0x56258ec8e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f25634fdb80 4 client.26803 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.098+0000 7f25634fdb80 1 -- 172.21.15.57:0/1984486246 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56258eb69ad0 con 0x56258ec8bcb0 2023-10-08T14:03:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.099+0000 7f25550b0700 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56258ec8bcb0 0x56258ec8e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.099+0000 7f2538ff9700 10 client.26803 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.099+0000 7f25550b0700 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56258ec8bcb0 0x56258ec8e0d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1bda8b72e2bd187 server_cookie=28d756ea7fb9cc7a in_seq=0 out_seq=0 2023-10-08T14:03:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f2538ff9700 1 -- 172.21.15.57:0/1984486246 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56258eb69ad0 con 0x56258ec8bcb0 2023-10-08T14:03:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f2538ff9700 10 client.26803 handle_command_reply: tid=0 2023-10-08T14:03:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f25634fdb80 1 client.26803 shutdown 2023-10-08T14:03:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f25634fdb80 1 -- 172.21.15.57:0/1984486246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec23880 msgr2=0x56258ec1f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec23880 0x56258ec1f800 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2540027990 tx=0x7f2540027ae0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f25634fdb80 1 -- 172.21.15.57:0/1984486246 shutdown_connections 2023-10-08T14:03:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x56258ec8bcb0 0x56258ec8e0d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56258ec241a0 0x56258ec1fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56258ec1e0b0 0x56258ec1f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f25634fdb80 1 --2- 172.21.15.57:0/1984486246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56258ec23880 0x56258ec1f800 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f2538ff9700 0 client.26803 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.112+0000 7f25634fdb80 1 -- 172.21.15.57:0/1984486246 >> 172.21.15.57:0/1984486246 conn(0x56258eb92200 msgr2=0x56258ec22560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f25634fdb80 1 -- 172.21.15.57:0/1984486246 shutdown_connections 2023-10-08T14:03:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f25634fdb80 1 -- 172.21.15.57:0/1984486246 wait complete. 2023-10-08T14:03:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f25634fdb80 20 client.26803 trim_cache size 0 max 16384 2023-10-08T14:03:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:03:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:03:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:03:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:03:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 -- 172.21.15.57:0/4218463959 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f253c0a15f0 msgr2=0x7f253c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f253c0a15f0 0x7f253c0a3ab0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2550197b10 tx=0x7f2540009260 comp rx=0 tx=0).stop 2023-10-08T14:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 -- 172.21.15.57:0/4218463959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500fec60 msgr2=0x7f255019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500fec60 0x7f255019c820 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f254c000c00 tx=0x7f254c004730 comp rx=0 tx=0).stop 2023-10-08T14:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 -- 172.21.15.57:0/4218463959 shutdown_connections 2023-10-08T14:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f253c0a15f0 0x7f253c0a3ab0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2550106760 0x7f2550196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f25500fec60 0x7f255019c820 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 --2- 172.21.15.57:0/4218463959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25500ff620 0x7f255019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 -- 172.21.15.57:0/4218463959 >> 172.21.15.57:0/4218463959 conn(0x7f2550075bf0 msgr2=0x7f25500fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 -- 172.21.15.57:0/4218463959 shutdown_connections 2023-10-08T14:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:12.113+0000 7f2557314700 1 -- 172.21.15.57:0/4218463959 wait complete. 2023-10-08T14:03:12.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:11 smithi115 ceph-mon[104408]: pgmap v658: 129 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 118 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-10-08T14:03:12.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:11 smithi057 ceph-mon[99476]: pgmap v658: 129 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 118 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-10-08T14:03:12.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:11 smithi099 ceph-mon[103849]: pgmap v658: 129 pgs: 7 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 118 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-10-08T14:03:14.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:13 smithi115 ceph-mon[104408]: pgmap v659: 129 pgs: 7 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 119 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 570 KiB/s wr, 142 op/s 2023-10-08T14:03:14.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:13 smithi057 ceph-mon[99476]: pgmap v659: 129 pgs: 7 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 119 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 570 KiB/s wr, 142 op/s 2023-10-08T14:03:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:13 smithi099 ceph-mon[103849]: pgmap v659: 129 pgs: 7 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 119 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 570 KiB/s wr, 142 op/s 2023-10-08T14:03:16.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:15 smithi115 ceph-mon[104408]: pgmap v660: 129 pgs: 7 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 119 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 943 KiB/s wr, 156 op/s 2023-10-08T14:03:16.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:15 smithi115 ceph-mon[104408]: osdmap e103: 12 total, 12 up, 12 in 2023-10-08T14:03:16.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:15 smithi057 ceph-mon[99476]: pgmap v660: 129 pgs: 7 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 119 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 943 KiB/s wr, 156 op/s 2023-10-08T14:03:16.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:15 smithi057 ceph-mon[99476]: osdmap e103: 12 total, 12 up, 12 in 2023-10-08T14:03:16.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:15 smithi099 ceph-mon[103849]: pgmap v660: 129 pgs: 7 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 119 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 943 KiB/s wr, 156 op/s 2023-10-08T14:03:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:15 smithi099 ceph-mon[103849]: osdmap e103: 12 total, 12 up, 12 in 2023-10-08T14:03:18.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:17 smithi115 ceph-mon[104408]: pgmap v662: 129 pgs: 28 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 93 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 944 KiB/s wr, 136 op/s 2023-10-08T14:03:18.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:17 smithi057 ceph-mon[99476]: pgmap v662: 129 pgs: 28 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 93 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 944 KiB/s wr, 136 op/s 2023-10-08T14:03:18.281 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:17 smithi099 ceph-mon[103849]: pgmap v662: 129 pgs: 28 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 93 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 944 KiB/s wr, 136 op/s 2023-10-08T14:03:19.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:19 smithi115 ceph-mon[104408]: pgmap v663: 129 pgs: 61 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 48 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2023-10-08T14:03:19.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:19 smithi057 ceph-mon[99476]: pgmap v663: 129 pgs: 61 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 48 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2023-10-08T14:03:19.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:19 smithi099 ceph-mon[103849]: pgmap v663: 129 pgs: 61 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 48 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2023-10-08T14:03:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:21 smithi115 ceph-mon[104408]: pgmap v664: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 934 KiB/s wr, 160 op/s 2023-10-08T14:03:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:21 smithi057 ceph-mon[99476]: pgmap v664: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 934 KiB/s wr, 160 op/s 2023-10-08T14:03:22.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:21 smithi099 ceph-mon[103849]: pgmap v664: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 934 KiB/s wr, 160 op/s 2023-10-08T14:03:24.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:23 smithi115 ceph-mon[104408]: pgmap v665: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2023-10-08T14:03:24.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:23 smithi057 ceph-mon[99476]: pgmap v665: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2023-10-08T14:03:24.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:23 smithi099 ceph-mon[103849]: pgmap v665: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2023-10-08T14:03:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.304+0000 7fb385880700 1 -- 172.21.15.57:0/3528533021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb380105680 msgr2=0x7fb380103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.304+0000 7fb385880700 1 --2- 172.21.15.57:0/3528533021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb380105680 0x7fb380103100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb374009990 tx=0x7fb374009ca0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.306+0000 7fb385880700 1 -- 172.21.15.57:0/3528533021 shutdown_connections 2023-10-08T14:03:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.306+0000 7fb385880700 1 --2- 172.21.15.57:0/3528533021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb380067b20 0x7fb380067fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.306+0000 7fb385880700 1 --2- 172.21.15.57:0/3528533021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb380105680 0x7fb380103100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.306+0000 7fb385880700 1 --2- 172.21.15.57:0/3528533021 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb380103640 0x7fb380103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.306+0000 7fb385880700 1 -- 172.21.15.57:0/3528533021 >> 172.21.15.57:0/3528533021 conn(0x7fb380076790 msgr2=0x7fb380076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.309+0000 7fb385880700 1 -- 172.21.15.57:0/3528533021 shutdown_connections 2023-10-08T14:03:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.310+0000 7fb385880700 1 -- 172.21.15.57:0/3528533021 wait complete. 2023-10-08T14:03:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.310+0000 7fb385880700 1 Processor -- start 2023-10-08T14:03:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb385880700 1 -- start start 2023-10-08T14:03:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb385880700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb380067b20 0x7fb38019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb385880700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb380103640 0x7fb38019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb385880700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb380105680 0x7fb3801a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb385880700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3801a21d0 con 0x7fb380105680 2023-10-08T14:03:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb385880700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3801a2340 con 0x7fb380103640 2023-10-08T14:03:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb385880700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3801a2500 con 0x7fb380067b20 2023-10-08T14:03:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb37effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb380067b20 0x7fb38019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb37f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb380105680 0x7fb3801a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb37effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb380067b20 0x7fb38019d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60818/0 (socket says 172.21.15.57:60818) 2023-10-08T14:03:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb37effd700 1 -- 172.21.15.57:0/4073476705 learned_addr learned my addr 172.21.15.57:0/4073476705 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.311+0000 7fb37e7fc700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb380103640 0x7fb38019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb37f7fe700 1 -- 172.21.15.57:0/4073476705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb380067b20 msgr2=0x7fb38019d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb37f7fe700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb380067b20 0x7fb38019d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb37f7fe700 1 -- 172.21.15.57:0/4073476705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb380103640 msgr2=0x7fb38019d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb37f7fe700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb380103640 0x7fb38019d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb37f7fe700 1 -- 172.21.15.57:0/4073476705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3740095f0 con 0x7fb380105680 2023-10-08T14:03:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb37effd700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb380067b20 0x7fb38019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:03:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb37f7fe700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb380105680 0x7fb3801a1b90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb37800bb00 tx=0x7fb37800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb38487e700 1 -- 172.21.15.57:0/4073476705 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800c9b0 con 0x7fb380105680 2023-10-08T14:03:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb385880700 1 -- 172.21.15.57:0/4073476705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3801a27e0 con 0x7fb380105680 2023-10-08T14:03:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb38487e700 1 -- 172.21.15.57:0/4073476705 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb378014440 con 0x7fb380105680 2023-10-08T14:03:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb38487e700 1 -- 172.21.15.57:0/4073476705 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3780125a0 con 0x7fb380105680 2023-10-08T14:03:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.312+0000 7fb385880700 1 -- 172.21.15.57:0/4073476705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3801aa020 con 0x7fb380105680 2023-10-08T14:03:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.317+0000 7fb391a69b80 1 Processor -- start 2023-10-08T14:03:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.317+0000 7fb38487e700 1 -- 172.21.15.57:0/4073476705 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb3780145b0 con 0x7fb380105680 2023-10-08T14:03:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.317+0000 7fb38487e700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb3680a1730 0x7fb3680a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.318+0000 7fb38487e700 1 -- 172.21.15.57:0/4073476705 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9501+0+0 (secure 0 0 0) 0x7fb3780cc9c0 con 0x7fb380105680 2023-10-08T14:03:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.320+0000 7fb391a69b80 1 -- start start 2023-10-08T14:03:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.320+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2afbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.320+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2afc1a0 0x557ba2af5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.320+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 0x557ba2af6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.320+0000 7fb391a69b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba2a5e400 con 0x557ba2af60b0 2023-10-08T14:03:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.320+0000 7fb391a69b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba2a4fbc0 con 0x557ba2afc1a0 2023-10-08T14:03:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.320+0000 7fb391a69b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba2a4d4e0 con 0x557ba2afb880 2023-10-08T14:03:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.321+0000 7fb37e7fc700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb3680a1730 0x7fb3680a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.322+0000 7fb37f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 0x557ba2af6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.322+0000 7fb37f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 0x557ba2af6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60016/0 (socket says 172.21.15.57:60016) 2023-10-08T14:03:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.322+0000 7fb37f7fe700 1 -- 172.21.15.57:0/3256456970 learned_addr learned my addr 172.21.15.57:0/3256456970 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.322+0000 7fb37effd700 1 --2- 172.21.15.57:0/3256456970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2afbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.323+0000 7fb37effd700 1 -- 172.21.15.57:0/3256456970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2afc1a0 msgr2=0x557ba2af5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.323+0000 7fb37effd700 1 --2- 172.21.15.57:0/3256456970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2afc1a0 0x557ba2af5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.323+0000 7fb37effd700 1 -- 172.21.15.57:0/3256456970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 msgr2=0x557ba2af6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.323+0000 7fb37effd700 1 --2- 172.21.15.57:0/3256456970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 0x557ba2af6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.323+0000 7fb37effd700 1 -- 172.21.15.57:0/3256456970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ba2873d80 con 0x557ba2afb880 2023-10-08T14:03:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.323+0000 7fb37effd700 1 --2- 172.21.15.57:0/3256456970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2afbc60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb3740098e0 tx=0x7fb374003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95860bce50cc7a1c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.323+0000 7fb365ffb700 1 -- 172.21.15.57:0/3256456970 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37401d030 con 0x557ba2afb880 2023-10-08T14:03:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.324+0000 7fb365ffb700 1 -- 172.21.15.57:0/3256456970 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb37401e040 con 0x557ba2afb880 2023-10-08T14:03:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.324+0000 7fb365ffb700 1 -- 172.21.15.57:0/3256456970 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400f6c0 con 0x557ba2afb880 2023-10-08T14:03:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.324+0000 7fb391a69b80 1 -- 172.21.15.57:0/3256456970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 msgr2=0x557ba2afbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.324+0000 7fb391a69b80 1 --2- 172.21.15.57:0/3256456970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2afbc60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb3740098e0 tx=0x7fb374003680 comp rx=0 tx=0).stop 2023-10-08T14:03:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.325+0000 7fb391a69b80 1 -- 172.21.15.57:0/3256456970 shutdown_connections 2023-10-08T14:03:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.325+0000 7fb391a69b80 1 --2- 172.21.15.57:0/3256456970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 0x557ba2af6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.325+0000 7fb391a69b80 1 --2- 172.21.15.57:0/3256456970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2afbc60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.325+0000 7fb391a69b80 1 --2- 172.21.15.57:0/3256456970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2afc1a0 0x557ba2af5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.325+0000 7fb391a69b80 1 -- 172.21.15.57:0/3256456970 >> 172.21.15.57:0/3256456970 conn(0x557ba2a6a200 msgr2=0x557ba2af0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.334+0000 7fb391a69b80 1 -- 172.21.15.57:0/3256456970 shutdown_connections 2023-10-08T14:03:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.340+0000 7fb37e7fc700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb3680a1730 0x7fb3680a3bf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb38019e800 tx=0x7fb370009db0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.341+0000 7fb391a69b80 1 -- 172.21.15.57:0/3256456970 wait complete. 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.341+0000 7fb391a69b80 1 -- start start 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.341+0000 7fb391a69b80 1 Processor -- start 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.342+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2aff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.342+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2afb880 0x557ba2affb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.342+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2b000b0 0x557ba2b2dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.342+0000 7fb391a69b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba2a48dd0 con 0x557ba2b000b0 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.342+0000 7fb391a69b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba29e56b0 con 0x557ba2afb880 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.342+0000 7fb391a69b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba29e5350 con 0x557ba2af60b0 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.342+0000 7fb391a69b80 10 client.0 _ll_register_callbacks cb 0x557ba28427a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.342+0000 7fb37effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2aff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.342+0000 7fb37effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2aff630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60832/0 (socket says 172.21.15.57:60832) 2023-10-08T14:03:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.342+0000 7fb37effd700 1 -- 172.21.15.57:0/3579843787 learned_addr learned my addr 172.21.15.57:0/3579843787 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.343+0000 7fb37effd700 1 -- 172.21.15.57:0/3579843787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2afb880 msgr2=0x557ba2affb70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:03:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.343+0000 7fb37effd700 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2afb880 0x557ba2affb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.343+0000 7fb37effd700 1 -- 172.21.15.57:0/3579843787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2b000b0 msgr2=0x557ba2b2dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.343+0000 7fb37effd700 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2b000b0 0x557ba2b2dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.343+0000 7fb37effd700 1 -- 172.21.15.57:0/3579843787 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb380114530 con 0x557ba2af60b0 2023-10-08T14:03:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.343+0000 7fb37effd700 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2aff630 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb37400bc60 tx=0x7fb374004ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c5b9c4793aac11 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.345+0000 7fb35effd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:03:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.345+0000 7fb35effd700 1 -- 172.21.15.57:0/3579843787 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37401d070 con 0x557ba2af60b0 2023-10-08T14:03:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.345+0000 7fb35effd700 1 -- 172.21.15.57:0/3579843787 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9501+0+0 (secure 0 0 0) 0x7fb37400f660 con 0x557ba2af60b0 2023-10-08T14:03:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.349+0000 7fb391a69b80 1 -- 172.21.15.57:0/3579843787 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ba2a10e50 con 0x557ba2af60b0 2023-10-08T14:03:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.349+0000 7fb35effd700 1 -- 172.21.15.57:0/3579843787 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb374005390 con 0x557ba2af60b0 2023-10-08T14:03:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.354+0000 7fb391a69b80 10 client.26815 fetch_fsmap learned FSMap version 48 2023-10-08T14:03:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.354+0000 7fb391a69b80 1 -- 172.21.15.57:0/3579843787 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x557ba285cc00 con 0x557ba2af60b0 2023-10-08T14:03:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.363+0000 7fb35effd700 1 -- 172.21.15.57:0/3579843787 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ba2a10e50 con 0x557ba2af60b0 2023-10-08T14:03:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.363+0000 7fb35effd700 1 -- 172.21.15.57:0/3579843787 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fb374033070 con 0x557ba2af60b0 2023-10-08T14:03:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.364+0000 7fb391a69b80 10 client.26815 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:03:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.364+0000 7fb391a69b80 10 client.26815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:03:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.365+0000 7fb391a69b80 20 client.26815 populate_metadata read hostname 'smithi057' 2023-10-08T14:03:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.365+0000 7fb391a69b80 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ba2b16760 0x557ba2b18b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.365+0000 7fb391a69b80 4 client.26815 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:03:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.365+0000 7fb391a69b80 1 -- 172.21.15.57:0/3579843787 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ba2a41db0 con 0x557ba2b16760 2023-10-08T14:03:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.372+0000 7fb37e7fc700 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ba2b16760 0x557ba2b18b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.372+0000 7fb37e7fc700 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ba2b16760 0x557ba2b18b80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd369aa562eed7bd server_cookie=9e2383d062ba3986 in_seq=0 out_seq=0 2023-10-08T14:03:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.374+0000 7fb35effd700 10 client.26815 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.374+0000 7fb35effd700 1 -- 172.21.15.57:0/3579843787 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557ba2a41db0 con 0x557ba2b16760 2023-10-08T14:03:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.374+0000 7fb35effd700 10 client.26815 handle_command_reply: tid=0 2023-10-08T14:03:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.383+0000 7fb391a69b80 1 client.26815 shutdown 2023-10-08T14:03:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.383+0000 7fb391a69b80 1 -- 172.21.15.57:0/3579843787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 msgr2=0x557ba2aff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.383+0000 7fb391a69b80 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2aff630 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb37400bc60 tx=0x7fb374004ca0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.385+0000 7fb391a69b80 1 -- 172.21.15.57:0/3579843787 shutdown_connections 2023-10-08T14:03:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.385+0000 7fb391a69b80 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ba2b16760 0x557ba2b18b80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.385+0000 7fb391a69b80 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2b000b0 0x557ba2b2dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.385+0000 7fb391a69b80 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2aff630 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.385+0000 7fb391a69b80 1 --2- 172.21.15.57:0/3579843787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2afb880 0x557ba2affb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.385+0000 7fb391a69b80 1 -- 172.21.15.57:0/3579843787 >> 172.21.15.57:0/3579843787 conn(0x557ba2a6a200 msgr2=0x557ba2af0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.386+0000 7fb35effd700 0 client.26815 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.395+0000 7fb391a69b80 1 -- 172.21.15.57:0/3579843787 shutdown_connections 2023-10-08T14:03:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.398+0000 7fb391a69b80 1 -- 172.21.15.57:0/3579843787 wait complete. 2023-10-08T14:03:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.402+0000 7fb391a69b80 20 client.26815 trim_cache size 0 max 16384 2023-10-08T14:03:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.436+0000 7fb391a69b80 1 Processor -- start 2023-10-08T14:03:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.437+0000 7fb391a69b80 1 -- start start 2023-10-08T14:03:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.437+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2afae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.437+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2afb880 0x557ba2af0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.437+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2b000b0 0x557ba2af0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.437+0000 7fb391a69b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba28c98c0 con 0x557ba2afb880 2023-10-08T14:03:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.437+0000 7fb391a69b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba2a10e50 con 0x557ba2b000b0 2023-10-08T14:03:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.437+0000 7fb391a69b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba2aeb2b0 con 0x557ba2af60b0 2023-10-08T14:03:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.438+0000 7fb37effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2afae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.438+0000 7fb37effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2afae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60838/0 (socket says 172.21.15.57:60838) 2023-10-08T14:03:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.438+0000 7fb37effd700 1 -- 172.21.15.57:0/1610490480 learned_addr learned my addr 172.21.15.57:0/1610490480 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.438+0000 7fb37effd700 1 -- 172.21.15.57:0/1610490480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2b000b0 msgr2=0x557ba2af0cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:03:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.438+0000 7fb37effd700 1 --2- 172.21.15.57:0/1610490480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2b000b0 0x557ba2af0cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.438+0000 7fb37effd700 1 -- 172.21.15.57:0/1610490480 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2afb880 msgr2=0x557ba2af0790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:03:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.438+0000 7fb37effd700 1 --2- 172.21.15.57:0/1610490480 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2afb880 0x557ba2af0790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.438+0000 7fb37effd700 1 -- 172.21.15.57:0/1610490480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ba289a1f0 con 0x557ba2af60b0 2023-10-08T14:03:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.438+0000 7fb37effd700 1 --2- 172.21.15.57:0/1610490480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2afae00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb3740098e0 tx=0x7fb374003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6366fb086e76935 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.439+0000 7fb3667fc700 1 -- 172.21.15.57:0/1610490480 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37401d040 con 0x557ba2af60b0 2023-10-08T14:03:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.439+0000 7fb3667fc700 1 -- 172.21.15.57:0/1610490480 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb374022a30 con 0x557ba2af60b0 2023-10-08T14:03:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.439+0000 7fb3667fc700 1 -- 172.21.15.57:0/1610490480 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400fb80 con 0x557ba2af60b0 2023-10-08T14:03:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.440+0000 7fb391a69b80 1 -- 172.21.15.57:0/1610490480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 msgr2=0x557ba2afae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.440+0000 7fb391a69b80 1 --2- 172.21.15.57:0/1610490480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2afae00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb3740098e0 tx=0x7fb374003680 comp rx=0 tx=0).stop 2023-10-08T14:03:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.443+0000 7fb391a69b80 1 -- 172.21.15.57:0/1610490480 shutdown_connections 2023-10-08T14:03:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.443+0000 7fb391a69b80 1 --2- 172.21.15.57:0/1610490480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2b000b0 0x557ba2af0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.443+0000 7fb391a69b80 1 --2- 172.21.15.57:0/1610490480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2af60b0 0x557ba2afae00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.443+0000 7fb391a69b80 1 --2- 172.21.15.57:0/1610490480 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2afb880 0x557ba2af0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.443+0000 7fb391a69b80 1 -- 172.21.15.57:0/1610490480 >> 172.21.15.57:0/1610490480 conn(0x557ba2a6a200 msgr2=0x557ba2afa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.456+0000 7fb391a69b80 1 -- 172.21.15.57:0/1610490480 shutdown_connections 2023-10-08T14:03:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.459+0000 7fb391a69b80 1 -- 172.21.15.57:0/1610490480 wait complete. 2023-10-08T14:03:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.460+0000 7fb391a69b80 1 -- start start 2023-10-08T14:03:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.460+0000 7fb391a69b80 1 Processor -- start 2023-10-08T14:03:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 0x557ba2b2d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2b2d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb391a69b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2b000b0 0x557ba2b2db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb391a69b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba2b95480 con 0x557ba2af60b0 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb391a69b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba2b955c0 con 0x557ba2b000b0 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb391a69b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ba2b95700 con 0x557ba2afb880 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb391a69b80 10 client.0 _ll_register_callbacks cb 0x7fb37400bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb37e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2b2d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb37e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2b2d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60844/0 (socket says 172.21.15.57:60844) 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb37e7fc700 1 -- 172.21.15.57:0/1071168672 learned_addr learned my addr 172.21.15.57:0/1071168672 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb37f7fe700 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2b000b0 0x557ba2b2db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.461+0000 7fb37effd700 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 0x557ba2b2d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.467+0000 7fb37e7fc700 1 -- 172.21.15.57:0/1071168672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2b000b0 msgr2=0x557ba2b2db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.467+0000 7fb37e7fc700 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2b000b0 0x557ba2b2db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.467+0000 7fb37e7fc700 1 -- 172.21.15.57:0/1071168672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 msgr2=0x557ba2b2d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.467+0000 7fb37e7fc700 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 0x557ba2b2d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.467+0000 7fb37e7fc700 1 -- 172.21.15.57:0/1071168672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ba285cc00 con 0x557ba2afb880 2023-10-08T14:03:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.474+0000 7fb37e7fc700 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2b2d5f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e4b0 tx=0x7fb370004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49556876e7266938 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.477+0000 7fb35effd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:03:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.477+0000 7fb35effd700 1 -- 172.21.15.57:0/1071168672 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37000e720 con 0x557ba2afb880 2023-10-08T14:03:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.477+0000 7fb35effd700 1 -- 172.21.15.57:0/1071168672 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9501+0+0 (secure 0 0 0) 0x7fb37000ed50 con 0x557ba2afb880 2023-10-08T14:03:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.477+0000 7fb391a69b80 1 -- 172.21.15.57:0/1071168672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb37401e040 con 0x557ba2afb880 2023-10-08T14:03:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.478+0000 7fb35effd700 1 -- 172.21.15.57:0/1071168672 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb370023070 con 0x557ba2afb880 2023-10-08T14:03:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.481+0000 7fb391a69b80 10 client.26821 fetch_fsmap learned FSMap version 48 2023-10-08T14:03:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.482+0000 7fb391a69b80 1 -- 172.21.15.57:0/1071168672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x557ba2872250 con 0x557ba2afb880 2023-10-08T14:03:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.495+0000 7fb35effd700 1 -- 172.21.15.57:0/1071168672 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb37401e040 con 0x557ba2afb880 2023-10-08T14:03:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.495+0000 7fb35effd700 1 -- 172.21.15.57:0/1071168672 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fb370015ce0 con 0x557ba2afb880 2023-10-08T14:03:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.495+0000 7fb391a69b80 10 client.26821 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:03:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.495+0000 7fb391a69b80 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:03:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.495+0000 7fb391a69b80 20 client.26821 populate_metadata read hostname 'smithi057' 2023-10-08T14:03:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.495+0000 7fb391a69b80 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ba2b96f40 0x557ba2b99360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.495+0000 7fb391a69b80 4 client.26821 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:03:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.495+0000 7fb391a69b80 1 -- 172.21.15.57:0/1071168672 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557ba2a41ad0 con 0x557ba2b96f40 2023-10-08T14:03:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.497+0000 7fb37effd700 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ba2b96f40 0x557ba2b99360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.499+0000 7fb37effd700 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ba2b96f40 0x557ba2b99360 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1d559fb94bb7f65 server_cookie=9119c7fca2cc64de in_seq=0 out_seq=0 2023-10-08T14:03:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.499+0000 7fb35effd700 10 client.26821 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.680+0000 7fb35effd700 1 -- 172.21.15.57:0/1071168672 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557ba2a41ad0 con 0x557ba2b96f40 2023-10-08T14:03:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.680+0000 7fb35effd700 10 client.26821 handle_command_reply: tid=0 2023-10-08T14:03:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.681+0000 7fb391a69b80 1 client.26821 shutdown 2023-10-08T14:03:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.681+0000 7fb391a69b80 1 -- 172.21.15.57:0/1071168672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 msgr2=0x557ba2b2d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.681+0000 7fb391a69b80 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2b2d5f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e4b0 tx=0x7fb370004750 comp rx=0 tx=0).stop 2023-10-08T14:03:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.683+0000 7fb391a69b80 1 -- 172.21.15.57:0/1071168672 shutdown_connections 2023-10-08T14:03:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.683+0000 7fb391a69b80 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x557ba2b96f40 0x557ba2b99360 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.683+0000 7fb391a69b80 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557ba2b000b0 0x557ba2b2db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.683+0000 7fb391a69b80 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557ba2af60b0 0x557ba2b2d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.683+0000 7fb391a69b80 1 --2- 172.21.15.57:0/1071168672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557ba2afb880 0x557ba2b2d5f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.683+0000 7fb391a69b80 1 -- 172.21.15.57:0/1071168672 >> 172.21.15.57:0/1071168672 conn(0x557ba2a6a200 msgr2=0x557ba2afdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.683+0000 7fb35effd700 0 client.26821 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.687+0000 7fb391a69b80 1 -- 172.21.15.57:0/1071168672 shutdown_connections 2023-10-08T14:03:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.690+0000 7fb391a69b80 1 -- 172.21.15.57:0/1071168672 wait complete. 2023-10-08T14:03:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.695+0000 7fb391a69b80 20 client.26821 trim_cache size 0 max 16384 2023-10-08T14:03:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:03:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:03:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:03:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:03:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.711+0000 7fb385880700 1 -- 172.21.15.57:0/4073476705 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb3680a1730 msgr2=0x7fb3680a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.711+0000 7fb385880700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb3680a1730 0x7fb3680a3bf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb38019e800 tx=0x7fb370009db0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.712+0000 7fb385880700 1 -- 172.21.15.57:0/4073476705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb380105680 msgr2=0x7fb3801a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.712+0000 7fb385880700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb380105680 0x7fb3801a1b90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb37800bb00 tx=0x7fb37800be10 comp rx=0 tx=0).stop 2023-10-08T14:03:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.714+0000 7fb385880700 1 -- 172.21.15.57:0/4073476705 shutdown_connections 2023-10-08T14:03:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.715+0000 7fb385880700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb380105680 0x7fb3801a1b90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.715+0000 7fb385880700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb380067b20 0x7fb38019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.715+0000 7fb385880700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fb3680a1730 0x7fb3680a3bf0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.715+0000 7fb385880700 1 --2- 172.21.15.57:0/4073476705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb380103640 0x7fb38019d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.715+0000 7fb385880700 1 -- 172.21.15.57:0/4073476705 >> 172.21.15.57:0/4073476705 conn(0x7fb380076790 msgr2=0x7fb3800fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.719+0000 7fb385880700 1 -- 172.21.15.57:0/4073476705 shutdown_connections 2023-10-08T14:03:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:25.725+0000 7fb385880700 1 -- 172.21.15.57:0/4073476705 wait complete. 2023-10-08T14:03:26.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:25 smithi115 ceph-mon[104408]: pgmap v666: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 962 KiB/s wr, 114 op/s 2023-10-08T14:03:26.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:25 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:03:26.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:25 smithi057 ceph-mon[99476]: pgmap v666: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 962 KiB/s wr, 114 op/s 2023-10-08T14:03:26.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:25 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:03:26.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:25 smithi099 ceph-mon[103849]: pgmap v666: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 962 KiB/s wr, 114 op/s 2023-10-08T14:03:26.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:25 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:03:28.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:27 smithi115 ceph-mon[104408]: pgmap v667: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 806 KiB/s wr, 101 op/s 2023-10-08T14:03:28.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:03:28.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:27 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:03:28.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:27 smithi057 ceph-mon[99476]: pgmap v667: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 806 KiB/s wr, 101 op/s 2023-10-08T14:03:28.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:03:28.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:27 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:03:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:27 smithi099 ceph-mon[103849]: pgmap v667: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 806 KiB/s wr, 101 op/s 2023-10-08T14:03:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:03:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:27 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:03:29.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:03:29.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:03:29.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:03:29.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:28 smithi115 ceph-mon[104408]: osdmap e104: 12 total, 12 up, 12 in 2023-10-08T14:03:29.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:28 smithi115 ceph-mon[104408]: pgmap v669: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 556 KiB/s wr, 74 op/s 2023-10-08T14:03:29.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:03:29.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:03:29.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:03:29.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:28 smithi057 ceph-mon[99476]: osdmap e104: 12 total, 12 up, 12 in 2023-10-08T14:03:29.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:28 smithi057 ceph-mon[99476]: pgmap v669: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 556 KiB/s wr, 74 op/s 2023-10-08T14:03:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:03:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:03:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:03:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:28 smithi099 ceph-mon[103849]: osdmap e104: 12 total, 12 up, 12 in 2023-10-08T14:03:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:28 smithi099 ceph-mon[103849]: pgmap v669: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 556 KiB/s wr, 74 op/s 2023-10-08T14:03:32.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:31 smithi115 ceph-mon[104408]: pgmap v670: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2023-10-08T14:03:32.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:31 smithi057 ceph-mon[99476]: pgmap v670: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2023-10-08T14:03:32.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:31 smithi099 ceph-mon[103849]: pgmap v670: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2023-10-08T14:03:34.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:33 smithi115 ceph-mon[104408]: pgmap v671: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 666 KiB/s wr, 74 op/s 2023-10-08T14:03:34.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:33 smithi057 ceph-mon[99476]: pgmap v671: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 666 KiB/s wr, 74 op/s 2023-10-08T14:03:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:33 smithi099 ceph-mon[103849]: pgmap v671: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 666 KiB/s wr, 74 op/s 2023-10-08T14:03:36.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:35 smithi115 ceph-mon[104408]: pgmap v672: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 550 KiB/s wr, 88 op/s 2023-10-08T14:03:36.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:35 smithi057 ceph-mon[99476]: pgmap v672: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 550 KiB/s wr, 88 op/s 2023-10-08T14:03:36.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:35 smithi099 ceph-mon[103849]: pgmap v672: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 550 KiB/s wr, 88 op/s 2023-10-08T14:03:37.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:36 smithi057 ceph-mon[99476]: pgmap v673: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-08T14:03:37.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:36 smithi099 ceph-mon[103849]: pgmap v673: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-08T14:03:37.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:36 smithi115 ceph-mon[104408]: pgmap v673: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 98 op/s 2023-10-08T14:03:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.934+0000 7f9851ffc700 1 -- 172.21.15.57:0/704828836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f984c105650 msgr2=0x7f984c105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.934+0000 7f9851ffc700 1 --2- 172.21.15.57:0/704828836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f984c105650 0x7f984c105a30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f983c009990 tx=0x7f983c009ca0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.937+0000 7f9851ffc700 1 -- 172.21.15.57:0/704828836 shutdown_connections 2023-10-08T14:03:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.937+0000 7f9851ffc700 1 --2- 172.21.15.57:0/704828836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f984c0ff540 0x7f984c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.937+0000 7f9851ffc700 1 --2- 172.21.15.57:0/704828836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f984c105650 0x7f984c105a30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.937+0000 7f9851ffc700 1 --2- 172.21.15.57:0/704828836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c0feb80 0x7f984c0ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.937+0000 7f9851ffc700 1 -- 172.21.15.57:0/704828836 >> 172.21.15.57:0/704828836 conn(0x7f984c075ad0 msgr2=0x7f984c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.948+0000 7f9851ffc700 1 -- 172.21.15.57:0/704828836 shutdown_connections 2023-10-08T14:03:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.958+0000 7f9851ffc700 1 -- 172.21.15.57:0/704828836 wait complete. 2023-10-08T14:03:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.959+0000 7f9851ffc700 1 Processor -- start 2023-10-08T14:03:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.967+0000 7f9851ffc700 1 -- start start 2023-10-08T14:03:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.967+0000 7f9851ffc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c0feb80 0x7f984c1144d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.967+0000 7f9851ffc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f984c0ff540 0x7f984c10e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.967+0000 7f9851ffc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f984c105650 0x7f984c10ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.967+0000 7f9851ffc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c10f3d0 con 0x7f984c0ff540 2023-10-08T14:03:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.967+0000 7f9851ffc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c06a210 con 0x7f984c105650 2023-10-08T14:03:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.967+0000 7f9851ffc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c06a3d0 con 0x7f984c0feb80 2023-10-08T14:03:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.968+0000 7f984bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f984c105650 0x7f984c10ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.968+0000 7f984b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c0feb80 0x7f984c1144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.968+0000 7f984bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f984c105650 0x7f984c10ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33234/0 (socket says 172.21.15.57:33234) 2023-10-08T14:03:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.968+0000 7f984bfff700 1 -- 172.21.15.57:0/4070042106 learned_addr learned my addr 172.21.15.57:0/4070042106 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.968+0000 7f984bfff700 1 -- 172.21.15.57:0/4070042106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c0feb80 msgr2=0x7f984c1144d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.968+0000 7f984bfff700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c0feb80 0x7f984c1144d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.968+0000 7f984bfff700 1 -- 172.21.15.57:0/4070042106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f984c0ff540 msgr2=0x7f984c10e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.968+0000 7f984bfff700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f984c0ff540 0x7f984c10e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.968+0000 7f984bfff700 1 -- 172.21.15.57:0/4070042106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c0095f0 con 0x7f984c105650 2023-10-08T14:03:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.969+0000 7f984bfff700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f984c105650 0x7f984c10ec00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f984400b970 tx=0x7f984400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.969+0000 7f9848ff9700 1 -- 172.21.15.57:0/4070042106 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984400c930 con 0x7f984c105650 2023-10-08T14:03:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.969+0000 7f9851ffc700 1 -- 172.21.15.57:0/4070042106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c06a6b0 con 0x7f984c105650 2023-10-08T14:03:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.969+0000 7f9851ffc700 1 -- 172.21.15.57:0/4070042106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c06abc0 con 0x7f984c105650 2023-10-08T14:03:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.969+0000 7f9848ff9700 1 -- 172.21.15.57:0/4070042106 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9844014440 con 0x7f984c105650 2023-10-08T14:03:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.969+0000 7f9848ff9700 1 -- 172.21.15.57:0/4070042106 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98440075b0 con 0x7f984c105650 2023-10-08T14:03:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.972+0000 7f9848ff9700 1 -- 172.21.15.57:0/4070042106 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9844007710 con 0x7f984c105650 2023-10-08T14:03:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.973+0000 7f9848ff9700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98380a1730 0x7f98380a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.973+0000 7f9848ff9700 1 -- 172.21.15.57:0/4070042106 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9441+0+0 (secure 0 0 0) 0x7f98440cd9f0 con 0x7f984c105650 2023-10-08T14:03:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.974+0000 7f984b7fe700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98380a1730 0x7f98380a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.974+0000 7f985e1e5b80 1 Processor -- start 2023-10-08T14:03:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.974+0000 7f985e1e5b80 1 -- start start 2023-10-08T14:03:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.974+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d1880 0x55655f1d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.974+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f134400 con 0x55655f1d21a0 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f125bc0 con 0x55655f1d1880 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f1234e0 con 0x55655f1cc0b0 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d1880 0x55655f1d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60048/0 (socket says 172.21.15.57:60048) 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984affd700 1 -- 172.21.15.57:0/637839477 learned_addr learned my addr 172.21.15.57:0/637839477 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984bfff700 1 --2- 172.21.15.57:0/637839477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984bfff700 1 -- 172.21.15.57:0/637839477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d1880 msgr2=0x55655f1d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984bfff700 1 --2- 172.21.15.57:0/637839477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d1880 0x55655f1d1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984bfff700 1 -- 172.21.15.57:0/637839477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 msgr2=0x55655f1cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984bfff700 1 --2- 172.21.15.57:0/637839477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984bfff700 1 -- 172.21.15.57:0/637839477 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55655ef49d80 con 0x55655f1cc0b0 2023-10-08T14:03:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984affd700 1 --2- 172.21.15.57:0/637839477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:03:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.975+0000 7f984b7fe700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98380a1730 0x7f98380a3bf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f983c003800 tx=0x7f983c00b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.976+0000 7f984bfff700 1 --2- 172.21.15.57:0/637839477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1cc510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f98440cae50 tx=0x7f984409a0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f31050db9c1b7d68 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.976+0000 7f9841ffb700 1 -- 172.21.15.57:0/637839477 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98440ba070 con 0x55655f1cc0b0 2023-10-08T14:03:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.976+0000 7f9841ffb700 1 -- 172.21.15.57:0/637839477 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f984409e830 con 0x55655f1cc0b0 2023-10-08T14:03:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.976+0000 7f9841ffb700 1 -- 172.21.15.57:0/637839477 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984409a940 con 0x55655f1cc0b0 2023-10-08T14:03:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.977+0000 7f985e1e5b80 1 -- 172.21.15.57:0/637839477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 msgr2=0x55655f1cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.977+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/637839477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1cc510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f98440cae50 tx=0x7f984409a0c0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.977+0000 7f985e1e5b80 1 -- 172.21.15.57:0/637839477 shutdown_connections 2023-10-08T14:03:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.977+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/637839477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1cc510 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.977+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/637839477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d1880 0x55655f1d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.977+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/637839477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.977+0000 7f985e1e5b80 1 -- 172.21.15.57:0/637839477 >> 172.21.15.57:0/637839477 conn(0x55655f140200 msgr2=0x55655f1c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.982+0000 7f985e1e5b80 1 -- 172.21.15.57:0/637839477 shutdown_connections 2023-10-08T14:03:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.982+0000 7f985e1e5b80 1 -- 172.21.15.57:0/637839477 wait complete. 2023-10-08T14:03:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.983+0000 7f985e1e5b80 1 -- start start 2023-10-08T14:03:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.983+0000 7f985e1e5b80 1 Processor -- start 2023-10-08T14:03:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.991+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1d5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.991+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d1880 0x55655f1d5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.991+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d21a0 0x55655f1d5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.991+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f11edd0 con 0x55655f1d1880 2023-10-08T14:03:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.991+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f0bb6b0 con 0x55655f1d21a0 2023-10-08T14:03:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.991+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f0bb350 con 0x55655f1cc0b0 2023-10-08T14:03:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.992+0000 7f985e1e5b80 10 client.0 _ll_register_callbacks cb 0x55655ef187a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:03:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.992+0000 7f984bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d21a0 0x55655f1d5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.992+0000 7f984bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d21a0 0x55655f1d5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33250/0 (socket says 172.21.15.57:33250) 2023-10-08T14:03:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.992+0000 7f984bfff700 1 -- 172.21.15.57:0/3804677816 learned_addr learned my addr 172.21.15.57:0/3804677816 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.994+0000 7f984b7fe700 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1d5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.994+0000 7f984b7fe700 1 -- 172.21.15.57:0/3804677816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d21a0 msgr2=0x55655f1d5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.994+0000 7f984b7fe700 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d21a0 0x55655f1d5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.994+0000 7f984b7fe700 1 -- 172.21.15.57:0/3804677816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d1880 msgr2=0x55655f1d5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.994+0000 7f984b7fe700 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d1880 0x55655f1d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.994+0000 7f984b7fe700 1 -- 172.21.15.57:0/3804677816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9834009440 con 0x55655f1cc0b0 2023-10-08T14:03:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.995+0000 7f984b7fe700 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1d5560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f983c0098e0 tx=0x7f983c005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77dc81f77c40fb3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.995+0000 7f9832ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:03:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f9832ffd700 1 -- 172.21.15.57:0/3804677816 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c023660 con 0x55655f1cc0b0 2023-10-08T14:03:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f9832ffd700 1 -- 172.21.15.57:0/3804677816 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9441+0+0 (secure 0 0 0) 0x7f983c0237c0 con 0x55655f1cc0b0 2023-10-08T14:03:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f985e1e5b80 1 -- 172.21.15.57:0/3804677816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55655f0e6e50 con 0x55655f1cc0b0 2023-10-08T14:03:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f9832ffd700 1 -- 172.21.15.57:0/3804677816 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f983c023980 con 0x55655f1cc0b0 2023-10-08T14:03:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f985e1e5b80 10 client.26830 fetch_fsmap learned FSMap version 48 2023-10-08T14:03:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f985e1e5b80 1 -- 172.21.15.57:0/3804677816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f984c06abc0 con 0x55655f1cc0b0 2023-10-08T14:03:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f9832ffd700 1 -- 172.21.15.57:0/3804677816 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55655f0e6e50 con 0x55655f1cc0b0 2023-10-08T14:03:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f9832ffd700 1 -- 172.21.15.57:0/3804677816 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f983c02fae0 con 0x55655f1cc0b0 2023-10-08T14:03:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f985e1e5b80 10 client.26830 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:03:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f985e1e5b80 10 client.26830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:03:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f985e1e5b80 20 client.26830 populate_metadata read hostname 'smithi057' 2023-10-08T14:03:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55655f1e5d60 0x55655f1e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f985e1e5b80 4 client.26830 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:03:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:38.997+0000 7f985e1e5b80 1 -- 172.21.15.57:0/3804677816 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55655f117db0 con 0x55655f1e5d60 2023-10-08T14:03:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.002+0000 7f984affd700 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55655f1e5d60 0x55655f1e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.004+0000 7f984affd700 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55655f1e5d60 0x55655f1e8200 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35ba756df22fb61 server_cookie=f25d9e1df6fcd96a in_seq=0 out_seq=0 2023-10-08T14:03:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.008+0000 7f9832ffd700 10 client.26830 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.008+0000 7f9832ffd700 1 -- 172.21.15.57:0/3804677816 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55655f117db0 con 0x55655f1e5d60 2023-10-08T14:03:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.008+0000 7f9832ffd700 10 client.26830 handle_command_reply: tid=0 2023-10-08T14:03:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.008+0000 7f985e1e5b80 1 client.26830 shutdown 2023-10-08T14:03:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.011+0000 7f985e1e5b80 1 -- 172.21.15.57:0/3804677816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 msgr2=0x55655f1d5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.011+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1d5560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f983c0098e0 tx=0x7f983c005050 comp rx=0 tx=0).stop 2023-10-08T14:03:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.019+0000 7f985e1e5b80 1 -- 172.21.15.57:0/3804677816 shutdown_connections 2023-10-08T14:03:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.019+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55655f1e5d60 0x55655f1e8200 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.019+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d21a0 0x55655f1d5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.019+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1d5560 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.019+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/3804677816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d1880 0x55655f1d5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.019+0000 7f985e1e5b80 1 -- 172.21.15.57:0/3804677816 >> 172.21.15.57:0/3804677816 conn(0x55655f140200 msgr2=0x55655f1c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.020+0000 7f9832ffd700 0 client.26830 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.026+0000 7f985e1e5b80 1 -- 172.21.15.57:0/3804677816 shutdown_connections 2023-10-08T14:03:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.026+0000 7f985e1e5b80 1 -- 172.21.15.57:0/3804677816 wait complete. 2023-10-08T14:03:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.031+0000 7f985e1e5b80 20 client.26830 trim_cache size 0 max 16384 2023-10-08T14:03:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.058+0000 7f985e1e5b80 1 Processor -- start 2023-10-08T14:03:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.058+0000 7f985e1e5b80 1 -- start start 2023-10-08T14:03:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d1880 0x55655f1c6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655ef9f8c0 con 0x55655f1d21a0 2023-10-08T14:03:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f0e6e50 con 0x55655f1d1880 2023-10-08T14:03:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f1d1eb0 con 0x55655f1cc0b0 2023-10-08T14:03:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f984bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f984bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60062/0 (socket says 172.21.15.57:60062) 2023-10-08T14:03:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f984bfff700 1 -- 172.21.15.57:0/1445032111 learned_addr learned my addr 172.21.15.57:0/1445032111 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f984b7fe700 1 --2- 172.21.15.57:0/1445032111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f984affd700 1 --2- 172.21.15.57:0/1445032111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d1880 0x55655f1c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f984b7fe700 1 -- 172.21.15.57:0/1445032111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d1880 msgr2=0x55655f1c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f984b7fe700 1 --2- 172.21.15.57:0/1445032111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d1880 0x55655f1c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f984b7fe700 1 -- 172.21.15.57:0/1445032111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 msgr2=0x55655f1c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.059+0000 7f984b7fe700 1 --2- 172.21.15.57:0/1445032111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.060+0000 7f984b7fe700 1 -- 172.21.15.57:0/1445032111 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55655ef32c00 con 0x55655f1cc0b0 2023-10-08T14:03:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.060+0000 7f984b7fe700 1 --2- 172.21.15.57:0/1445032111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1cbb70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f983c000c00 tx=0x7f983c005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=290f16f2971b6703 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.060+0000 7f98427fc700 1 -- 172.21.15.57:0/1445032111 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c010710 con 0x55655f1cc0b0 2023-10-08T14:03:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.060+0000 7f98427fc700 1 -- 172.21.15.57:0/1445032111 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f983c02a070 con 0x55655f1cc0b0 2023-10-08T14:03:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.060+0000 7f98427fc700 1 -- 172.21.15.57:0/1445032111 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c02f700 con 0x55655f1cc0b0 2023-10-08T14:03:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.061+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1445032111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 msgr2=0x55655f1cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.061+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/1445032111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1cbb70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f983c000c00 tx=0x7f983c005e50 comp rx=0 tx=0).stop 2023-10-08T14:03:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.064+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1445032111 shutdown_connections 2023-10-08T14:03:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.064+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/1445032111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.064+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/1445032111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1cc0b0 0x55655f1cbb70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.064+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/1445032111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1d1880 0x55655f1c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.064+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1445032111 >> 172.21.15.57:0/1445032111 conn(0x55655f140200 msgr2=0x55655f1cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.065+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1445032111 shutdown_connections 2023-10-08T14:03:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.065+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1445032111 wait complete. 2023-10-08T14:03:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.065+0000 7f985e1e5b80 1 -- start start 2023-10-08T14:03:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.066+0000 7f985e1e5b80 1 Processor -- start 2023-10-08T14:03:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.066+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1cc0b0 0x55655f1dcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.066+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1d1880 0x55655f1dac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.066+0000 7f985e1e5b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1db140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.066+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f1db680 con 0x55655f1d21a0 2023-10-08T14:03:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.066+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f1db7f0 con 0x55655f1cc0b0 2023-10-08T14:03:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.066+0000 7f985e1e5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55655f1db9b0 con 0x55655f1d1880 2023-10-08T14:03:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.066+0000 7f985e1e5b80 10 client.0 _ll_register_callbacks cb 0x7f983c00bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:03:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.068+0000 7f984bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1db140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.068+0000 7f984bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1db140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60068/0 (socket says 172.21.15.57:60068) 2023-10-08T14:03:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.068+0000 7f984bfff700 1 -- 172.21.15.57:0/1175764597 learned_addr learned my addr 172.21.15.57:0/1175764597 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.069+0000 7f984bfff700 1 -- 172.21.15.57:0/1175764597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1d1880 msgr2=0x55655f1dac00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:03:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.069+0000 7f984bfff700 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1d1880 0x55655f1dac00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.069+0000 7f984bfff700 1 -- 172.21.15.57:0/1175764597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1cc0b0 msgr2=0x55655f1dcc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:03:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.069+0000 7f984bfff700 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1cc0b0 0x55655f1dcc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.069+0000 7f984bfff700 1 -- 172.21.15.57:0/1175764597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c06abc0 con 0x55655f1d21a0 2023-10-08T14:03:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.075+0000 7f984bfff700 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1db140 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f984409a3e0 tx=0x7f9844085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49beac3fe36ef1b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.076+0000 7f9832ffd700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:03:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.076+0000 7f9832ffd700 1 -- 172.21.15.57:0/1175764597 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844099dd0 con 0x55655f1d21a0 2023-10-08T14:03:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.076+0000 7f9832ffd700 1 -- 172.21.15.57:0/1175764597 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9441+0+0 (secure 0 0 0) 0x7f984409a940 con 0x55655f1d21a0 2023-10-08T14:03:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.076+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1175764597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f984409e830 con 0x55655f1d21a0 2023-10-08T14:03:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.083+0000 7f9832ffd700 1 -- 172.21.15.57:0/1175764597 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f984409ab00 con 0x55655f1d21a0 2023-10-08T14:03:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.086+0000 7f985e1e5b80 10 client.17262 fetch_fsmap learned FSMap version 48 2023-10-08T14:03:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.086+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1175764597 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55655ef701f0 con 0x55655f1d21a0 2023-10-08T14:03:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.086+0000 7f9832ffd700 1 -- 172.21.15.57:0/1175764597 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f984409e830 con 0x55655f1d21a0 2023-10-08T14:03:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.087+0000 7f9832ffd700 1 -- 172.21.15.57:0/1175764597 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f984400ca90 con 0x55655f1d21a0 2023-10-08T14:03:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.087+0000 7f985e1e5b80 10 client.17262 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:03:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.087+0000 7f985e1e5b80 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:03:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.087+0000 7f985e1e5b80 20 client.17262 populate_metadata read hostname 'smithi057' 2023-10-08T14:03:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.087+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55655f239ee0 0x55655f23c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.087+0000 7f985e1e5b80 4 client.17262 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:03:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.087+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1175764597 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55655f117ad0 con 0x55655f239ee0 2023-10-08T14:03:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.089+0000 7f984b7fe700 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55655f239ee0 0x55655f23c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.089+0000 7f9832ffd700 10 client.17262 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.089+0000 7f984b7fe700 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55655f239ee0 0x55655f23c300 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb8d2ef615dad664 server_cookie=ea1d65cd379b1689 in_seq=0 out_seq=0 2023-10-08T14:03:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.180+0000 7f9832ffd700 1 -- 172.21.15.57:0/1175764597 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55655f117ad0 con 0x55655f239ee0 2023-10-08T14:03:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.180+0000 7f9832ffd700 10 client.17262 handle_command_reply: tid=0 2023-10-08T14:03:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.182+0000 7f985e1e5b80 1 client.17262 shutdown 2023-10-08T14:03:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.191+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1175764597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 msgr2=0x55655f1db140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.191+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1db140 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f984409a3e0 tx=0x7f9844085040 comp rx=0 tx=0).stop 2023-10-08T14:03:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.199+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1175764597 shutdown_connections 2023-10-08T14:03:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.199+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55655f239ee0 0x55655f23c300 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.199+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55655f1d21a0 0x55655f1db140 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.199+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55655f1cc0b0 0x55655f1dcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.199+0000 7f985e1e5b80 1 --2- 172.21.15.57:0/1175764597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55655f1d1880 0x55655f1dac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.199+0000 7f9832ffd700 0 client.17262 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.199+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1175764597 >> 172.21.15.57:0/1175764597 conn(0x55655f140200 msgr2=0x55655f1d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.203+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1175764597 shutdown_connections 2023-10-08T14:03:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.208+0000 7f985e1e5b80 1 -- 172.21.15.57:0/1175764597 wait complete. 2023-10-08T14:03:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.208+0000 7f985e1e5b80 20 client.17262 trim_cache size 0 max 16384 2023-10-08T14:03:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:03:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:03:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:03:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:03:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.224+0000 7f9851ffc700 1 -- 172.21.15.57:0/4070042106 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98380a1730 msgr2=0x7f98380a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.224+0000 7f9851ffc700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98380a1730 0x7f98380a3bf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f983c003800 tx=0x7f983c00b540 comp rx=0 tx=0).stop 2023-10-08T14:03:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.224+0000 7f9851ffc700 1 -- 172.21.15.57:0/4070042106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f984c105650 msgr2=0x7f984c10ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.224+0000 7f9851ffc700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f984c105650 0x7f984c10ec00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f984400b970 tx=0x7f984400bc80 comp rx=0 tx=0).stop 2023-10-08T14:03:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.225+0000 7f9851ffc700 1 -- 172.21.15.57:0/4070042106 shutdown_connections 2023-10-08T14:03:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.225+0000 7f9851ffc700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f98380a1730 0x7f98380a3bf0 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f983c003800 tx=0x7f983c00b540 comp rx=0 tx=0).stop 2023-10-08T14:03:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.225+0000 7f9851ffc700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f984c105650 0x7f984c10ec00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.225+0000 7f9851ffc700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f984c0feb80 0x7f984c1144d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.225+0000 7f9851ffc700 1 --2- 172.21.15.57:0/4070042106 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f984c0ff540 0x7f984c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.225+0000 7f9851ffc700 1 -- 172.21.15.57:0/4070042106 >> 172.21.15.57:0/4070042106 conn(0x7f984c075ad0 msgr2=0x7f984c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.231+0000 7f9851ffc700 1 -- 172.21.15.57:0/4070042106 shutdown_connections 2023-10-08T14:03:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:39.231+0000 7f9851ffc700 1 -- 172.21.15.57:0/4070042106 wait complete. 2023-10-08T14:03:39.700 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:39 smithi115 ceph-mon[104408]: pgmap v674: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 953 KiB/s wr, 96 op/s 2023-10-08T14:03:39.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:39 smithi057 ceph-mon[99476]: pgmap v674: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 953 KiB/s wr, 96 op/s 2023-10-08T14:03:39.781 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:39 smithi099 ceph-mon[103849]: pgmap v674: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 953 KiB/s wr, 96 op/s 2023-10-08T14:03:42.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:41 smithi115 ceph-mon[104408]: pgmap v675: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 101 op/s 2023-10-08T14:03:42.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:41 smithi057 ceph-mon[99476]: pgmap v675: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 101 op/s 2023-10-08T14:03:42.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:41 smithi099 ceph-mon[103849]: pgmap v675: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 101 op/s 2023-10-08T14:03:44.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:43 smithi115 ceph-mon[104408]: pgmap v676: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 805 KiB/s wr, 84 op/s 2023-10-08T14:03:44.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:43 smithi057 ceph-mon[99476]: pgmap v676: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 805 KiB/s wr, 84 op/s 2023-10-08T14:03:44.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:43 smithi099 ceph-mon[103849]: pgmap v676: 129 pgs: 65 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 805 KiB/s wr, 84 op/s 2023-10-08T14:03:46.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:45 smithi115 ceph-mon[104408]: pgmap v677: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2023-10-08T14:03:46.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:45 smithi057 ceph-mon[99476]: pgmap v677: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2023-10-08T14:03:46.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:45 smithi099 ceph-mon[103849]: pgmap v677: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2023-10-08T14:03:47.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:46 smithi057 ceph-mon[99476]: pgmap v678: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 110 op/s 2023-10-08T14:03:47.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:46 smithi099 ceph-mon[103849]: pgmap v678: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 110 op/s 2023-10-08T14:03:47.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:46 smithi115 ceph-mon[104408]: pgmap v678: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 110 op/s 2023-10-08T14:03:49.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:49 smithi115 ceph-mon[104408]: pgmap v679: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1001 KiB/s wr, 107 op/s 2023-10-08T14:03:49.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:49 smithi057 ceph-mon[99476]: pgmap v679: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1001 KiB/s wr, 107 op/s 2023-10-08T14:03:49.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:49 smithi099 ceph-mon[103849]: pgmap v679: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1001 KiB/s wr, 107 op/s 2023-10-08T14:03:52.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:51 smithi115 ceph-mon[104408]: pgmap v680: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 120 op/s 2023-10-08T14:03:52.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:51 smithi057 ceph-mon[99476]: pgmap v680: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 120 op/s 2023-10-08T14:03:52.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:51 smithi099 ceph-mon[103849]: pgmap v680: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 120 op/s 2023-10-08T14:03:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.434+0000 7f829e54e700 1 -- 172.21.15.57:0/3587832667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82981045e0 msgr2=0x7f82981049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.434+0000 7f829e54e700 1 --2- 172.21.15.57:0/3587832667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82981045e0 0x7f82981049c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8288009990 tx=0x7f8288009ca0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.435+0000 7f829e54e700 1 -- 172.21.15.57:0/3587832667 shutdown_connections 2023-10-08T14:03:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.435+0000 7f829e54e700 1 --2- 172.21.15.57:0/3587832667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82980fff90 0x7f829810c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.435+0000 7f829e54e700 1 --2- 172.21.15.57:0/3587832667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82981045e0 0x7f82981049c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.435+0000 7f829e54e700 1 --2- 172.21.15.57:0/3587832667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82980ff5d0 0x7f82980ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.435+0000 7f829e54e700 1 -- 172.21.15.57:0/3587832667 >> 172.21.15.57:0/3587832667 conn(0x7f8298076790 msgr2=0x7f8298076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.438+0000 7f829e54e700 1 -- 172.21.15.57:0/3587832667 shutdown_connections 2023-10-08T14:03:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.440+0000 7f829e54e700 1 -- 172.21.15.57:0/3587832667 wait complete. 2023-10-08T14:03:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.440+0000 7f829e54e700 1 Processor -- start 2023-10-08T14:03:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f829e54e700 1 -- start start 2023-10-08T14:03:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f829e54e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82980ff5d0 0x7f8298071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f829e54e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82980fff90 0x7f8298072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f829e54e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82981045e0 0x7f829806ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f829e54e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829806b650 con 0x7f82981045e0 2023-10-08T14:03:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f829e54e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829806b7c0 con 0x7f82980fff90 2023-10-08T14:03:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f829e54e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829806b980 con 0x7f82980ff5d0 2023-10-08T14:03:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f8297fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82980ff5d0 0x7f8298071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f8297fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82980ff5d0 0x7f8298071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60892/0 (socket says 172.21.15.57:60892) 2023-10-08T14:03:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f8297fff700 1 -- 172.21.15.57:0/1654527648 learned_addr learned my addr 172.21.15.57:0/1654527648 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.441+0000 7f82977fe700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82980fff90 0x7f8298072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.442+0000 7f8297fff700 1 -- 172.21.15.57:0/1654527648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82980fff90 msgr2=0x7f8298072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.442+0000 7f8297fff700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82980fff90 0x7f8298072320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.442+0000 7f8297fff700 1 -- 172.21.15.57:0/1654527648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82981045e0 msgr2=0x7f829806ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.442+0000 7f8297fff700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82981045e0 0x7f829806ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.442+0000 7f8297fff700 1 -- 172.21.15.57:0/1654527648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82880095f0 con 0x7f82980ff5d0 2023-10-08T14:03:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.442+0000 7f8297fff700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82980ff5d0 0x7f8298071de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8288000c00 tx=0x7f8288003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.442+0000 7f82957fa700 1 -- 172.21.15.57:0/1654527648 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82880054a0 con 0x7f82980ff5d0 2023-10-08T14:03:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.442+0000 7f82957fa700 1 -- 172.21.15.57:0/1654527648 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8288005600 con 0x7f82980ff5d0 2023-10-08T14:03:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.442+0000 7f829e54e700 1 -- 172.21.15.57:0/1654527648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829806bc00 con 0x7f82980ff5d0 2023-10-08T14:03:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.442+0000 7f829e54e700 1 -- 172.21.15.57:0/1654527648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829806fb90 con 0x7f82980ff5d0 2023-10-08T14:03:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.447+0000 7f82aa737b80 1 Processor -- start 2023-10-08T14:03:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.447+0000 7f82957fa700 1 -- 172.21.15.57:0/1654527648 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288018690 con 0x7f82980ff5d0 2023-10-08T14:03:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.453+0000 7f82aa737b80 1 -- start start 2023-10-08T14:03:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.454+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13d3880 0x55e3b13d3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.454+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d41a0 0x55e3b13cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.454+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13ce0b0 0x55e3b13ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.454+0000 7f82aa737b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b1336400 con 0x55e3b13d3880 2023-10-08T14:03:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.454+0000 7f82aa737b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b1327bc0 con 0x55e3b13ce0b0 2023-10-08T14:03:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.454+0000 7f82aa737b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b13254e0 con 0x55e3b13d41a0 2023-10-08T14:03:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.454+0000 7f82977fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13ce0b0 0x55e3b13ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.454+0000 7f82977fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13ce0b0 0x55e3b13ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33280/0 (socket says 172.21.15.57:33280) 2023-10-08T14:03:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.454+0000 7f82977fe700 1 -- 172.21.15.57:0/1389706897 learned_addr learned my addr 172.21.15.57:0/1389706897 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.454+0000 7f829caeb700 1 --2- 172.21.15.57:0/1389706897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13d3880 0x55e3b13d3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.455+0000 7f829caeb700 1 -- 172.21.15.57:0/1389706897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d41a0 msgr2=0x55e3b13cdb70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:03:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.455+0000 7f829caeb700 1 --2- 172.21.15.57:0/1389706897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d41a0 0x55e3b13cdb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.455+0000 7f829caeb700 1 -- 172.21.15.57:0/1389706897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13ce0b0 msgr2=0x55e3b13ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.455+0000 7f829caeb700 1 --2- 172.21.15.57:0/1389706897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13ce0b0 0x55e3b13ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.455+0000 7f829caeb700 1 -- 172.21.15.57:0/1389706897 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3b114bd80 con 0x55e3b13d3880 2023-10-08T14:03:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.455+0000 7f829caeb700 1 --2- 172.21.15.57:0/1389706897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13d3880 0x55e3b13d3c60 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f829000e410 tx=0x7f829000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f266641076e2993 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.455+0000 7f827e7fc700 1 -- 172.21.15.57:0/1389706897 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290010040 con 0x55e3b13d3880 2023-10-08T14:03:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.455+0000 7f827e7fc700 1 -- 172.21.15.57:0/1389706897 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8290004db0 con 0x55e3b13d3880 2023-10-08T14:03:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.455+0000 7f827e7fc700 1 -- 172.21.15.57:0/1389706897 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900044f0 con 0x55e3b13d3880 2023-10-08T14:03:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.456+0000 7f82aa737b80 1 -- 172.21.15.57:0/1389706897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13d3880 msgr2=0x55e3b13d3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.456+0000 7f82aa737b80 1 --2- 172.21.15.57:0/1389706897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13d3880 0x55e3b13d3c60 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f829000e410 tx=0x7f829000b040 comp rx=0 tx=0).stop 2023-10-08T14:03:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.457+0000 7f82aa737b80 1 -- 172.21.15.57:0/1389706897 shutdown_connections 2023-10-08T14:03:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.457+0000 7f82aa737b80 1 --2- 172.21.15.57:0/1389706897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13ce0b0 0x55e3b13ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.457+0000 7f82aa737b80 1 --2- 172.21.15.57:0/1389706897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13d3880 0x55e3b13d3c60 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.457+0000 7f82aa737b80 1 --2- 172.21.15.57:0/1389706897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d41a0 0x55e3b13cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.457+0000 7f82aa737b80 1 -- 172.21.15.57:0/1389706897 >> 172.21.15.57:0/1389706897 conn(0x55e3b1342200 msgr2=0x55e3b13c8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.457+0000 7f82957fa700 1 -- 172.21.15.57:0/1654527648 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f828800b4f0 con 0x7f82980ff5d0 2023-10-08T14:03:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.458+0000 7f82957fa700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82800a16f0 0x7f82800a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.458+0000 7f82957fa700 1 -- 172.21.15.57:0/1654527648 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9441+0+0 (secure 0 0 0) 0x7f82880cea90 con 0x7f82980ff5d0 2023-10-08T14:03:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.461+0000 7f82aa737b80 1 -- 172.21.15.57:0/1389706897 shutdown_connections 2023-10-08T14:03:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.461+0000 7f82977fe700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82800a16f0 0x7f82800a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.461+0000 7f82977fe700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82800a16f0 0x7f82800a3bb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55e3b13cea70 tx=0x7f828c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.471+0000 7f82aa737b80 1 -- 172.21.15.57:0/1389706897 wait complete. 2023-10-08T14:03:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.472+0000 7f82aa737b80 1 -- start start 2023-10-08T14:03:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.472+0000 7f82aa737b80 1 Processor -- start 2023-10-08T14:03:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.472+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 0x55e3b13d7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.472+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13d7cd0 0x55e3b13d8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.472+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13e1d40 0x55e3b13e4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.472+0000 7f82aa737b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b1320dd0 con 0x55e3b13d7cd0 2023-10-08T14:03:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.472+0000 7f82aa737b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b12bd6b0 con 0x55e3b13e1d40 2023-10-08T14:03:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.472+0000 7f82aa737b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b12bd350 con 0x55e3b13d3880 2023-10-08T14:03:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.473+0000 7f82aa737b80 10 client.0 _ll_register_callbacks cb 0x55e3b111a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:03:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.474+0000 7f829caeb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13e1d40 0x55e3b13e4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.475+0000 7f829caeb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13e1d40 0x55e3b13e4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33284/0 (socket says 172.21.15.57:33284) 2023-10-08T14:03:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.475+0000 7f829caeb700 1 -- 172.21.15.57:0/2331888822 learned_addr learned my addr 172.21.15.57:0/2331888822 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.475+0000 7f829caeb700 1 -- 172.21.15.57:0/2331888822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 msgr2=0x55e3b13d7790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:03:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.475+0000 7f829caeb700 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 0x55e3b13d7790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.475+0000 7f829caeb700 1 -- 172.21.15.57:0/2331888822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13d7cd0 msgr2=0x55e3b13d8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.475+0000 7f829caeb700 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13d7cd0 0x55e3b13d8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.475+0000 7f829caeb700 1 -- 172.21.15.57:0/2331888822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829000e0c0 con 0x55e3b13e1d40 2023-10-08T14:03:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.475+0000 7f829caeb700 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13e1d40 0x55e3b13e4130 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f829000eab0 tx=0x7f8290004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3fb95c60461072a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.476+0000 7f82777fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:03:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.476+0000 7f82777fe700 1 -- 172.21.15.57:0/2331888822 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290010040 con 0x55e3b13e1d40 2023-10-08T14:03:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.476+0000 7f82aa737b80 1 -- 172.21.15.57:0/2331888822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3b12e8e50 con 0x55e3b13e1d40 2023-10-08T14:03:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.480+0000 7f82777fe700 1 -- 172.21.15.57:0/2331888822 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9441+0+0 (secure 0 0 0) 0x7f8290003ee0 con 0x55e3b13e1d40 2023-10-08T14:03:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.481+0000 7f82777fe700 1 -- 172.21.15.57:0/2331888822 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8290009360 con 0x55e3b13e1d40 2023-10-08T14:03:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.481+0000 7f82aa737b80 10 client.26795 fetch_fsmap learned FSMap version 48 2023-10-08T14:03:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.481+0000 7f82aa737b80 1 -- 172.21.15.57:0/2331888822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55e3b1134c00 con 0x55e3b13e1d40 2023-10-08T14:03:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.481+0000 7f82777fe700 1 -- 172.21.15.57:0/2331888822 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3b12e8e50 con 0x55e3b13e1d40 2023-10-08T14:03:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.489+0000 7f82777fe700 1 -- 172.21.15.57:0/2331888822 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f8290007500 con 0x55e3b13e1d40 2023-10-08T14:03:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.489+0000 7f82aa737b80 10 client.26795 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:03:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.489+0000 7f82aa737b80 10 client.26795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:03:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.489+0000 7f82aa737b80 20 client.26795 populate_metadata read hostname 'smithi057' 2023-10-08T14:03:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.489+0000 7f82aa737b80 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e3b13e8790 0x55e3b13eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.489+0000 7f82aa737b80 4 client.26795 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:03:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.489+0000 7f82aa737b80 1 -- 172.21.15.57:0/2331888822 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3b1319db0 con 0x55e3b13e8790 2023-10-08T14:03:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.506+0000 7f8297fff700 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e3b13e8790 0x55e3b13eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.506+0000 7f8297fff700 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e3b13e8790 0x55e3b13eabb0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802fd17d0607d42a server_cookie=8abc3b5deccc9d06 in_seq=0 out_seq=0 2023-10-08T14:03:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.514+0000 7f82777fe700 10 client.26795 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.514+0000 7f82777fe700 1 -- 172.21.15.57:0/2331888822 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e3b1319db0 con 0x55e3b13e8790 2023-10-08T14:03:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.514+0000 7f82777fe700 10 client.26795 handle_command_reply: tid=0 2023-10-08T14:03:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.517+0000 7f82aa737b80 1 client.26795 shutdown 2023-10-08T14:03:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.522+0000 7f82aa737b80 1 -- 172.21.15.57:0/2331888822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13e1d40 msgr2=0x55e3b13e4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.522+0000 7f82aa737b80 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13e1d40 0x55e3b13e4130 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f829000eab0 tx=0x7f8290004570 comp rx=0 tx=0).stop 2023-10-08T14:03:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.523+0000 7f82aa737b80 1 -- 172.21.15.57:0/2331888822 shutdown_connections 2023-10-08T14:03:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.524+0000 7f82aa737b80 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e3b13e8790 0x55e3b13eabb0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.524+0000 7f82aa737b80 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13e1d40 0x55e3b13e4130 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.524+0000 7f82aa737b80 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 0x55e3b13d7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.524+0000 7f82aa737b80 1 --2- 172.21.15.57:0/2331888822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13d7cd0 0x55e3b13d8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.524+0000 7f82aa737b80 1 -- 172.21.15.57:0/2331888822 >> 172.21.15.57:0/2331888822 conn(0x55e3b1342200 msgr2=0x55e3b13c8010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.524+0000 7f82777fe700 0 client.26795 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.527+0000 7f82aa737b80 1 -- 172.21.15.57:0/2331888822 shutdown_connections 2023-10-08T14:03:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.528+0000 7f82aa737b80 1 -- 172.21.15.57:0/2331888822 wait complete. 2023-10-08T14:03:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.536+0000 7f82aa737b80 20 client.26795 trim_cache size 0 max 16384 2023-10-08T14:03:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.564+0000 7f82aa737b80 1 Processor -- start 2023-10-08T14:03:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.566+0000 7f82aa737b80 1 -- start start 2023-10-08T14:03:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.566+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 0x55e3b13cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.566+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13d7cd0 0x55e3b13deca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.566+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 0x55e3b13d6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.566+0000 7f82aa737b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b11a18c0 con 0x55e3b13e1d40 2023-10-08T14:03:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.566+0000 7f82aa737b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b13da520 con 0x55e3b13d7cd0 2023-10-08T14:03:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.566+0000 7f82aa737b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b13da6d0 con 0x55e3b13d3880 2023-10-08T14:03:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.567+0000 7f8297fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 0x55e3b13cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.567+0000 7f8297fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 0x55e3b13cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:60908/0 (socket says 172.21.15.57:60908) 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.567+0000 7f8297fff700 1 -- 172.21.15.57:0/722041245 learned_addr learned my addr 172.21.15.57:0/722041245 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.567+0000 7f8297fff700 1 -- 172.21.15.57:0/722041245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13d7cd0 msgr2=0x55e3b13deca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.567+0000 7f8297fff700 1 --2- 172.21.15.57:0/722041245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13d7cd0 0x55e3b13deca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.567+0000 7f8297fff700 1 -- 172.21.15.57:0/722041245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 msgr2=0x55e3b13d6cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.567+0000 7f8297fff700 1 --2- 172.21.15.57:0/722041245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 0x55e3b13d6cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.567+0000 7f8297fff700 1 -- 172.21.15.57:0/722041245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3b11721f0 con 0x55e3b13d3880 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.567+0000 7f8297fff700 1 --2- 172.21.15.57:0/722041245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 0x55e3b13cdb70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8288098280 tx=0x7f8288010640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74abbf839ce4f21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.568+0000 7f827effd700 1 -- 172.21.15.57:0/722041245 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828809b810 con 0x55e3b13d3880 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.568+0000 7f827effd700 1 -- 172.21.15.57:0/722041245 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f828809f960 con 0x55e3b13d3880 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.568+0000 7f827effd700 1 -- 172.21.15.57:0/722041245 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82880860a0 con 0x55e3b13d3880 2023-10-08T14:03:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.569+0000 7f82aa737b80 1 -- 172.21.15.57:0/722041245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 msgr2=0x55e3b13cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.569+0000 7f82aa737b80 1 --2- 172.21.15.57:0/722041245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 0x55e3b13cdb70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8288098280 tx=0x7f8288010640 comp rx=0 tx=0).stop 2023-10-08T14:03:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.569+0000 7f82aa737b80 1 -- 172.21.15.57:0/722041245 shutdown_connections 2023-10-08T14:03:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.569+0000 7f82aa737b80 1 --2- 172.21.15.57:0/722041245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 0x55e3b13d6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.569+0000 7f82aa737b80 1 --2- 172.21.15.57:0/722041245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13d3880 0x55e3b13cdb70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.569+0000 7f82aa737b80 1 --2- 172.21.15.57:0/722041245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13d7cd0 0x55e3b13deca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.569+0000 7f82aa737b80 1 -- 172.21.15.57:0/722041245 >> 172.21.15.57:0/722041245 conn(0x55e3b1342200 msgr2=0x55e3b13ef510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.575+0000 7f82aa737b80 1 -- 172.21.15.57:0/722041245 shutdown_connections 2023-10-08T14:03:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.583+0000 7f82aa737b80 1 -- 172.21.15.57:0/722041245 wait complete. 2023-10-08T14:03:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.583+0000 7f82aa737b80 1 -- start start 2023-10-08T14:03:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.583+0000 7f82aa737b80 1 Processor -- start 2023-10-08T14:03:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13d3880 0x55e3b13dcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 0x55e3b13dd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82aa737b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13dd700 0x55e3b143b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82aa737b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b143bb00 con 0x55e3b13e1d40 2023-10-08T14:03:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82aa737b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b143bc40 con 0x55e3b13d3880 2023-10-08T14:03:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82aa737b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b143bdb0 con 0x55e3b13dd700 2023-10-08T14:03:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82aa737b80 10 client.0 _ll_register_callbacks cb 0x7f829000e2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:03:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f829caeb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13dd700 0x55e3b143b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82977fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 0x55e3b13dd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82977fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 0x55e3b13dd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60102/0 (socket says 172.21.15.57:60102) 2023-10-08T14:03:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82977fe700 1 -- 172.21.15.57:0/3720423203 learned_addr learned my addr 172.21.15.57:0/3720423203 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:03:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f8297fff700 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13d3880 0x55e3b13dcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.584+0000 7f82977fe700 1 -- 172.21.15.57:0/3720423203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13dd700 msgr2=0x55e3b143b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.585+0000 7f82977fe700 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13dd700 0x55e3b143b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.585+0000 7f82977fe700 1 -- 172.21.15.57:0/3720423203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13d3880 msgr2=0x55e3b13dcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:03:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.585+0000 7f82977fe700 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13d3880 0x55e3b13dcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.585+0000 7f82977fe700 1 -- 172.21.15.57:0/3720423203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829806fb90 con 0x55e3b13e1d40 2023-10-08T14:03:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.585+0000 7f82977fe700 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 0x55e3b13dd1c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x55e3b1393ae0 tx=0x7f828c004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8101207cde1c63f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:03:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.585+0000 7f82777fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:03:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.585+0000 7f82777fe700 1 -- 172.21.15.57:0/3720423203 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c016070 con 0x55e3b13e1d40 2023-10-08T14:03:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.585+0000 7f82777fe700 1 -- 172.21.15.57:0/3720423203 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9441+0+0 (secure 0 0 0) 0x7f828c003890 con 0x55e3b13e1d40 2023-10-08T14:03:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.585+0000 7f82aa737b80 1 -- 172.21.15.57:0/3720423203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8290004db0 con 0x55e3b13e1d40 2023-10-08T14:03:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.585+0000 7f82777fe700 1 -- 172.21.15.57:0/3720423203 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f828c003a50 con 0x55e3b13e1d40 2023-10-08T14:03:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.586+0000 7f82aa737b80 10 client.17271 fetch_fsmap learned FSMap version 48 2023-10-08T14:03:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.586+0000 7f82aa737b80 1 -- 172.21.15.57:0/3720423203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55e3b114a250 con 0x55e3b13e1d40 2023-10-08T14:03:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.597+0000 7f82777fe700 1 -- 172.21.15.57:0/3720423203 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8290004db0 con 0x55e3b13e1d40 2023-10-08T14:03:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.597+0000 7f82777fe700 1 -- 172.21.15.57:0/3720423203 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f828c028440 con 0x55e3b13e1d40 2023-10-08T14:03:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.597+0000 7f82aa737b80 10 client.17271 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:03:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.597+0000 7f82aa737b80 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:03:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.597+0000 7f82aa737b80 20 client.17271 populate_metadata read hostname 'smithi057' 2023-10-08T14:03:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.597+0000 7f82aa737b80 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e3b143d7e0 0x55e3b143fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:03:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.597+0000 7f82aa737b80 4 client.17271 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:03:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.597+0000 7f82aa737b80 1 -- 172.21.15.57:0/3720423203 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e3b1319ad0 con 0x55e3b143d7e0 2023-10-08T14:03:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.610+0000 7f8297fff700 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e3b143d7e0 0x55e3b143fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:03:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.610+0000 7f82777fe700 10 client.17271 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.610+0000 7f8297fff700 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e3b143d7e0 0x55e3b143fc00 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b214ea112dd51fc5 server_cookie=7cfa98a0dc8c49fb in_seq=0 out_seq=0 2023-10-08T14:03:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.783+0000 7f82777fe700 1 -- 172.21.15.57:0/3720423203 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e3b1319ad0 con 0x55e3b143d7e0 2023-10-08T14:03:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.783+0000 7f82777fe700 10 client.17271 handle_command_reply: tid=0 2023-10-08T14:03:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.783+0000 7f82aa737b80 1 client.17271 shutdown 2023-10-08T14:03:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.789+0000 7f82aa737b80 1 -- 172.21.15.57:0/3720423203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 msgr2=0x55e3b13dd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.789+0000 7f82aa737b80 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 0x55e3b13dd1c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x55e3b1393ae0 tx=0x7f828c004310 comp rx=0 tx=0).stop 2023-10-08T14:03:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.795+0000 7f82aa737b80 1 -- 172.21.15.57:0/3720423203 shutdown_connections 2023-10-08T14:03:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.795+0000 7f82aa737b80 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e3b143d7e0 0x55e3b143fc00 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.795+0000 7f82aa737b80 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e3b13dd700 0x55e3b143b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.795+0000 7f82aa737b80 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3b13d3880 0x55e3b13dcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.795+0000 7f82aa737b80 1 --2- 172.21.15.57:0/3720423203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e3b13e1d40 0x55e3b13dd1c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.795+0000 7f82aa737b80 1 -- 172.21.15.57:0/3720423203 >> 172.21.15.57:0/3720423203 conn(0x55e3b1342200 msgr2=0x55e3b13d6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.799+0000 7f82777fe700 0 client.17271 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:03:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.799+0000 7f82aa737b80 1 -- 172.21.15.57:0/3720423203 shutdown_connections 2023-10-08T14:03:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.803+0000 7f82aa737b80 1 -- 172.21.15.57:0/3720423203 wait complete. 2023-10-08T14:03:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.804+0000 7f82aa737b80 20 client.17271 trim_cache size 0 max 16384 2023-10-08T14:03:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:03:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:03:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:03:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:03:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.810+0000 7f829e54e700 1 -- 172.21.15.57:0/1654527648 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82800a16f0 msgr2=0x7f82800a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.810+0000 7f829e54e700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82800a16f0 0x7f82800a3bb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55e3b13cea70 tx=0x7f828c009260 comp rx=0 tx=0).stop 2023-10-08T14:03:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.810+0000 7f829e54e700 1 -- 172.21.15.57:0/1654527648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82980ff5d0 msgr2=0x7f8298071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:03:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.810+0000 7f829e54e700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82980ff5d0 0x7f8298071de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8288000c00 tx=0x7f8288003b20 comp rx=0 tx=0).stop 2023-10-08T14:03:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.815+0000 7f829e54e700 1 -- 172.21.15.57:0/1654527648 shutdown_connections 2023-10-08T14:03:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.815+0000 7f829e54e700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82981045e0 0x7f829806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.815+0000 7f829e54e700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82980ff5d0 0x7f8298071de0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.815+0000 7f829e54e700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82800a16f0 0x7f82800a3bb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.815+0000 7f829e54e700 1 --2- 172.21.15.57:0/1654527648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82980fff90 0x7f8298072320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:03:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.815+0000 7f829e54e700 1 -- 172.21.15.57:0/1654527648 >> 172.21.15.57:0/1654527648 conn(0x7f8298076790 msgr2=0x7f82980fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:03:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.820+0000 7f829e54e700 1 -- 172.21.15.57:0/1654527648 shutdown_connections 2023-10-08T14:03:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:03:52.823+0000 7f829e54e700 1 -- 172.21.15.57:0/1654527648 wait complete. 2023-10-08T14:03:54.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:53 smithi115 ceph-mon[104408]: pgmap v681: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 109 op/s 2023-10-08T14:03:54.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:53 smithi057 ceph-mon[99476]: pgmap v681: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 109 op/s 2023-10-08T14:03:54.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:53 smithi099 ceph-mon[103849]: pgmap v681: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 109 op/s 2023-10-08T14:03:56.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:55 smithi115 ceph-mon[104408]: pgmap v682: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 114 op/s 2023-10-08T14:03:56.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:55 smithi057 ceph-mon[99476]: pgmap v682: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 114 op/s 2023-10-08T14:03:56.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:55 smithi099 ceph-mon[103849]: pgmap v682: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 114 op/s 2023-10-08T14:03:58.191 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:57 smithi115 ceph-mon[104408]: pgmap v683: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 807 KiB/s wr, 90 op/s 2023-10-08T14:03:58.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:57 smithi057 ceph-mon[99476]: pgmap v683: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 807 KiB/s wr, 90 op/s 2023-10-08T14:03:58.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:57 smithi099 ceph-mon[103849]: pgmap v683: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 807 KiB/s wr, 90 op/s 2023-10-08T14:03:59.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:03:59 smithi115 ceph-mon[104408]: pgmap v684: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 86 op/s 2023-10-08T14:03:59.741 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:03:59 smithi057 ceph-mon[99476]: pgmap v684: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 86 op/s 2023-10-08T14:03:59.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:03:59 smithi099 ceph-mon[103849]: pgmap v684: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 86 op/s 2023-10-08T14:04:02.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:01 smithi115 ceph-mon[104408]: pgmap v685: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 960 KiB/s wr, 97 op/s 2023-10-08T14:04:02.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:01 smithi057 ceph-mon[99476]: pgmap v685: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 960 KiB/s wr, 97 op/s 2023-10-08T14:04:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:01 smithi099 ceph-mon[103849]: pgmap v685: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 960 KiB/s wr, 97 op/s 2023-10-08T14:04:04.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:03 smithi115 ceph-mon[104408]: pgmap v686: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 886 KiB/s wr, 90 op/s 2023-10-08T14:04:04.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:03 smithi115 ceph-mon[104408]: osdmap e105: 12 total, 12 up, 12 in 2023-10-08T14:04:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:03 smithi057 ceph-mon[99476]: pgmap v686: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 886 KiB/s wr, 90 op/s 2023-10-08T14:04:04.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:03 smithi057 ceph-mon[99476]: osdmap e105: 12 total, 12 up, 12 in 2023-10-08T14:04:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:03 smithi099 ceph-mon[103849]: pgmap v686: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 886 KiB/s wr, 90 op/s 2023-10-08T14:04:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:03 smithi099 ceph-mon[103849]: osdmap e105: 12 total, 12 up, 12 in 2023-10-08T14:04:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.003+0000 7ff33f303700 1 -- 172.21.15.57:0/1824462590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3381040e0 msgr2=0x7ff33810bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.003+0000 7ff33f303700 1 --2- 172.21.15.57:0/1824462590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3381040e0 0x7ff33810bc60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff330009f90 tx=0x7ff33000b330 comp rx=0 tx=0).stop 2023-10-08T14:04:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.011+0000 7ff33f303700 1 -- 172.21.15.57:0/1824462590 shutdown_connections 2023-10-08T14:04:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.011+0000 7ff33f303700 1 --2- 172.21.15.57:0/1824462590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3381040e0 0x7ff33810bc60 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.011+0000 7ff33f303700 1 --2- 172.21.15.57:0/1824462590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3380fefc0 0x7ff3380ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.011+0000 7ff33f303700 1 --2- 172.21.15.57:0/1824462590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3380ff970 0x7ff3381039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.011+0000 7ff33f303700 1 -- 172.21.15.57:0/1824462590 >> 172.21.15.57:0/1824462590 conn(0x7ff338075bf0 msgr2=0x7ff3380fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.017+0000 7ff33f303700 1 -- 172.21.15.57:0/1824462590 shutdown_connections 2023-10-08T14:04:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.018+0000 7ff33f303700 1 -- 172.21.15.57:0/1824462590 wait complete. 2023-10-08T14:04:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.018+0000 7ff33f303700 1 Processor -- start 2023-10-08T14:04:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33f303700 1 -- start start 2023-10-08T14:04:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33f303700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3380fefc0 0x7ff338106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33f303700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3381040e0 0x7ff338107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33f303700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff338107840 0x7ff3381a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33f303700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33810bfc0 con 0x7ff3380fefc0 2023-10-08T14:04:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33f303700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3381a11c0 con 0x7ff3381040e0 2023-10-08T14:04:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33f303700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3381a1330 con 0x7ff338107840 2023-10-08T14:04:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33d09f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3380fefc0 0x7ff338106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33d09f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3380fefc0 0x7ff338106dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60112/0 (socket says 172.21.15.57:60112) 2023-10-08T14:04:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33d09f700 1 -- 172.21.15.57:0/1799755079 learned_addr learned my addr 172.21.15.57:0/1799755079 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.019+0000 7ff33d8a0700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff338107840 0x7ff3381a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.020+0000 7ff33d09f700 1 -- 172.21.15.57:0/1799755079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff338107840 msgr2=0x7ff3381a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.020+0000 7ff33d09f700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff338107840 0x7ff3381a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.020+0000 7ff33d09f700 1 -- 172.21.15.57:0/1799755079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3381040e0 msgr2=0x7ff338107300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.020+0000 7ff33d09f700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3381040e0 0x7ff338107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.020+0000 7ff33d09f700 1 -- 172.21.15.57:0/1799755079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff330009c40 con 0x7ff3380fefc0 2023-10-08T14:04:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.020+0000 7ff33d09f700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3380fefc0 0x7ff338106dc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff32801faf0 tx=0x7ff32801fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.023+0000 7ff32e7fc700 1 -- 172.21.15.57:0/1799755079 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32801daa0 con 0x7ff3380fefc0 2023-10-08T14:04:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.023+0000 7ff32e7fc700 1 -- 172.21.15.57:0/1799755079 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff328004cf0 con 0x7ff3380fefc0 2023-10-08T14:04:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.023+0000 7ff32e7fc700 1 -- 172.21.15.57:0/1799755079 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328005540 con 0x7ff3380fefc0 2023-10-08T14:04:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.024+0000 7ff33f303700 1 -- 172.21.15.57:0/1799755079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3381a1610 con 0x7ff3380fefc0 2023-10-08T14:04:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.024+0000 7ff33f303700 1 -- 172.21.15.57:0/1799755079 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3381a1aa0 con 0x7ff3380fefc0 2023-10-08T14:04:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.027+0000 7ff32e7fc700 1 -- 172.21.15.57:0/1799755079 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff328004750 con 0x7ff3380fefc0 2023-10-08T14:04:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.028+0000 7ff32e7fc700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff3240a1780 0x7ff3240a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.028+0000 7ff32e7fc700 1 -- 172.21.15.57:0/1799755079 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9441+0+0 (secure 0 0 0) 0x7ff328025070 con 0x7ff3380fefc0 2023-10-08T14:04:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff34b4ecb80 1 Processor -- start 2023-10-08T14:04:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff34b4ecb80 1 -- start start 2023-10-08T14:04:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bf3880 0x5578c8bf3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 0x5578c8bedb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bee0b0 0x5578c8bee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8b56400 con 0x5578c8bf3880 2023-10-08T14:04:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8b47bc0 con 0x5578c8bf41a0 2023-10-08T14:04:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8b454e0 con 0x5578c8bee0b0 2023-10-08T14:04:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff33c89e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 0x5578c8bedb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff33c89e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 0x5578c8bedb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33316/0 (socket says 172.21.15.57:33316) 2023-10-08T14:04:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff33c89e700 1 -- 172.21.15.57:0/4067789118 learned_addr learned my addr 172.21.15.57:0/4067789118 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.029+0000 7ff33d8a0700 1 --2- 172.21.15.57:0/4067789118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bee0b0 0x5578c8bee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.030+0000 7ff33c89e700 1 -- 172.21.15.57:0/4067789118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bee0b0 msgr2=0x5578c8bee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.030+0000 7ff33c89e700 1 --2- 172.21.15.57:0/4067789118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bee0b0 0x5578c8bee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.030+0000 7ff33c89e700 1 -- 172.21.15.57:0/4067789118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bf3880 msgr2=0x5578c8bf3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.030+0000 7ff33c89e700 1 --2- 172.21.15.57:0/4067789118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bf3880 0x5578c8bf3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.030+0000 7ff33c89e700 1 -- 172.21.15.57:0/4067789118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578c896bd80 con 0x5578c8bf41a0 2023-10-08T14:04:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.030+0000 7ff33c89e700 1 --2- 172.21.15.57:0/4067789118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 0x5578c8bedb70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff33400bd20 tx=0x7ff33400d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b9170e124e34a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.031+0000 7ff33c89e700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff3240a1780 0x7ff3240a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.032+0000 7ff33c89e700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff3240a1780 0x7ff3240a3c40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff334006910 tx=0x7ff334014410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.036+0000 7ff3237fe700 1 -- 172.21.15.57:0/4067789118 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334010070 con 0x5578c8bf41a0 2023-10-08T14:04:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.036+0000 7ff3237fe700 1 -- 172.21.15.57:0/4067789118 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33400be70 con 0x5578c8bf41a0 2023-10-08T14:04:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.036+0000 7ff3237fe700 1 -- 172.21.15.57:0/4067789118 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3340065a0 con 0x5578c8bf41a0 2023-10-08T14:04:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.037+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/4067789118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 msgr2=0x5578c8bedb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.037+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/4067789118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 0x5578c8bedb70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff33400bd20 tx=0x7ff33400d330 comp rx=0 tx=0).stop 2023-10-08T14:04:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.042+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/4067789118 shutdown_connections 2023-10-08T14:04:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.042+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/4067789118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bee0b0 0x5578c8bee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.042+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/4067789118 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bf3880 0x5578c8bf3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.042+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/4067789118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 0x5578c8bedb70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.042+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/4067789118 >> 172.21.15.57:0/4067789118 conn(0x5578c8b62200 msgr2=0x5578c8be8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.043+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/4067789118 shutdown_connections 2023-10-08T14:04:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.043+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/4067789118 wait complete. 2023-10-08T14:04:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.043+0000 7ff34b4ecb80 1 -- start start 2023-10-08T14:04:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.043+0000 7ff34b4ecb80 1 Processor -- start 2023-10-08T14:04:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bf7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf3880 0x5578c8bf7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 0x5578c8bf7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8b40dd0 con 0x5578c8bee0b0 2023-10-08T14:04:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8add6b0 con 0x5578c8bf41a0 2023-10-08T14:04:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8add350 con 0x5578c8bf3880 2023-10-08T14:04:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff34b4ecb80 10 client.0 _ll_register_callbacks cb 0x5578c893a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:04:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff33d09f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bf7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff33d09f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bf7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60126/0 (socket says 172.21.15.57:60126) 2023-10-08T14:04:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff33d09f700 1 -- 172.21.15.57:0/81936995 learned_addr learned my addr 172.21.15.57:0/81936995 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.044+0000 7ff33c89e700 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf3880 0x5578c8bf7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.045+0000 7ff33d8a0700 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 0x5578c8bf7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.045+0000 7ff33d09f700 1 -- 172.21.15.57:0/81936995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf3880 msgr2=0x5578c8bf7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.045+0000 7ff33d09f700 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf3880 0x5578c8bf7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.045+0000 7ff33d09f700 1 -- 172.21.15.57:0/81936995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 msgr2=0x5578c8bf7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.045+0000 7ff33d09f700 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 0x5578c8bf7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.045+0000 7ff33d09f700 1 -- 172.21.15.57:0/81936995 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33400b9d0 con 0x5578c8bee0b0 2023-10-08T14:04:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.045+0000 7ff33d09f700 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bf7540 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff3280a7d40 tx=0x7ff32801df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41eb59347cec3ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.048+0000 7ff320ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:04:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.048+0000 7ff320ff9700 1 -- 172.21.15.57:0/81936995 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3280b3660 con 0x5578c8bee0b0 2023-10-08T14:04:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.048+0000 7ff320ff9700 1 -- 172.21.15.57:0/81936995 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9441+0+0 (secure 0 0 0) 0x7ff3280dd040 con 0x5578c8bee0b0 2023-10-08T14:04:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.048+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/81936995 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578c8b08e50 con 0x5578c8bee0b0 2023-10-08T14:04:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.049+0000 7ff320ff9700 1 -- 172.21.15.57:0/81936995 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3280af170 con 0x5578c8bee0b0 2023-10-08T14:04:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.050+0000 7ff34b4ecb80 10 client.17280 fetch_fsmap learned FSMap version 48 2023-10-08T14:04:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.050+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/81936995 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5578c8954c00 con 0x5578c8bee0b0 2023-10-08T14:04:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.051+0000 7ff320ff9700 1 -- 172.21.15.57:0/81936995 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3280b7080 con 0x5578c8bee0b0 2023-10-08T14:04:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.051+0000 7ff320ff9700 1 -- 172.21.15.57:0/81936995 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff3280de3b0 con 0x5578c8bee0b0 2023-10-08T14:04:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.051+0000 7ff34b4ecb80 10 client.17280 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:04:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.051+0000 7ff34b4ecb80 10 client.17280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:04:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.051+0000 7ff34b4ecb80 20 client.17280 populate_metadata read hostname 'smithi057' 2023-10-08T14:04:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.051+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578c8c18d10 0x5578c8c1b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.051+0000 7ff34b4ecb80 4 client.17280 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:04:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.051+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/81936995 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578c8b39db0 con 0x5578c8c18d10 2023-10-08T14:04:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.051+0000 7ff33c89e700 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578c8c18d10 0x5578c8c1b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.052+0000 7ff320ff9700 10 client.17280 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.052+0000 7ff33c89e700 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578c8c18d10 0x5578c8c1b130 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851c55b3fa0e77d server_cookie=5a49061af73cf803 in_seq=0 out_seq=0 2023-10-08T14:04:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.057+0000 7ff320ff9700 1 -- 172.21.15.57:0/81936995 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5578c8b39db0 con 0x5578c8c18d10 2023-10-08T14:04:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.057+0000 7ff320ff9700 10 client.17280 handle_command_reply: tid=0 2023-10-08T14:04:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.057+0000 7ff34b4ecb80 1 client.17280 shutdown 2023-10-08T14:04:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.060+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/81936995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 msgr2=0x5578c8bf7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.060+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bf7540 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff3280a7d40 tx=0x7ff32801df50 comp rx=0 tx=0).stop 2023-10-08T14:04:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.060+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/81936995 shutdown_connections 2023-10-08T14:04:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.060+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578c8c18d10 0x5578c8c1b130 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.060+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf41a0 0x5578c8bf7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.060+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bf7540 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.060+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/81936995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf3880 0x5578c8bf7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.060+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/81936995 >> 172.21.15.57:0/81936995 conn(0x5578c8b62200 msgr2=0x5578c8be8010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.062+0000 7ff320ff9700 0 client.17280 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.062+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/81936995 shutdown_connections 2023-10-08T14:04:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.066+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/81936995 wait complete. 2023-10-08T14:04:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.067+0000 7ff34b4ecb80 20 client.17280 trim_cache size 0 max 16384 2023-10-08T14:04:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.093+0000 7ff34b4ecb80 1 Processor -- start 2023-10-08T14:04:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.095+0000 7ff34b4ecb80 1 -- start start 2023-10-08T14:04:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.095+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bedb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.095+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.095+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf41a0 0x5578c8bf6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.095+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c89c18c0 con 0x5578c8bee0b0 2023-10-08T14:04:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.095+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8bf8510 con 0x5578c8bf3880 2023-10-08T14:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.095+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8be32b0 con 0x5578c8bf41a0 2023-10-08T14:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.097+0000 7ff33d8a0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf41a0 0x5578c8bf6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.097+0000 7ff33c89e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.097+0000 7ff33c89e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33332/0 (socket says 172.21.15.57:33332) 2023-10-08T14:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.097+0000 7ff33c89e700 1 -- 172.21.15.57:0/4140763621 learned_addr learned my addr 172.21.15.57:0/4140763621 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.097+0000 7ff33c89e700 1 -- 172.21.15.57:0/4140763621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf41a0 msgr2=0x5578c8bf6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.097+0000 7ff33c89e700 1 --2- 172.21.15.57:0/4140763621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf41a0 0x5578c8bf6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.097+0000 7ff33c89e700 1 -- 172.21.15.57:0/4140763621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 msgr2=0x5578c8bedb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.097+0000 7ff33c89e700 1 --2- 172.21.15.57:0/4140763621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bedb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.097+0000 7ff33c89e700 1 -- 172.21.15.57:0/4140763621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578c8c1fd20 con 0x5578c8bf3880 2023-10-08T14:04:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.098+0000 7ff33c89e700 1 --2- 172.21.15.57:0/4140763621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfec10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff334016000 tx=0x7ff334022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e96fb529a43109 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.099+0000 7ff323fff700 1 -- 172.21.15.57:0/4140763621 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400dea0 con 0x5578c8bf3880 2023-10-08T14:04:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.099+0000 7ff323fff700 1 -- 172.21.15.57:0/4140763621 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff334014880 con 0x5578c8bf3880 2023-10-08T14:04:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.099+0000 7ff323fff700 1 -- 172.21.15.57:0/4140763621 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334010040 con 0x5578c8bf3880 2023-10-08T14:04:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.100+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/4140763621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 msgr2=0x5578c8bfec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.100+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/4140763621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfec10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff334016000 tx=0x7ff334022000 comp rx=0 tx=0).stop 2023-10-08T14:04:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.104+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/4140763621 shutdown_connections 2023-10-08T14:04:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.104+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/4140763621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf41a0 0x5578c8bf6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.104+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/4140763621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bedb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.104+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/4140763621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfec10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.104+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/4140763621 >> 172.21.15.57:0/4140763621 conn(0x5578c8b62200 msgr2=0x5578c8bf16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.117+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/4140763621 shutdown_connections 2023-10-08T14:04:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.117+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/4140763621 wait complete. 2023-10-08T14:04:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.117+0000 7ff34b4ecb80 1 -- start start 2023-10-08T14:04:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.117+0000 7ff34b4ecb80 1 Processor -- start 2023-10-08T14:04:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.118+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bfc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.118+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.118+0000 7ff34b4ecb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf41a0 0x5578c8bfcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.118+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8c6bb00 con 0x5578c8bee0b0 2023-10-08T14:04:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.118+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8c6bc70 con 0x5578c8bf3880 2023-10-08T14:04:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.118+0000 7ff34b4ecb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578c8c6be30 con 0x5578c8bf41a0 2023-10-08T14:04:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.118+0000 7ff34b4ecb80 10 client.0 _ll_register_callbacks cb 0x7ff3280aac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:04:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.118+0000 7ff33c89e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.118+0000 7ff33c89e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33334/0 (socket says 172.21.15.57:33334) 2023-10-08T14:04:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.118+0000 7ff33c89e700 1 -- 172.21.15.57:0/3629118944 learned_addr learned my addr 172.21.15.57:0/3629118944 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.119+0000 7ff33c89e700 1 -- 172.21.15.57:0/3629118944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf41a0 msgr2=0x5578c8bfcef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:04:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.119+0000 7ff33c89e700 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf41a0 0x5578c8bfcef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.119+0000 7ff33c89e700 1 -- 172.21.15.57:0/3629118944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 msgr2=0x5578c8bfc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.119+0000 7ff33c89e700 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bfc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.119+0000 7ff33c89e700 1 -- 172.21.15.57:0/3629118944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578c896bd80 con 0x5578c8bf3880 2023-10-08T14:04:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.119+0000 7ff33c89e700 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfc9b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff334014a10 tx=0x7ff33400de90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce922b36829b7f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.119+0000 7ff320ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:04:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.119+0000 7ff320ff9700 1 -- 172.21.15.57:0/3629118944 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334010040 con 0x5578c8bf3880 2023-10-08T14:04:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.119+0000 7ff320ff9700 1 -- 172.21.15.57:0/3629118944 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9441+0+0 (secure 0 0 0) 0x7ff334026040 con 0x5578c8bf3880 2023-10-08T14:04:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.125+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/3629118944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff33400be70 con 0x5578c8bf3880 2023-10-08T14:04:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.134+0000 7ff320ff9700 1 -- 172.21.15.57:0/3629118944 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff33402d130 con 0x5578c8bf3880 2023-10-08T14:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.134+0000 7ff34b4ecb80 10 client.26810 fetch_fsmap learned FSMap version 48 2023-10-08T14:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.134+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/3629118944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5578c89921f0 con 0x5578c8bf3880 2023-10-08T14:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.135+0000 7ff320ff9700 1 -- 172.21.15.57:0/3629118944 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff33400be70 con 0x5578c8bf3880 2023-10-08T14:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.135+0000 7ff320ff9700 1 -- 172.21.15.57:0/3629118944 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff33403d050 con 0x5578c8bf3880 2023-10-08T14:04:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.135+0000 7ff34b4ecb80 10 client.26810 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.135+0000 7ff34b4ecb80 10 client.26810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.135+0000 7ff34b4ecb80 20 client.26810 populate_metadata read hostname 'smithi057' 2023-10-08T14:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.135+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578c8c6d230 0x5578c8c6f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.135+0000 7ff34b4ecb80 4 client.26810 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:04:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.135+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/3629118944 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5578c8b39ad0 con 0x5578c8c6d230 2023-10-08T14:04:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.136+0000 7ff33d09f700 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578c8c6d230 0x5578c8c6f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.136+0000 7ff33d09f700 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578c8c6d230 0x5578c8c6f650 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38df107295e6163b server_cookie=1af4fd7c288858cb in_seq=0 out_seq=0 2023-10-08T14:04:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.136+0000 7ff320ff9700 10 client.26810 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:06.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:05 smithi115 ceph-mon[104408]: pgmap v688: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2023-10-08T14:04:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.223+0000 7ff320ff9700 1 -- 172.21.15.57:0/3629118944 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5578c8b39ad0 con 0x5578c8c6d230 2023-10-08T14:04:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.223+0000 7ff320ff9700 10 client.26810 handle_command_reply: tid=0 2023-10-08T14:04:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.224+0000 7ff34b4ecb80 1 client.26810 shutdown 2023-10-08T14:04:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/3629118944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 msgr2=0x5578c8bfc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfc9b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff334014a10 tx=0x7ff33400de90 comp rx=0 tx=0).stop 2023-10-08T14:04:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/3629118944 shutdown_connections 2023-10-08T14:04:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5578c8c6d230 0x5578c8c6f650 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578c8bf41a0 0x5578c8bfcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5578c8bee0b0 0x5578c8bfc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff34b4ecb80 1 --2- 172.21.15.57:0/3629118944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578c8bf3880 0x5578c8bfc9b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/3629118944 >> 172.21.15.57:0/3629118944 conn(0x5578c8b62200 msgr2=0x5578c8bf7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff320ff9700 0 client.26810 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/3629118944 shutdown_connections 2023-10-08T14:04:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.225+0000 7ff34b4ecb80 1 -- 172.21.15.57:0/3629118944 wait complete. 2023-10-08T14:04:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:04:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:04:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:04:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:04:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.226+0000 7ff34b4ecb80 20 client.26810 trim_cache size 0 max 16384 2023-10-08T14:04:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.227+0000 7ff33f303700 1 -- 172.21.15.57:0/1799755079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff3240a1780 msgr2=0x7ff3240a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.227+0000 7ff33f303700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff3240a1780 0x7ff3240a3c40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff334006910 tx=0x7ff334014410 comp rx=0 tx=0).stop 2023-10-08T14:04:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.227+0000 7ff33f303700 1 -- 172.21.15.57:0/1799755079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3380fefc0 msgr2=0x7ff338106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.227+0000 7ff33f303700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3380fefc0 0x7ff338106dc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff32801faf0 tx=0x7ff32801fe00 comp rx=0 tx=0).stop 2023-10-08T14:04:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.227+0000 7ff33f303700 1 -- 172.21.15.57:0/1799755079 shutdown_connections 2023-10-08T14:04:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.227+0000 7ff33f303700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff338107840 0x7ff3381a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.227+0000 7ff33f303700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3380fefc0 0x7ff338106dc0 secure :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff32801faf0 tx=0x7ff32801fe00 comp rx=0 tx=0).stop 2023-10-08T14:04:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.227+0000 7ff33f303700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff3240a1780 0x7ff3240a3c40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.227+0000 7ff33f303700 1 --2- 172.21.15.57:0/1799755079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3381040e0 0x7ff338107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.227+0000 7ff33f303700 1 -- 172.21.15.57:0/1799755079 >> 172.21.15.57:0/1799755079 conn(0x7ff338075bf0 msgr2=0x7ff338102210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.236+0000 7ff33f303700 1 -- 172.21.15.57:0/1799755079 shutdown_connections 2023-10-08T14:04:06.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:05 smithi057 ceph-mon[99476]: pgmap v688: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2023-10-08T14:04:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:06.243+0000 7ff33f303700 1 -- 172.21.15.57:0/1799755079 wait complete. 2023-10-08T14:04:06.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:05 smithi099 ceph-mon[103849]: pgmap v688: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2023-10-08T14:04:08.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:07 smithi115 ceph-mon[104408]: pgmap v689: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 110 op/s 2023-10-08T14:04:08.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:07 smithi057 ceph-mon[99476]: pgmap v689: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 110 op/s 2023-10-08T14:04:08.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:07 smithi099 ceph-mon[103849]: pgmap v689: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 110 op/s 2023-10-08T14:04:09.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:09 smithi057 ceph-mon[99476]: pgmap v690: 129 pgs: 51 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 62 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1019 KiB/s wr, 122 op/s 2023-10-08T14:04:09.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:09 smithi099 ceph-mon[103849]: pgmap v690: 129 pgs: 51 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 62 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1019 KiB/s wr, 122 op/s 2023-10-08T14:04:09.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:09 smithi115 ceph-mon[104408]: pgmap v690: 129 pgs: 51 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 62 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1019 KiB/s wr, 122 op/s 2023-10-08T14:04:10.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:10 smithi057 ceph-mon[99476]: osdmap e106: 12 total, 12 up, 12 in 2023-10-08T14:04:10.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:10 smithi099 ceph-mon[103849]: osdmap e106: 12 total, 12 up, 12 in 2023-10-08T14:04:10.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:10 smithi115 ceph-mon[104408]: osdmap e106: 12 total, 12 up, 12 in 2023-10-08T14:04:11.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:11 smithi057 ceph-mon[99476]: pgmap v692: 129 pgs: 48 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 64 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 144 op/s 2023-10-08T14:04:11.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:11 smithi099 ceph-mon[103849]: pgmap v692: 129 pgs: 48 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 64 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 144 op/s 2023-10-08T14:04:11.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:11 smithi115 ceph-mon[104408]: pgmap v692: 129 pgs: 48 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 64 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 144 op/s 2023-10-08T14:04:14.190 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:13 smithi115 ceph-mon[104408]: pgmap v693: 129 pgs: 50 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 60 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-10-08T14:04:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:13 smithi057 ceph-mon[99476]: pgmap v693: 129 pgs: 50 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 60 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-10-08T14:04:14.278 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:13 smithi099 ceph-mon[103849]: pgmap v693: 129 pgs: 50 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 60 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-10-08T14:04:16.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:15 smithi115 ceph-mon[104408]: pgmap v694: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 798 KiB/s wr, 133 op/s 2023-10-08T14:04:16.240 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:04:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T14:04:15.835+0000 7fe4ce904700 -1 calc_pg_upmaps abort due to max <= 0 2023-10-08T14:04:16.240 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:04:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T14:04:15.835+0000 7fe4ce904700 -1 calc_pg_upmaps abort due to max <= 0 2023-10-08T14:04:16.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:15 smithi057 ceph-mon[99476]: pgmap v694: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 798 KiB/s wr, 133 op/s 2023-10-08T14:04:16.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:15 smithi099 ceph-mon[103849]: pgmap v694: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 798 KiB/s wr, 133 op/s 2023-10-08T14:04:18.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:17 smithi057 ceph-mon[99476]: pgmap v695: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 743 KiB/s wr, 129 op/s 2023-10-08T14:04:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:17 smithi099 ceph-mon[103849]: pgmap v695: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 743 KiB/s wr, 129 op/s 2023-10-08T14:04:18.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:17 smithi115 ceph-mon[104408]: pgmap v695: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 743 KiB/s wr, 129 op/s 2023-10-08T14:04:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.469+0000 7fe8ff76e700 1 -- 172.21.15.57:0/2968678506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f810ef10 msgr2=0x7fe8f81077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.469+0000 7fe8ff76e700 1 --2- 172.21.15.57:0/2968678506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f810ef10 0x7fe8f81077f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe8f001c0d0 tx=0x7fe8f001c3e0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.469+0000 7fe8ff76e700 1 -- 172.21.15.57:0/2968678506 shutdown_connections 2023-10-08T14:04:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.469+0000 7fe8ff76e700 1 --2- 172.21.15.57:0/2968678506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f8119fd0 0x7fe8f811c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.469+0000 7fe8ff76e700 1 --2- 172.21.15.57:0/2968678506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f810ef10 0x7fe8f81077f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.469+0000 7fe8ff76e700 1 --2- 172.21.15.57:0/2968678506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f810f380 0x7fe8f8107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.469+0000 7fe8ff76e700 1 -- 172.21.15.57:0/2968678506 >> 172.21.15.57:0/2968678506 conn(0x7fe8f806b7c0 msgr2=0x7fe8f806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.482+0000 7fe8ff76e700 1 -- 172.21.15.57:0/2968678506 shutdown_connections 2023-10-08T14:04:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.482+0000 7fe8ff76e700 1 -- 172.21.15.57:0/2968678506 wait complete. 2023-10-08T14:04:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.483+0000 7fe8ff76e700 1 Processor -- start 2023-10-08T14:04:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.483+0000 7fe8ff76e700 1 -- start start 2023-10-08T14:04:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8ff76e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f810ef10 0x7fe8f8119520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8ff76e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f810f380 0x7fe8f8119a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8ff76e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f8119fd0 0x7fe8f8115810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8ff76e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f8115fe0 con 0x7fe8f810f380 2023-10-08T14:04:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8ff76e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f8116150 con 0x7fe8f8119fd0 2023-10-08T14:04:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8ff76e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f8116310 con 0x7fe8f810ef10 2023-10-08T14:04:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8fcd09700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f810f380 0x7fe8f8119a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8fcd09700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f810f380 0x7fe8f8119a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60154/0 (socket says 172.21.15.57:60154) 2023-10-08T14:04:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8fcd09700 1 -- 172.21.15.57:0/3619626757 learned_addr learned my addr 172.21.15.57:0/3619626757 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8fdd0b700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f8119fd0 0x7fe8f8115810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8fcd09700 1 -- 172.21.15.57:0/3619626757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f810ef10 msgr2=0x7fe8f8119520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f810ef10 0x7fe8f8119520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8fcd09700 1 -- 172.21.15.57:0/3619626757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f8119fd0 msgr2=0x7fe8f8115810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f8119fd0 0x7fe8f8115810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.484+0000 7fe8fcd09700 1 -- 172.21.15.57:0/3619626757 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f001ad10 con 0x7fe8f810f380 2023-10-08T14:04:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.485+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f810f380 0x7fe8f8119a60 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400b550 tx=0x7fe8f400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.485+0000 7fe8ee7fc700 1 -- 172.21.15.57:0/3619626757 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4011830 con 0x7fe8f810f380 2023-10-08T14:04:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.485+0000 7fe8ff76e700 1 -- 172.21.15.57:0/3619626757 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f81a9c00 con 0x7fe8f810f380 2023-10-08T14:04:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.485+0000 7fe8ff76e700 1 -- 172.21.15.57:0/3619626757 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f81aa030 con 0x7fe8f810f380 2023-10-08T14:04:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.490+0000 7fe90b957b80 1 Processor -- start 2023-10-08T14:04:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.490+0000 7fe8ee7fc700 1 -- 172.21.15.57:0/3619626757 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8f4011e50 con 0x7fe8f810f380 2023-10-08T14:04:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.490+0000 7fe8ee7fc700 1 -- 172.21.15.57:0/3619626757 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4010e40 con 0x7fe8f810f380 2023-10-08T14:04:19.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.496+0000 7fe8ee7fc700 1 -- 172.21.15.57:0/3619626757 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe8f4011990 con 0x7fe8f810f380 2023-10-08T14:04:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.496+0000 7fe90b957b80 1 -- start start 2023-10-08T14:04:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8d8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 0x55650c8d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d30b0 0x55650c8d3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe90b957b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c83b400 con 0x55650c8d8880 2023-10-08T14:04:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe90b957b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c82cbc0 con 0x55650c8d30b0 2023-10-08T14:04:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe90b957b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c82a4e0 con 0x55650c8d91a0 2023-10-08T14:04:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe8fdd0b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8d8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe8fdd0b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8d8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60160/0 (socket says 172.21.15.57:60160) 2023-10-08T14:04:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe8fdd0b700 1 -- 172.21.15.57:0/3431988658 learned_addr learned my addr 172.21.15.57:0/3431988658 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe8fdd0b700 1 -- 172.21.15.57:0/3431988658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 msgr2=0x55650c8d2b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:04:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe8fdd0b700 1 --2- 172.21.15.57:0/3431988658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 0x55650c8d2b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe8fdd0b700 1 -- 172.21.15.57:0/3431988658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d30b0 msgr2=0x55650c8d3510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:04:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.497+0000 7fe8fdd0b700 1 --2- 172.21.15.57:0/3431988658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d30b0 0x55650c8d3510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.498+0000 7fe8ee7fc700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8e40a1670 0x7fe8e40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.498+0000 7fe8fdd0b700 1 -- 172.21.15.57:0/3431988658 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55650c650d80 con 0x55650c8d8880 2023-10-08T14:04:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.498+0000 7fe8ee7fc700 1 -- 172.21.15.57:0/3619626757 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7fe8f40ccd20 con 0x7fe8f810f380 2023-10-08T14:04:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.498+0000 7fe8fdd0b700 1 --2- 172.21.15.57:0/3431988658 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8d8c60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800b7c0 tx=0x7fe8e800bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0f3622051cd01a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.506+0000 7fe8e37fe700 1 -- 172.21.15.57:0/3431988658 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80125f0 con 0x55650c8d8880 2023-10-08T14:04:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.506+0000 7fe8e37fe700 1 -- 172.21.15.57:0/3431988658 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8e8012c10 con 0x55650c8d8880 2023-10-08T14:04:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.506+0000 7fe8e37fe700 1 -- 172.21.15.57:0/3431988658 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8013df0 con 0x55650c8d8880 2023-10-08T14:04:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.507+0000 7fe8fd50a700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8e40a1670 0x7fe8e40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.507+0000 7fe90b957b80 1 -- 172.21.15.57:0/3431988658 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 msgr2=0x55650c8d8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.507+0000 7fe90b957b80 1 --2- 172.21.15.57:0/3431988658 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8d8c60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800b7c0 tx=0x7fe8e800bad0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.511+0000 7fe90b957b80 1 -- 172.21.15.57:0/3431988658 shutdown_connections 2023-10-08T14:04:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.511+0000 7fe90b957b80 1 --2- 172.21.15.57:0/3431988658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d30b0 0x55650c8d3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.511+0000 7fe90b957b80 1 --2- 172.21.15.57:0/3431988658 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8d8c60 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.511+0000 7fe90b957b80 1 --2- 172.21.15.57:0/3431988658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 0x55650c8d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.511+0000 7fe90b957b80 1 -- 172.21.15.57:0/3431988658 >> 172.21.15.57:0/3431988658 conn(0x55650c847200 msgr2=0x55650c8cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.511+0000 7fe90b957b80 1 -- 172.21.15.57:0/3431988658 shutdown_connections 2023-10-08T14:04:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.511+0000 7fe90b957b80 1 -- 172.21.15.57:0/3431988658 wait complete. 2023-10-08T14:04:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.512+0000 7fe90b957b80 1 -- start start 2023-10-08T14:04:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.512+0000 7fe90b957b80 1 Processor -- start 2023-10-08T14:04:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.512+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d30b0 0x55650c8dc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.512+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d91a0 0x55650c8dd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe90b957b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c825dd0 con 0x55650c8d91a0 2023-10-08T14:04:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe90b957b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c7c26b0 con 0x55650c8d8880 2023-10-08T14:04:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe90b957b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c7c2350 con 0x55650c8d30b0 2023-10-08T14:04:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe90b957b80 10 client.0 _ll_register_callbacks cb 0x55650c61f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:04:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe8fcd09700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe8fcd09700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8dcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33364/0 (socket says 172.21.15.57:33364) 2023-10-08T14:04:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe8fcd09700 1 -- 172.21.15.57:0/829210347 learned_addr learned my addr 172.21.15.57:0/829210347 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe8fdd0b700 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d91a0 0x55650c8dd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe8fcd09700 1 -- 172.21.15.57:0/829210347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d30b0 msgr2=0x55650c8dc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d30b0 0x55650c8dc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe8fcd09700 1 -- 172.21.15.57:0/829210347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d91a0 msgr2=0x55650c8dd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d91a0 0x55650c8dd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.513+0000 7fe8fcd09700 1 -- 172.21.15.57:0/829210347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e800e040 con 0x55650c8d8880 2023-10-08T14:04:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.514+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8dcb90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40ca680 tx=0x7fe8f4018310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4980723ff999187e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.515+0000 7fe8fd50a700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8e40a1670 0x7fe8e40a3b30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe8f001a040 tx=0x7fe8f0019d90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.521+0000 7fe8e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:04:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.521+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/829210347 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4099e80 con 0x55650c8d8880 2023-10-08T14:04:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.521+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/829210347 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7fe8f409db70 con 0x55650c8d8880 2023-10-08T14:04:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.522+0000 7fe90b957b80 1 -- 172.21.15.57:0/829210347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55650c7ede50 con 0x55650c8d8880 2023-10-08T14:04:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.532+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/829210347 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8f40a1db0 con 0x55650c8d8880 2023-10-08T14:04:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.538+0000 7fe90b957b80 10 client.26819 fetch_fsmap learned FSMap version 48 2023-10-08T14:04:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.538+0000 7fe90b957b80 1 -- 172.21.15.57:0/829210347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55650c639c00 con 0x55650c8d8880 2023-10-08T14:04:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.542+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/829210347 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55650c7ede50 con 0x55650c8d8880 2023-10-08T14:04:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.542+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/829210347 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fe8f40a1920 con 0x55650c8d8880 2023-10-08T14:04:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.542+0000 7fe90b957b80 10 client.26819 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:04:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.542+0000 7fe90b957b80 10 client.26819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:04:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.542+0000 7fe90b957b80 20 client.26819 populate_metadata read hostname 'smithi057' 2023-10-08T14:04:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.542+0000 7fe90b957b80 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55650c8ecf20 0x55650c8ef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.542+0000 7fe90b957b80 4 client.26819 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:04:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.542+0000 7fe90b957b80 1 -- 172.21.15.57:0/829210347 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55650c81edb0 con 0x55650c8ecf20 2023-10-08T14:04:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.543+0000 7fe8fd50a700 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55650c8ecf20 0x55650c8ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.555+0000 7fe8fd50a700 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55650c8ecf20 0x55650c8ef340 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f87da2ec1f6564 server_cookie=9cb447b78cd20dd3 in_seq=0 out_seq=0 2023-10-08T14:04:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.555+0000 7fe8e0ff9700 10 client.26819 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.556+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/829210347 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55650c81edb0 con 0x55650c8ecf20 2023-10-08T14:04:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.556+0000 7fe8e0ff9700 10 client.26819 handle_command_reply: tid=0 2023-10-08T14:04:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.560+0000 7fe90b957b80 1 client.26819 shutdown 2023-10-08T14:04:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe90b957b80 1 -- 172.21.15.57:0/829210347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 msgr2=0x55650c8dcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe90b957b80 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8dcb90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40ca680 tx=0x7fe8f4018310 comp rx=0 tx=0).stop 2023-10-08T14:04:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe90b957b80 1 -- 172.21.15.57:0/829210347 shutdown_connections 2023-10-08T14:04:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe90b957b80 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55650c8ecf20 0x55650c8ef340 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe90b957b80 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d91a0 0x55650c8dd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe90b957b80 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d30b0 0x55650c8dc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe90b957b80 1 --2- 172.21.15.57:0/829210347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8dcb90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe8e0ff9700 0 client.26819 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe90b957b80 1 -- 172.21.15.57:0/829210347 >> 172.21.15.57:0/829210347 conn(0x55650c847200 msgr2=0x55650c8cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe90b957b80 1 -- 172.21.15.57:0/829210347 shutdown_connections 2023-10-08T14:04:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.561+0000 7fe90b957b80 1 -- 172.21.15.57:0/829210347 wait complete. 2023-10-08T14:04:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.562+0000 7fe90b957b80 20 client.26819 trim_cache size 0 max 16384 2023-10-08T14:04:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe90b957b80 1 Processor -- start 2023-10-08T14:04:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe90b957b80 1 -- start start 2023-10-08T14:04:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d30b0 0x55650c8cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 0x55650c8daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe90b957b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c6a68c0 con 0x55650c8d8880 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe90b957b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c8ce610 con 0x55650c8d30b0 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe90b957b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c8ea4e0 con 0x55650c8d91a0 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe8fcd09700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe8fcd09700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60180/0 (socket says 172.21.15.57:60180) 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.584+0000 7fe8fcd09700 1 -- 172.21.15.57:0/3509769319 learned_addr learned my addr 172.21.15.57:0/3509769319 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8fcd09700 1 -- 172.21.15.57:0/3509769319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 msgr2=0x55650c8daf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/3509769319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 0x55650c8daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8fcd09700 1 -- 172.21.15.57:0/3509769319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d30b0 msgr2=0x55650c8cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8fd50a700 1 --2- 172.21.15.57:0/3509769319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d30b0 0x55650c8cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/3509769319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d30b0 0x55650c8cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8fcd09700 1 -- 172.21.15.57:0/3509769319 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55650c6771f0 con 0x55650c8d8880 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8fd50a700 1 --2- 172.21.15.57:0/3509769319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d30b0 0x55650c8cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/3509769319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8cdcd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4004860 tx=0x7fe8f4006d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446581c7b6c1e6ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8e3fff700 1 -- 172.21.15.57:0/3509769319 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f400d040 con 0x55650c8d8880 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8e3fff700 1 -- 172.21.15.57:0/3509769319 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8f4095ba0 con 0x55650c8d8880 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.585+0000 7fe8e3fff700 1 -- 172.21.15.57:0/3509769319 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40a1db0 con 0x55650c8d8880 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 -- 172.21.15.57:0/3509769319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 msgr2=0x55650c8cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 --2- 172.21.15.57:0/3509769319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8cdcd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4004860 tx=0x7fe8f4006d90 comp rx=0 tx=0).stop 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 -- 172.21.15.57:0/3509769319 shutdown_connections 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 --2- 172.21.15.57:0/3509769319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 0x55650c8daf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 --2- 172.21.15.57:0/3509769319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d30b0 0x55650c8cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 --2- 172.21.15.57:0/3509769319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d8880 0x55650c8cdcd0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 -- 172.21.15.57:0/3509769319 >> 172.21.15.57:0/3509769319 conn(0x55650c847200 msgr2=0x55650c8d6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 -- 172.21.15.57:0/3509769319 shutdown_connections 2023-10-08T14:04:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 -- 172.21.15.57:0/3509769319 wait complete. 2023-10-08T14:04:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 -- start start 2023-10-08T14:04:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.586+0000 7fe90b957b80 1 Processor -- start 2023-10-08T14:04:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d30b0 0x55650c8e1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8e2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe90b957b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 0x55650c93fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe90b957b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c8e2ae0 con 0x55650c8d30b0 2023-10-08T14:04:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe90b957b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c940250 con 0x55650c8d8880 2023-10-08T14:04:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe90b957b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650c940410 con 0x55650c8d91a0 2023-10-08T14:04:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe90b957b80 10 client.0 _ll_register_callbacks cb 0x55650c61f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:04:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fd50a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d30b0 0x55650c8e1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fd50a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d30b0 0x55650c8e1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60182/0 (socket says 172.21.15.57:60182) 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fd50a700 1 -- 172.21.15.57:0/1673228051 learned_addr learned my addr 172.21.15.57:0/1673228051 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8e2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fdd0b700 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 0x55650c93fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fd50a700 1 -- 172.21.15.57:0/1673228051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 msgr2=0x55650c93fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fd50a700 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 0x55650c93fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fd50a700 1 -- 172.21.15.57:0/1673228051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 msgr2=0x55650c8e2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fd50a700 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8e2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fd50a700 1 -- 172.21.15.57:0/1673228051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55650c639c00 con 0x55650c8d30b0 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8e2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:04:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8fd50a700 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d30b0 0x55650c8e1e50 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe8f001e010 tx=0x7fe8f001a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78041d82cc79b80 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.587+0000 7fe8e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:04:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/1673228051 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f001f040 con 0x55650c8d30b0 2023-10-08T14:04:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/1673228051 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7fe8f0031040 con 0x55650c8d30b0 2023-10-08T14:04:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe90b957b80 1 -- 172.21.15.57:0/1673228051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8e8012c10 con 0x55650c8d30b0 2023-10-08T14:04:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/1673228051 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8f003bd30 con 0x55650c8d30b0 2023-10-08T14:04:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe90b957b80 10 client.17295 fetch_fsmap learned FSMap version 48 2023-10-08T14:04:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe90b957b80 1 -- 172.21.15.57:0/1673228051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55650c64f250 con 0x55650c8d30b0 2023-10-08T14:04:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/1673228051 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8e8012c10 con 0x55650c8d30b0 2023-10-08T14:04:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/1673228051 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fe8f00189a0 con 0x55650c8d30b0 2023-10-08T14:04:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe90b957b80 10 client.17295 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:04:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe90b957b80 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:04:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe90b957b80 20 client.17295 populate_metadata read hostname 'smithi057' 2023-10-08T14:04:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe90b957b80 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55650c941c50 0x55650c944070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe90b957b80 4 client.17295 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:04:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe90b957b80 1 -- 172.21.15.57:0/1673228051 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55650c81ead0 con 0x55650c941c50 2023-10-08T14:04:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.588+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55650c941c50 0x55650c944070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.589+0000 7fe8e0ff9700 10 client.17295 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.589+0000 7fe8fcd09700 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55650c941c50 0x55650c944070 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4385da77837d98e7 server_cookie=521784a7c3afab5e in_seq=0 out_seq=0 2023-10-08T14:04:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.722+0000 7fe8e0ff9700 1 -- 172.21.15.57:0/1673228051 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55650c81ead0 con 0x55650c941c50 2023-10-08T14:04:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.722+0000 7fe8e0ff9700 10 client.17295 handle_command_reply: tid=0 2023-10-08T14:04:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.722+0000 7fe90b957b80 1 client.17295 shutdown 2023-10-08T14:04:19.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:19 smithi057 ceph-mon[99476]: pgmap v696: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 827 KiB/s wr, 116 op/s 2023-10-08T14:04:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.736+0000 7fe90b957b80 1 -- 172.21.15.57:0/1673228051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d30b0 msgr2=0x55650c8e1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.736+0000 7fe90b957b80 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d30b0 0x55650c8e1e50 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe8f001e010 tx=0x7fe8f001a9f0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.740+0000 7fe90b957b80 1 -- 172.21.15.57:0/1673228051 shutdown_connections 2023-10-08T14:04:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.740+0000 7fe90b957b80 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55650c941c50 0x55650c944070 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.740+0000 7fe90b957b80 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55650c8d91a0 0x55650c93fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.740+0000 7fe90b957b80 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55650c8d30b0 0x55650c8e1e50 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.740+0000 7fe90b957b80 1 --2- 172.21.15.57:0/1673228051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55650c8d8880 0x55650c8e2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.740+0000 7fe90b957b80 1 -- 172.21.15.57:0/1673228051 >> 172.21.15.57:0/1673228051 conn(0x55650c847200 msgr2=0x55650c8d7700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.740+0000 7fe8e0ff9700 0 client.17295 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.742+0000 7fe90b957b80 1 -- 172.21.15.57:0/1673228051 shutdown_connections 2023-10-08T14:04:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.742+0000 7fe90b957b80 1 -- 172.21.15.57:0/1673228051 wait complete. 2023-10-08T14:04:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.747+0000 7fe90b957b80 20 client.17295 trim_cache size 0 max 16384 2023-10-08T14:04:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:04:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:04:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:04:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:04:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.760+0000 7fe8ff76e700 1 -- 172.21.15.57:0/3619626757 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8e40a1670 msgr2=0x7fe8e40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.760+0000 7fe8ff76e700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8e40a1670 0x7fe8e40a3b30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe8f001a040 tx=0x7fe8f0019d90 comp rx=0 tx=0).stop 2023-10-08T14:04:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.760+0000 7fe8ff76e700 1 -- 172.21.15.57:0/3619626757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f810f380 msgr2=0x7fe8f8119a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.760+0000 7fe8ff76e700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f810f380 0x7fe8f8119a60 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400b550 tx=0x7fe8f400b910 comp rx=0 tx=0).stop 2023-10-08T14:04:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.761+0000 7fe8ff76e700 1 -- 172.21.15.57:0/3619626757 shutdown_connections 2023-10-08T14:04:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.761+0000 7fe8ff76e700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe8e40a1670 0x7fe8e40a3b30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.761+0000 7fe8ff76e700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f8119fd0 0x7fe8f8115810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.761+0000 7fe8ff76e700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f810ef10 0x7fe8f8119520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.761+0000 7fe8ff76e700 1 --2- 172.21.15.57:0/3619626757 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe8f810f380 0x7fe8f8119a60 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.761+0000 7fe8ff76e700 1 -- 172.21.15.57:0/3619626757 >> 172.21.15.57:0/3619626757 conn(0x7fe8f806b7c0 msgr2=0x7fe8f811b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.762+0000 7fe8ff76e700 1 -- 172.21.15.57:0/3619626757 shutdown_connections 2023-10-08T14:04:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:19.765+0000 7fe8ff76e700 1 -- 172.21.15.57:0/3619626757 wait complete. 2023-10-08T14:04:19.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:19 smithi099 ceph-mon[103849]: pgmap v696: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 827 KiB/s wr, 116 op/s 2023-10-08T14:04:19.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:19 smithi115 ceph-mon[104408]: pgmap v696: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 827 KiB/s wr, 116 op/s 2023-10-08T14:04:22.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:21 smithi115 ceph-mon[104408]: pgmap v697: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 943 KiB/s wr, 105 op/s 2023-10-08T14:04:22.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:21 smithi057 ceph-mon[99476]: pgmap v697: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 943 KiB/s wr, 105 op/s 2023-10-08T14:04:22.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:21 smithi099 ceph-mon[103849]: pgmap v697: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 943 KiB/s wr, 105 op/s 2023-10-08T14:04:24.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:23 smithi115 ceph-mon[104408]: pgmap v698: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 899 KiB/s wr, 101 op/s 2023-10-08T14:04:24.242 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:23 smithi057 ceph-mon[99476]: pgmap v698: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 899 KiB/s wr, 101 op/s 2023-10-08T14:04:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:23 smithi099 ceph-mon[103849]: pgmap v698: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 899 KiB/s wr, 101 op/s 2023-10-08T14:04:26.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:25 smithi057 ceph-mon[99476]: pgmap v699: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 122 op/s 2023-10-08T14:04:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:25 smithi099 ceph-mon[103849]: pgmap v699: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 122 op/s 2023-10-08T14:04:26.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:25 smithi115 ceph-mon[104408]: pgmap v699: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 122 op/s 2023-10-08T14:04:28.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:27 smithi057 ceph-mon[99476]: pgmap v700: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 972 KiB/s wr, 97 op/s 2023-10-08T14:04:28.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:27 smithi099 ceph-mon[103849]: pgmap v700: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 972 KiB/s wr, 97 op/s 2023-10-08T14:04:28.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:27 smithi115 ceph-mon[104408]: pgmap v700: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 972 KiB/s wr, 97 op/s 2023-10-08T14:04:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:28 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:04:29.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:28 smithi099 ceph-mon[103849]: pgmap v701: 129 pgs: 63 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 45 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-10-08T14:04:29.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:28 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:04:29.443 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:28 smithi115 ceph-mon[104408]: pgmap v701: 129 pgs: 63 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 45 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-10-08T14:04:29.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:28 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:04:29.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:28 smithi057 ceph-mon[99476]: pgmap v701: 129 pgs: 63 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 45 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-10-08T14:04:30.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:30.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:31.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:31 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:04:31.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:31 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:04:31.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:31 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:31.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:31 smithi115 ceph-mon[104408]: pgmap v702: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 976 KiB/s wr, 124 op/s 2023-10-08T14:04:31.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:31 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:04:31.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:31 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:04:31.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:31 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:31.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:31 smithi057 ceph-mon[99476]: pgmap v702: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 976 KiB/s wr, 124 op/s 2023-10-08T14:04:31.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:31 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:04:31.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:31 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:04:31.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:31 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:04:31.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:31 smithi099 ceph-mon[103849]: pgmap v702: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 976 KiB/s wr, 124 op/s 2023-10-08T14:04:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:32.973+0000 7ff287a18700 1 -- 172.21.15.57:0/1651973043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff280112740 msgr2=0x7ff280114b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:32.973+0000 7ff287a18700 1 --2- 172.21.15.57:0/1651973043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff280112740 0x7ff280114b30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff270009990 tx=0x7ff270009ca0 comp rx=0 tx=0).stop 2023-10-08T14:04:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:32.973+0000 7ff287a18700 1 -- 172.21.15.57:0/1651973043 shutdown_connections 2023-10-08T14:04:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:32.973+0000 7ff287a18700 1 --2- 172.21.15.57:0/1651973043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff280112740 0x7ff280114b30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:32.973+0000 7ff287a18700 1 --2- 172.21.15.57:0/1651973043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff280109190 0x7ff280109570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:32.973+0000 7ff287a18700 1 --2- 172.21.15.57:0/1651973043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff280109ab0 0x7ff280112200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:32.973+0000 7ff287a18700 1 -- 172.21.15.57:0/1651973043 >> 172.21.15.57:0/1651973043 conn(0x7ff28006b7c0 msgr2=0x7ff28006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:32.978+0000 7ff287a18700 1 -- 172.21.15.57:0/1651973043 shutdown_connections 2023-10-08T14:04:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:32.995+0000 7ff287a18700 1 -- 172.21.15.57:0/1651973043 wait complete. 2023-10-08T14:04:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:32.995+0000 7ff287a18700 1 Processor -- start 2023-10-08T14:04:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.001+0000 7ff287a18700 1 -- start start 2023-10-08T14:04:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.001+0000 7ff287a18700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff280109ab0 0x7ff2801a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.001+0000 7ff287a18700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff280112740 0x7ff2801a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.001+0000 7ff287a18700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2801a1000 0x7ff2801ab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.001+0000 7ff287a18700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2801abc90 con 0x7ff2801a1000 2023-10-08T14:04:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.001+0000 7ff287a18700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2801abe00 con 0x7ff280112740 2023-10-08T14:04:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.001+0000 7ff287a18700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2801abfc0 con 0x7ff280109ab0 2023-10-08T14:04:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.002+0000 7ff285fb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2801a1000 0x7ff2801ab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.003+0000 7ff285fb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2801a1000 0x7ff2801ab600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60200/0 (socket says 172.21.15.57:60200) 2023-10-08T14:04:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.003+0000 7ff285fb5700 1 -- 172.21.15.57:0/1793256163 learned_addr learned my addr 172.21.15.57:0/1793256163 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.003+0000 7ff285fb5700 1 -- 172.21.15.57:0/1793256163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff280109ab0 msgr2=0x7ff2801a0580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T14:04:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.003+0000 7ff285fb5700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff280109ab0 0x7ff2801a0580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.003+0000 7ff285fb5700 1 -- 172.21.15.57:0/1793256163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff280112740 msgr2=0x7ff2801a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.003+0000 7ff285fb5700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff280112740 0x7ff2801a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.003+0000 7ff285fb5700 1 -- 172.21.15.57:0/1793256163 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2700095f0 con 0x7ff2801a1000 2023-10-08T14:04:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.003+0000 7ff285fb5700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2801a1000 0x7ff2801ab600 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff2700098e0 tx=0x7ff2700037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.004+0000 7ff2767fc700 1 -- 172.21.15.57:0/1793256163 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270027a50 con 0x7ff2801a1000 2023-10-08T14:04:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.004+0000 7ff2767fc700 1 -- 172.21.15.57:0/1793256163 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff270005580 con 0x7ff2801a1000 2023-10-08T14:04:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.004+0000 7ff287a18700 1 -- 172.21.15.57:0/1793256163 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28019e370 con 0x7ff2801a1000 2023-10-08T14:04:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.004+0000 7ff2767fc700 1 -- 172.21.15.57:0/1793256163 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27000fa00 con 0x7ff2801a1000 2023-10-08T14:04:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.004+0000 7ff287a18700 1 -- 172.21.15.57:0/1793256163 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28019e820 con 0x7ff2801a1000 2023-10-08T14:04:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.012+0000 7ff293c01b80 1 Processor -- start 2023-10-08T14:04:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.012+0000 7ff2767fc700 1 -- 172.21.15.57:0/1793256163 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff27000fb60 con 0x7ff2801a1000 2023-10-08T14:04:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.013+0000 7ff2767fc700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff26c0b2700 0x7ff26c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.013+0000 7ff2767fc700 1 -- 172.21.15.57:0/1793256163 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7ff2700cf830 con 0x7ff2801a1000 2023-10-08T14:04:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.013+0000 7ff293c01b80 1 -- start start 2023-10-08T14:04:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.013+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d93880 0x55e4e8d93c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.013+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d941a0 0x55e4e8d8db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.013+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d8e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.013+0000 7ff293c01b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8cf6400 con 0x55e4e8d8e0b0 2023-10-08T14:04:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.013+0000 7ff293c01b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8ce7bc0 con 0x55e4e8d941a0 2023-10-08T14:04:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.013+0000 7ff293c01b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8ce54e0 con 0x55e4e8d93880 2023-10-08T14:04:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.014+0000 7ff285fb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d8e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.014+0000 7ff285fb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d8e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60202/0 (socket says 172.21.15.57:60202) 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.014+0000 7ff285fb5700 1 -- 172.21.15.57:0/1920756319 learned_addr learned my addr 172.21.15.57:0/1920756319 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.016+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1920756319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d941a0 0x55e4e8d8db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.016+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff26c0b2700 0x7ff26c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.016+0000 7ff284fb3700 1 -- 172.21.15.57:0/1920756319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d93880 msgr2=0x55e4e8d93c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.016+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1920756319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d93880 0x55e4e8d93c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.016+0000 7ff284fb3700 1 -- 172.21.15.57:0/1920756319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d8e0b0 msgr2=0x55e4e8d8e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.016+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1920756319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d8e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.016+0000 7ff284fb3700 1 -- 172.21.15.57:0/1920756319 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4e8b0bd80 con 0x55e4e8d941a0 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.016+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff26c0b2700 0x7ff26c0b4bc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff27c014fd0 tx=0x7ff27c008000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.016+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1920756319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d941a0 0x55e4e8d8db70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff280108820 tx=0x7ff27c014990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e991a17b89bea1eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.017+0000 7ff26b7fe700 1 -- 172.21.15.57:0/1920756319 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c004830 con 0x55e4e8d941a0 2023-10-08T14:04:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.017+0000 7ff26b7fe700 1 -- 172.21.15.57:0/1920756319 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27c004e50 con 0x55e4e8d941a0 2023-10-08T14:04:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.017+0000 7ff26b7fe700 1 -- 172.21.15.57:0/1920756319 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c01f450 con 0x55e4e8d941a0 2023-10-08T14:04:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.018+0000 7ff293c01b80 1 -- 172.21.15.57:0/1920756319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d941a0 msgr2=0x55e4e8d8db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.018+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1920756319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d941a0 0x55e4e8d8db70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff280108820 tx=0x7ff27c014990 comp rx=0 tx=0).stop 2023-10-08T14:04:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.018+0000 7ff293c01b80 1 -- 172.21.15.57:0/1920756319 shutdown_connections 2023-10-08T14:04:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.018+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1920756319 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d8e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.018+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1920756319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d93880 0x55e4e8d93c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.018+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1920756319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d941a0 0x55e4e8d8db70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.018+0000 7ff293c01b80 1 -- 172.21.15.57:0/1920756319 >> 172.21.15.57:0/1920756319 conn(0x55e4e8d02200 msgr2=0x55e4e8d88b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.027+0000 7ff293c01b80 1 -- 172.21.15.57:0/1920756319 shutdown_connections 2023-10-08T14:04:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.040+0000 7ff293c01b80 1 -- 172.21.15.57:0/1920756319 wait complete. 2023-10-08T14:04:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.041+0000 7ff293c01b80 1 -- start start 2023-10-08T14:04:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.041+0000 7ff293c01b80 1 Processor -- start 2023-10-08T14:04:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.042+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d97670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.042+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d941a0 0x55e4e8d97bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.042+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 0x55e4e8dc5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.042+0000 7ff293c01b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8ce0dd0 con 0x55e4e8d941a0 2023-10-08T14:04:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.042+0000 7ff293c01b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8c7d6b0 con 0x55e4e8d8e0b0 2023-10-08T14:04:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.042+0000 7ff293c01b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8c7d350 con 0x55e4e8d980f0 2023-10-08T14:04:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.042+0000 7ff293c01b80 10 client.0 _ll_register_callbacks cb 0x55e4e8ada7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:04:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.047+0000 7ff285fb5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 0x55e4e8dc5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.047+0000 7ff285fb5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 0x55e4e8dc5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:32792/0 (socket says 172.21.15.57:32792) 2023-10-08T14:04:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.047+0000 7ff285fb5700 1 -- 172.21.15.57:0/1642675491 learned_addr learned my addr 172.21.15.57:0/1642675491 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.048+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d941a0 0x55e4e8d97bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.048+0000 7ff284fb3700 1 -- 172.21.15.57:0/1642675491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 msgr2=0x55e4e8dc5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.048+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 0x55e4e8dc5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.048+0000 7ff284fb3700 1 -- 172.21.15.57:0/1642675491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d8e0b0 msgr2=0x55e4e8d97670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.048+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d97670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.048+0000 7ff284fb3700 1 -- 172.21.15.57:0/1642675491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c00de20 con 0x55e4e8d941a0 2023-10-08T14:04:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.048+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d941a0 0x55e4e8d97bb0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff27c021040 tx=0x7ff27c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95786424500e04ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.048+0000 7ff268ff9700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.049+0000 7ff268ff9700 1 -- 172.21.15.57:0/1642675491 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c004a30 con 0x55e4e8d941a0 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.049+0000 7ff268ff9700 1 -- 172.21.15.57:0/1642675491 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7ff27c004b90 con 0x55e4e8d941a0 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.049+0000 7ff293c01b80 1 -- 172.21.15.57:0/1642675491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4e8ca8e50 con 0x55e4e8d941a0 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.049+0000 7ff268ff9700 1 -- 172.21.15.57:0/1642675491 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff27c0052f0 con 0x55e4e8d941a0 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.049+0000 7ff293c01b80 10 client.17307 fetch_fsmap learned FSMap version 48 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.049+0000 7ff293c01b80 1 -- 172.21.15.57:0/1642675491 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55e4e8af4c00 con 0x55e4e8d941a0 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.049+0000 7ff268ff9700 1 -- 172.21.15.57:0/1642675491 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4e8ca8e50 con 0x55e4e8d941a0 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.049+0000 7ff268ff9700 1 -- 172.21.15.57:0/1642675491 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff27c01f450 con 0x55e4e8d941a0 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.050+0000 7ff293c01b80 10 client.17307 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:04:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.050+0000 7ff293c01b80 10 client.17307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.050+0000 7ff293c01b80 20 client.17307 populate_metadata read hostname 'smithi057' 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.050+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e4e8dca2b0 0x55e4e8dcc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.050+0000 7ff293c01b80 4 client.17307 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.050+0000 7ff293c01b80 1 -- 172.21.15.57:0/1642675491 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4e8cd9db0 con 0x55e4e8dca2b0 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.050+0000 7ff2857b4700 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e4e8dca2b0 0x55e4e8dcc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.051+0000 7ff2857b4700 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e4e8dca2b0 0x55e4e8dcc770 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b1330e005911af3 server_cookie=f69f1c0cb55c91de in_seq=0 out_seq=0 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.052+0000 7ff268ff9700 10 client.17307 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.052+0000 7ff268ff9700 1 -- 172.21.15.57:0/1642675491 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e4e8cd9db0 con 0x55e4e8dca2b0 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.052+0000 7ff268ff9700 10 client.17307 handle_command_reply: tid=0 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.052+0000 7ff293c01b80 1 client.17307 shutdown 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.052+0000 7ff293c01b80 1 -- 172.21.15.57:0/1642675491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d941a0 msgr2=0x55e4e8d97bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.052+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d941a0 0x55e4e8d97bb0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff27c021040 tx=0x7ff27c005f90 comp rx=0 tx=0).stop 2023-10-08T14:04:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.054+0000 7ff293c01b80 1 -- 172.21.15.57:0/1642675491 shutdown_connections 2023-10-08T14:04:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.054+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e4e8dca2b0 0x55e4e8dcc770 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.054+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 0x55e4e8dc5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.054+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d97670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.054+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1642675491 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d941a0 0x55e4e8d97bb0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.054+0000 7ff293c01b80 1 -- 172.21.15.57:0/1642675491 >> 172.21.15.57:0/1642675491 conn(0x55e4e8d02200 msgr2=0x55e4e8d88010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.055+0000 7ff268ff9700 0 client.17307 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.064+0000 7ff293c01b80 1 -- 172.21.15.57:0/1642675491 shutdown_connections 2023-10-08T14:04:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.066+0000 7ff293c01b80 1 -- 172.21.15.57:0/1642675491 wait complete. 2023-10-08T14:04:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.069+0000 7ff293c01b80 20 client.17307 trim_cache size 0 max 16384 2023-10-08T14:04:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.097+0000 7ff293c01b80 1 Processor -- start 2023-10-08T14:04:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.097+0000 7ff293c01b80 1 -- start start 2023-10-08T14:04:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.097+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d8db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.097+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d980f0 0x55e4e8d88790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.097+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d91e10 0x55e4e8d88cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.097+0000 7ff293c01b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8b618c0 con 0x55e4e8d980f0 2023-10-08T14:04:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.097+0000 7ff293c01b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8d85830 con 0x55e4e8d8e0b0 2023-10-08T14:04:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.097+0000 7ff293c01b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8dc6c10 con 0x55e4e8d91e10 2023-10-08T14:04:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.098+0000 7ff285fb5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d91e10 0x55e4e8d88cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.098+0000 7ff285fb5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d91e10 0x55e4e8d88cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:32796/0 (socket says 172.21.15.57:32796) 2023-10-08T14:04:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.098+0000 7ff285fb5700 1 -- 172.21.15.57:0/2802164605 learned_addr learned my addr 172.21.15.57:0/2802164605 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.098+0000 7ff285fb5700 1 -- 172.21.15.57:0/2802164605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d8e0b0 msgr2=0x55e4e8d8db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.098+0000 7ff285fb5700 1 --2- 172.21.15.57:0/2802164605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d8db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.098+0000 7ff285fb5700 1 -- 172.21.15.57:0/2802164605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d980f0 msgr2=0x55e4e8d88790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.098+0000 7ff285fb5700 1 --2- 172.21.15.57:0/2802164605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d980f0 0x55e4e8d88790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.098+0000 7ff285fb5700 1 -- 172.21.15.57:0/2802164605 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4e8b321f0 con 0x55e4e8d91e10 2023-10-08T14:04:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.098+0000 7ff285fb5700 1 --2- 172.21.15.57:0/2802164605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d91e10 0x55e4e8d88cd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff27000c010 tx=0x7ff270098890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2758d725d60144 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.105+0000 7ff26bfff700 1 -- 172.21.15.57:0/2802164605 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27009bdb0 con 0x55e4e8d91e10 2023-10-08T14:04:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.105+0000 7ff26bfff700 1 -- 172.21.15.57:0/2802164605 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27009cb50 con 0x55e4e8d91e10 2023-10-08T14:04:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.105+0000 7ff26bfff700 1 -- 172.21.15.57:0/2802164605 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27000fea0 con 0x55e4e8d91e10 2023-10-08T14:04:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.106+0000 7ff293c01b80 1 -- 172.21.15.57:0/2802164605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d91e10 msgr2=0x55e4e8d88cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.106+0000 7ff293c01b80 1 --2- 172.21.15.57:0/2802164605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d91e10 0x55e4e8d88cd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff27000c010 tx=0x7ff270098890 comp rx=0 tx=0).stop 2023-10-08T14:04:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.110+0000 7ff293c01b80 1 -- 172.21.15.57:0/2802164605 shutdown_connections 2023-10-08T14:04:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.110+0000 7ff293c01b80 1 --2- 172.21.15.57:0/2802164605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d91e10 0x55e4e8d88cd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.110+0000 7ff293c01b80 1 --2- 172.21.15.57:0/2802164605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8d8db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.110+0000 7ff293c01b80 1 --2- 172.21.15.57:0/2802164605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d980f0 0x55e4e8d88790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.110+0000 7ff293c01b80 1 -- 172.21.15.57:0/2802164605 >> 172.21.15.57:0/2802164605 conn(0x55e4e8d02200 msgr2=0x55e4e8d91700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.111+0000 7ff293c01b80 1 -- 172.21.15.57:0/2802164605 shutdown_connections 2023-10-08T14:04:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.112+0000 7ff293c01b80 1 -- 172.21.15.57:0/2802164605 wait complete. 2023-10-08T14:04:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.112+0000 7ff293c01b80 1 -- start start 2023-10-08T14:04:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.112+0000 7ff293c01b80 1 Processor -- start 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8dbffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d91e10 0x55e4e8dc0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff293c01b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 0x55e4e8e1d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff293c01b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8dc0c50 con 0x55e4e8d8e0b0 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff293c01b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8e1d700 con 0x55e4e8d91e10 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff293c01b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e8e1d840 con 0x55e4e8d980f0 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff293c01b80 10 client.0 _ll_register_callbacks cb 0x7ff270005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff284fb3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d91e10 0x55e4e8dc0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff285fb5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 0x55e4e8e1d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff284fb3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d91e10 0x55e4e8dc0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33416/0 (socket says 172.21.15.57:33416) 2023-10-08T14:04:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff285fb5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 0x55e4e8e1d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:32804/0 (socket says 172.21.15.57:32804) 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.113+0000 7ff284fb3700 1 -- 172.21.15.57:0/1344471887 learned_addr learned my addr 172.21.15.57:0/1344471887 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff284fb3700 1 -- 172.21.15.57:0/1344471887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 msgr2=0x55e4e8e1d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 0x55e4e8e1d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff284fb3700 1 -- 172.21.15.57:0/1344471887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d8e0b0 msgr2=0x55e4e8dbffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8dbffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff284fb3700 1 -- 172.21.15.57:0/1344471887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28019e820 con 0x55e4e8d91e10 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff284fb3700 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d91e10 0x55e4e8dc0520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff27c012010 tx=0x7ff27c014e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6502a6450ddb0ecd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff268ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff268ff9700 1 -- 172.21.15.57:0/1344471887 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c01f650 con 0x55e4e8d91e10 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff293c01b80 1 -- 172.21.15.57:0/1344471887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff27c004e50 con 0x55e4e8d91e10 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff268ff9700 1 -- 172.21.15.57:0/1344471887 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7ff27c004830 con 0x55e4e8d91e10 2023-10-08T14:04:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff268ff9700 1 -- 172.21.15.57:0/1344471887 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff27c0052f0 con 0x55e4e8d91e10 2023-10-08T14:04:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff293c01b80 10 client.26831 fetch_fsmap learned FSMap version 48 2023-10-08T14:04:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.114+0000 7ff293c01b80 1 -- 172.21.15.57:0/1344471887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55e4e8b0a250 con 0x55e4e8d91e10 2023-10-08T14:04:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.115+0000 7ff268ff9700 1 -- 172.21.15.57:0/1344471887 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27c004e50 con 0x55e4e8d91e10 2023-10-08T14:04:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.115+0000 7ff268ff9700 1 -- 172.21.15.57:0/1344471887 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff27c028a40 con 0x55e4e8d91e10 2023-10-08T14:04:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.116+0000 7ff293c01b80 10 client.26831 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:04:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.116+0000 7ff293c01b80 10 client.26831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:04:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.116+0000 7ff293c01b80 20 client.26831 populate_metadata read hostname 'smithi057' 2023-10-08T14:04:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.116+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e4e8e1ecd0 0x55e4e8e21190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.116+0000 7ff293c01b80 4 client.26831 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:04:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.116+0000 7ff293c01b80 1 -- 172.21.15.57:0/1344471887 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e4e8cd9ad0 con 0x55e4e8e1ecd0 2023-10-08T14:04:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.120+0000 7ff2857b4700 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e4e8e1ecd0 0x55e4e8e21190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.120+0000 7ff268ff9700 10 client.26831 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.120+0000 7ff2857b4700 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e4e8e1ecd0 0x55e4e8e21190 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d3c0b8d6d82cb8 server_cookie=c6cc4c4402183410 in_seq=0 out_seq=0 2023-10-08T14:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.159+0000 7ff268ff9700 1 -- 172.21.15.57:0/1344471887 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e4e8cd9ad0 con 0x55e4e8e1ecd0 2023-10-08T14:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.159+0000 7ff268ff9700 10 client.26831 handle_command_reply: tid=0 2023-10-08T14:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.159+0000 7ff293c01b80 1 client.26831 shutdown 2023-10-08T14:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.161+0000 7ff293c01b80 1 -- 172.21.15.57:0/1344471887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d91e10 msgr2=0x55e4e8dc0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.161+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d91e10 0x55e4e8dc0520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff27c012010 tx=0x7ff27c014e90 comp rx=0 tx=0).stop 2023-10-08T14:04:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.172+0000 7ff293c01b80 1 -- 172.21.15.57:0/1344471887 shutdown_connections 2023-10-08T14:04:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.172+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e4e8e1ecd0 0x55e4e8e21190 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.172+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4e8d980f0 0x55e4e8e1d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.172+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4e8d8e0b0 0x55e4e8dbffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.172+0000 7ff293c01b80 1 --2- 172.21.15.57:0/1344471887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4e8d91e10 0x55e4e8dc0520 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.172+0000 7ff293c01b80 1 -- 172.21.15.57:0/1344471887 >> 172.21.15.57:0/1344471887 conn(0x55e4e8d02200 msgr2=0x55e4e8d95df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.178+0000 7ff268ff9700 0 client.26831 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.178+0000 7ff293c01b80 1 -- 172.21.15.57:0/1344471887 shutdown_connections 2023-10-08T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.181+0000 7ff293c01b80 1 -- 172.21.15.57:0/1344471887 wait complete. 2023-10-08T14:04:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.187+0000 7ff293c01b80 20 client.26831 trim_cache size 0 max 16384 2023-10-08T14:04:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:04:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:04:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:04:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:04:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.205+0000 7ff287a18700 1 -- 172.21.15.57:0/1793256163 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff26c0b2700 msgr2=0x7ff26c0b4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.205+0000 7ff287a18700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff26c0b2700 0x7ff26c0b4bc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff27c014fd0 tx=0x7ff27c008000 comp rx=0 tx=0).stop 2023-10-08T14:04:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.205+0000 7ff287a18700 1 -- 172.21.15.57:0/1793256163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2801a1000 msgr2=0x7ff2801ab600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.205+0000 7ff287a18700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2801a1000 0x7ff2801ab600 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff2700098e0 tx=0x7ff2700037c0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.218+0000 7ff287a18700 1 -- 172.21.15.57:0/1793256163 shutdown_connections 2023-10-08T14:04:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.218+0000 7ff287a18700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2801a1000 0x7ff2801ab600 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.218+0000 7ff287a18700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff280109ab0 0x7ff2801a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.218+0000 7ff287a18700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff26c0b2700 0x7ff26c0b4bc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.218+0000 7ff287a18700 1 --2- 172.21.15.57:0/1793256163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff280112740 0x7ff2801a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.218+0000 7ff287a18700 1 -- 172.21.15.57:0/1793256163 >> 172.21.15.57:0/1793256163 conn(0x7ff28006b7c0 msgr2=0x7ff28006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.229+0000 7ff287a18700 1 -- 172.21.15.57:0/1793256163 shutdown_connections 2023-10-08T14:04:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:33.239+0000 7ff287a18700 1 -- 172.21.15.57:0/1793256163 wait complete. 2023-10-08T14:04:34.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:33 smithi115 ceph-mon[104408]: pgmap v703: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 777 KiB/s wr, 121 op/s 2023-10-08T14:04:34.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:33 smithi057 ceph-mon[99476]: pgmap v703: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 777 KiB/s wr, 121 op/s 2023-10-08T14:04:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:33 smithi099 ceph-mon[103849]: pgmap v703: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 777 KiB/s wr, 121 op/s 2023-10-08T14:04:36.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:35 smithi115 ceph-mon[104408]: pgmap v704: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 974 KiB/s wr, 134 op/s 2023-10-08T14:04:36.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:35 smithi057 ceph-mon[99476]: pgmap v704: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 974 KiB/s wr, 134 op/s 2023-10-08T14:04:36.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:35 smithi099 ceph-mon[103849]: pgmap v704: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 974 KiB/s wr, 134 op/s 2023-10-08T14:04:38.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:37 smithi057 ceph-mon[99476]: pgmap v705: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 493 KiB/s wr, 106 op/s 2023-10-08T14:04:38.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:37 smithi099 ceph-mon[103849]: pgmap v705: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 493 KiB/s wr, 106 op/s 2023-10-08T14:04:38.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:37 smithi115 ceph-mon[104408]: pgmap v705: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 493 KiB/s wr, 106 op/s 2023-10-08T14:04:39.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:39 smithi057 ceph-mon[99476]: pgmap v706: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 653 KiB/s wr, 122 op/s 2023-10-08T14:04:39.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:39 smithi099 ceph-mon[103849]: pgmap v706: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 653 KiB/s wr, 122 op/s 2023-10-08T14:04:39.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:39 smithi115 ceph-mon[104408]: pgmap v706: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 653 KiB/s wr, 122 op/s 2023-10-08T14:04:42.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:41 smithi115 ceph-mon[104408]: pgmap v707: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 637 KiB/s wr, 128 op/s 2023-10-08T14:04:42.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:41 smithi057 ceph-mon[99476]: pgmap v707: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 637 KiB/s wr, 128 op/s 2023-10-08T14:04:42.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:41 smithi099 ceph-mon[103849]: pgmap v707: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 637 KiB/s wr, 128 op/s 2023-10-08T14:04:44.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:43 smithi115 ceph-mon[104408]: pgmap v708: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 809 KiB/s wr, 119 op/s 2023-10-08T14:04:44.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:43 smithi057 ceph-mon[99476]: pgmap v708: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 809 KiB/s wr, 119 op/s 2023-10-08T14:04:44.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:43 smithi099 ceph-mon[103849]: pgmap v708: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 809 KiB/s wr, 119 op/s 2023-10-08T14:04:46.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:45 smithi057 ceph-mon[99476]: pgmap v709: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 876 KiB/s wr, 140 op/s 2023-10-08T14:04:46.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:45 smithi099 ceph-mon[103849]: pgmap v709: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 876 KiB/s wr, 140 op/s 2023-10-08T14:04:46.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:45 smithi115 ceph-mon[104408]: pgmap v709: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 876 KiB/s wr, 140 op/s 2023-10-08T14:04:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.468+0000 7efe23c55700 1 -- 172.21.15.57:0/232086959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe1c081370 msgr2=0x7efe1c079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.469+0000 7efe23c55700 1 --2- 172.21.15.57:0/232086959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe1c081370 0x7efe1c079150 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efe18009990 tx=0x7efe18009ca0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.480+0000 7efe23c55700 1 -- 172.21.15.57:0/232086959 shutdown_connections 2023-10-08T14:04:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.480+0000 7efe23c55700 1 --2- 172.21.15.57:0/232086959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c119c10 0x7efe1c11c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.480+0000 7efe23c55700 1 --2- 172.21.15.57:0/232086959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe1c081370 0x7efe1c079150 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.480+0000 7efe23c55700 1 --2- 172.21.15.57:0/232086959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe1c079690 0x7efe1c079b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.480+0000 7efe23c55700 1 -- 172.21.15.57:0/232086959 >> 172.21.15.57:0/232086959 conn(0x7efe1c0764f0 msgr2=0x7efe1c078910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.480+0000 7efe23c55700 1 -- 172.21.15.57:0/232086959 shutdown_connections 2023-10-08T14:04:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.483+0000 7efe23c55700 1 -- 172.21.15.57:0/232086959 wait complete. 2023-10-08T14:04:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.483+0000 7efe23c55700 1 Processor -- start 2023-10-08T14:04:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.483+0000 7efe23c55700 1 -- start start 2023-10-08T14:04:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe23c55700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe1c079690 0x7efe1c071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe23c55700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c081370 0x7efe1c072410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe23c55700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe1c119c10 0x7efe1c06c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe23c55700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c06c900 con 0x7efe1c119c10 2023-10-08T14:04:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe23c55700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c06ca70 con 0x7efe1c081370 2023-10-08T14:04:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe23c55700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c06cc30 con 0x7efe1c079690 2023-10-08T14:04:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe219f1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe1c079690 0x7efe1c071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe219f1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe1c079690 0x7efe1c071ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:32816/0 (socket says 172.21.15.57:32816) 2023-10-08T14:04:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe219f1700 1 -- 172.21.15.57:0/3981404533 learned_addr learned my addr 172.21.15.57:0/3981404533 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe221f2700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe1c119c10 0x7efe1c06c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe219f1700 1 -- 172.21.15.57:0/3981404533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c081370 msgr2=0x7efe1c072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe219f1700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c081370 0x7efe1c072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe219f1700 1 -- 172.21.15.57:0/3981404533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe1c119c10 msgr2=0x7efe1c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe219f1700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe1c119c10 0x7efe1c06c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.484+0000 7efe219f1700 1 -- 172.21.15.57:0/3981404533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180095f0 con 0x7efe1c079690 2023-10-08T14:04:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.494+0000 7efe219f1700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe1c079690 0x7efe1c071ed0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe18003b90 tx=0x7efe18003c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.498+0000 7efe12ffd700 1 -- 172.21.15.57:0/3981404533 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1801d070 con 0x7efe1c079690 2023-10-08T14:04:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.498+0000 7efe12ffd700 1 -- 172.21.15.57:0/3981404533 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe18003e40 con 0x7efe1c079690 2023-10-08T14:04:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.498+0000 7efe12ffd700 1 -- 172.21.15.57:0/3981404533 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1800fe00 con 0x7efe1c079690 2023-10-08T14:04:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.498+0000 7efe23c55700 1 -- 172.21.15.57:0/3981404533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1c1babe0 con 0x7efe1c079690 2023-10-08T14:04:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.498+0000 7efe23c55700 1 -- 172.21.15.57:0/3981404533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1c1baf50 con 0x7efe1c079690 2023-10-08T14:04:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.503+0000 7efe2fe3eb80 1 Processor -- start 2023-10-08T14:04:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.503+0000 7efe12ffd700 1 -- 172.21.15.57:0/3981404533 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7efe1802b440 con 0x7efe1c079690 2023-10-08T14:04:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.504+0000 7efe12ffd700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efe080a1800 0x7efe080a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.504+0000 7efe12ffd700 1 -- 172.21.15.57:0/3981404533 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7efe180ce5b0 con 0x7efe1c079690 2023-10-08T14:04:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.504+0000 7efe221f2700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efe080a1800 0x7efe080a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe2fe3eb80 1 -- start start 2023-10-08T14:04:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe221f2700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efe080a1800 0x7efe080a3cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efe0c006fd0 tx=0x7efe0c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fb880 0x55a5a93fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fc1a0 0x55a5a93f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 0x55a5a93f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a935e400 con 0x55a5a93fc1a0 2023-10-08T14:04:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a934fbc0 con 0x55a5a93fb880 2023-10-08T14:04:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a934d4e0 con 0x55a5a93f60b0 2023-10-08T14:04:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe211f0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fc1a0 0x55a5a93f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe211f0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fc1a0 0x55a5a93f5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60246/0 (socket says 172.21.15.57:60246) 2023-10-08T14:04:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.505+0000 7efe211f0700 1 -- 172.21.15.57:0/4028198137 learned_addr learned my addr 172.21.15.57:0/4028198137 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe211f0700 1 -- 172.21.15.57:0/4028198137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 msgr2=0x55a5a93f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe211f0700 1 --2- 172.21.15.57:0/4028198137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 0x55a5a93f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe211f0700 1 -- 172.21.15.57:0/4028198137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fb880 msgr2=0x55a5a93fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe219f1700 1 --2- 172.21.15.57:0/4028198137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fb880 0x55a5a93fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe211f0700 1 --2- 172.21.15.57:0/4028198137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fb880 0x55a5a93fbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe211f0700 1 -- 172.21.15.57:0/4028198137 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5a9173d80 con 0x55a5a93fc1a0 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe219f1700 1 --2- 172.21.15.57:0/4028198137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fb880 0x55a5a93fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe211f0700 1 --2- 172.21.15.57:0/4028198137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fc1a0 0x55a5a93f5b70 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efe14009d70 tx=0x7efe1400b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff242d314399c00 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe07fff700 1 -- 172.21.15.57:0/4028198137 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14013410 con 0x55a5a93fc1a0 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe07fff700 1 -- 172.21.15.57:0/4028198137 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe14013a30 con 0x55a5a93fc1a0 2023-10-08T14:04:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.506+0000 7efe07fff700 1 -- 172.21.15.57:0/4028198137 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140129d0 con 0x55a5a93fc1a0 2023-10-08T14:04:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.507+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/4028198137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fc1a0 msgr2=0x55a5a93f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.507+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/4028198137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fc1a0 0x55a5a93f5b70 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efe14009d70 tx=0x7efe1400b700 comp rx=0 tx=0).stop 2023-10-08T14:04:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.507+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/4028198137 shutdown_connections 2023-10-08T14:04:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.507+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/4028198137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 0x55a5a93f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.507+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/4028198137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fb880 0x55a5a93fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.507+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/4028198137 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fc1a0 0x55a5a93f5b70 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.507+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/4028198137 >> 172.21.15.57:0/4028198137 conn(0x55a5a936a200 msgr2=0x55a5a93f0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.507+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/4028198137 shutdown_connections 2023-10-08T14:04:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.507+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/4028198137 wait complete. 2023-10-08T14:04:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.508+0000 7efe2fe3eb80 1 -- start start 2023-10-08T14:04:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.508+0000 7efe2fe3eb80 1 Processor -- start 2023-10-08T14:04:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93f60b0 0x55a5a9402220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93fb880 0x55a5a93ff510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fc1a0 0x55a5a93ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9348dd0 con 0x55a5a93f60b0 2023-10-08T14:04:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a92e56b0 con 0x55a5a93fc1a0 2023-10-08T14:04:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a92e5350 con 0x55a5a93fb880 2023-10-08T14:04:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe2fe3eb80 10 client.0 _ll_register_callbacks cb 0x55a5a91427a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:04:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe219f1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93f60b0 0x55a5a9402220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe219f1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93f60b0 0x55a5a9402220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60254/0 (socket says 172.21.15.57:60254) 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe219f1700 1 -- 172.21.15.57:0/2471531016 learned_addr learned my addr 172.21.15.57:0/2471531016 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe219f1700 1 -- 172.21.15.57:0/2471531016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93fb880 msgr2=0x55a5a93ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe219f1700 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93fb880 0x55a5a93ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe219f1700 1 -- 172.21.15.57:0/2471531016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fc1a0 msgr2=0x55a5a93ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe219f1700 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fc1a0 0x55a5a93ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.509+0000 7efe219f1700 1 -- 172.21.15.57:0/2471531016 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe140099a0 con 0x55a5a93f60b0 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.510+0000 7efe219f1700 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93f60b0 0x55a5a9402220 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7efe1801f000 tx=0x7efe1809ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33c4a95899bcaf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.510+0000 7efe057fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.510+0000 7efe057fa700 1 -- 172.21.15.57:0/2471531016 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1809b630 con 0x55a5a93f60b0 2023-10-08T14:04:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.510+0000 7efe057fa700 1 -- 172.21.15.57:0/2471531016 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7efe1809f320 con 0x55a5a93f60b0 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.510+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/2471531016 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5a9310e50 con 0x55a5a93f60b0 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.510+0000 7efe057fa700 1 -- 172.21.15.57:0/2471531016 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe180a34d0 con 0x55a5a93f60b0 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.510+0000 7efe2fe3eb80 10 client.17319 fetch_fsmap learned FSMap version 48 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.510+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/2471531016 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a5a915cc00 con 0x55a5a93f60b0 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.510+0000 7efe057fa700 1 -- 172.21.15.57:0/2471531016 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a5a9310e50 con 0x55a5a93f60b0 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.510+0000 7efe057fa700 1 -- 172.21.15.57:0/2471531016 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7efe180a10e0 con 0x55a5a93f60b0 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.511+0000 7efe2fe3eb80 10 client.17319 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.511+0000 7efe2fe3eb80 10 client.17319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.511+0000 7efe2fe3eb80 20 client.17319 populate_metadata read hostname 'smithi057' 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.511+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a5a940fa00 0x55a5a9411e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.511+0000 7efe2fe3eb80 4 client.17319 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:04:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.511+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/2471531016 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5a9341db0 con 0x55a5a940fa00 2023-10-08T14:04:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.511+0000 7efe211f0700 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a5a940fa00 0x55a5a9411e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.526+0000 7efe211f0700 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a5a940fa00 0x55a5a9411e20 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7c584fb52283ef5 server_cookie=84d3339cf878962c in_seq=0 out_seq=0 2023-10-08T14:04:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.526+0000 7efe057fa700 10 client.17319 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.536+0000 7efe057fa700 1 -- 172.21.15.57:0/2471531016 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a5a9341db0 con 0x55a5a940fa00 2023-10-08T14:04:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.536+0000 7efe057fa700 10 client.17319 handle_command_reply: tid=0 2023-10-08T14:04:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.536+0000 7efe2fe3eb80 1 client.17319 shutdown 2023-10-08T14:04:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.538+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/2471531016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93f60b0 msgr2=0x55a5a9402220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.538+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93f60b0 0x55a5a9402220 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7efe1801f000 tx=0x7efe1809ba00 comp rx=0 tx=0).stop 2023-10-08T14:04:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.541+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/2471531016 shutdown_connections 2023-10-08T14:04:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.541+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a5a940fa00 0x55a5a9411e20 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.541+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fc1a0 0x55a5a93ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.541+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93f60b0 0x55a5a9402220 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.541+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/2471531016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93fb880 0x55a5a93ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.541+0000 7efe057fa700 0 client.17319 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.541+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/2471531016 >> 172.21.15.57:0/2471531016 conn(0x55a5a936a200 msgr2=0x55a5a93f0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.542+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/2471531016 shutdown_connections 2023-10-08T14:04:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.543+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/2471531016 wait complete. 2023-10-08T14:04:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.552+0000 7efe2fe3eb80 20 client.17319 trim_cache size 0 max 16384 2023-10-08T14:04:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.579+0000 7efe2fe3eb80 1 Processor -- start 2023-10-08T14:04:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.579+0000 7efe2fe3eb80 1 -- start start 2023-10-08T14:04:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.579+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93f60b0 0x55a5a93fedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.579+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 0x55a5a9401d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.579+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93fc1a0 0x55a5a93f0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.579+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a91c98c0 con 0x55a5a93fb880 2023-10-08T14:04:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.579+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9341db0 con 0x55a5a93f60b0 2023-10-08T14:04:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.579+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a9310e50 con 0x55a5a93fc1a0 2023-10-08T14:04:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.580+0000 7efe219f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93f60b0 0x55a5a93fedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.580+0000 7efe219f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93f60b0 0x55a5a93fedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33452/0 (socket says 172.21.15.57:33452) 2023-10-08T14:04:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.580+0000 7efe219f1700 1 -- 172.21.15.57:0/3371576531 learned_addr learned my addr 172.21.15.57:0/3371576531 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.580+0000 7efe211f0700 1 --2- 172.21.15.57:0/3371576531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 0x55a5a9401d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.580+0000 7efe211f0700 1 -- 172.21.15.57:0/3371576531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93fc1a0 msgr2=0x55a5a93f0790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:04:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.580+0000 7efe211f0700 1 --2- 172.21.15.57:0/3371576531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93fc1a0 0x55a5a93f0790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.580+0000 7efe211f0700 1 -- 172.21.15.57:0/3371576531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93f60b0 msgr2=0x55a5a93fedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.580+0000 7efe211f0700 1 --2- 172.21.15.57:0/3371576531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93f60b0 0x55a5a93fedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.580+0000 7efe211f0700 1 -- 172.21.15.57:0/3371576531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5a919a1f0 con 0x55a5a93fb880 2023-10-08T14:04:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.581+0000 7efe211f0700 1 --2- 172.21.15.57:0/3371576531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 0x55a5a9401d50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x55a5a93f26d0 tx=0x7efe1400bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fcfce44690e8d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.581+0000 7efe10d82700 1 -- 172.21.15.57:0/3371576531 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14022ad0 con 0x55a5a93fb880 2023-10-08T14:04:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.581+0000 7efe10d82700 1 -- 172.21.15.57:0/3371576531 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe14022c30 con 0x55a5a93fb880 2023-10-08T14:04:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.581+0000 7efe10d82700 1 -- 172.21.15.57:0/3371576531 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14013410 con 0x55a5a93fb880 2023-10-08T14:04:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.582+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3371576531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 msgr2=0x55a5a9401d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.582+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/3371576531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 0x55a5a9401d50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x55a5a93f26d0 tx=0x7efe1400bb00 comp rx=0 tx=0).stop 2023-10-08T14:04:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.584+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3371576531 shutdown_connections 2023-10-08T14:04:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.584+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/3371576531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93fc1a0 0x55a5a93f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.584+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/3371576531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93f60b0 0x55a5a93fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.584+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/3371576531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 0x55a5a9401d50 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.584+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3371576531 >> 172.21.15.57:0/3371576531 conn(0x55a5a936a200 msgr2=0x55a5a9416780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.593+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3371576531 shutdown_connections 2023-10-08T14:04:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.593+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3371576531 wait complete. 2023-10-08T14:04:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.594+0000 7efe2fe3eb80 1 -- start start 2023-10-08T14:04:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.594+0000 7efe2fe3eb80 1 Processor -- start 2023-10-08T14:04:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.594+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 0x55a5a9404e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.594+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 0x55a5a9405390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.594+0000 7efe2fe3eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fc1a0 0x55a5a945c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a945cdb0 con 0x55a5a93fb880 2023-10-08T14:04:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a945cf20 con 0x55a5a93fc1a0 2023-10-08T14:04:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe2fe3eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a945d0e0 con 0x55a5a93f60b0 2023-10-08T14:04:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe2fe3eb80 10 client.0 _ll_register_callbacks cb 0x7efe1809ad10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:04:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe219f1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 0x55a5a9404e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe219f1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 0x55a5a9404e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:32844/0 (socket says 172.21.15.57:32844) 2023-10-08T14:04:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe219f1700 1 -- 172.21.15.57:0/3746610498 learned_addr learned my addr 172.21.15.57:0/3746610498 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe211f0700 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 0x55a5a9405390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe219f1700 1 -- 172.21.15.57:0/3746610498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fc1a0 msgr2=0x55a5a945c870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:04:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe219f1700 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fc1a0 0x55a5a945c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe219f1700 1 -- 172.21.15.57:0/3746610498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 msgr2=0x55a5a9405390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe219f1700 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 0x55a5a9405390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.595+0000 7efe219f1700 1 -- 172.21.15.57:0/3746610498 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5a9173d80 con 0x55a5a93f60b0 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.596+0000 7efe219f1700 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 0x55a5a9404e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efe180098e0 tx=0x7efe1809abe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8759db449b2fc5ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.596+0000 7efe057fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.596+0000 7efe057fa700 1 -- 172.21.15.57:0/3746610498 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe180ce360 con 0x55a5a93f60b0 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.596+0000 7efe057fa700 1 -- 172.21.15.57:0/3746610498 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7efe1809b430 con 0x55a5a93f60b0 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.596+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3746610498 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe14013a30 con 0x55a5a93f60b0 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.596+0000 7efe057fa700 1 -- 172.21.15.57:0/3746610498 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe1809b5f0 con 0x55a5a93f60b0 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.596+0000 7efe2fe3eb80 10 client.26875 fetch_fsmap learned FSMap version 48 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.596+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3746610498 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a5a9172250 con 0x55a5a93f60b0 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.596+0000 7efe057fa700 1 -- 172.21.15.57:0/3746610498 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe14013a30 con 0x55a5a93f60b0 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.596+0000 7efe057fa700 1 -- 172.21.15.57:0/3746610498 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7efe180052d0 con 0x55a5a93f60b0 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.597+0000 7efe2fe3eb80 10 client.26875 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.597+0000 7efe2fe3eb80 10 client.26875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.597+0000 7efe2fe3eb80 20 client.26875 populate_metadata read hostname 'smithi057' 2023-10-08T14:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.597+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a5a9464470 0x55a5a9466860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.597+0000 7efe2fe3eb80 4 client.26875 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.597+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3746610498 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a5a9341ad0 con 0x55a5a9464470 2023-10-08T14:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.597+0000 7efe211f0700 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a5a9464470 0x55a5a9466860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.598+0000 7efe211f0700 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a5a9464470 0x55a5a9466860 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d10f13eb290084 server_cookie=b9cf1910afb3bdb3 in_seq=0 out_seq=0 2023-10-08T14:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.598+0000 7efe057fa700 10 client.26875 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.638+0000 7efe057fa700 1 -- 172.21.15.57:0/3746610498 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a5a9341ad0 con 0x55a5a9464470 2023-10-08T14:04:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.638+0000 7efe057fa700 10 client.26875 handle_command_reply: tid=0 2023-10-08T14:04:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.641+0000 7efe2fe3eb80 1 client.26875 shutdown 2023-10-08T14:04:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.646+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3746610498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 msgr2=0x55a5a9404e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.646+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 0x55a5a9404e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efe180098e0 tx=0x7efe1809abe0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.646+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3746610498 shutdown_connections 2023-10-08T14:04:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.646+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55a5a9464470 0x55a5a9466860 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.646+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5a93fc1a0 0x55a5a945c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.646+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5a93f60b0 0x55a5a9404e50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.646+0000 7efe2fe3eb80 1 --2- 172.21.15.57:0/3746610498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a5a93fb880 0x55a5a9405390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.646+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3746610498 >> 172.21.15.57:0/3746610498 conn(0x55a5a936a200 msgr2=0x55a5a93fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.650+0000 7efe057fa700 0 client.26875 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.663+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3746610498 shutdown_connections 2023-10-08T14:04:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.663+0000 7efe2fe3eb80 1 -- 172.21.15.57:0/3746610498 wait complete. 2023-10-08T14:04:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.663+0000 7efe2fe3eb80 20 client.26875 trim_cache size 0 max 16384 2023-10-08T14:04:46.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:04:46.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:04:46.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:04:46.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:04:46.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.670+0000 7efe23c55700 1 -- 172.21.15.57:0/3981404533 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efe080a1800 msgr2=0x7efe080a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.670+0000 7efe23c55700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efe080a1800 0x7efe080a3cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efe0c006fd0 tx=0x7efe0c008040 comp rx=0 tx=0).stop 2023-10-08T14:04:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.670+0000 7efe23c55700 1 -- 172.21.15.57:0/3981404533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe1c079690 msgr2=0x7efe1c071ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.670+0000 7efe23c55700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe1c079690 0x7efe1c071ed0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe18003b90 tx=0x7efe18003c70 comp rx=0 tx=0).stop 2023-10-08T14:04:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.674+0000 7efe23c55700 1 -- 172.21.15.57:0/3981404533 shutdown_connections 2023-10-08T14:04:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.674+0000 7efe23c55700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe1c119c10 0x7efe1c06c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.674+0000 7efe23c55700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe1c079690 0x7efe1c071ed0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.674+0000 7efe23c55700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7efe080a1800 0x7efe080a3cc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.674+0000 7efe23c55700 1 --2- 172.21.15.57:0/3981404533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c081370 0x7efe1c072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.674+0000 7efe23c55700 1 -- 172.21.15.57:0/3981404533 >> 172.21.15.57:0/3981404533 conn(0x7efe1c0764f0 msgr2=0x7efe1c080480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.680+0000 7efe23c55700 1 -- 172.21.15.57:0/3981404533 shutdown_connections 2023-10-08T14:04:46.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:46.685+0000 7efe23c55700 1 -- 172.21.15.57:0/3981404533 wait complete. 2023-10-08T14:04:48.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:47 smithi057 ceph-mon[99476]: pgmap v710: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 620 KiB/s wr, 112 op/s 2023-10-08T14:04:48.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:47 smithi099 ceph-mon[103849]: pgmap v710: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 620 KiB/s wr, 112 op/s 2023-10-08T14:04:48.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:47 smithi115 ceph-mon[104408]: pgmap v710: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 620 KiB/s wr, 112 op/s 2023-10-08T14:04:49.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:49 smithi057 ceph-mon[99476]: pgmap v711: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-10-08T14:04:49.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:49 smithi099 ceph-mon[103849]: pgmap v711: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-10-08T14:04:49.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:49 smithi115 ceph-mon[104408]: pgmap v711: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-10-08T14:04:52.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:51 smithi115 ceph-mon[104408]: pgmap v712: 129 pgs: 54 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 55 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 963 KiB/s wr, 125 op/s 2023-10-08T14:04:52.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:51 smithi057 ceph-mon[99476]: pgmap v712: 129 pgs: 54 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 55 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 963 KiB/s wr, 125 op/s 2023-10-08T14:04:52.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:51 smithi099 ceph-mon[103849]: pgmap v712: 129 pgs: 54 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 55 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 963 KiB/s wr, 125 op/s 2023-10-08T14:04:54.189 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:53 smithi115 ceph-mon[104408]: pgmap v713: 129 pgs: 54 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 55 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 798 KiB/s wr, 105 op/s 2023-10-08T14:04:54.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:53 smithi057 ceph-mon[99476]: pgmap v713: 129 pgs: 54 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 55 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 798 KiB/s wr, 105 op/s 2023-10-08T14:04:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:53 smithi099 ceph-mon[103849]: pgmap v713: 129 pgs: 54 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 55 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 798 KiB/s wr, 105 op/s 2023-10-08T14:04:56.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:55 smithi057 ceph-mon[99476]: pgmap v714: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 835 KiB/s wr, 126 op/s 2023-10-08T14:04:56.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:55 smithi099 ceph-mon[103849]: pgmap v714: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 835 KiB/s wr, 126 op/s 2023-10-08T14:04:56.441 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:55 smithi115 ceph-mon[104408]: pgmap v714: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 835 KiB/s wr, 126 op/s 2023-10-08T14:04:58.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:57 smithi057 ceph-mon[99476]: pgmap v715: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 801 KiB/s wr, 96 op/s 2023-10-08T14:04:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:57 smithi099 ceph-mon[103849]: pgmap v715: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 801 KiB/s wr, 96 op/s 2023-10-08T14:04:58.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:57 smithi115 ceph-mon[104408]: pgmap v715: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 801 KiB/s wr, 96 op/s 2023-10-08T14:04:59.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:04:59 smithi099 ceph-mon[103849]: pgmap v716: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 806 KiB/s wr, 105 op/s 2023-10-08T14:04:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.899+0000 7effd8a7b700 1 -- 172.21.15.57:0/748223716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4104180 msgr2=0x7effd410bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.899+0000 7effd8a7b700 1 --2- 172.21.15.57:0/748223716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4104180 0x7effd410bd00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7effcc01c0d0 tx=0x7effcc01c3e0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.901+0000 7effd8a7b700 1 -- 172.21.15.57:0/748223716 shutdown_connections 2023-10-08T14:04:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.901+0000 7effd8a7b700 1 --2- 172.21.15.57:0/748223716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4104180 0x7effd410bd00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.901+0000 7effd8a7b700 1 --2- 172.21.15.57:0/748223716 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effd40ff060 0x7effd40ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.901+0000 7effd8a7b700 1 --2- 172.21.15.57:0/748223716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd40ffa10 0x7effd4103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.901+0000 7effd8a7b700 1 -- 172.21.15.57:0/748223716 >> 172.21.15.57:0/748223716 conn(0x7effd4075c40 msgr2=0x7effd40fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.911+0000 7effd8a7b700 1 -- 172.21.15.57:0/748223716 shutdown_connections 2023-10-08T14:04:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.915+0000 7effd8a7b700 1 -- 172.21.15.57:0/748223716 wait complete. 2023-10-08T14:04:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd8a7b700 1 Processor -- start 2023-10-08T14:04:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd8a7b700 1 -- start start 2023-10-08T14:04:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd8a7b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4104180 0x7effd4107db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd8a7b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd41082f0 0x7effd4108770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd8a7b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effd41bc920 0x7effd41bed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd8a7b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd41bf250 con 0x7effd41bc920 2023-10-08T14:04:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd8a7b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd41bf3c0 con 0x7effd41082f0 2023-10-08T14:04:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd8a7b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd41bf580 con 0x7effd4104180 2023-10-08T14:04:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effd41bc920 0x7effd41bed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effd41bc920 0x7effd41bed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60280/0 (socket says 172.21.15.57:60280) 2023-10-08T14:04:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.916+0000 7effd2d9d700 1 -- 172.21.15.57:0/1006124234 learned_addr learned my addr 172.21.15.57:0/1006124234 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effd2d9d700 1 -- 172.21.15.57:0/1006124234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4104180 msgr2=0x7effd4107db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effd2d9d700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4104180 0x7effd4107db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effd2d9d700 1 -- 172.21.15.57:0/1006124234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd41082f0 msgr2=0x7effd4108770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effd2d9d700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd41082f0 0x7effd4108770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effd2d9d700 1 -- 172.21.15.57:0/1006124234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc01ad10 con 0x7effd41bc920 2023-10-08T14:04:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effd2d9d700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effd41bc920 0x7effd41bed10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7effcc004c40 tx=0x7effcc004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effc37fe700 1 -- 172.21.15.57:0/1006124234 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc01f040 con 0x7effd41bc920 2023-10-08T14:04:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effd8a7b700 1 -- 172.21.15.57:0/1006124234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd41bf800 con 0x7effd41bc920 2023-10-08T14:04:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effc37fe700 1 -- 172.21.15.57:0/1006124234 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effcc00b070 con 0x7effd41bc920 2023-10-08T14:04:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effc37fe700 1 -- 172.21.15.57:0/1006124234 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc007e50 con 0x7effd41bc920 2023-10-08T14:04:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.917+0000 7effd8a7b700 1 -- 172.21.15.57:0/1006124234 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd41bfcd0 con 0x7effd41bc920 2023-10-08T14:04:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.922+0000 7effe4c64b80 1 Processor -- start 2023-10-08T14:04:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.923+0000 7effc37fe700 1 -- 172.21.15.57:0/1006124234 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7effcc00d030 con 0x7effd41bc920 2023-10-08T14:04:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.924+0000 7effc37fe700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7effbc0a1670 0x7effbc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.924+0000 7effc37fe700 1 -- 172.21.15.57:0/1006124234 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7effcc0ce370 con 0x7effd41bc920 2023-10-08T14:04:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effe4c64b80 1 -- start start 2023-10-08T14:04:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac2d090 0x55e2fac2d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d9b0 0x55e2fac27380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 0x55e2fac27d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effe4c64b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fab87400 con 0x55e2fac278c0 2023-10-08T14:04:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effe4c64b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fab78bc0 con 0x55e2fac2d090 2023-10-08T14:04:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effe4c64b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fab764e0 con 0x55e2fac2d9b0 2023-10-08T14:04:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effd1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d9b0 0x55e2fac27380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effd1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d9b0 0x55e2fac27380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:32866/0 (socket says 172.21.15.57:32866) 2023-10-08T14:04:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effd1d9b700 1 -- 172.21.15.57:0/3702655372 learned_addr learned my addr 172.21.15.57:0/3702655372 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.925+0000 7effd1d9b700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7effbc0a1670 0x7effbc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.926+0000 7effd1d9b700 1 -- 172.21.15.57:0/3702655372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac2d090 msgr2=0x55e2fac2d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.926+0000 7effd1d9b700 1 --2- 172.21.15.57:0/3702655372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac2d090 0x55e2fac2d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.926+0000 7effd1d9b700 1 -- 172.21.15.57:0/3702655372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 msgr2=0x55e2fac27d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.926+0000 7effd1d9b700 1 --2- 172.21.15.57:0/3702655372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 0x55e2fac27d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.926+0000 7effd1d9b700 1 -- 172.21.15.57:0/3702655372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2fa99cd80 con 0x55e2fac2d9b0 2023-10-08T14:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.926+0000 7effd1d9b700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7effbc0a1670 0x7effbc0a3b30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7effc8005950 tx=0x7effc8008000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.926+0000 7effd1d9b700 1 --2- 172.21.15.57:0/3702655372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d9b0 0x55e2fac27380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7effc8005fd0 tx=0x7effc8012e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=179c1fd2b76a85dd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.927+0000 7effc0ff9700 1 -- 172.21.15.57:0/3702655372 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc8022070 con 0x55e2fac2d9b0 2023-10-08T14:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.927+0000 7effc0ff9700 1 -- 172.21.15.57:0/3702655372 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effc8018040 con 0x55e2fac2d9b0 2023-10-08T14:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.927+0000 7effc0ff9700 1 -- 172.21.15.57:0/3702655372 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc8005280 con 0x55e2fac2d9b0 2023-10-08T14:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.928+0000 7effe4c64b80 1 -- 172.21.15.57:0/3702655372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d9b0 msgr2=0x55e2fac27380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.928+0000 7effe4c64b80 1 --2- 172.21.15.57:0/3702655372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d9b0 0x55e2fac27380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7effc8005fd0 tx=0x7effc8012e10 comp rx=0 tx=0).stop 2023-10-08T14:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.928+0000 7effe4c64b80 1 -- 172.21.15.57:0/3702655372 shutdown_connections 2023-10-08T14:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.928+0000 7effe4c64b80 1 --2- 172.21.15.57:0/3702655372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 0x55e2fac27d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.928+0000 7effe4c64b80 1 --2- 172.21.15.57:0/3702655372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac2d090 0x55e2fac2d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.928+0000 7effe4c64b80 1 --2- 172.21.15.57:0/3702655372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d9b0 0x55e2fac27380 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7effc8005fd0 tx=0x7effc8012e10 comp rx=0 tx=0).stop 2023-10-08T14:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.928+0000 7effe4c64b80 1 -- 172.21.15.57:0/3702655372 >> 172.21.15.57:0/3702655372 conn(0x55e2fab93200 msgr2=0x55e2fac19b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:59.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:04:59 smithi115 ceph-mon[104408]: pgmap v716: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 806 KiB/s wr, 105 op/s 2023-10-08T14:04:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.936+0000 7effe4c64b80 1 -- 172.21.15.57:0/3702655372 shutdown_connections 2023-10-08T14:04:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.938+0000 7effe4c64b80 1 -- 172.21.15.57:0/3702655372 wait complete. 2023-10-08T14:04:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.938+0000 7effe4c64b80 1 -- start start 2023-10-08T14:04:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.938+0000 7effe4c64b80 1 Processor -- start 2023-10-08T14:04:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.939+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 0x55e2fac310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.939+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac2d090 0x55e2fac31620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.939+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 0x55e2fac3d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.939+0000 7effe4c64b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fab71dd0 con 0x55e2fac2d090 2023-10-08T14:04:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.939+0000 7effe4c64b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fab0e6b0 con 0x55e2fac31b60 2023-10-08T14:04:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.939+0000 7effe4c64b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fab0e350 con 0x55e2fac278c0 2023-10-08T14:04:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.939+0000 7effe4c64b80 10 client.0 _ll_register_callbacks cb 0x55e2fa96b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:04:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.939+0000 7effd2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 0x55e2fac3d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.939+0000 7effd2d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 0x55e2fac3d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33486/0 (socket says 172.21.15.57:33486) 2023-10-08T14:04:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.939+0000 7effd2d9d700 1 -- 172.21.15.57:0/3518134814 learned_addr learned my addr 172.21.15.57:0/3518134814 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:04:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effd259c700 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 0x55e2fac310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effd2d9d700 1 -- 172.21.15.57:0/3518134814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 msgr2=0x55e2fac310e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effd2d9d700 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 0x55e2fac310e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effd2d9d700 1 -- 172.21.15.57:0/3518134814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac2d090 msgr2=0x55e2fac31620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effd2d9d700 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac2d090 0x55e2fac31620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effd2d9d700 1 -- 172.21.15.57:0/3518134814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc8012710 con 0x55e2fac31b60 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effd2d9d700 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 0x55e2fac3d450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7effcc0a2f30 tx=0x7effcc01a4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf164969c405db8d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effb9ffb700 1 -- 172.21.15.57:0/3518134814 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc0cbaf0 con 0x55e2fac31b60 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effb9ffb700 1 -- 172.21.15.57:0/3518134814 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7effcc09f2b0 con 0x55e2fac31b60 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.940+0000 7effe4c64b80 1 -- 172.21.15.57:0/3518134814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2fab39e50 con 0x55e2fac31b60 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effb9ffb700 1 -- 172.21.15.57:0/3518134814 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7effcc0b92b0 con 0x55e2fac31b60 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effe4c64b80 10 client.26840 fetch_fsmap learned FSMap version 48 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effe4c64b80 1 -- 172.21.15.57:0/3518134814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55e2fa985c00 con 0x55e2fac31b60 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effb9ffb700 1 -- 172.21.15.57:0/3518134814 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effcc058080 con 0x55e2fac31b60 2023-10-08T14:04:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effb9ffb700 1 -- 172.21.15.57:0/3518134814 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7effcc09b4e0 con 0x55e2fac31b60 2023-10-08T14:04:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effe4c64b80 10 client.26840 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:04:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effe4c64b80 10 client.26840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:04:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effe4c64b80 20 client.26840 populate_metadata read hostname 'smithi057' 2023-10-08T14:04:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effe4c64b80 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e2fac41980 0x55e2fac43da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:04:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effe4c64b80 4 client.26840 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:04:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effe4c64b80 1 -- 172.21.15.57:0/3518134814 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2fab6adb0 con 0x55e2fac41980 2023-10-08T14:04:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.941+0000 7effd259c700 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e2fac41980 0x55e2fac43da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:04:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.942+0000 7effd259c700 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e2fac41980 0x55e2fac43da0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e416229f436e39 server_cookie=2a1bf0a6f839b3c4 in_seq=0 out_seq=0 2023-10-08T14:04:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.942+0000 7effb9ffb700 10 client.26840 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effb9ffb700 1 -- 172.21.15.57:0/3518134814 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e2fab6adb0 con 0x55e2fac41980 2023-10-08T14:04:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effb9ffb700 10 client.26840 handle_command_reply: tid=0 2023-10-08T14:04:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effe4c64b80 1 client.26840 shutdown 2023-10-08T14:04:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effe4c64b80 1 -- 172.21.15.57:0/3518134814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 msgr2=0x55e2fac3d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:04:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effe4c64b80 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 0x55e2fac3d450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7effcc0a2f30 tx=0x7effcc01a4c0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effe4c64b80 1 -- 172.21.15.57:0/3518134814 shutdown_connections 2023-10-08T14:04:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effe4c64b80 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e2fac41980 0x55e2fac43da0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effe4c64b80 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 0x55e2fac3d450 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effe4c64b80 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 0x55e2fac310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effe4c64b80 1 --2- 172.21.15.57:0/3518134814 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac2d090 0x55e2fac31620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:04:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effe4c64b80 1 -- 172.21.15.57:0/3518134814 >> 172.21.15.57:0/3518134814 conn(0x55e2fab93200 msgr2=0x55e2fac19010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:04:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.943+0000 7effb9ffb700 0 client.26840 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:04:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.953+0000 7effe4c64b80 1 -- 172.21.15.57:0/3518134814 shutdown_connections 2023-10-08T14:04:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.960+0000 7effe4c64b80 1 -- 172.21.15.57:0/3518134814 wait complete. 2023-10-08T14:04:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.974+0000 7effe4c64b80 20 client.26840 trim_cache size 0 max 16384 2023-10-08T14:04:59.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:04:59 smithi057 ceph-mon[99476]: pgmap v716: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 806 KiB/s wr, 105 op/s 2023-10-08T14:05:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:04:59.999+0000 7effe4c64b80 1 Processor -- start 2023-10-08T14:05:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effe4c64b80 1 -- start start 2023-10-08T14:05:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 0x55e2fac27380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d090 0x55e2fac3d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 0x55e2fac19790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effe4c64b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fa9f28c0 con 0x55e2fac278c0 2023-10-08T14:05:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effe4c64b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fac1a610 con 0x55e2fac31b60 2023-10-08T14:05:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effe4c64b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fab6adb0 con 0x55e2fac2d090 2023-10-08T14:05:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effd259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 0x55e2fac27380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effd259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 0x55e2fac27380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60300/0 (socket says 172.21.15.57:60300) 2023-10-08T14:05:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effd259c700 1 -- 172.21.15.57:0/1268667688 learned_addr learned my addr 172.21.15.57:0/1268667688 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.006+0000 7effd1d9b700 1 --2- 172.21.15.57:0/1268667688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d090 0x55e2fac3d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.007+0000 7effd259c700 1 -- 172.21.15.57:0/1268667688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d090 msgr2=0x55e2fac3d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.007+0000 7effd259c700 1 --2- 172.21.15.57:0/1268667688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d090 0x55e2fac3d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.007+0000 7effd259c700 1 -- 172.21.15.57:0/1268667688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 msgr2=0x55e2fac19790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:05:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.007+0000 7effd259c700 1 --2- 172.21.15.57:0/1268667688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 0x55e2fac19790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.007+0000 7effd259c700 1 -- 172.21.15.57:0/1268667688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2fac48990 con 0x55e2fac278c0 2023-10-08T14:05:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.007+0000 7effd1d9b700 1 --2- 172.21.15.57:0/1268667688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d090 0x55e2fac3d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T14:05:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.007+0000 7effd259c700 1 --2- 172.21.15.57:0/1268667688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 0x55e2fac27380 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7effc4018c20 tx=0x7effc4018c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b25fb45888ff28f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.016+0000 7effc17fa700 1 -- 172.21.15.57:0/1268667688 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4009ce0 con 0x55e2fac278c0 2023-10-08T14:05:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.016+0000 7effc17fa700 1 -- 172.21.15.57:0/1268667688 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effc40185a0 con 0x55e2fac278c0 2023-10-08T14:05:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.016+0000 7effc17fa700 1 -- 172.21.15.57:0/1268667688 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4016c30 con 0x55e2fac278c0 2023-10-08T14:05:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.017+0000 7effe4c64b80 1 -- 172.21.15.57:0/1268667688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 msgr2=0x55e2fac27380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.017+0000 7effe4c64b80 1 --2- 172.21.15.57:0/1268667688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 0x55e2fac27380 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7effc4018c20 tx=0x7effc4018c50 comp rx=0 tx=0).stop 2023-10-08T14:05:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.019+0000 7effe4c64b80 1 -- 172.21.15.57:0/1268667688 shutdown_connections 2023-10-08T14:05:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.019+0000 7effe4c64b80 1 --2- 172.21.15.57:0/1268667688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac31b60 0x55e2fac19790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.019+0000 7effe4c64b80 1 --2- 172.21.15.57:0/1268667688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac278c0 0x55e2fac27380 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.019+0000 7effe4c64b80 1 --2- 172.21.15.57:0/1268667688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac2d090 0x55e2fac3d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.019+0000 7effe4c64b80 1 -- 172.21.15.57:0/1268667688 >> 172.21.15.57:0/1268667688 conn(0x55e2fab93200 msgr2=0x55e2fac3caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.022+0000 7effe4c64b80 1 -- 172.21.15.57:0/1268667688 shutdown_connections 2023-10-08T14:05:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.022+0000 7effe4c64b80 1 -- 172.21.15.57:0/1268667688 wait complete. 2023-10-08T14:05:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.023+0000 7effe4c64b80 1 -- start start 2023-10-08T14:05:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.023+0000 7effe4c64b80 1 Processor -- start 2023-10-08T14:05:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 0x55e2fac90ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac2d090 0x55e2fac91020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effe4c64b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac31b60 0x55e2fac365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effe4c64b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fac36cb0 con 0x55e2fac31b60 2023-10-08T14:05:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effe4c64b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fac36e20 con 0x55e2fac2d090 2023-10-08T14:05:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effe4c64b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2fac36fe0 con 0x55e2fac278c0 2023-10-08T14:05:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effe4c64b80 10 client.0 _ll_register_callbacks cb 0x55e2fa96b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:05:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effd2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac31b60 0x55e2fac365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effd259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 0x55e2fac90ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effd2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac31b60 0x55e2fac365b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60308/0 (socket says 172.21.15.57:60308) 2023-10-08T14:05:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.028+0000 7effd2d9d700 1 -- 172.21.15.57:0/1371076503 learned_addr learned my addr 172.21.15.57:0/1371076503 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.029+0000 7effd2d9d700 1 -- 172.21.15.57:0/1371076503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 msgr2=0x55e2fac90ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.029+0000 7effd2d9d700 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 0x55e2fac90ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.029+0000 7effd2d9d700 1 -- 172.21.15.57:0/1371076503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac2d090 msgr2=0x55e2fac91020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.029+0000 7effd2d9d700 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac2d090 0x55e2fac91020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.029+0000 7effd2d9d700 1 -- 172.21.15.57:0/1371076503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc4009550 con 0x55e2fac31b60 2023-10-08T14:05:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.029+0000 7effd259c700 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 0x55e2fac90ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:05:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.029+0000 7effd2d9d700 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac31b60 0x55e2fac365b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7effcc004e40 tx=0x7effcc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de3dbc71d55b304 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.029+0000 7effb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:05:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.029+0000 7effb9ffb700 1 -- 172.21.15.57:0/1371076503 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc09b5c0 con 0x55e2fac31b60 2023-10-08T14:05:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.029+0000 7effb9ffb700 1 -- 172.21.15.57:0/1371076503 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7effcc0b92b0 con 0x55e2fac31b60 2023-10-08T14:05:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.030+0000 7effe4c64b80 1 -- 172.21.15.57:0/1371076503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effc8018040 con 0x55e2fac31b60 2023-10-08T14:05:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effb9ffb700 1 -- 172.21.15.57:0/1371076503 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7effcc0974a0 con 0x55e2fac31b60 2023-10-08T14:05:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effe4c64b80 10 client.17334 fetch_fsmap learned FSMap version 48 2023-10-08T14:05:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effe4c64b80 1 -- 172.21.15.57:0/1371076503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55e2fa9c31f0 con 0x55e2fac31b60 2023-10-08T14:05:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effb9ffb700 1 -- 172.21.15.57:0/1371076503 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effc8018040 con 0x55e2fac31b60 2023-10-08T14:05:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effb9ffb700 1 -- 172.21.15.57:0/1371076503 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7effcc0a2fd0 con 0x55e2fac31b60 2023-10-08T14:05:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effe4c64b80 10 client.17334 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:05:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effe4c64b80 10 client.17334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:05:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effe4c64b80 20 client.17334 populate_metadata read hostname 'smithi057' 2023-10-08T14:05:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effe4c64b80 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e2fac37ab0 0x55e2fac23b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effe4c64b80 4 client.17334 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:05:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.038+0000 7effe4c64b80 1 -- 172.21.15.57:0/1371076503 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e2fab6aad0 con 0x55e2fac37ab0 2023-10-08T14:05:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.039+0000 7effd259c700 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e2fac37ab0 0x55e2fac23b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.039+0000 7effb9ffb700 10 client.17334 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.039+0000 7effd259c700 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e2fac37ab0 0x55e2fac23b90 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b440d1e6c628a7dc server_cookie=df493fae06c62978 in_seq=0 out_seq=0 2023-10-08T14:05:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.126+0000 7effb9ffb700 1 -- 172.21.15.57:0/1371076503 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e2fab6aad0 con 0x55e2fac37ab0 2023-10-08T14:05:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.126+0000 7effb9ffb700 10 client.17334 handle_command_reply: tid=0 2023-10-08T14:05:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.126+0000 7effe4c64b80 1 client.17334 shutdown 2023-10-08T14:05:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.127+0000 7effe4c64b80 1 -- 172.21.15.57:0/1371076503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac31b60 msgr2=0x55e2fac365b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.127+0000 7effe4c64b80 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac31b60 0x55e2fac365b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7effcc004e40 tx=0x7effcc01c3e0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.127+0000 7effe4c64b80 1 -- 172.21.15.57:0/1371076503 shutdown_connections 2023-10-08T14:05:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.127+0000 7effe4c64b80 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55e2fac37ab0 0x55e2fac23b90 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.127+0000 7effe4c64b80 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e2fac31b60 0x55e2fac365b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.127+0000 7effe4c64b80 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e2fac278c0 0x55e2fac90ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.127+0000 7effe4c64b80 1 --2- 172.21.15.57:0/1371076503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2fac2d090 0x55e2fac91020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.127+0000 7effe4c64b80 1 -- 172.21.15.57:0/1371076503 >> 172.21.15.57:0/1371076503 conn(0x55e2fab93200 msgr2=0x55e2fac2f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.127+0000 7effb9ffb700 0 client.17334 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.135+0000 7effe4c64b80 1 -- 172.21.15.57:0/1371076503 shutdown_connections 2023-10-08T14:05:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.137+0000 7effe4c64b80 1 -- 172.21.15.57:0/1371076503 wait complete. 2023-10-08T14:05:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:05:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.143+0000 7effe4c64b80 20 client.17334 trim_cache size 0 max 16384 2023-10-08T14:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.143+0000 7effd8a7b700 1 -- 172.21.15.57:0/1006124234 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7effbc0a1670 msgr2=0x7effbc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.143+0000 7effd8a7b700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7effbc0a1670 0x7effbc0a3b30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7effc8005950 tx=0x7effc8008000 comp rx=0 tx=0).stop 2023-10-08T14:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.143+0000 7effd8a7b700 1 -- 172.21.15.57:0/1006124234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effd41bc920 msgr2=0x7effd41bed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.143+0000 7effd8a7b700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effd41bc920 0x7effd41bed10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7effcc004c40 tx=0x7effcc004d20 comp rx=0 tx=0).stop 2023-10-08T14:05:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.146+0000 7effd8a7b700 1 -- 172.21.15.57:0/1006124234 shutdown_connections 2023-10-08T14:05:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.146+0000 7effd8a7b700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effd41bc920 0x7effd41bed10 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.146+0000 7effd8a7b700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4104180 0x7effd4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.146+0000 7effd8a7b700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7effbc0a1670 0x7effbc0a3b30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.146+0000 7effd8a7b700 1 --2- 172.21.15.57:0/1006124234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd41082f0 0x7effd4108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.146+0000 7effd8a7b700 1 -- 172.21.15.57:0/1006124234 >> 172.21.15.57:0/1006124234 conn(0x7effd4075c40 msgr2=0x7effd4106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.149+0000 7effd8a7b700 1 -- 172.21.15.57:0/1006124234 shutdown_connections 2023-10-08T14:05:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:00.158+0000 7effd8a7b700 1 -- 172.21.15.57:0/1006124234 wait complete. 2023-10-08T14:05:02.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:01 smithi115 ceph-mon[104408]: pgmap v717: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 850 KiB/s wr, 116 op/s 2023-10-08T14:05:02.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:01 smithi057 ceph-mon[99476]: pgmap v717: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 850 KiB/s wr, 116 op/s 2023-10-08T14:05:02.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:01 smithi099 ceph-mon[103849]: pgmap v717: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 850 KiB/s wr, 116 op/s 2023-10-08T14:05:04.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:03 smithi115 ceph-mon[104408]: pgmap v718: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 860 KiB/s wr, 108 op/s 2023-10-08T14:05:04.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:03 smithi057 ceph-mon[99476]: pgmap v718: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 860 KiB/s wr, 108 op/s 2023-10-08T14:05:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:03 smithi099 ceph-mon[103849]: pgmap v718: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 860 KiB/s wr, 108 op/s 2023-10-08T14:05:06.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:05 smithi057 ceph-mon[99476]: pgmap v719: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 138 op/s 2023-10-08T14:05:06.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:05 smithi099 ceph-mon[103849]: pgmap v719: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 138 op/s 2023-10-08T14:05:06.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:05 smithi115 ceph-mon[104408]: pgmap v719: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 138 op/s 2023-10-08T14:05:08.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:07 smithi057 ceph-mon[99476]: pgmap v720: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-08T14:05:08.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:07 smithi099 ceph-mon[103849]: pgmap v720: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-08T14:05:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:07 smithi115 ceph-mon[104408]: pgmap v720: 129 pgs: 54 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 56 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-10-08T14:05:09.739 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:09 smithi057 ceph-mon[99476]: pgmap v721: 129 pgs: 54 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 57 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 114 op/s 2023-10-08T14:05:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:09 smithi099 ceph-mon[103849]: pgmap v721: 129 pgs: 54 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 57 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 114 op/s 2023-10-08T14:05:09.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:09 smithi115 ceph-mon[104408]: pgmap v721: 129 pgs: 54 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 57 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 114 op/s 2023-10-08T14:05:12.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:11 smithi115 ceph-mon[104408]: pgmap v722: 129 pgs: 50 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 62 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2023-10-08T14:05:12.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:11 smithi057 ceph-mon[99476]: pgmap v722: 129 pgs: 50 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 62 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2023-10-08T14:05:12.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:11 smithi099 ceph-mon[103849]: pgmap v722: 129 pgs: 50 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 62 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2023-10-08T14:05:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.484+0000 7ff397bfd700 1 -- 172.21.15.57:0/2670612522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff390119fd0 msgr2=0x7ff39011c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.484+0000 7ff397bfd700 1 --2- 172.21.15.57:0/2670612522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff390119fd0 0x7ff39011c3c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff38801c100 tx=0x7ff38801c410 comp rx=0 tx=0).stop 2023-10-08T14:05:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.485+0000 7ff397bfd700 1 -- 172.21.15.57:0/2670612522 shutdown_connections 2023-10-08T14:05:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.485+0000 7ff397bfd700 1 --2- 172.21.15.57:0/2670612522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff390119fd0 0x7ff39011c3c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.485+0000 7ff397bfd700 1 --2- 172.21.15.57:0/2670612522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39010ef10 0x7ff3901077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.485+0000 7ff397bfd700 1 --2- 172.21.15.57:0/2670612522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff39010f380 0x7ff390107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.485+0000 7ff397bfd700 1 -- 172.21.15.57:0/2670612522 >> 172.21.15.57:0/2670612522 conn(0x7ff39006b7c0 msgr2=0x7ff39006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.505+0000 7ff397bfd700 1 -- 172.21.15.57:0/2670612522 shutdown_connections 2023-10-08T14:05:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.507+0000 7ff397bfd700 1 -- 172.21.15.57:0/2670612522 wait complete. 2023-10-08T14:05:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.507+0000 7ff397bfd700 1 Processor -- start 2023-10-08T14:05:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.508+0000 7ff397bfd700 1 -- start start 2023-10-08T14:05:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.508+0000 7ff397bfd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39010ef10 0x7ff3901a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.508+0000 7ff397bfd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff39010f380 0x7ff3901a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.508+0000 7ff397bfd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff390119fd0 0x7ff390115400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.508+0000 7ff397bfd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff390115bd0 con 0x7ff39010ef10 2023-10-08T14:05:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.508+0000 7ff397bfd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff390115d40 con 0x7ff390119fd0 2023-10-08T14:05:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.508+0000 7ff397bfd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff390115f00 con 0x7ff39010f380 2023-10-08T14:05:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.508+0000 7ff39619a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff390119fd0 0x7ff390115400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.508+0000 7ff39619a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff390119fd0 0x7ff390115400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33512/0 (socket says 172.21.15.57:33512) 2023-10-08T14:05:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.508+0000 7ff39619a700 1 -- 172.21.15.57:0/3934709140 learned_addr learned my addr 172.21.15.57:0/3934709140 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.509+0000 7ff39619a700 1 -- 172.21.15.57:0/3934709140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff39010f380 msgr2=0x7ff3901a34c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:05:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.509+0000 7ff39619a700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff39010f380 0x7ff3901a34c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.509+0000 7ff39619a700 1 -- 172.21.15.57:0/3934709140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39010ef10 msgr2=0x7ff3901a2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.509+0000 7ff39619a700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39010ef10 0x7ff3901a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.509+0000 7ff39619a700 1 -- 172.21.15.57:0/3934709140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38801ad10 con 0x7ff390119fd0 2023-10-08T14:05:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.509+0000 7ff39619a700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff390119fd0 0x7ff390115400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff38801c7a0 tx=0x7ff3880188b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.509+0000 7ff386ffd700 1 -- 172.21.15.57:0/3934709140 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38801f050 con 0x7ff390119fd0 2023-10-08T14:05:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.509+0000 7ff386ffd700 1 -- 172.21.15.57:0/3934709140 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff38801de50 con 0x7ff390119fd0 2023-10-08T14:05:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.510+0000 7ff386ffd700 1 -- 172.21.15.57:0/3934709140 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38802c6b0 con 0x7ff390119fd0 2023-10-08T14:05:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.510+0000 7ff397bfd700 1 -- 172.21.15.57:0/3934709140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff390116180 con 0x7ff390119fd0 2023-10-08T14:05:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.510+0000 7ff397bfd700 1 -- 172.21.15.57:0/3934709140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3901a9f00 con 0x7ff390119fd0 2023-10-08T14:05:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.515+0000 7ff3a3de6b80 1 Processor -- start 2023-10-08T14:05:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.516+0000 7ff386ffd700 1 -- 172.21.15.57:0/3934709140 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff38802c810 con 0x7ff390119fd0 2023-10-08T14:05:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.517+0000 7ff386ffd700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff37c0a1740 0x7ff37c0a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.518+0000 7ff386ffd700 1 -- 172.21.15.57:0/3934709140 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7ff3880dfa50 con 0x7ff390119fd0 2023-10-08T14:05:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.518+0000 7ff3a3de6b80 1 -- start start 2023-10-08T14:05:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.518+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1880 0x55cd0d9b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.518+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.518+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9ac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.518+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0d914400 con 0x55cd0d9b21a0 2023-10-08T14:05:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.518+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0d905bc0 con 0x55cd0d9b1880 2023-10-08T14:05:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.518+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0d9034e0 con 0x55cd0d9ac0b0 2023-10-08T14:05:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.519+0000 7ff395198700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.519+0000 7ff395198700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60324/0 (socket says 172.21.15.57:60324) 2023-10-08T14:05:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.519+0000 7ff395198700 1 -- 172.21.15.57:0/1517696451 learned_addr learned my addr 172.21.15.57:0/1517696451 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.519+0000 7ff395198700 1 -- 172.21.15.57:0/1517696451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9ac0b0 msgr2=0x55cd0d9ac510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:05:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.519+0000 7ff395198700 1 --2- 172.21.15.57:0/1517696451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9ac510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.519+0000 7ff395198700 1 -- 172.21.15.57:0/1517696451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1880 msgr2=0x55cd0d9b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.519+0000 7ff395198700 1 --2- 172.21.15.57:0/1517696451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1880 0x55cd0d9b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.519+0000 7ff395198700 1 -- 172.21.15.57:0/1517696451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd0d729d80 con 0x55cd0d9b21a0 2023-10-08T14:05:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.520+0000 7ff395198700 1 --2- 172.21.15.57:0/1517696451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9abb70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff39006e060 tx=0x7ff38c00d410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d96584c7cc7689 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.524+0000 7ff37bfff700 1 -- 172.21.15.57:0/1517696451 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c004890 con 0x55cd0d9b21a0 2023-10-08T14:05:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.524+0000 7ff37bfff700 1 -- 172.21.15.57:0/1517696451 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff38c006070 con 0x55cd0d9b21a0 2023-10-08T14:05:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.524+0000 7ff37bfff700 1 -- 172.21.15.57:0/1517696451 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c015740 con 0x55cd0d9b21a0 2023-10-08T14:05:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.525+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/1517696451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b21a0 msgr2=0x55cd0d9abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.525+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/1517696451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9abb70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff39006e060 tx=0x7ff38c00d410 comp rx=0 tx=0).stop 2023-10-08T14:05:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.526+0000 7ff395198700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff37c0a1740 0x7ff37c0a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.527+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/1517696451 shutdown_connections 2023-10-08T14:05:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.527+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/1517696451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.527+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/1517696451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1880 0x55cd0d9b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.527+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/1517696451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9abb70 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.527+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/1517696451 >> 172.21.15.57:0/1517696451 conn(0x55cd0d920200 msgr2=0x55cd0d9a6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.531+0000 7ff395198700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff37c0a1740 0x7ff37c0a3c00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00b8b0 tx=0x7ff38c0061e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.538+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/1517696451 shutdown_connections 2023-10-08T14:05:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.549+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/1517696451 wait complete. 2023-10-08T14:05:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.549+0000 7ff3a3de6b80 1 -- start start 2023-10-08T14:05:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.549+0000 7ff3a3de6b80 1 Processor -- start 2023-10-08T14:05:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.560+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.560+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1880 0x55cd0d9b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.560+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.560+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0d8fedd0 con 0x55cd0d9ac0b0 2023-10-08T14:05:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.560+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0d89b6b0 con 0x55cd0d9b1880 2023-10-08T14:05:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.560+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0d89b350 con 0x55cd0d9b21a0 2023-10-08T14:05:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.560+0000 7ff3a3de6b80 10 client.0 _ll_register_callbacks cb 0x55cd0d6f87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:05:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.561+0000 7ff39619a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.561+0000 7ff39619a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:32906/0 (socket says 172.21.15.57:32906) 2023-10-08T14:05:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.561+0000 7ff39619a700 1 -- 172.21.15.57:0/587635176 learned_addr learned my addr 172.21.15.57:0/587635176 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff395198700 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1880 0x55cd0d9b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff39619a700 1 -- 172.21.15.57:0/587635176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1880 msgr2=0x55cd0d9b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff39619a700 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1880 0x55cd0d9b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff39619a700 1 -- 172.21.15.57:0/587635176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9ac0b0 msgr2=0x55cd0d9b8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff39619a700 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff39619a700 1 -- 172.21.15.57:0/587635176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c00b970 con 0x55cd0d9b21a0 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff39619a700 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9b5be0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff3880dce10 tx=0x7ff3880a8b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1061728699c6c43 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff3797fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff3797fa700 1 -- 172.21.15.57:0/587635176 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3880acb30 con 0x55cd0d9b21a0 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff3797fa700 1 -- 172.21.15.57:0/587635176 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7ff3880ac6d0 con 0x55cd0d9b21a0 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/587635176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd0d8c6e50 con 0x55cd0d9b21a0 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff3797fa700 1 -- 172.21.15.57:0/587635176 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3880b0820 con 0x55cd0d9b21a0 2023-10-08T14:05:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff3a3de6b80 10 client.26893 fetch_fsmap learned FSMap version 48 2023-10-08T14:05:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.564+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/587635176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55cd0d712c00 con 0x55cd0d9b21a0 2023-10-08T14:05:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.565+0000 7ff3797fa700 1 -- 172.21.15.57:0/587635176 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd0d8c6e50 con 0x55cd0d9b21a0 2023-10-08T14:05:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.565+0000 7ff3797fa700 1 -- 172.21.15.57:0/587635176 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff3880b4540 con 0x55cd0d9b21a0 2023-10-08T14:05:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.565+0000 7ff3a3de6b80 10 client.26893 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:05:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.565+0000 7ff3a3de6b80 10 client.26893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:05:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.565+0000 7ff3a3de6b80 20 client.26893 populate_metadata read hostname 'smithi057' 2023-10-08T14:05:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.565+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd0d9e7b70 0x55cd0d9e9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.565+0000 7ff3a3de6b80 4 client.26893 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:05:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.565+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/587635176 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd0d8f7db0 con 0x55cd0d9e7b70 2023-10-08T14:05:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.568+0000 7ff395999700 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd0d9e7b70 0x55cd0d9e9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.568+0000 7ff395999700 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd0d9e7b70 0x55cd0d9e9f90 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d668787f700324f server_cookie=bdeb7127e0f8fd90 in_seq=0 out_seq=0 2023-10-08T14:05:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.570+0000 7ff3797fa700 10 client.26893 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.572+0000 7ff3797fa700 1 -- 172.21.15.57:0/587635176 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cd0d8f7db0 con 0x55cd0d9e7b70 2023-10-08T14:05:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.572+0000 7ff3797fa700 10 client.26893 handle_command_reply: tid=0 2023-10-08T14:05:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.572+0000 7ff3a3de6b80 1 client.26893 shutdown 2023-10-08T14:05:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.574+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/587635176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b21a0 msgr2=0x55cd0d9b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.574+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9b5be0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff3880dce10 tx=0x7ff3880a8b40 comp rx=0 tx=0).stop 2023-10-08T14:05:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.575+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/587635176 shutdown_connections 2023-10-08T14:05:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.575+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd0d9e7b70 0x55cd0d9e9f90 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.575+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b21a0 0x55cd0d9b5be0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.575+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9b8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.575+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/587635176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1880 0x55cd0d9b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.575+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/587635176 >> 172.21.15.57:0/587635176 conn(0x55cd0d920200 msgr2=0x55cd0d9a6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.575+0000 7ff3797fa700 0 client.26893 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.583+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/587635176 shutdown_connections 2023-10-08T14:05:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.586+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/587635176 wait complete. 2023-10-08T14:05:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.589+0000 7ff3a3de6b80 20 client.26893 trim_cache size 0 max 16384 2023-10-08T14:05:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.615+0000 7ff3a3de6b80 1 Processor -- start 2023-10-08T14:05:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.619+0000 7ff3a3de6b80 1 -- start start 2023-10-08T14:05:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.619+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.619+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b1880 0x55cd0d9b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.619+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1f40 0x55cd0d9dec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.619+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0d77f8c0 con 0x55cd0d9b1880 2023-10-08T14:05:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.619+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0d9b6b10 con 0x55cd0d9b1f40 2023-10-08T14:05:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.619+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0d9a8580 con 0x55cd0d9ac0b0 2023-10-08T14:05:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.620+0000 7ff39619a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1f40 0x55cd0d9dec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.620+0000 7ff39619a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1f40 0x55cd0d9dec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33532/0 (socket says 172.21.15.57:33532) 2023-10-08T14:05:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.620+0000 7ff39619a700 1 -- 172.21.15.57:0/4286509374 learned_addr learned my addr 172.21.15.57:0/4286509374 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.620+0000 7ff39619a700 1 -- 172.21.15.57:0/4286509374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9ac0b0 msgr2=0x55cd0d9abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.621+0000 7ff39619a700 1 --2- 172.21.15.57:0/4286509374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.621+0000 7ff39619a700 1 -- 172.21.15.57:0/4286509374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b1880 msgr2=0x55cd0d9b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.621+0000 7ff39619a700 1 --2- 172.21.15.57:0/4286509374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b1880 0x55cd0d9b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.621+0000 7ff39619a700 1 -- 172.21.15.57:0/4286509374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd0d7501f0 con 0x55cd0d9b1f40 2023-10-08T14:05:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.622+0000 7ff39619a700 1 --2- 172.21.15.57:0/4286509374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1f40 0x55cd0d9dec80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff3880ac3d0 tx=0x7ff38801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc7d333bcd8c45c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.625+0000 7ff384d82700 1 -- 172.21.15.57:0/4286509374 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3880acdd0 con 0x55cd0d9b1f40 2023-10-08T14:05:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.625+0000 7ff384d82700 1 -- 172.21.15.57:0/4286509374 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3880ac6d0 con 0x55cd0d9b1f40 2023-10-08T14:05:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.625+0000 7ff384d82700 1 -- 172.21.15.57:0/4286509374 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3880b4a40 con 0x55cd0d9b1f40 2023-10-08T14:05:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.626+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/4286509374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1f40 msgr2=0x55cd0d9dec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.626+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/4286509374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1f40 0x55cd0d9dec80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff3880ac3d0 tx=0x7ff38801c410 comp rx=0 tx=0).stop 2023-10-08T14:05:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.637+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/4286509374 shutdown_connections 2023-10-08T14:05:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.637+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/4286509374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9b1f40 0x55cd0d9dec80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.637+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/4286509374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.637+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/4286509374 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b1880 0x55cd0d9b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.637+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/4286509374 >> 172.21.15.57:0/4286509374 conn(0x55cd0d920200 msgr2=0x55cd0d9ee8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.642+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/4286509374 shutdown_connections 2023-10-08T14:05:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.642+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/4286509374 wait complete. 2023-10-08T14:05:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.643+0000 7ff3a3de6b80 1 -- start start 2023-10-08T14:05:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.643+0000 7ff3a3de6b80 1 Processor -- start 2023-10-08T14:05:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.643+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9dce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.643+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b1880 0x55cd0d9dd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.643+0000 7ff3a3de6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b1f40 0x55cd0da3a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.644+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0d9ddb20 con 0x55cd0d9b1880 2023-10-08T14:05:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.644+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0da3aed0 con 0x55cd0d9ac0b0 2023-10-08T14:05:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.644+0000 7ff3a3de6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0da3b090 con 0x55cd0d9b1f40 2023-10-08T14:05:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.644+0000 7ff3a3de6b80 10 client.0 _ll_register_callbacks cb 0x55cd0d6f87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:05:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff39619a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b1f40 0x55cd0da3a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff39619a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b1f40 0x55cd0da3a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:32920/0 (socket says 172.21.15.57:32920) 2023-10-08T14:05:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff39619a700 1 -- 172.21.15.57:0/3411186742 learned_addr learned my addr 172.21.15.57:0/3411186742 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff395999700 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9dce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff39619a700 1 -- 172.21.15.57:0/3411186742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9ac0b0 msgr2=0x55cd0d9dce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff39619a700 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9dce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff39619a700 1 -- 172.21.15.57:0/3411186742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b1880 msgr2=0x55cd0d9dd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff39619a700 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b1880 0x55cd0d9dd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff39619a700 1 -- 172.21.15.57:0/3411186742 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd0d712c00 con 0x55cd0d9b1f40 2023-10-08T14:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff39619a700 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b1f40 0x55cd0da3a960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3880ac610 tx=0x7ff3880b0820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c44f26d9bfc5e139 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.646+0000 7ff3797fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.647+0000 7ff3797fa700 1 -- 172.21.15.57:0/3411186742 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3880cad40 con 0x55cd0d9b1f40 2023-10-08T14:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.647+0000 7ff3797fa700 1 -- 172.21.15.57:0/3411186742 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7ff3880acdd0 con 0x55cd0d9b1f40 2023-10-08T14:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.647+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/3411186742 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff38c006070 con 0x55cd0d9b1f40 2023-10-08T14:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.647+0000 7ff3797fa700 1 -- 172.21.15.57:0/3411186742 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3880caea0 con 0x55cd0d9b1f40 2023-10-08T14:05:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.651+0000 7ff3a3de6b80 10 client.26896 fetch_fsmap learned FSMap version 48 2023-10-08T14:05:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.651+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/3411186742 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55cd0d728250 con 0x55cd0d9b1f40 2023-10-08T14:05:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.655+0000 7ff3797fa700 1 -- 172.21.15.57:0/3411186742 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3880bd030 con 0x55cd0d9b1f40 2023-10-08T14:05:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.655+0000 7ff3797fa700 1 -- 172.21.15.57:0/3411186742 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff38802cac0 con 0x55cd0d9b1f40 2023-10-08T14:05:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.655+0000 7ff3a3de6b80 10 client.26896 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:05:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.655+0000 7ff3a3de6b80 10 client.26896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:05:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.655+0000 7ff3a3de6b80 20 client.26896 populate_metadata read hostname 'smithi057' 2023-10-08T14:05:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.656+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd0da3c350 0x55cd0da3e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.656+0000 7ff3a3de6b80 4 client.26896 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:05:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.656+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/3411186742 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd0d8f7ad0 con 0x55cd0da3c350 2023-10-08T14:05:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.656+0000 7ff395999700 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd0da3c350 0x55cd0da3e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.662+0000 7ff395999700 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd0da3c350 0x55cd0da3e740 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19428b78da9b978a server_cookie=62919a3d000579fc in_seq=0 out_seq=0 2023-10-08T14:05:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:13.667+0000 7ff3797fa700 10 client.26896 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:14.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:13 smithi115 ceph-mon[104408]: pgmap v723: 129 pgs: 50 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 62 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 106 op/s 2023-10-08T14:05:14.257 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:13 smithi057 ceph-mon[99476]: pgmap v723: 129 pgs: 50 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 62 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 106 op/s 2023-10-08T14:05:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:13 smithi099 ceph-mon[103849]: pgmap v723: 129 pgs: 50 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 62 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 106 op/s 2023-10-08T14:05:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.641+0000 7ff3797fa700 1 -- 172.21.15.57:0/3411186742 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd0d8f7ad0 con 0x55cd0da3c350 2023-10-08T14:05:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.641+0000 7ff3797fa700 10 client.26896 handle_command_reply: tid=0 2023-10-08T14:05:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.641+0000 7ff3a3de6b80 1 client.26896 shutdown 2023-10-08T14:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.643+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/3411186742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b1f40 msgr2=0x55cd0da3a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.643+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b1f40 0x55cd0da3a960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3880ac610 tx=0x7ff3880b0820 comp rx=0 tx=0).stop 2023-10-08T14:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.643+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/3411186742 shutdown_connections 2023-10-08T14:05:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.643+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55cd0da3c350 0x55cd0da3e740 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.643+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cd0d9b1f40 0x55cd0da3a960 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.643+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cd0d9ac0b0 0x55cd0d9dce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.643+0000 7ff3a3de6b80 1 --2- 172.21.15.57:0/3411186742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd0d9b1880 0x55cd0d9dd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.643+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/3411186742 >> 172.21.15.57:0/3411186742 conn(0x55cd0d920200 msgr2=0x55cd0d9b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.644+0000 7ff3797fa700 0 client.26896 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.644+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/3411186742 shutdown_connections 2023-10-08T14:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.644+0000 7ff3a3de6b80 1 -- 172.21.15.57:0/3411186742 wait complete. 2023-10-08T14:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.644+0000 7ff3a3de6b80 20 client.26896 trim_cache size 0 max 16384 2023-10-08T14:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:05:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:05:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:05:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.649+0000 7ff397bfd700 1 -- 172.21.15.57:0/3934709140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff37c0a1740 msgr2=0x7ff37c0a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.649+0000 7ff397bfd700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff37c0a1740 0x7ff37c0a3c00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00b8b0 tx=0x7ff38c0061e0 comp rx=0 tx=0).stop 2023-10-08T14:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.649+0000 7ff397bfd700 1 -- 172.21.15.57:0/3934709140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff390119fd0 msgr2=0x7ff390115400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.649+0000 7ff397bfd700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff390119fd0 0x7ff390115400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff38801c7a0 tx=0x7ff3880188b0 comp rx=0 tx=0).stop 2023-10-08T14:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.652+0000 7ff397bfd700 1 -- 172.21.15.57:0/3934709140 shutdown_connections 2023-10-08T14:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.652+0000 7ff397bfd700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7ff37c0a1740 0x7ff37c0a3c00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.652+0000 7ff397bfd700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff390119fd0 0x7ff390115400 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.652+0000 7ff397bfd700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff39010ef10 0x7ff3901a2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.652+0000 7ff397bfd700 1 --2- 172.21.15.57:0/3934709140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff39010f380 0x7ff3901a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.652+0000 7ff397bfd700 1 -- 172.21.15.57:0/3934709140 >> 172.21.15.57:0/3934709140 conn(0x7ff39006b7c0 msgr2=0x7ff39011b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.663+0000 7ff397bfd700 1 -- 172.21.15.57:0/3934709140 shutdown_connections 2023-10-08T14:05:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:14.664+0000 7ff397bfd700 1 -- 172.21.15.57:0/3934709140 wait complete. 2023-10-08T14:05:16.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:15 smithi057 ceph-mon[99476]: pgmap v724: 129 pgs: 43 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-10-08T14:05:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:15 smithi099 ceph-mon[103849]: pgmap v724: 129 pgs: 43 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-10-08T14:05:16.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:15 smithi115 ceph-mon[104408]: pgmap v724: 129 pgs: 43 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-10-08T14:05:18.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:17 smithi057 ceph-mon[99476]: pgmap v725: 129 pgs: 43 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 954 KiB/s wr, 88 op/s 2023-10-08T14:05:18.271 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:17 smithi099 ceph-mon[103849]: pgmap v725: 129 pgs: 43 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 954 KiB/s wr, 88 op/s 2023-10-08T14:05:18.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:17 smithi115 ceph-mon[104408]: pgmap v725: 129 pgs: 43 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 954 KiB/s wr, 88 op/s 2023-10-08T14:05:19.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:19 smithi099 ceph-mon[103849]: pgmap v726: 129 pgs: 43 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 103 op/s 2023-10-08T14:05:19.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:19 smithi115 ceph-mon[104408]: pgmap v726: 129 pgs: 43 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 103 op/s 2023-10-08T14:05:19.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:19 smithi057 ceph-mon[99476]: pgmap v726: 129 pgs: 43 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 103 op/s 2023-10-08T14:05:22.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:21 smithi115 ceph-mon[104408]: pgmap v727: 129 pgs: 39 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 77 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 999 KiB/s wr, 134 op/s 2023-10-08T14:05:22.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:21 smithi057 ceph-mon[99476]: pgmap v727: 129 pgs: 39 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 77 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 999 KiB/s wr, 134 op/s 2023-10-08T14:05:22.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:21 smithi099 ceph-mon[103849]: pgmap v727: 129 pgs: 39 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 77 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 999 KiB/s wr, 134 op/s 2023-10-08T14:05:24.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:23 smithi057 ceph-mon[99476]: pgmap v728: 129 pgs: 39 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 77 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 924 KiB/s wr, 138 op/s 2023-10-08T14:05:24.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:23 smithi099 ceph-mon[103849]: pgmap v728: 129 pgs: 39 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 77 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 924 KiB/s wr, 138 op/s 2023-10-08T14:05:24.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:23 smithi115 ceph-mon[104408]: pgmap v728: 129 pgs: 39 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 77 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 924 KiB/s wr, 138 op/s 2023-10-08T14:05:26.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:25 smithi057 ceph-mon[99476]: pgmap v729: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 738 KiB/s wr, 201 op/s 2023-10-08T14:05:26.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:25 smithi099 ceph-mon[103849]: pgmap v729: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 738 KiB/s wr, 201 op/s 2023-10-08T14:05:26.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:25 smithi115 ceph-mon[104408]: pgmap v729: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 738 KiB/s wr, 201 op/s 2023-10-08T14:05:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.035+0000 7f0c791b2700 1 -- 172.21.15.57:0/355786368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c7410ef10 msgr2=0x7f0c741077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.035+0000 7f0c791b2700 1 --2- 172.21.15.57:0/355786368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c7410ef10 0x7f0c741077f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009fc0 tx=0x7f0c6c00b330 comp rx=0 tx=0).stop 2023-10-08T14:05:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.035+0000 7f0c791b2700 1 -- 172.21.15.57:0/355786368 shutdown_connections 2023-10-08T14:05:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.035+0000 7f0c791b2700 1 --2- 172.21.15.57:0/355786368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c74119fd0 0x7f0c7411c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.035+0000 7f0c791b2700 1 --2- 172.21.15.57:0/355786368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c7410ef10 0x7f0c741077f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.035+0000 7f0c791b2700 1 --2- 172.21.15.57:0/355786368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c7410f380 0x7f0c74107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.035+0000 7f0c791b2700 1 -- 172.21.15.57:0/355786368 >> 172.21.15.57:0/355786368 conn(0x7f0c7406b7c0 msgr2=0x7f0c7406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.035+0000 7f0c791b2700 1 -- 172.21.15.57:0/355786368 shutdown_connections 2023-10-08T14:05:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.035+0000 7f0c791b2700 1 -- 172.21.15.57:0/355786368 wait complete. 2023-10-08T14:05:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.036+0000 7f0c791b2700 1 Processor -- start 2023-10-08T14:05:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.036+0000 7f0c791b2700 1 -- start start 2023-10-08T14:05:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.037+0000 7f0c791b2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c7410ef10 0x7f0c74119550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.037+0000 7f0c791b2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c7410f380 0x7f0c74119a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.037+0000 7f0c791b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c74119fd0 0x7f0c74115870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.037+0000 7f0c791b2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c74116040 con 0x7f0c7410ef10 2023-10-08T14:05:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.037+0000 7f0c791b2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c741161b0 con 0x7f0c74119fd0 2023-10-08T14:05:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.037+0000 7f0c791b2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c74116370 con 0x7f0c7410f380 2023-10-08T14:05:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.037+0000 7f0c72d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c7410ef10 0x7f0c74119550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.037+0000 7f0c72d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c7410ef10 0x7f0c74119550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60354/0 (socket says 172.21.15.57:60354) 2023-10-08T14:05:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.037+0000 7f0c72d9d700 1 -- 172.21.15.57:0/3156017865 learned_addr learned my addr 172.21.15.57:0/3156017865 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.037+0000 7f0c7359e700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c74119fd0 0x7f0c74115870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c72d9d700 1 -- 172.21.15.57:0/3156017865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c7410f380 msgr2=0x7f0c74119a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:05:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c7410f380 0x7f0c74119a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c72d9d700 1 -- 172.21.15.57:0/3156017865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c74119fd0 msgr2=0x7f0c74115870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c74119fd0 0x7f0c74115870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c72d9d700 1 -- 172.21.15.57:0/3156017865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c009c40 con 0x7f0c7410ef10 2023-10-08T14:05:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c7410ef10 0x7f0c74119550 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c007960 tx=0x7f0c6c0079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c5bfff700 1 -- 172.21.15.57:0/3156017865 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c00e040 con 0x7f0c7410ef10 2023-10-08T14:05:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c5bfff700 1 -- 172.21.15.57:0/3156017865 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c6c00bbf0 con 0x7f0c7410ef10 2023-10-08T14:05:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c5bfff700 1 -- 172.21.15.57:0/3156017865 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c01c660 con 0x7f0c7410ef10 2023-10-08T14:05:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c791b2700 1 -- 172.21.15.57:0/3156017865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c741a9c00 con 0x7f0c7410ef10 2023-10-08T14:05:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.038+0000 7f0c791b2700 1 -- 172.21.15.57:0/3156017865 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c741a9f70 con 0x7f0c7410ef10 2023-10-08T14:05:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.044+0000 7f0c8539bb80 1 Processor -- start 2023-10-08T14:05:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.047+0000 7f0c5bfff700 1 -- 172.21.15.57:0/3156017865 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0c6c005040 con 0x7f0c7410ef10 2023-10-08T14:05:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.048+0000 7f0c5bfff700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c5c0a1730 0x7f0c5c0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.048+0000 7f0c5bfff700 1 -- 172.21.15.57:0/3156017865 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7f0c6c0cda10 con 0x7f0c7410ef10 2023-10-08T14:05:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.049+0000 7f0c8539bb80 1 -- start start 2023-10-08T14:05:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.049+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e4383880 0x5633e4383c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.049+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.049+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e437e0b0 0x5633e437e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.049+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e42e6400 con 0x5633e437e0b0 2023-10-08T14:05:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.049+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e42d7bc0 con 0x5633e43841a0 2023-10-08T14:05:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.049+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e42d54e0 con 0x5633e4383880 2023-10-08T14:05:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.050+0000 7f0c7259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.050+0000 7f0c7259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33554/0 (socket says 172.21.15.57:33554) 2023-10-08T14:05:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.050+0000 7f0c7259c700 1 -- 172.21.15.57:0/2374652181 learned_addr learned my addr 172.21.15.57:0/2374652181 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.050+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/2374652181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e4383880 0x5633e4383c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.050+0000 7f0c7259c700 1 -- 172.21.15.57:0/2374652181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e4383880 msgr2=0x5633e4383c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.050+0000 7f0c7259c700 1 --2- 172.21.15.57:0/2374652181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e4383880 0x5633e4383c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.050+0000 7f0c7259c700 1 -- 172.21.15.57:0/2374652181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e437e0b0 msgr2=0x5633e437e510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:05:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.050+0000 7f0c7259c700 1 --2- 172.21.15.57:0/2374652181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e437e0b0 0x5633e437e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.050+0000 7f0c7259c700 1 -- 172.21.15.57:0/2374652181 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e40fbd80 con 0x5633e43841a0 2023-10-08T14:05:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.050+0000 7f0c7259c700 1 --2- 172.21.15.57:0/2374652181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437db70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0c6401aaa0 tx=0x7f0c6401adb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac34434e858f5e05 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.057+0000 7f0c7359e700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c5c0a1730 0x7f0c5c0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.058+0000 7f0c7359e700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c5c0a1730 0x7f0c5c0a3bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0c74069db0 tx=0x7f0c68009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.062+0000 7f0c597fa700 1 -- 172.21.15.57:0/2374652181 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6401fa60 con 0x5633e43841a0 2023-10-08T14:05:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.070+0000 7f0c597fa700 1 -- 172.21.15.57:0/2374652181 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c640044f0 con 0x5633e43841a0 2023-10-08T14:05:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.070+0000 7f0c597fa700 1 -- 172.21.15.57:0/2374652181 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64005020 con 0x5633e43841a0 2023-10-08T14:05:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.071+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2374652181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 msgr2=0x5633e437db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.071+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/2374652181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437db70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0c6401aaa0 tx=0x7f0c6401adb0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.080+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2374652181 shutdown_connections 2023-10-08T14:05:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.080+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/2374652181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e437e0b0 0x5633e437e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.080+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/2374652181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e4383880 0x5633e4383c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.080+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/2374652181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437db70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.080+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2374652181 >> 172.21.15.57:0/2374652181 conn(0x5633e42f2200 msgr2=0x5633e4378b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.085+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2374652181 shutdown_connections 2023-10-08T14:05:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.088+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2374652181 wait complete. 2023-10-08T14:05:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.088+0000 7f0c8539bb80 1 -- start start 2023-10-08T14:05:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.088+0000 7f0c8539bb80 1 Processor -- start 2023-10-08T14:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e437e0b0 0x5633e438a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e43875a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e4387c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e42d0dd0 con 0x5633e4383880 2023-10-08T14:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e426d6b0 con 0x5633e43841a0 2023-10-08T14:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e426d350 con 0x5633e437e0b0 2023-10-08T14:05:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c8539bb80 10 client.0 _ll_register_callbacks cb 0x5633e40ca7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c7359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e4387c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c7259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e43875a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c7359e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e4387c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33560/0 (socket says 172.21.15.57:33560) 2023-10-08T14:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.089+0000 7f0c7359e700 1 -- 172.21.15.57:0/2798648089 learned_addr learned my addr 172.21.15.57:0/2798648089 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.090+0000 7f0c7359e700 1 -- 172.21.15.57:0/2798648089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e437e0b0 msgr2=0x5633e438a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.090+0000 7f0c7359e700 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e437e0b0 0x5633e438a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.090+0000 7f0c7359e700 1 -- 172.21.15.57:0/2798648089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 msgr2=0x5633e43875a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.090+0000 7f0c7359e700 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e43875a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.090+0000 7f0c7359e700 1 -- 172.21.15.57:0/2798648089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6401a780 con 0x5633e43841a0 2023-10-08T14:05:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.090+0000 7f0c7359e700 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e4387c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c68012700 tx=0x7f0c680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26466ce99a17a28e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.090+0000 7f0c56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:05:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.090+0000 7f0c56ff5700 1 -- 172.21.15.57:0/2798648089 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68004220 con 0x5633e43841a0 2023-10-08T14:05:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.090+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2798648089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633e4298e50 con 0x5633e43841a0 2023-10-08T14:05:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.091+0000 7f0c56ff5700 1 -- 172.21.15.57:0/2798648089 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7f0c68018070 con 0x5633e43841a0 2023-10-08T14:05:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.091+0000 7f0c56ff5700 1 -- 172.21.15.57:0/2798648089 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c68004380 con 0x5633e43841a0 2023-10-08T14:05:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.091+0000 7f0c8539bb80 10 client.26864 fetch_fsmap learned FSMap version 48 2023-10-08T14:05:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.091+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2798648089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5633e40e4c00 con 0x5633e43841a0 2023-10-08T14:05:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.096+0000 7f0c56ff5700 1 -- 172.21.15.57:0/2798648089 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633e4298e50 con 0x5633e43841a0 2023-10-08T14:05:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.096+0000 7f0c56ff5700 1 -- 172.21.15.57:0/2798648089 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f0c68025070 con 0x5633e43841a0 2023-10-08T14:05:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.096+0000 7f0c8539bb80 10 client.26864 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:05:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.096+0000 7f0c8539bb80 10 client.26864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:05:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.096+0000 7f0c8539bb80 20 client.26864 populate_metadata read hostname 'smithi057' 2023-10-08T14:05:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.096+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5633e4397e10 0x5633e439a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.096+0000 7f0c8539bb80 4 client.26864 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:05:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.096+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2798648089 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633e42c9db0 con 0x5633e4397e10 2023-10-08T14:05:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.097+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5633e4397e10 0x5633e439a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.097+0000 7f0c56ff5700 10 client.26864 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.097+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5633e4397e10 0x5633e439a230 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff01f89fb8f1b14 server_cookie=bbdedbb8d8ddfc60 in_seq=0 out_seq=0 2023-10-08T14:05:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c56ff5700 1 -- 172.21.15.57:0/2798648089 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5633e42c9db0 con 0x5633e4397e10 2023-10-08T14:05:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c56ff5700 10 client.26864 handle_command_reply: tid=0 2023-10-08T14:05:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c8539bb80 1 client.26864 shutdown 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2798648089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 msgr2=0x5633e4387c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e4387c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c68012700 tx=0x7f0c680047c0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2798648089 shutdown_connections 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5633e4397e10 0x5633e439a230 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e4387c90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e437e0b0 0x5633e438a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c56ff5700 0 client.26864 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/2798648089 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e43875a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2798648089 >> 172.21.15.57:0/2798648089 conn(0x5633e42f2200 msgr2=0x5633e4378010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.098+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2798648089 shutdown_connections 2023-10-08T14:05:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.099+0000 7f0c8539bb80 1 -- 172.21.15.57:0/2798648089 wait complete. 2023-10-08T14:05:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.099+0000 7f0c8539bb80 20 client.26864 trim_cache size 0 max 16384 2023-10-08T14:05:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.126+0000 7f0c8539bb80 1 Processor -- start 2023-10-08T14:05:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.126+0000 7f0c8539bb80 1 -- start start 2023-10-08T14:05:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e437e0b0 0x5633e437db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e4378790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e43841a0 0x5633e4378cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e41518c0 con 0x5633e4383880 2023-10-08T14:05:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e43885b0 con 0x5633e437e0b0 2023-10-08T14:05:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e4379610 con 0x5633e43841a0 2023-10-08T14:05:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c72d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e437e0b0 0x5633e437db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c72d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e437e0b0 0x5633e437db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33566/0 (socket says 172.21.15.57:33566) 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c72d9d700 1 -- 172.21.15.57:0/1048290677 learned_addr learned my addr 172.21.15.57:0/1048290677 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c7359e700 1 --2- 172.21.15.57:0/1048290677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e43841a0 0x5633e4378cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c72d9d700 1 -- 172.21.15.57:0/1048290677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e43841a0 msgr2=0x5633e4378cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/1048290677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e43841a0 0x5633e4378cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c72d9d700 1 -- 172.21.15.57:0/1048290677 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 msgr2=0x5633e4378790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/1048290677 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e4378790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.127+0000 7f0c72d9d700 1 -- 172.21.15.57:0/1048290677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e41221f0 con 0x5633e437e0b0 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.128+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/1048290677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e437e0b0 0x5633e437db70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c096b30 tx=0x7f0c6c085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f67615fdab8a464 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.128+0000 7f0c59ffb700 1 -- 172.21.15.57:0/1048290677 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c0850b0 con 0x5633e437e0b0 2023-10-08T14:05:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.128+0000 7f0c59ffb700 1 -- 172.21.15.57:0/1048290677 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c6c09ad30 con 0x5633e437e0b0 2023-10-08T14:05:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.128+0000 7f0c59ffb700 1 -- 172.21.15.57:0/1048290677 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c0b8850 con 0x5633e437e0b0 2023-10-08T14:05:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 -- 172.21.15.57:0/1048290677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e437e0b0 msgr2=0x5633e437db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/1048290677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e437e0b0 0x5633e437db70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c096b30 tx=0x7f0c6c085040 comp rx=0 tx=0).stop 2023-10-08T14:05:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 -- 172.21.15.57:0/1048290677 shutdown_connections 2023-10-08T14:05:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/1048290677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e43841a0 0x5633e4378cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/1048290677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e437e0b0 0x5633e437db70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/1048290677 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e4378790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 -- 172.21.15.57:0/1048290677 >> 172.21.15.57:0/1048290677 conn(0x5633e42f2200 msgr2=0x5633e437e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 -- 172.21.15.57:0/1048290677 shutdown_connections 2023-10-08T14:05:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 -- 172.21.15.57:0/1048290677 wait complete. 2023-10-08T14:05:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 -- start start 2023-10-08T14:05:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.129+0000 7f0c8539bb80 1 Processor -- start 2023-10-08T14:05:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.130+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e437e0b0 0x5633e437f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.130+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e437f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.130+0000 7f0c8539bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.130+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e43804a0 con 0x5633e4383880 2023-10-08T14:05:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.130+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e4380610 con 0x5633e43841a0 2023-10-08T14:05:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.130+0000 7f0c8539bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e43807d0 con 0x5633e437e0b0 2023-10-08T14:05:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.130+0000 7f0c8539bb80 10 client.0 _ll_register_callbacks cb 0x7f0c68011dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:05:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.135+0000 7f0c7259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e437f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.135+0000 7f0c7259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e437f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60380/0 (socket says 172.21.15.57:60380) 2023-10-08T14:05:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.135+0000 7f0c7259c700 1 -- 172.21.15.57:0/4183698894 learned_addr learned my addr 172.21.15.57:0/4183698894 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.135+0000 7f0c7359e700 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.135+0000 7f0c7359e700 1 -- 172.21.15.57:0/4183698894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e437e0b0 msgr2=0x5633e437f400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:05:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.135+0000 7f0c7359e700 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e437e0b0 0x5633e437f400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.135+0000 7f0c7359e700 1 -- 172.21.15.57:0/4183698894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 msgr2=0x5633e437f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.135+0000 7f0c7359e700 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e437f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.135+0000 7f0c7359e700 1 -- 172.21.15.57:0/4183698894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c741a9f70 con 0x5633e43841a0 2023-10-08T14:05:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.136+0000 7f0c7359e700 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437fe80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c6801fa00 tx=0x7f0c680099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf6db0f11743ddec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.136+0000 7f0c56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:05:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.137+0000 7f0c56ff5700 1 -- 172.21.15.57:0/4183698894 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68004450 con 0x5633e43841a0 2023-10-08T14:05:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.137+0000 7f0c56ff5700 1 -- 172.21.15.57:0/4183698894 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9533+0+0 (secure 0 0 0) 0x7f0c68018070 con 0x5633e43841a0 2023-10-08T14:05:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.137+0000 7f0c8539bb80 1 -- 172.21.15.57:0/4183698894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c640044f0 con 0x5633e43841a0 2023-10-08T14:05:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.137+0000 7f0c56ff5700 1 -- 172.21.15.57:0/4183698894 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c680045b0 con 0x5633e43841a0 2023-10-08T14:05:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.137+0000 7f0c8539bb80 10 client.26870 fetch_fsmap learned FSMap version 48 2023-10-08T14:05:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.137+0000 7f0c8539bb80 1 -- 172.21.15.57:0/4183698894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5633e40fa250 con 0x5633e43841a0 2023-10-08T14:05:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.138+0000 7f0c56ff5700 1 -- 172.21.15.57:0/4183698894 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c640044f0 con 0x5633e43841a0 2023-10-08T14:05:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.138+0000 7f0c56ff5700 1 -- 172.21.15.57:0/4183698894 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f0c68012730 con 0x5633e43841a0 2023-10-08T14:05:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.138+0000 7f0c8539bb80 10 client.26870 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T14:05:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.138+0000 7f0c8539bb80 10 client.26870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:05:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.138+0000 7f0c8539bb80 20 client.26870 populate_metadata read hostname 'smithi057' 2023-10-08T14:05:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.138+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5633e4397e10 0x5633e43ee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.138+0000 7f0c8539bb80 4 client.26870 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:05:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.138+0000 7f0c8539bb80 1 -- 172.21.15.57:0/4183698894 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633e42c9ad0 con 0x5633e4397e10 2023-10-08T14:05:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.143+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5633e4397e10 0x5633e43ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.145+0000 7f0c56ff5700 10 client.26870 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:28.145+0000 7f0c72d9d700 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5633e4397e10 0x5633e43ee890 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8c37d1151ac53c server_cookie=1d840e5d9b86a11a in_seq=0 out_seq=0 2023-10-08T14:05:28.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:28 smithi099 ceph-mon[103849]: pgmap v730: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 705 KiB/s wr, 194 op/s 2023-10-08T14:05:28.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:27 smithi115 ceph-mon[104408]: pgmap v730: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 705 KiB/s wr, 194 op/s 2023-10-08T14:05:28.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:27 smithi057 ceph-mon[99476]: pgmap v730: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 705 KiB/s wr, 194 op/s 2023-10-08T14:05:29.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.491+0000 7f0c56ff5700 1 -- 172.21.15.57:0/4183698894 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5633e42c9ad0 con 0x5633e4397e10 2023-10-08T14:05:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.491+0000 7f0c56ff5700 10 client.26870 handle_command_reply: tid=0 2023-10-08T14:05:29.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.492+0000 7f0c8539bb80 1 client.26870 shutdown 2023-10-08T14:05:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.494+0000 7f0c8539bb80 1 -- 172.21.15.57:0/4183698894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 msgr2=0x5633e437fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.494+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437fe80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c6801fa00 tx=0x7f0c680099c0 comp rx=0 tx=0).stop 2023-10-08T14:05:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.494+0000 7f0c8539bb80 1 -- 172.21.15.57:0/4183698894 shutdown_connections 2023-10-08T14:05:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.494+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5633e4397e10 0x5633e43ee890 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.494+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633e43841a0 0x5633e437fe80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.494+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5633e437e0b0 0x5633e437f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.494+0000 7f0c8539bb80 1 --2- 172.21.15.57:0/4183698894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5633e4383880 0x5633e437f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.494+0000 7f0c8539bb80 1 -- 172.21.15.57:0/4183698894 >> 172.21.15.57:0/4183698894 conn(0x5633e42f2200 msgr2=0x5633e4385dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.494+0000 7f0c56ff5700 0 client.26870 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.498+0000 7f0c8539bb80 1 -- 172.21.15.57:0/4183698894 shutdown_connections 2023-10-08T14:05:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.498+0000 7f0c8539bb80 1 -- 172.21.15.57:0/4183698894 wait complete. 2023-10-08T14:05:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.500+0000 7f0c8539bb80 20 client.26870 trim_cache size 0 max 16384 2023-10-08T14:05:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:05:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:05:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:05:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.507+0000 7f0c791b2700 1 -- 172.21.15.57:0/3156017865 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c5c0a1730 msgr2=0x7f0c5c0a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.507+0000 7f0c791b2700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c5c0a1730 0x7f0c5c0a3bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0c74069db0 tx=0x7f0c68009260 comp rx=0 tx=0).stop 2023-10-08T14:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.507+0000 7f0c791b2700 1 -- 172.21.15.57:0/3156017865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c7410ef10 msgr2=0x7f0c74119550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.507+0000 7f0c791b2700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c7410ef10 0x7f0c74119550 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c007960 tx=0x7f0c6c0079d0 comp rx=0 tx=0).stop 2023-10-08T14:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.507+0000 7f0c791b2700 1 -- 172.21.15.57:0/3156017865 shutdown_connections 2023-10-08T14:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.507+0000 7f0c791b2700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f0c5c0a1730 0x7f0c5c0a3bf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.507+0000 7f0c791b2700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c74119fd0 0x7f0c74115870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.507+0000 7f0c791b2700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c7410ef10 0x7f0c74119550 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.507+0000 7f0c791b2700 1 --2- 172.21.15.57:0/3156017865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c7410f380 0x7f0c74119a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.507+0000 7f0c791b2700 1 -- 172.21.15.57:0/3156017865 >> 172.21.15.57:0/3156017865 conn(0x7f0c7406b7c0 msgr2=0x7f0c7411a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.508+0000 7f0c791b2700 1 -- 172.21.15.57:0/3156017865 shutdown_connections 2023-10-08T14:05:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:29.508+0000 7f0c791b2700 1 -- 172.21.15.57:0/3156017865 wait complete. 2023-10-08T14:05:29.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:29 smithi115 ceph-mon[104408]: pgmap v731: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 600 KiB/s wr, 214 op/s 2023-10-08T14:05:29.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:29 smithi057 ceph-mon[99476]: pgmap v731: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 600 KiB/s wr, 214 op/s 2023-10-08T14:05:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:29 smithi099 ceph-mon[103849]: pgmap v731: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 600 KiB/s wr, 214 op/s 2023-10-08T14:05:30.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:30 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:05:30.998 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:30 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:05:31.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:30 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:05:31.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:31 smithi057 ceph-mon[99476]: pgmap v732: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 822 KiB/s wr, 321 op/s 2023-10-08T14:05:32.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:31 smithi099 ceph-mon[103849]: pgmap v732: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 822 KiB/s wr, 321 op/s 2023-10-08T14:05:32.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:31 smithi115 ceph-mon[104408]: pgmap v732: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 822 KiB/s wr, 321 op/s 2023-10-08T14:05:32.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:05:32.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:05:32.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:05:33.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:05:33.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:05:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:32 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:05:33.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:05:33.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:05:33.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:05:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:33 smithi099 ceph-mon[103849]: pgmap v733: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 752 KiB/s wr, 316 op/s 2023-10-08T14:05:34.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:33 smithi099 ceph-mon[103849]: osdmap e107: 12 total, 12 up, 12 in 2023-10-08T14:05:34.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:33 smithi115 ceph-mon[104408]: pgmap v733: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 752 KiB/s wr, 316 op/s 2023-10-08T14:05:34.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:33 smithi115 ceph-mon[104408]: osdmap e107: 12 total, 12 up, 12 in 2023-10-08T14:05:34.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:33 smithi057 ceph-mon[99476]: pgmap v733: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 752 KiB/s wr, 316 op/s 2023-10-08T14:05:34.253 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:33 smithi057 ceph-mon[99476]: osdmap e107: 12 total, 12 up, 12 in 2023-10-08T14:05:36.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:35 smithi057 ceph-mon[99476]: pgmap v735: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 860 KiB/s wr, 336 op/s 2023-10-08T14:05:36.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:35 smithi099 ceph-mon[103849]: pgmap v735: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 860 KiB/s wr, 336 op/s 2023-10-08T14:05:36.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:35 smithi115 ceph-mon[104408]: pgmap v735: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 860 KiB/s wr, 336 op/s 2023-10-08T14:05:38.239 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:37 smithi057 ceph-mon[99476]: pgmap v736: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 860 KiB/s wr, 331 op/s 2023-10-08T14:05:38.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:37 smithi099 ceph-mon[103849]: pgmap v736: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 860 KiB/s wr, 331 op/s 2023-10-08T14:05:38.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:37 smithi115 ceph-mon[104408]: pgmap v736: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 860 KiB/s wr, 331 op/s 2023-10-08T14:05:39.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:39 smithi099 ceph-mon[103849]: pgmap v737: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 846 KiB/s wr, 324 op/s 2023-10-08T14:05:39.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:39 smithi115 ceph-mon[104408]: pgmap v737: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 846 KiB/s wr, 324 op/s 2023-10-08T14:05:39.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:39 smithi057 ceph-mon[99476]: pgmap v737: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 846 KiB/s wr, 324 op/s 2023-10-08T14:05:41.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:41 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:05:41.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:41 smithi115 ceph-mon[104408]: pgmap v738: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 730 KiB/s wr, 251 op/s 2023-10-08T14:05:41.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:41 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:05:41.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:41 smithi057 ceph-mon[99476]: pgmap v738: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 730 KiB/s wr, 251 op/s 2023-10-08T14:05:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:41 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:05:42.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:41 smithi099 ceph-mon[103849]: pgmap v738: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 730 KiB/s wr, 251 op/s 2023-10-08T14:05:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.898+0000 7f9b3c40b700 1 -- 172.21.15.57:0/871356252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b34113940 msgr2=0x7f9b34115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.898+0000 7f9b3c40b700 1 --2- 172.21.15.57:0/871356252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b34113940 0x7f9b34115d30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00b0d0 tx=0x7f9b2c00b3e0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.901+0000 7f9b3c40b700 1 -- 172.21.15.57:0/871356252 shutdown_connections 2023-10-08T14:05:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.901+0000 7f9b3c40b700 1 --2- 172.21.15.57:0/871356252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b34113940 0x7f9b34115d30 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.901+0000 7f9b3c40b700 1 --2- 172.21.15.57:0/871356252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b341077f0 0x7f9b34107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.901+0000 7f9b3c40b700 1 --2- 172.21.15.57:0/871356252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34108110 0x7f9b34113400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.901+0000 7f9b3c40b700 1 -- 172.21.15.57:0/871356252 >> 172.21.15.57:0/871356252 conn(0x7f9b3406b7c0 msgr2=0x7f9b3406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.910+0000 7f9b3c40b700 1 -- 172.21.15.57:0/871356252 shutdown_connections 2023-10-08T14:05:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.921+0000 7f9b3c40b700 1 -- 172.21.15.57:0/871356252 wait complete. 2023-10-08T14:05:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.921+0000 7f9b3c40b700 1 Processor -- start 2023-10-08T14:05:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.921+0000 7f9b3c40b700 1 -- start start 2023-10-08T14:05:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.922+0000 7f9b3c40b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b341077f0 0x7f9b341ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.922+0000 7f9b3c40b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b34108110 0x7f9b341abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.922+0000 7f9b3c40b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34113940 0x7f9b341a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.922+0000 7f9b3c40b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b341a5e40 con 0x7f9b341077f0 2023-10-08T14:05:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.922+0000 7f9b3c40b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b341a5fb0 con 0x7f9b34113940 2023-10-08T14:05:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.922+0000 7f9b3c40b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b341a6170 con 0x7f9b34108110 2023-10-08T14:05:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.922+0000 7f9b399a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b34108110 0x7f9b341abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.922+0000 7f9b399a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b34108110 0x7f9b341abb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:32974/0 (socket says 172.21.15.57:32974) 2023-10-08T14:05:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.922+0000 7f9b399a6700 1 -- 172.21.15.57:0/2732244745 learned_addr learned my addr 172.21.15.57:0/2732244745 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.923+0000 7f9b399a6700 1 -- 172.21.15.57:0/2732244745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34113940 msgr2=0x7f9b341a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.923+0000 7f9b399a6700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34113940 0x7f9b341a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.923+0000 7f9b399a6700 1 -- 172.21.15.57:0/2732244745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b341077f0 msgr2=0x7f9b341ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.923+0000 7f9b399a6700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b341077f0 0x7f9b341ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.923+0000 7f9b399a6700 1 -- 172.21.15.57:0/2732244745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c009d00 con 0x7f9b34108110 2023-10-08T14:05:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.923+0000 7f9b399a6700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b34108110 0x7f9b341abb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000d950 tx=0x7f9b3000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.927+0000 7f9b2b7fe700 1 -- 172.21.15.57:0/2732244745 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30009b40 con 0x7f9b34108110 2023-10-08T14:05:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.927+0000 7f9b2b7fe700 1 -- 172.21.15.57:0/2732244745 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b30009ca0 con 0x7f9b34108110 2023-10-08T14:05:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.927+0000 7f9b2b7fe700 1 -- 172.21.15.57:0/2732244745 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3000f620 con 0x7f9b34108110 2023-10-08T14:05:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.928+0000 7f9b3c40b700 1 -- 172.21.15.57:0/2732244745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b341a6450 con 0x7f9b34108110 2023-10-08T14:05:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.928+0000 7f9b3c40b700 1 -- 172.21.15.57:0/2732244745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b341b2640 con 0x7f9b34108110 2023-10-08T14:05:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.932+0000 7f9b2b7fe700 1 -- 172.21.15.57:0/2732244745 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9b30010460 con 0x7f9b34108110 2023-10-08T14:05:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.933+0000 7f9b2b7fe700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9b200a1800 0x7f9b200a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.933+0000 7f9b2b7fe700 1 -- 172.21.15.57:0/2732244745 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9473+0+0 (secure 0 0 0) 0x7f9b300cd230 con 0x7f9b34108110 2023-10-08T14:05:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.933+0000 7f9b3a1a7700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9b200a1800 0x7f9b200a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.934+0000 7f9b3a1a7700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9b200a1800 0x7f9b200a3cc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009770 tx=0x7f9b24006cd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.963+0000 7f9b485f4b80 1 Processor -- start 2023-10-08T14:05:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.968+0000 7f9b485f4b80 1 -- start start 2023-10-08T14:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.968+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c27880 0x556ba0c27c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.968+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c281a0 0x556ba0c21b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.968+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c22510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.968+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0b8a400 con 0x556ba0c27880 2023-10-08T14:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.968+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0b7bbc0 con 0x556ba0c281a0 2023-10-08T14:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.968+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0b794e0 con 0x556ba0c220b0 2023-10-08T14:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.969+0000 7f9b3a9a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c22510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.969+0000 7f9b3a9a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c22510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:32980/0 (socket says 172.21.15.57:32980) 2023-10-08T14:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.969+0000 7f9b3a9a8700 1 -- 172.21.15.57:0/3623456506 learned_addr learned my addr 172.21.15.57:0/3623456506 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.969+0000 7f9b3a9a8700 1 -- 172.21.15.57:0/3623456506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c281a0 msgr2=0x556ba0c21b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:05:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.969+0000 7f9b3a9a8700 1 --2- 172.21.15.57:0/3623456506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c281a0 0x556ba0c21b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.969+0000 7f9b3a9a8700 1 -- 172.21.15.57:0/3623456506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c27880 msgr2=0x556ba0c27c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:05:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.969+0000 7f9b3a9a8700 1 --2- 172.21.15.57:0/3623456506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c27880 0x556ba0c27c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.969+0000 7f9b3a9a8700 1 -- 172.21.15.57:0/3623456506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ba099fd80 con 0x556ba0c220b0 2023-10-08T14:05:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.979+0000 7f9b3a9a8700 1 --2- 172.21.15.57:0/3623456506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c22510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009fd0 tx=0x7f9b2c0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f2fa907077d88d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.979+0000 7f9b28d82700 1 -- 172.21.15.57:0/3623456506 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0093a0 con 0x556ba0c220b0 2023-10-08T14:05:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.979+0000 7f9b28d82700 1 -- 172.21.15.57:0/3623456506 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2c010250 con 0x556ba0c220b0 2023-10-08T14:05:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.979+0000 7f9b28d82700 1 -- 172.21.15.57:0/3623456506 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0078c0 con 0x556ba0c220b0 2023-10-08T14:05:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.980+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3623456506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 msgr2=0x556ba0c22510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.980+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/3623456506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c22510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009fd0 tx=0x7f9b2c0096c0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.980+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3623456506 shutdown_connections 2023-10-08T14:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.980+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/3623456506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c22510 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.980+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/3623456506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c27880 0x556ba0c27c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.980+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/3623456506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c281a0 0x556ba0c21b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.981+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3623456506 >> 172.21.15.57:0/3623456506 conn(0x556ba0b96200 msgr2=0x556ba0c1cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.981+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3623456506 shutdown_connections 2023-10-08T14:05:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.981+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3623456506 wait complete. 2023-10-08T14:05:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.981+0000 7f9b485f4b80 1 -- start start 2023-10-08T14:05:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.981+0000 7f9b485f4b80 1 Processor -- start 2023-10-08T14:05:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c27880 0x556ba0c2bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c281a0 0x556ba0c2c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0b74dd0 con 0x556ba0c281a0 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0b116b0 con 0x556ba0c27880 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0b11350 con 0x556ba0c220b0 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b485f4b80 10 client.0 _ll_register_callbacks cb 0x556ba096e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b399a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c27880 0x556ba0c2bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b399a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c27880 0x556ba0c2bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33598/0 (socket says 172.21.15.57:33598) 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b399a6700 1 -- 172.21.15.57:0/2977662384 learned_addr learned my addr 172.21.15.57:0/2977662384 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.982+0000 7f9b3a9a8700 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c281a0 0x556ba0c2c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.983+0000 7f9b399a6700 1 -- 172.21.15.57:0/2977662384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 msgr2=0x556ba0c2b660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.983+0000 7f9b399a6700 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c2b660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.983+0000 7f9b399a6700 1 -- 172.21.15.57:0/2977662384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c281a0 msgr2=0x556ba0c2c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.983+0000 7f9b399a6700 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c281a0 0x556ba0c2c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.983+0000 7f9b399a6700 1 -- 172.21.15.57:0/2977662384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b34116680 con 0x556ba0c27880 2023-10-08T14:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.983+0000 7f9b399a6700 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c27880 0x556ba0c2bba0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9b30099d80 tx=0x7f9b30099a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ebec2c7e0d1fd2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.983+0000 7f9b1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.983+0000 7f9b1dffb700 1 -- 172.21.15.57:0/2977662384 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30010910 con 0x556ba0c27880 2023-10-08T14:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.983+0000 7f9b1dffb700 1 -- 172.21.15.57:0/2977662384 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9473+0+0 (secure 0 0 0) 0x7f9b3009df40 con 0x556ba0c27880 2023-10-08T14:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.983+0000 7f9b485f4b80 1 -- 172.21.15.57:0/2977662384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ba0b3ce50 con 0x556ba0c27880 2023-10-08T14:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.986+0000 7f9b1dffb700 1 -- 172.21.15.57:0/2977662384 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b300ca9f0 con 0x556ba0c27880 2023-10-08T14:05:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.986+0000 7f9b485f4b80 10 client.26876 fetch_fsmap learned FSMap version 49 2023-10-08T14:05:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.986+0000 7f9b485f4b80 1 -- 172.21.15.57:0/2977662384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x556ba0988c00 con 0x556ba0c27880 2023-10-08T14:05:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.989+0000 7f9b1dffb700 1 -- 172.21.15.57:0/2977662384 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ba0b3ce50 con 0x556ba0c27880 2023-10-08T14:05:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.989+0000 7f9b1dffb700 1 -- 172.21.15.57:0/2977662384 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 49) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f9b300992e0 con 0x556ba0c27880 2023-10-08T14:05:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.989+0000 7f9b485f4b80 10 client.26876 fetch_fsmap finished waiting for FSMap version 49 2023-10-08T14:05:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.989+0000 7f9b485f4b80 10 client.26876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:05:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.989+0000 7f9b485f4b80 20 client.26876 populate_metadata read hostname 'smithi057' 2023-10-08T14:05:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.989+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ba0c4ccc0 0x556ba0c4f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.989+0000 7f9b485f4b80 4 client.26876 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:05:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.989+0000 7f9b485f4b80 1 -- 172.21.15.57:0/2977662384 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ba0b6ddb0 con 0x556ba0c4ccc0 2023-10-08T14:05:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.990+0000 7f9b3a1a7700 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ba0c4ccc0 0x556ba0c4f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.990+0000 7f9b3a1a7700 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ba0c4ccc0 0x556ba0c4f0e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb4b500d8dc3670 server_cookie=8a72918525977900 in_seq=0 out_seq=0 2023-10-08T14:05:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.990+0000 7f9b1dffb700 10 client.26876 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.991+0000 7f9b1dffb700 1 -- 172.21.15.57:0/2977662384 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556ba0b6ddb0 con 0x556ba0c4ccc0 2023-10-08T14:05:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.991+0000 7f9b1dffb700 10 client.26876 handle_command_reply: tid=0 2023-10-08T14:05:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.991+0000 7f9b485f4b80 1 client.26876 shutdown 2023-10-08T14:05:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.993+0000 7f9b485f4b80 1 -- 172.21.15.57:0/2977662384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c27880 msgr2=0x556ba0c2bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.993+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c27880 0x556ba0c2bba0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9b30099d80 tx=0x7f9b30099a20 comp rx=0 tx=0).stop 2023-10-08T14:05:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.993+0000 7f9b485f4b80 1 -- 172.21.15.57:0/2977662384 shutdown_connections 2023-10-08T14:05:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.993+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ba0c4ccc0 0x556ba0c4f0e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.993+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c281a0 0x556ba0c2c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.993+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c2b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.993+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/2977662384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c27880 0x556ba0c2bba0 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9b30099d80 tx=0x7f9b30099a20 comp rx=0 tx=0).stop 2023-10-08T14:05:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.994+0000 7f9b485f4b80 1 -- 172.21.15.57:0/2977662384 >> 172.21.15.57:0/2977662384 conn(0x556ba0b96200 msgr2=0x556ba0c1c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.998+0000 7f9b1dffb700 0 client.26876 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:42.999+0000 7f9b485f4b80 1 -- 172.21.15.57:0/2977662384 shutdown_connections 2023-10-08T14:05:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.001+0000 7f9b485f4b80 1 -- 172.21.15.57:0/2977662384 wait complete. 2023-10-08T14:05:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.002+0000 7f9b485f4b80 20 client.26876 trim_cache size 0 max 16384 2023-10-08T14:05:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.029+0000 7f9b485f4b80 1 Processor -- start 2023-10-08T14:05:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.036+0000 7f9b485f4b80 1 -- start start 2023-10-08T14:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c21b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c281a0 0x556ba0c1c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c1ccd0 0x556ba0c257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba09f58c0 con 0x556ba0c1ccd0 2023-10-08T14:05:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0b6dde0 con 0x556ba0c281a0 2023-10-08T14:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0c19830 con 0x556ba0c220b0 2023-10-08T14:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b399a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c281a0 0x556ba0c1c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b3a9a8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c1ccd0 0x556ba0c257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b399a6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c281a0 0x556ba0c1c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33606/0 (socket says 172.21.15.57:33606) 2023-10-08T14:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b399a6700 1 -- 172.21.15.57:0/1538180097 learned_addr learned my addr 172.21.15.57:0/1538180097 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b3a1a7700 1 --2- 172.21.15.57:0/1538180097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c21b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b3a9a8700 1 -- 172.21.15.57:0/1538180097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 msgr2=0x556ba0c21b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b3a9a8700 1 --2- 172.21.15.57:0/1538180097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c21b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b3a9a8700 1 -- 172.21.15.57:0/1538180097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c281a0 msgr2=0x556ba0c1c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b3a9a8700 1 --2- 172.21.15.57:0/1538180097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c281a0 0x556ba0c1c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.037+0000 7f9b3a9a8700 1 -- 172.21.15.57:0/1538180097 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ba0c22760 con 0x556ba0c1ccd0 2023-10-08T14:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.038+0000 7f9b3a9a8700 1 --2- 172.21.15.57:0/1538180097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c1ccd0 0x556ba0c257a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c044040 tx=0x7f9b2c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0abf50a901cfd8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.038+0000 7f9b29583700 1 -- 172.21.15.57:0/1538180097 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0093a0 con 0x556ba0c1ccd0 2023-10-08T14:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.038+0000 7f9b29583700 1 -- 172.21.15.57:0/1538180097 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2c0103c0 con 0x556ba0c1ccd0 2023-10-08T14:05:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.038+0000 7f9b29583700 1 -- 172.21.15.57:0/1538180097 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0050b0 con 0x556ba0c1ccd0 2023-10-08T14:05:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.049+0000 7f9b485f4b80 1 -- 172.21.15.57:0/1538180097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c1ccd0 msgr2=0x556ba0c257a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.049+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/1538180097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c1ccd0 0x556ba0c257a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c044040 tx=0x7f9b2c009620 comp rx=0 tx=0).stop 2023-10-08T14:05:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.060+0000 7f9b485f4b80 1 -- 172.21.15.57:0/1538180097 shutdown_connections 2023-10-08T14:05:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.060+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/1538180097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c1ccd0 0x556ba0c257a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.060+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/1538180097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c220b0 0x556ba0c21b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.060+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/1538180097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c281a0 0x556ba0c1c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.060+0000 7f9b485f4b80 1 -- 172.21.15.57:0/1538180097 >> 172.21.15.57:0/1538180097 conn(0x556ba0b96200 msgr2=0x556ba0c2aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.063+0000 7f9b485f4b80 1 -- 172.21.15.57:0/1538180097 shutdown_connections 2023-10-08T14:05:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.063+0000 7f9b485f4b80 1 -- 172.21.15.57:0/1538180097 wait complete. 2023-10-08T14:05:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.064+0000 7f9b485f4b80 1 -- start start 2023-10-08T14:05:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.064+0000 7f9b485f4b80 1 Processor -- start 2023-10-08T14:05:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.064+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c1ccd0 0x556ba0c30b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.064+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c220b0 0x556ba0c310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.064+0000 7f9b485f4b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c281a0 0x556ba0c315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.064+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0c33170 con 0x556ba0c220b0 2023-10-08T14:05:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.064+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0c9fab0 con 0x556ba0c1ccd0 2023-10-08T14:05:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.064+0000 7f9b485f4b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba0c9fc40 con 0x556ba0c281a0 2023-10-08T14:05:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.064+0000 7f9b485f4b80 10 client.0 _ll_register_callbacks cb 0x556ba096e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:05:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.065+0000 7f9b3a1a7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c1ccd0 0x556ba0c30b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.065+0000 7f9b3a1a7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c1ccd0 0x556ba0c30b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33610/0 (socket says 172.21.15.57:33610) 2023-10-08T14:05:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.065+0000 7f9b3a1a7700 1 -- 172.21.15.57:0/3169715763 learned_addr learned my addr 172.21.15.57:0/3169715763 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.066+0000 7f9b3a1a7700 1 -- 172.21.15.57:0/3169715763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c281a0 msgr2=0x556ba0c315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.066+0000 7f9b3a1a7700 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c281a0 0x556ba0c315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.066+0000 7f9b3a1a7700 1 -- 172.21.15.57:0/3169715763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c220b0 msgr2=0x556ba0c310b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.066+0000 7f9b3a1a7700 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c220b0 0x556ba0c310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.066+0000 7f9b3a1a7700 1 -- 172.21.15.57:0/3169715763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ba099fd80 con 0x556ba0c1ccd0 2023-10-08T14:05:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.066+0000 7f9b3a1a7700 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c1ccd0 0x556ba0c30b70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009110 tx=0x7f9b240047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ccee5e4b37408fe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.066+0000 7f9b1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:05:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.066+0000 7f9b1dffb700 1 -- 172.21.15.57:0/3169715763 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2401a040 con 0x556ba0c1ccd0 2023-10-08T14:05:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.066+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3169715763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b2c010250 con 0x556ba0c1ccd0 2023-10-08T14:05:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.066+0000 7f9b1dffb700 1 -- 172.21.15.57:0/3169715763 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9473+0+0 (secure 0 0 0) 0x7f9b2401fe40 con 0x556ba0c1ccd0 2023-10-08T14:05:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.067+0000 7f9b1dffb700 1 -- 172.21.15.57:0/3169715763 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b24013ea0 con 0x556ba0c1ccd0 2023-10-08T14:05:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.067+0000 7f9b485f4b80 10 client.26882 fetch_fsmap learned FSMap version 49 2023-10-08T14:05:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.067+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3169715763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x556ba09c61f0 con 0x556ba0c1ccd0 2023-10-08T14:05:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.079+0000 7f9b1dffb700 1 -- 172.21.15.57:0/3169715763 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2c010250 con 0x556ba0c1ccd0 2023-10-08T14:05:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.079+0000 7f9b1dffb700 1 -- 172.21.15.57:0/3169715763 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 49) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f9b24032530 con 0x556ba0c1ccd0 2023-10-08T14:05:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.079+0000 7f9b485f4b80 10 client.26882 fetch_fsmap finished waiting for FSMap version 49 2023-10-08T14:05:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.079+0000 7f9b485f4b80 10 client.26882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:05:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.079+0000 7f9b485f4b80 20 client.26882 populate_metadata read hostname 'smithi057' 2023-10-08T14:05:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.079+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ba0ca15e0 0x556ba0ca3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.079+0000 7f9b485f4b80 4 client.26882 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:05:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.079+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3169715763 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ba0b6dad0 con 0x556ba0ca15e0 2023-10-08T14:05:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.087+0000 7f9b399a6700 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ba0ca15e0 0x556ba0ca3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.087+0000 7f9b1dffb700 10 client.26882 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:43.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:43.087+0000 7f9b399a6700 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ba0ca15e0 0x556ba0ca3a60 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=664570b718c1eb0c server_cookie=76152a98f6ec5c98 in_seq=0 out_seq=0 2023-10-08T14:05:44.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:43 smithi057 ceph-mon[99476]: pgmap v739: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 692 KiB/s wr, 229 op/s 2023-10-08T14:05:44.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:43 smithi099 ceph-mon[103849]: pgmap v739: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 692 KiB/s wr, 229 op/s 2023-10-08T14:05:44.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:43 smithi115 ceph-mon[104408]: pgmap v739: 129 pgs: 34 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 84 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 692 KiB/s wr, 229 op/s 2023-10-08T14:05:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.478+0000 7f9b1dffb700 1 -- 172.21.15.57:0/3169715763 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ba0b6dad0 con 0x556ba0ca15e0 2023-10-08T14:05:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.478+0000 7f9b1dffb700 10 client.26882 handle_command_reply: tid=0 2023-10-08T14:05:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.478+0000 7f9b485f4b80 1 client.26882 shutdown 2023-10-08T14:05:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.478+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3169715763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c1ccd0 msgr2=0x556ba0c30b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.478+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c1ccd0 0x556ba0c30b70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009110 tx=0x7f9b240047e0 comp rx=0 tx=0).stop 2023-10-08T14:05:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.480+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3169715763 shutdown_connections 2023-10-08T14:05:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.480+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x556ba0ca15e0 0x556ba0ca3a60 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.480+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556ba0c281a0 0x556ba0c315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.480+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ba0c1ccd0 0x556ba0c30b70 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009110 tx=0x7f9b240047e0 comp rx=0 tx=0).stop 2023-10-08T14:05:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.480+0000 7f9b485f4b80 1 --2- 172.21.15.57:0/3169715763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556ba0c220b0 0x556ba0c310b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.480+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3169715763 >> 172.21.15.57:0/3169715763 conn(0x556ba0b96200 msgr2=0x556ba0c29ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.480+0000 7f9b1dffb700 0 client.26882 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.487+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3169715763 shutdown_connections 2023-10-08T14:05:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.487+0000 7f9b485f4b80 1 -- 172.21.15.57:0/3169715763 wait complete. 2023-10-08T14:05:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.494+0000 7f9b485f4b80 20 client.26882 trim_cache size 0 max 16384 2023-10-08T14:05:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:05:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:05:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:05:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:05:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.499+0000 7f9b3c40b700 1 -- 172.21.15.57:0/2732244745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9b200a1800 msgr2=0x7f9b200a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.499+0000 7f9b3c40b700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9b200a1800 0x7f9b200a3cc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009770 tx=0x7f9b24006cd0 comp rx=0 tx=0).stop 2023-10-08T14:05:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.499+0000 7f9b3c40b700 1 -- 172.21.15.57:0/2732244745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b34108110 msgr2=0x7f9b341abb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.499+0000 7f9b3c40b700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b34108110 0x7f9b341abb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000d950 tx=0x7f9b3000dd10 comp rx=0 tx=0).stop 2023-10-08T14:05:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.499+0000 7f9b3c40b700 1 -- 172.21.15.57:0/2732244745 shutdown_connections 2023-10-08T14:05:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.499+0000 7f9b3c40b700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f9b200a1800 0x7f9b200a3cc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.499+0000 7f9b3c40b700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34113940 0x7f9b341a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.499+0000 7f9b3c40b700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9b341077f0 0x7f9b341ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.499+0000 7f9b3c40b700 1 --2- 172.21.15.57:0/2732244745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b34108110 0x7f9b341abb90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.499+0000 7f9b3c40b700 1 -- 172.21.15.57:0/2732244745 >> 172.21.15.57:0/2732244745 conn(0x7f9b3406b7c0 msgr2=0x7f9b3406f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.506+0000 7f9b3c40b700 1 -- 172.21.15.57:0/2732244745 shutdown_connections 2023-10-08T14:05:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:44.506+0000 7f9b3c40b700 1 -- 172.21.15.57:0/2732244745 wait complete. 2023-10-08T14:05:46.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:45 smithi057 ceph-mon[99476]: pgmap v740: 129 pgs: 26 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 930 KiB/s wr, 310 op/s 2023-10-08T14:05:46.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:45 smithi099 ceph-mon[103849]: pgmap v740: 129 pgs: 26 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 930 KiB/s wr, 310 op/s 2023-10-08T14:05:46.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:45 smithi115 ceph-mon[104408]: pgmap v740: 129 pgs: 26 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 930 KiB/s wr, 310 op/s 2023-10-08T14:05:48.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:47 smithi057 ceph-mon[99476]: pgmap v741: 129 pgs: 26 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 570 KiB/s wr, 256 op/s 2023-10-08T14:05:48.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:47 smithi099 ceph-mon[103849]: pgmap v741: 129 pgs: 26 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 570 KiB/s wr, 256 op/s 2023-10-08T14:05:48.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:47 smithi115 ceph-mon[104408]: pgmap v741: 129 pgs: 26 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 94 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 570 KiB/s wr, 256 op/s 2023-10-08T14:05:49.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:49 smithi099 ceph-mon[103849]: pgmap v742: 129 pgs: 19 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 102 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 895 KiB/s wr, 271 op/s 2023-10-08T14:05:49.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:49 smithi115 ceph-mon[104408]: pgmap v742: 129 pgs: 19 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 102 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 895 KiB/s wr, 271 op/s 2023-10-08T14:05:49.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:49 smithi057 ceph-mon[99476]: pgmap v742: 129 pgs: 19 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 102 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 895 KiB/s wr, 271 op/s 2023-10-08T14:05:52.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:51 smithi057 ceph-mon[99476]: pgmap v743: 129 pgs: 13 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 111 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 886 KiB/s wr, 315 op/s 2023-10-08T14:05:52.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:51 smithi099 ceph-mon[103849]: pgmap v743: 129 pgs: 13 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 111 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 886 KiB/s wr, 315 op/s 2023-10-08T14:05:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:51 smithi115 ceph-mon[104408]: pgmap v743: 129 pgs: 13 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 111 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 886 KiB/s wr, 315 op/s 2023-10-08T14:05:54.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:53 smithi057 ceph-mon[99476]: pgmap v744: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 707 KiB/s wr, 278 op/s 2023-10-08T14:05:54.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:53 smithi057 ceph-mon[99476]: osdmap e108: 12 total, 12 up, 12 in 2023-10-08T14:05:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:53 smithi099 ceph-mon[103849]: pgmap v744: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 707 KiB/s wr, 278 op/s 2023-10-08T14:05:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:53 smithi099 ceph-mon[103849]: osdmap e108: 12 total, 12 up, 12 in 2023-10-08T14:05:54.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:53 smithi115 ceph-mon[104408]: pgmap v744: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 707 KiB/s wr, 278 op/s 2023-10-08T14:05:54.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:53 smithi115 ceph-mon[104408]: osdmap e108: 12 total, 12 up, 12 in 2023-10-08T14:05:56.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:55 smithi099 ceph-mon[103849]: pgmap v746: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 992 KiB/s wr, 288 op/s 2023-10-08T14:05:56.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:55 smithi115 ceph-mon[104408]: pgmap v746: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 992 KiB/s wr, 288 op/s 2023-10-08T14:05:56.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:55 smithi057 ceph-mon[99476]: pgmap v746: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 992 KiB/s wr, 288 op/s 2023-10-08T14:05:57.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:56 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:05:57.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:56 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:05:57.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:56 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:05:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.032+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/1523795800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc48119fd0 msgr2=0x7fbc4811c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.032+0000 7fbc4f0c1700 1 --2- 172.21.15.57:0/1523795800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc48119fd0 0x7fbc4811c3c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbc4001a9d0 tx=0x7fbc4001ace0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.032+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/1523795800 shutdown_connections 2023-10-08T14:05:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.032+0000 7fbc4f0c1700 1 --2- 172.21.15.57:0/1523795800 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc48119fd0 0x7fbc4811c3c0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.032+0000 7fbc4f0c1700 1 --2- 172.21.15.57:0/1523795800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4810ef10 0x7fbc481077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.032+0000 7fbc4f0c1700 1 --2- 172.21.15.57:0/1523795800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4810f380 0x7fbc48107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.032+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/1523795800 >> 172.21.15.57:0/1523795800 conn(0x7fbc4806b7c0 msgr2=0x7fbc4806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.032+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/1523795800 shutdown_connections 2023-10-08T14:05:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.032+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/1523795800 wait complete. 2023-10-08T14:05:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.032+0000 7fbc4f0c1700 1 Processor -- start 2023-10-08T14:05:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4f0c1700 1 -- start start 2023-10-08T14:05:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4f0c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4810ef10 0x7fbc48115660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4f0c1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4810f380 0x7fbc48115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4f0c1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc48119fd0 0x7fbc48118f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4f0c1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc481194d0 con 0x7fbc48119fd0 2023-10-08T14:05:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4f0c1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48119610 con 0x7fbc4810ef10 2023-10-08T14:05:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4f0c1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc481197d0 con 0x7fbc4810f380 2023-10-08T14:05:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4d65e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc48119fd0 0x7fbc48118f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4d65e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc48119fd0 0x7fbc48118f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60434/0 (socket says 172.21.15.57:60434) 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4d65e700 1 -- 172.21.15.57:0/985935538 learned_addr learned my addr 172.21.15.57:0/985935538 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc47fff700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4810f380 0x7fbc48115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4ce5d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4810ef10 0x7fbc48115660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4d65e700 1 -- 172.21.15.57:0/985935538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4810f380 msgr2=0x7fbc48115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4d65e700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4810f380 0x7fbc48115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4d65e700 1 -- 172.21.15.57:0/985935538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4810ef10 msgr2=0x7fbc48115660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4d65e700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4810ef10 0x7fbc48115660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.033+0000 7fbc4d65e700 1 -- 172.21.15.57:0/985935538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4001a600 con 0x7fbc48119fd0 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.034+0000 7fbc4d65e700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc48119fd0 0x7fbc48118f00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbc4001a9a0 tx=0x7fbc4001a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.034+0000 7fbc45ffb700 1 -- 172.21.15.57:0/985935538 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40003680 con 0x7fbc48119fd0 2023-10-08T14:05:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.034+0000 7fbc45ffb700 1 -- 172.21.15.57:0/985935538 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc40004cf0 con 0x7fbc48119fd0 2023-10-08T14:05:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.034+0000 7fbc45ffb700 1 -- 172.21.15.57:0/985935538 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40005020 con 0x7fbc48119fd0 2023-10-08T14:05:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.034+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/985935538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc48119a50 con 0x7fbc48119fd0 2023-10-08T14:05:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.034+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/985935538 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc481cbd80 con 0x7fbc48119fd0 2023-10-08T14:05:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.037+0000 7fbc45ffb700 1 -- 172.21.15.57:0/985935538 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbc40004830 con 0x7fbc48119fd0 2023-10-08T14:05:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.038+0000 7fbc45ffb700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbc300a15f0 0x7fbc300a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.038+0000 7fbc45ffb700 1 -- 172.21.15.57:0/985935538 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9473+0+0 (secure 0 0 0) 0x7fbc400ce4a0 con 0x7fbc48119fd0 2023-10-08T14:05:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.039+0000 7fbc5b2aab80 1 Processor -- start 2023-10-08T14:05:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.039+0000 7fbc5b2aab80 1 -- start start 2023-10-08T14:05:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.039+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3078a880 0x55fc3078ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.039+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3078b1a0 0x55fc30784b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.039+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30785510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.039+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc306ed400 con 0x55fc3078b1a0 2023-10-08T14:05:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.039+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc306debc0 con 0x55fc3078a880 2023-10-08T14:05:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.039+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc306dc4e0 con 0x55fc307850b0 2023-10-08T14:05:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.040+0000 7fbc4d65e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30785510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.040+0000 7fbc4d65e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30785510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33020/0 (socket says 172.21.15.57:33020) 2023-10-08T14:05:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.040+0000 7fbc4d65e700 1 -- 172.21.15.57:0/520386892 learned_addr learned my addr 172.21.15.57:0/520386892 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.040+0000 7fbc4ce5d700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbc300a15f0 0x7fbc300a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.041+0000 7fbc4ce5d700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbc300a15f0 0x7fbc300a3ab0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009ef0 tx=0x7fbc3800b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.042+0000 7fbc4d65e700 1 -- 172.21.15.57:0/520386892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3078a880 msgr2=0x55fc3078ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.042+0000 7fbc4d65e700 1 --2- 172.21.15.57:0/520386892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3078a880 0x55fc3078ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.042+0000 7fbc4d65e700 1 -- 172.21.15.57:0/520386892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3078b1a0 msgr2=0x55fc30784b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:05:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.042+0000 7fbc4d65e700 1 --2- 172.21.15.57:0/520386892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3078b1a0 0x55fc30784b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.042+0000 7fbc4d65e700 1 -- 172.21.15.57:0/520386892 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc30502d80 con 0x55fc307850b0 2023-10-08T14:05:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.043+0000 7fbc4d65e700 1 --2- 172.21.15.57:0/520386892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30785510 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc40097c60 tx=0x7fbc40097600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca6fb619ae9ea80 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.044+0000 7fbc2effd700 1 -- 172.21.15.57:0/520386892 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40016360 con 0x55fc307850b0 2023-10-08T14:05:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.044+0000 7fbc2effd700 1 -- 172.21.15.57:0/520386892 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc4009b8f0 con 0x55fc307850b0 2023-10-08T14:05:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.044+0000 7fbc2effd700 1 -- 172.21.15.57:0/520386892 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc400ce120 con 0x55fc307850b0 2023-10-08T14:05:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.045+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/520386892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 msgr2=0x55fc30785510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.045+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/520386892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30785510 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc40097c60 tx=0x7fbc40097600 comp rx=0 tx=0).stop 2023-10-08T14:05:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.046+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/520386892 shutdown_connections 2023-10-08T14:05:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.046+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/520386892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30785510 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.046+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/520386892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3078a880 0x55fc3078ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.046+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/520386892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3078b1a0 0x55fc30784b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.046+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/520386892 >> 172.21.15.57:0/520386892 conn(0x55fc306f9200 msgr2=0x55fc3077fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.046+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/520386892 shutdown_connections 2023-10-08T14:05:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.046+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/520386892 wait complete. 2023-10-08T14:05:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.046+0000 7fbc5b2aab80 1 -- start start 2023-10-08T14:05:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.046+0000 7fbc5b2aab80 1 Processor -- start 2023-10-08T14:05:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc307850b0 0x55fc30791220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078a880 0x55fc3078e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3078b1a0 0x55fc3078eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc306d7dd0 con 0x55fc3078b1a0 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc306746b0 con 0x55fc307850b0 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc30674350 con 0x55fc3078a880 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc5b2aab80 10 client.0 _ll_register_callbacks cb 0x55fc304d17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc4ce5d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc307850b0 0x55fc30791220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078a880 0x55fc3078e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078a880 0x55fc3078e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33024/0 (socket says 172.21.15.57:33024) 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc4ce5d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc307850b0 0x55fc30791220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33636/0 (socket says 172.21.15.57:33636) 2023-10-08T14:05:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.047+0000 7fbc47fff700 1 -- 172.21.15.57:0/945164765 learned_addr learned my addr 172.21.15.57:0/945164765 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc47fff700 1 -- 172.21.15.57:0/945164765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc307850b0 msgr2=0x55fc30791220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc47fff700 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc307850b0 0x55fc30791220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc47fff700 1 -- 172.21.15.57:0/945164765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3078b1a0 msgr2=0x55fc3078eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc47fff700 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3078b1a0 0x55fc3078eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc47fff700 1 -- 172.21.15.57:0/945164765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc481cbd80 con 0x55fc3078a880 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc4ce5d700 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc307850b0 0x55fc30791220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc47fff700 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078a880 0x55fc3078e560 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbc4806e060 tx=0x7fbc3c0106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91bfa777344ebd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc2c7f8700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/945164765 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c004250 con 0x55fc3078a880 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/945164765 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9473+0+0 (secure 0 0 0) 0x7fbc3c010e50 con 0x55fc3078a880 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.048+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/945164765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc3069fe50 con 0x55fc3078a880 2023-10-08T14:05:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.049+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/945164765 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc3c0043b0 con 0x55fc3078a880 2023-10-08T14:05:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.049+0000 7fbc5b2aab80 10 client.26926 fetch_fsmap learned FSMap version 50 2023-10-08T14:05:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.049+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/945164765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55fc304ebc00 con 0x55fc3078a880 2023-10-08T14:05:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.056+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/945164765 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc3069fe50 con 0x55fc3078a880 2023-10-08T14:05:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.056+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/945164765 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 50) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fbc3c011660 con 0x55fc3078a880 2023-10-08T14:05:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.056+0000 7fbc5b2aab80 10 client.26926 fetch_fsmap finished waiting for FSMap version 50 2023-10-08T14:05:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.056+0000 7fbc5b2aab80 10 client.26926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:05:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.056+0000 7fbc5b2aab80 20 client.26926 populate_metadata read hostname 'smithi057' 2023-10-08T14:05:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.056+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fc3079ea70 0x55fc307a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.056+0000 7fbc5b2aab80 4 client.26926 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:05:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.056+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/945164765 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc306d0db0 con 0x55fc3079ea70 2023-10-08T14:05:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.056+0000 7fbc4ce5d700 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fc3079ea70 0x55fc307a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.057+0000 7fbc2c7f8700 10 client.26926 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.057+0000 7fbc4ce5d700 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fc3079ea70 0x55fc307a0e90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c745f89c6879abb server_cookie=aef369772157220b in_seq=0 out_seq=0 2023-10-08T14:05:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.058+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/945164765 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fc306d0db0 con 0x55fc3079ea70 2023-10-08T14:05:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.058+0000 7fbc2c7f8700 10 client.26926 handle_command_reply: tid=0 2023-10-08T14:05:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.058+0000 7fbc5b2aab80 1 client.26926 shutdown 2023-10-08T14:05:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.059+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/945164765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078a880 msgr2=0x55fc3078e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.059+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078a880 0x55fc3078e560 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbc4806e060 tx=0x7fbc3c0106b0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.059+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/945164765 shutdown_connections 2023-10-08T14:05:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.059+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fc3079ea70 0x55fc307a0e90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.059+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3078b1a0 0x55fc3078eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.059+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc307850b0 0x55fc30791220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.059+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/945164765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078a880 0x55fc3078e560 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbc4806e060 tx=0x7fbc3c0106b0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.059+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/945164765 >> 172.21.15.57:0/945164765 conn(0x55fc306f9200 msgr2=0x55fc3077f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.059+0000 7fbc2c7f8700 0 client.26926 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.062+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/945164765 shutdown_connections 2023-10-08T14:05:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.062+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/945164765 wait complete. 2023-10-08T14:05:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.063+0000 7fbc5b2aab80 20 client.26926 trim_cache size 0 max 16384 2023-10-08T14:05:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.086+0000 7fbc5b2aab80 1 Processor -- start 2023-10-08T14:05:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.086+0000 7fbc5b2aab80 1 -- start start 2023-10-08T14:05:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.086+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc307850b0 0x55fc30784b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.086+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078b1a0 0x55fc3077f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.086+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3077fcd0 0x55fc307886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.086+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc305588c0 con 0x55fc307850b0 2023-10-08T14:05:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.086+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3077a2b0 con 0x55fc3077fcd0 2023-10-08T14:05:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.086+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc307812a0 con 0x55fc3078b1a0 2023-10-08T14:05:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.089+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078b1a0 0x55fc3077f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.090+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078b1a0 0x55fc3077f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33032/0 (socket says 172.21.15.57:33032) 2023-10-08T14:05:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.090+0000 7fbc47fff700 1 -- 172.21.15.57:0/492127324 learned_addr learned my addr 172.21.15.57:0/492127324 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.090+0000 7fbc4d65e700 1 --2- 172.21.15.57:0/492127324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3077fcd0 0x55fc307886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.090+0000 7fbc4d65e700 1 -- 172.21.15.57:0/492127324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078b1a0 msgr2=0x55fc3077f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.090+0000 7fbc4d65e700 1 --2- 172.21.15.57:0/492127324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078b1a0 0x55fc3077f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.090+0000 7fbc4d65e700 1 -- 172.21.15.57:0/492127324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc307850b0 msgr2=0x55fc30784b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.090+0000 7fbc4d65e700 1 --2- 172.21.15.57:0/492127324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc307850b0 0x55fc30784b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.090+0000 7fbc4d65e700 1 -- 172.21.15.57:0/492127324 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc305291f0 con 0x55fc3077fcd0 2023-10-08T14:05:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.091+0000 7fbc4d65e700 1 --2- 172.21.15.57:0/492127324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3077fcd0 0x55fc307886a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbc40097c60 tx=0x7fbc4001a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b2782125104474 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.091+0000 7fbc2f7fe700 1 -- 172.21.15.57:0/492127324 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4009b1f0 con 0x55fc3077fcd0 2023-10-08T14:05:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.091+0000 7fbc2f7fe700 1 -- 172.21.15.57:0/492127324 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc4009f160 con 0x55fc3077fcd0 2023-10-08T14:05:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.091+0000 7fbc2f7fe700 1 -- 172.21.15.57:0/492127324 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc400ce120 con 0x55fc3077fcd0 2023-10-08T14:05:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.092+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/492127324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3077fcd0 msgr2=0x55fc307886a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.092+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/492127324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3077fcd0 0x55fc307886a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbc40097c60 tx=0x7fbc4001a320 comp rx=0 tx=0).stop 2023-10-08T14:05:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.094+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/492127324 shutdown_connections 2023-10-08T14:05:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.094+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/492127324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3077fcd0 0x55fc307886a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.094+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/492127324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc307850b0 0x55fc30784b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.094+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/492127324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc3078b1a0 0x55fc3077f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.095+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/492127324 >> 172.21.15.57:0/492127324 conn(0x55fc306f9200 msgr2=0x55fc307a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.101+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/492127324 shutdown_connections 2023-10-08T14:05:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.101+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/492127324 wait complete. 2023-10-08T14:05:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.102+0000 7fbc5b2aab80 1 -- start start 2023-10-08T14:05:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.102+0000 7fbc5b2aab80 1 Processor -- start 2023-10-08T14:05:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.107+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3077fcd0 0x55fc30793940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.107+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30793e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.107+0000 7fbc5b2aab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3078b1a0 0x55fc307943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.107+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc307f19f0 con 0x55fc3077fcd0 2023-10-08T14:05:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.107+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc307f1b60 con 0x55fc3078b1a0 2023-10-08T14:05:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.107+0000 7fbc5b2aab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc307f1d20 con 0x55fc307850b0 2023-10-08T14:05:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.107+0000 7fbc5b2aab80 10 client.0 _ll_register_callbacks cb 0x7fbc3c007850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:05:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.108+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30793e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.108+0000 7fbc4ce5d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3077fcd0 0x55fc30793940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.108+0000 7fbc4ce5d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3077fcd0 0x55fc30793940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60458/0 (socket says 172.21.15.57:60458) 2023-10-08T14:05:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.108+0000 7fbc4ce5d700 1 -- 172.21.15.57:0/3859004603 learned_addr learned my addr 172.21.15.57:0/3859004603 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:05:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.108+0000 7fbc4ce5d700 1 -- 172.21.15.57:0/3859004603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 msgr2=0x55fc30793e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.108+0000 7fbc4ce5d700 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30793e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.108+0000 7fbc4ce5d700 1 -- 172.21.15.57:0/3859004603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3078b1a0 msgr2=0x55fc307943c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:05:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.108+0000 7fbc4ce5d700 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3078b1a0 0x55fc307943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.108+0000 7fbc4ce5d700 1 -- 172.21.15.57:0/3859004603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc30502d80 con 0x55fc3077fcd0 2023-10-08T14:05:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.108+0000 7fbc47fff700 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30793e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:05:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.109+0000 7fbc4ce5d700 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3077fcd0 0x55fc30793940 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009910 tx=0x7fbc38009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc0902fe3730084 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:05:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.109+0000 7fbc2c7f8700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:05:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.109+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/3859004603 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38017660 con 0x55fc3077fcd0 2023-10-08T14:05:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.109+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/3859004603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc306d0ad0 con 0x55fc3077fcd0 2023-10-08T14:05:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.109+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/3859004603 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9473+0+0 (secure 0 0 0) 0x7fbc380177c0 con 0x55fc3077fcd0 2023-10-08T14:05:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.109+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/3859004603 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc380179a0 con 0x55fc3077fcd0 2023-10-08T14:05:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.109+0000 7fbc5b2aab80 10 client.17370 fetch_fsmap learned FSMap version 50 2023-10-08T14:05:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.109+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/3859004603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55fc30501250 con 0x55fc3077fcd0 2023-10-08T14:05:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.110+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/3859004603 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc306d0ad0 con 0x55fc3077fcd0 2023-10-08T14:05:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.111+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/3859004603 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 50) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fbc38024af0 con 0x55fc3077fcd0 2023-10-08T14:05:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.111+0000 7fbc5b2aab80 10 client.17370 fetch_fsmap finished waiting for FSMap version 50 2023-10-08T14:05:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.111+0000 7fbc5b2aab80 10 client.17370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:05:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.111+0000 7fbc5b2aab80 20 client.17370 populate_metadata read hostname 'smithi057' 2023-10-08T14:05:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.111+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fc307f3660 0x55fc307f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:05:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.111+0000 7fbc5b2aab80 4 client.17370 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:05:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.111+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/3859004603 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc30523f10 con 0x55fc307f3660 2023-10-08T14:05:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.114+0000 7fbc47fff700 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fc307f3660 0x55fc307f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:05:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.121+0000 7fbc2c7f8700 10 client.17370 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:58.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:58.121+0000 7fbc47fff700 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fc307f3660 0x55fc307f5a80 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1d9a1a326b7e32 server_cookie=6251a89db4e5031f in_seq=0 out_seq=0 2023-10-08T14:05:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:57 smithi099 ceph-mon[103849]: pgmap v747: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 972 KiB/s wr, 279 op/s 2023-10-08T14:05:58.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:57 smithi115 ceph-mon[104408]: pgmap v747: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 972 KiB/s wr, 279 op/s 2023-10-08T14:05:58.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:57 smithi057 ceph-mon[99476]: pgmap v747: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 972 KiB/s wr, 279 op/s 2023-10-08T14:05:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc2c7f8700 1 -- 172.21.15.57:0/3859004603 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc30523f10 con 0x55fc307f3660 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc2c7f8700 10 client.17370 handle_command_reply: tid=0 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 client.17370 shutdown 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/3859004603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3077fcd0 msgr2=0x55fc30793940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3077fcd0 0x55fc30793940 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009910 tx=0x7fbc38009c90 comp rx=0 tx=0).stop 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/3859004603 shutdown_connections 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55fc307f3660 0x55fc307f5a80 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc3078b1a0 0x55fc307943c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc2c7f8700 0 client.17370 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55fc3077fcd0 0x55fc30793940 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 --2- 172.21.15.57:0/3859004603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fc307850b0 0x55fc30793e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/3859004603 >> 172.21.15.57:0/3859004603 conn(0x55fc306f9200 msgr2=0x55fc3078ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/3859004603 shutdown_connections 2023-10-08T14:05:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 1 -- 172.21.15.57:0/3859004603 wait complete. 2023-10-08T14:05:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.555+0000 7fbc5b2aab80 20 client.17370 trim_cache size 0 max 16384 2023-10-08T14:05:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:05:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:05:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:05:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:05:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/985935538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbc300a15f0 msgr2=0x7fbc300a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbc300a15f0 0x7fbc300a3ab0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009ef0 tx=0x7fbc3800b040 comp rx=0 tx=0).stop 2023-10-08T14:05:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/985935538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc48119fd0 msgr2=0x7fbc48118f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:05:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc48119fd0 0x7fbc48118f00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbc4001a9a0 tx=0x7fbc4001a590 comp rx=0 tx=0).stop 2023-10-08T14:05:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/985935538 shutdown_connections 2023-10-08T14:05:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc48119fd0 0x7fbc48118f00 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fbc300a15f0 0x7fbc300a3ab0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4810ef10 0x7fbc48115660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 --2- 172.21.15.57:0/985935538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc4810f380 0x7fbc48115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:05:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/985935538 >> 172.21.15.57:0/985935538 conn(0x7fbc4806b7c0 msgr2=0x7fbc4811a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:05:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/985935538 shutdown_connections 2023-10-08T14:05:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:05:59.556+0000 7fbc4f0c1700 1 -- 172.21.15.57:0/985935538 wait complete. 2023-10-08T14:05:59.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:05:59 smithi099 ceph-mon[103849]: pgmap v748: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 618 KiB/s wr, 280 op/s 2023-10-08T14:05:59.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:05:59 smithi115 ceph-mon[104408]: pgmap v748: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 618 KiB/s wr, 280 op/s 2023-10-08T14:05:59.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:05:59 smithi057 ceph-mon[99476]: pgmap v748: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 618 KiB/s wr, 280 op/s 2023-10-08T14:06:02.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:01 smithi115 ceph-mon[104408]: pgmap v749: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1012 KiB/s wr, 364 op/s 2023-10-08T14:06:02.241 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:01 smithi057 ceph-mon[99476]: pgmap v749: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1012 KiB/s wr, 364 op/s 2023-10-08T14:06:02.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:01 smithi099 ceph-mon[103849]: pgmap v749: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1012 KiB/s wr, 364 op/s 2023-10-08T14:06:04.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:03 smithi115 ceph-mon[104408]: pgmap v750: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2023-10-08T14:06:04.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:03 smithi057 ceph-mon[99476]: pgmap v750: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2023-10-08T14:06:04.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:03 smithi099 ceph-mon[103849]: pgmap v750: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2023-10-08T14:06:06.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:05 smithi115 ceph-mon[104408]: pgmap v751: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2023-10-08T14:06:06.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:05 smithi057 ceph-mon[99476]: pgmap v751: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2023-10-08T14:06:06.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:05 smithi099 ceph-mon[103849]: pgmap v751: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2023-10-08T14:06:08.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:07 smithi057 ceph-mon[99476]: pgmap v752: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 831 KiB/s wr, 326 op/s 2023-10-08T14:06:08.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:07 smithi099 ceph-mon[103849]: pgmap v752: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 831 KiB/s wr, 326 op/s 2023-10-08T14:06:08.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:07 smithi115 ceph-mon[104408]: pgmap v752: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 831 KiB/s wr, 326 op/s 2023-10-08T14:06:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:09 smithi099 ceph-mon[103849]: pgmap v753: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 329 op/s 2023-10-08T14:06:09.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:09 smithi115 ceph-mon[104408]: pgmap v753: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 329 op/s 2023-10-08T14:06:09.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:09 smithi057 ceph-mon[99476]: pgmap v753: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 329 op/s 2023-10-08T14:06:10.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:10 smithi115 ceph-mon[104408]: osdmap e109: 12 total, 12 up, 12 in 2023-10-08T14:06:10.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:10 smithi057 ceph-mon[99476]: osdmap e109: 12 total, 12 up, 12 in 2023-10-08T14:06:11.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:10 smithi099 ceph-mon[103849]: osdmap e109: 12 total, 12 up, 12 in 2023-10-08T14:06:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:11 smithi115 ceph-mon[104408]: pgmap v755: 129 pgs: 30 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 89 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 276 op/s 2023-10-08T14:06:11.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:11 smithi057 ceph-mon[99476]: pgmap v755: 129 pgs: 30 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 89 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 276 op/s 2023-10-08T14:06:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:11 smithi099 ceph-mon[103849]: pgmap v755: 129 pgs: 30 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 89 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 276 op/s 2023-10-08T14:06:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.752+0000 7f7abfcfd700 1 -- 172.21.15.57:0/2670895777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ab80ff630 msgr2=0x7f7ab80ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.752+0000 7f7abfcfd700 1 --2- 172.21.15.57:0/2670895777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ab80ff630 0x7f7ab80ffab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009990 tx=0x7f7aac009ca0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.753+0000 7f7abfcfd700 1 -- 172.21.15.57:0/2670895777 shutdown_connections 2023-10-08T14:06:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.753+0000 7f7abfcfd700 1 --2- 172.21.15.57:0/2670895777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab80ffff0 0x7f7ab810c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.753+0000 7f7abfcfd700 1 --2- 172.21.15.57:0/2670895777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ab81056d0 0x7f7ab8105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.753+0000 7f7abfcfd700 1 --2- 172.21.15.57:0/2670895777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ab80ff630 0x7f7ab80ffab0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.753+0000 7f7abfcfd700 1 -- 172.21.15.57:0/2670895777 >> 172.21.15.57:0/2670895777 conn(0x7f7ab8076810 msgr2=0x7f7ab8076c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.763+0000 7f7abfcfd700 1 -- 172.21.15.57:0/2670895777 shutdown_connections 2023-10-08T14:06:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.763+0000 7f7abfcfd700 1 -- 172.21.15.57:0/2670895777 wait complete. 2023-10-08T14:06:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.764+0000 7f7abfcfd700 1 Processor -- start 2023-10-08T14:06:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.764+0000 7f7abfcfd700 1 -- start start 2023-10-08T14:06:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.764+0000 7f7abfcfd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab80ff630 0x7f7ab810cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.764+0000 7f7abfcfd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ab80ffff0 0x7f7ab810d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.764+0000 7f7abfcfd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ab81056d0 0x7f7ab810d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.764+0000 7f7abfcfd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab810ec60 con 0x7f7ab80ffff0 2023-10-08T14:06:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.764+0000 7f7abfcfd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab819ae00 con 0x7f7ab80ff630 2023-10-08T14:06:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.764+0000 7f7abfcfd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab819afc0 con 0x7f7ab81056d0 2023-10-08T14:06:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.767+0000 7f7abe29a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ab81056d0 0x7f7ab810d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.771+0000 7f7abe29a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ab81056d0 0x7f7ab810d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33048/0 (socket says 172.21.15.57:33048) 2023-10-08T14:06:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.771+0000 7f7abe29a700 1 -- 172.21.15.57:0/1279485117 learned_addr learned my addr 172.21.15.57:0/1279485117 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.773+0000 7f7abd298700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ab80ffff0 0x7f7ab810d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.773+0000 7f7abd298700 1 -- 172.21.15.57:0/1279485117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ab81056d0 msgr2=0x7f7ab810d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.773+0000 7f7abd298700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ab81056d0 0x7f7ab810d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.773+0000 7f7abd298700 1 -- 172.21.15.57:0/1279485117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab80ff630 msgr2=0x7f7ab810cc90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:06:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.773+0000 7f7abd298700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab80ff630 0x7f7ab810cc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.774+0000 7f7abd298700 1 -- 172.21.15.57:0/1279485117 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac0095f0 con 0x7f7ab80ffff0 2023-10-08T14:06:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.774+0000 7f7abd298700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ab80ffff0 0x7f7ab810d1d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000c00 tx=0x7f7aac004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.774+0000 7f7ab6ffd700 1 -- 172.21.15.57:0/1279485117 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac01d070 con 0x7f7ab80ffff0 2023-10-08T14:06:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.775+0000 7f7ab6ffd700 1 -- 172.21.15.57:0/1279485117 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aac003c80 con 0x7f7ab80ffff0 2023-10-08T14:06:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.775+0000 7f7abfcfd700 1 -- 172.21.15.57:0/1279485117 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab819b240 con 0x7f7ab80ffff0 2023-10-08T14:06:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.775+0000 7f7ab6ffd700 1 -- 172.21.15.57:0/1279485117 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00fdc0 con 0x7f7ab80ffff0 2023-10-08T14:06:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.775+0000 7f7abfcfd700 1 -- 172.21.15.57:0/1279485117 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab819b6f0 con 0x7f7ab80ffff0 2023-10-08T14:06:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.781+0000 7f7ab6ffd700 1 -- 172.21.15.57:0/1279485117 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7aac02b440 con 0x7f7ab80ffff0 2023-10-08T14:06:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.782+0000 7f7ab6ffd700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7aa00a17c0 0x7f7aa00a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.782+0000 7f7ab6ffd700 1 -- 172.21.15.57:0/1279485117 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9597+0+0 (secure 0 0 0) 0x7f7aac0cf2e0 con 0x7f7ab80ffff0 2023-10-08T14:06:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.782+0000 7f7abda99700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7aa00a17c0 0x7f7aa00a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.795+0000 7f7acbee6b80 1 Processor -- start 2023-10-08T14:06:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.796+0000 7f7acbee6b80 1 -- start start 2023-10-08T14:06:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.796+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f349c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.796+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34a1a0 0x55967f343b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.796+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f344510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.796+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f2ac400 con 0x55967f3440b0 2023-10-08T14:06:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.796+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f29dbc0 con 0x55967f34a1a0 2023-10-08T14:06:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.796+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f29b4e0 con 0x55967f349880 2023-10-08T14:06:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abda99700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f349c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abe29a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f344510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abe29a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f344510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60476/0 (socket says 172.21.15.57:60476) 2023-10-08T14:06:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abe29a700 1 -- 172.21.15.57:0/137762562 learned_addr learned my addr 172.21.15.57:0/137762562 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abda99700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7aa00a17c0 0x7f7aa00a3c80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800adf0 tx=0x7f7aa800c710 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abe29a700 1 -- 172.21.15.57:0/137762562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 msgr2=0x55967f349c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abe29a700 1 --2- 172.21.15.57:0/137762562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f349c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abe29a700 1 -- 172.21.15.57:0/137762562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34a1a0 msgr2=0x55967f343b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abe29a700 1 --2- 172.21.15.57:0/137762562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34a1a0 0x55967f343b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abe29a700 1 -- 172.21.15.57:0/137762562 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55967f0c1d80 con 0x55967f3440b0 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.797+0000 7f7abe29a700 1 --2- 172.21.15.57:0/137762562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f344510 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0007880 tx=0x7f7ab0007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f57c44358cfa497 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.798+0000 7f7a9ffff700 1 -- 172.21.15.57:0/137762562 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00125f0 con 0x55967f3440b0 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.798+0000 7f7a9ffff700 1 -- 172.21.15.57:0/137762562 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab0012c10 con 0x55967f3440b0 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.798+0000 7f7a9ffff700 1 -- 172.21.15.57:0/137762562 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0013de0 con 0x55967f3440b0 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.799+0000 7f7acbee6b80 1 -- 172.21.15.57:0/137762562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 msgr2=0x55967f344510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.799+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/137762562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f344510 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0007880 tx=0x7f7ab0007b90 comp rx=0 tx=0).stop 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.799+0000 7f7acbee6b80 1 -- 172.21.15.57:0/137762562 shutdown_connections 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.799+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/137762562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f344510 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.799+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/137762562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f349c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.799+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/137762562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34a1a0 0x55967f343b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.799+0000 7f7acbee6b80 1 -- 172.21.15.57:0/137762562 >> 172.21.15.57:0/137762562 conn(0x55967f2b8200 msgr2=0x55967f33eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.814+0000 7f7acbee6b80 1 -- 172.21.15.57:0/137762562 shutdown_connections 2023-10-08T14:06:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.817+0000 7f7acbee6b80 1 -- 172.21.15.57:0/137762562 wait complete. 2023-10-08T14:06:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.817+0000 7f7acbee6b80 1 -- start start 2023-10-08T14:06:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.817+0000 7f7acbee6b80 1 Processor -- start 2023-10-08T14:06:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.824+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f34d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.824+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f34db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.824+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 0x55967f359d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.824+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f296dd0 con 0x55967f3440b0 2023-10-08T14:06:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.824+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f2336b0 con 0x55967f34e090 2023-10-08T14:06:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.824+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f233350 con 0x55967f349880 2023-10-08T14:06:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.824+0000 7f7acbee6b80 10 client.0 _ll_register_callbacks cb 0x55967f0907a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:06:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.825+0000 7f7abda99700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f34d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.825+0000 7f7abda99700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f34d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60482/0 (socket says 172.21.15.57:60482) 2023-10-08T14:06:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.825+0000 7f7abda99700 1 -- 172.21.15.57:0/1674193880 learned_addr learned my addr 172.21.15.57:0/1674193880 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.828+0000 7f7abe29a700 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 0x55967f359d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.829+0000 7f7abd298700 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f34db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.829+0000 7f7abe29a700 1 -- 172.21.15.57:0/1674193880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 msgr2=0x55967f34db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.829+0000 7f7abe29a700 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f34db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.829+0000 7f7abe29a700 1 -- 172.21.15.57:0/1674193880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 msgr2=0x55967f34d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.829+0000 7f7abe29a700 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f34d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.829+0000 7f7abe29a700 1 -- 172.21.15.57:0/1674193880 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab000e040 con 0x55967f34e090 2023-10-08T14:06:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.836+0000 7f7abe29a700 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 0x55967f359d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0007880 tx=0x7f7ab0013c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c2ad84f311f2ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.836+0000 7f7a9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2023-10-08T14:06:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.845+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/1674193880 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0013780 con 0x55967f34e090 2023-10-08T14:06:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.845+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/1674193880 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9597+0+0 (secure 0 0 0) 0x7f7ab0013480 con 0x55967f34e090 2023-10-08T14:06:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.846+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1674193880 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55967f25ee50 con 0x55967f34e090 2023-10-08T14:06:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.851+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/1674193880 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ab0012d80 con 0x55967f34e090 2023-10-08T14:06:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.854+0000 7f7acbee6b80 10 client.26897 fetch_fsmap learned FSMap version 50 2023-10-08T14:06:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.854+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1674193880 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55967f0aac00 con 0x55967f34e090 2023-10-08T14:06:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.856+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/1674193880 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55967f25ee50 con 0x55967f34e090 2023-10-08T14:06:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.856+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/1674193880 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 50) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f7ab0031a30 con 0x55967f34e090 2023-10-08T14:06:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.856+0000 7f7acbee6b80 10 client.26897 fetch_fsmap finished waiting for FSMap version 50 2023-10-08T14:06:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.857+0000 7f7acbee6b80 10 client.26897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:06:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.857+0000 7f7acbee6b80 20 client.26897 populate_metadata read hostname 'smithi057' 2023-10-08T14:06:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.857+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55967f35e1a0 0x55967f3605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.857+0000 7f7acbee6b80 4 client.26897 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:06:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.857+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1674193880 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55967f28fdb0 con 0x55967f35e1a0 2023-10-08T14:06:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.858+0000 7f7abda99700 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55967f35e1a0 0x55967f3605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.862+0000 7f7a9d7fa700 10 client.26897 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.862+0000 7f7abda99700 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55967f35e1a0 0x55967f3605c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d17aee4af57caa0b server_cookie=492dc39992536888 in_seq=0 out_seq=0 2023-10-08T14:06:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.867+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/1674193880 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55967f28fdb0 con 0x55967f35e1a0 2023-10-08T14:06:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.867+0000 7f7a9d7fa700 10 client.26897 handle_command_reply: tid=0 2023-10-08T14:06:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.867+0000 7f7acbee6b80 1 client.26897 shutdown 2023-10-08T14:06:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1674193880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 msgr2=0x55967f359d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 0x55967f359d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0007880 tx=0x7f7ab0013c90 comp rx=0 tx=0).stop 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1674193880 shutdown_connections 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55967f35e1a0 0x55967f3605c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 0x55967f359d80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f34d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/1674193880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f34db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1674193880 >> 172.21.15.57:0/1674193880 conn(0x55967f2b8200 msgr2=0x55967f33e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7a9d7fa700 0 client.26897 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1674193880 shutdown_connections 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1674193880 wait complete. 2023-10-08T14:06:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.868+0000 7f7acbee6b80 20 client.26897 trim_cache size 0 max 16384 2023-10-08T14:06:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.897+0000 7f7acbee6b80 1 Processor -- start 2023-10-08T14:06:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.897+0000 7f7acbee6b80 1 -- start start 2023-10-08T14:06:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.897+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f3440b0 0x55967f343b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.897+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f349880 0x55967f354cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.897+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f34e090 0x55967f34f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.897+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f1178c0 con 0x55967f349880 2023-10-08T14:06:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.897+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f25ee80 con 0x55967f3440b0 2023-10-08T14:06:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.897+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f3406a0 con 0x55967f34e090 2023-10-08T14:06:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.900+0000 7f7abda99700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f3440b0 0x55967f343b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.902+0000 7f7abd298700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f349880 0x55967f354cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.902+0000 7f7abd298700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f349880 0x55967f354cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60490/0 (socket says 172.21.15.57:60490) 2023-10-08T14:06:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.902+0000 7f7abd298700 1 -- 172.21.15.57:0/1771922138 learned_addr learned my addr 172.21.15.57:0/1771922138 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.902+0000 7f7abe29a700 1 --2- 172.21.15.57:0/1771922138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f34e090 0x55967f34f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.903+0000 7f7abe29a700 1 -- 172.21.15.57:0/1771922138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f3440b0 msgr2=0x55967f343b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.903+0000 7f7abe29a700 1 --2- 172.21.15.57:0/1771922138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f3440b0 0x55967f343b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.903+0000 7f7abe29a700 1 -- 172.21.15.57:0/1771922138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f349880 msgr2=0x55967f354cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.903+0000 7f7abe29a700 1 --2- 172.21.15.57:0/1771922138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f349880 0x55967f354cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.903+0000 7f7abe29a700 1 -- 172.21.15.57:0/1771922138 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55967f0e81f0 con 0x55967f34e090 2023-10-08T14:06:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.903+0000 7f7abe29a700 1 --2- 172.21.15.57:0/1771922138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f34e090 0x55967f34f5c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0013a30 tx=0x7f7ab0001ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94fd315a2173b662 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.904+0000 7f7ab4ff9700 1 -- 172.21.15.57:0/1771922138 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0013480 con 0x55967f34e090 2023-10-08T14:06:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.904+0000 7f7ab4ff9700 1 -- 172.21.15.57:0/1771922138 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab0006fe0 con 0x55967f34e090 2023-10-08T14:06:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.904+0000 7f7ab4ff9700 1 -- 172.21.15.57:0/1771922138 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0012a50 con 0x55967f34e090 2023-10-08T14:06:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.905+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1771922138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f34e090 msgr2=0x55967f34f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.905+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/1771922138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f34e090 0x55967f34f5c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0013a30 tx=0x7f7ab0001ef0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.905+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1771922138 shutdown_connections 2023-10-08T14:06:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.905+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/1771922138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f34e090 0x55967f34f5c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.905+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/1771922138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f3440b0 0x55967f343b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.905+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/1771922138 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f349880 0x55967f354cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.905+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1771922138 >> 172.21.15.57:0/1771922138 conn(0x55967f2b8200 msgr2=0x55967f34eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.905+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1771922138 shutdown_connections 2023-10-08T14:06:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.905+0000 7f7acbee6b80 1 -- 172.21.15.57:0/1771922138 wait complete. 2023-10-08T14:06:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.905+0000 7f7acbee6b80 1 -- start start 2023-10-08T14:06:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.906+0000 7f7acbee6b80 1 Processor -- start 2023-10-08T14:06:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.906+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f3a6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.906+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f3a6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.906+0000 7f7acbee6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 0x55967f3b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.906+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f3b14e0 con 0x55967f3440b0 2023-10-08T14:06:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.906+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f3b1620 con 0x55967f34e090 2023-10-08T14:06:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.906+0000 7f7acbee6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f3b17b0 con 0x55967f349880 2023-10-08T14:06:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.906+0000 7f7acbee6b80 10 client.0 _ll_register_callbacks cb 0x7f7ab0013670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:06:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.907+0000 7f7abd298700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f3a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.907+0000 7f7abd298700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f3a6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33076/0 (socket says 172.21.15.57:33076) 2023-10-08T14:06:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.907+0000 7f7abd298700 1 -- 172.21.15.57:0/2550253302 learned_addr learned my addr 172.21.15.57:0/2550253302 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.909+0000 7f7abe29a700 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 0x55967f3b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.909+0000 7f7abd298700 1 -- 172.21.15.57:0/2550253302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 msgr2=0x55967f3b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.909+0000 7f7abd298700 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 0x55967f3b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.909+0000 7f7abd298700 1 -- 172.21.15.57:0/2550253302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 msgr2=0x55967f3a6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.909+0000 7f7abd298700 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f3a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.909+0000 7f7abd298700 1 -- 172.21.15.57:0/2550253302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55967f0aac00 con 0x55967f349880 2023-10-08T14:06:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.909+0000 7f7abd298700 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f3a6650 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009990 tx=0x7f7aac0ccb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9d50226d7afb225 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.910+0000 7f7a9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:06:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.910+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/2550253302 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac09bec0 con 0x55967f349880 2023-10-08T14:06:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.910+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/2550253302 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9597+0+0 (secure 0 0 0) 0x7f7aac0a0010 con 0x55967f349880 2023-10-08T14:06:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.910+0000 7f7acbee6b80 1 -- 172.21.15.57:0/2550253302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ab0012c10 con 0x55967f349880 2023-10-08T14:06:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.911+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/2550253302 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aac0a41c0 con 0x55967f349880 2023-10-08T14:06:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.924+0000 7f7acbee6b80 10 client.26947 fetch_fsmap learned FSMap version 50 2023-10-08T14:06:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.924+0000 7f7acbee6b80 1 -- 172.21.15.57:0/2550253302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55967f0c0250 con 0x55967f349880 2023-10-08T14:06:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.924+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/2550253302 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab0012c10 con 0x55967f349880 2023-10-08T14:06:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.924+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/2550253302 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 50) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f7aac00f460 con 0x55967f349880 2023-10-08T14:06:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.924+0000 7f7acbee6b80 10 client.26947 fetch_fsmap finished waiting for FSMap version 50 2023-10-08T14:06:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.924+0000 7f7acbee6b80 10 client.26947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:06:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.924+0000 7f7acbee6b80 20 client.26947 populate_metadata read hostname 'smithi057' 2023-10-08T14:06:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.924+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55967f3b2d90 0x55967f3b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.924+0000 7f7acbee6b80 4 client.26947 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:06:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.924+0000 7f7acbee6b80 1 -- 172.21.15.57:0/2550253302 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55967f28fad0 con 0x55967f3b2d90 2023-10-08T14:06:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.926+0000 7f7abda99700 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55967f3b2d90 0x55967f3b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.927+0000 7f7a9d7fa700 10 client.26947 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:12.927+0000 7f7abda99700 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55967f3b2d90 0x55967f3b51b0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ca8d2934cb7d06 server_cookie=22ab1dc295b1c895 in_seq=0 out_seq=0 2023-10-08T14:06:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:13 smithi057 ceph-mon[99476]: pgmap v756: 129 pgs: 45 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 689 KiB/s wr, 245 op/s 2023-10-08T14:06:14.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:13 smithi057 ceph-mon[99476]: osdmap e110: 12 total, 12 up, 12 in 2023-10-08T14:06:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:13 smithi099 ceph-mon[103849]: pgmap v756: 129 pgs: 45 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 689 KiB/s wr, 245 op/s 2023-10-08T14:06:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:13 smithi099 ceph-mon[103849]: osdmap e110: 12 total, 12 up, 12 in 2023-10-08T14:06:14.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:13 smithi115 ceph-mon[104408]: pgmap v756: 129 pgs: 45 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 689 KiB/s wr, 245 op/s 2023-10-08T14:06:14.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:13 smithi115 ceph-mon[104408]: osdmap e110: 12 total, 12 up, 12 in 2023-10-08T14:06:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.599+0000 7f7a9d7fa700 1 -- 172.21.15.57:0/2550253302 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55967f28fad0 con 0x55967f3b2d90 2023-10-08T14:06:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.599+0000 7f7a9d7fa700 10 client.26947 handle_command_reply: tid=0 2023-10-08T14:06:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.601+0000 7f7acbee6b80 1 client.26947 shutdown 2023-10-08T14:06:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.602+0000 7f7acbee6b80 1 -- 172.21.15.57:0/2550253302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 msgr2=0x55967f3a6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.602+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f3a6650 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009990 tx=0x7f7aac0ccb00 comp rx=0 tx=0).stop 2023-10-08T14:06:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.602+0000 7f7acbee6b80 1 -- 172.21.15.57:0/2550253302 shutdown_connections 2023-10-08T14:06:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.602+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x55967f3b2d90 0x55967f3b51b0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.602+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55967f34e090 0x55967f3b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.602+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55967f3440b0 0x55967f3a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.602+0000 7f7acbee6b80 1 --2- 172.21.15.57:0/2550253302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55967f349880 0x55967f3a6650 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009990 tx=0x7f7aac0ccb00 comp rx=0 tx=0).stop 2023-10-08T14:06:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.602+0000 7f7acbee6b80 1 -- 172.21.15.57:0/2550253302 >> 172.21.15.57:0/2550253302 conn(0x55967f2b8200 msgr2=0x55967f348df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.602+0000 7f7a9d7fa700 0 client.26947 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.610+0000 7f7acbee6b80 1 -- 172.21.15.57:0/2550253302 shutdown_connections 2023-10-08T14:06:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.610+0000 7f7acbee6b80 1 -- 172.21.15.57:0/2550253302 wait complete. 2023-10-08T14:06:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.617+0000 7f7acbee6b80 20 client.26947 trim_cache size 0 max 16384 2023-10-08T14:06:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:06:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:06:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:06:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:06:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.625+0000 7f7abfcfd700 1 -- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7aa00a17c0 msgr2=0x7f7aa00a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.625+0000 7f7abfcfd700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7aa00a17c0 0x7f7aa00a3c80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800adf0 tx=0x7f7aa800c710 comp rx=0 tx=0).stop 2023-10-08T14:06:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.625+0000 7f7abfcfd700 1 -- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ab80ffff0 msgr2=0x7f7ab810d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.625+0000 7f7abfcfd700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ab80ffff0 0x7f7ab810d1d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000c00 tx=0x7f7aac004320 comp rx=0 tx=0).stop 2023-10-08T14:06:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.629+0000 7f7abfcfd700 1 -- 172.21.15.57:0/1279485117 shutdown_connections 2023-10-08T14:06:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.629+0000 7f7abfcfd700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ab81056d0 0x7f7ab810d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.629+0000 7f7abfcfd700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7aa00a17c0 0x7f7aa00a3c80 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800adf0 tx=0x7f7aa800c710 comp rx=0 tx=0).stop 2023-10-08T14:06:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.629+0000 7f7abfcfd700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab80ff630 0x7f7ab810cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.629+0000 7f7abfcfd700 1 --2- 172.21.15.57:0/1279485117 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7ab80ffff0 0x7f7ab810d1d0 secure :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000c00 tx=0x7f7aac004320 comp rx=0 tx=0).stop 2023-10-08T14:06:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.629+0000 7f7abfcfd700 1 -- 172.21.15.57:0/1279485117 >> 172.21.15.57:0/1279485117 conn(0x7f7ab8076810 msgr2=0x7f7ab80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.637+0000 7f7abfcfd700 1 -- 172.21.15.57:0/1279485117 shutdown_connections 2023-10-08T14:06:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:14.643+0000 7f7abfcfd700 1 -- 172.21.15.57:0/1279485117 wait complete. 2023-10-08T14:06:16.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:15 smithi057 ceph-mon[99476]: pgmap v758: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 297 op/s 2023-10-08T14:06:16.241 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:06:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T14:06:15.842+0000 7fe4ce904700 -1 calc_pg_upmaps abort due to max <= 0 2023-10-08T14:06:16.241 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:06:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T14:06:15.842+0000 7fe4ce904700 -1 calc_pg_upmaps abort due to max <= 0 2023-10-08T14:06:16.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:15 smithi099 ceph-mon[103849]: pgmap v758: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 297 op/s 2023-10-08T14:06:16.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:15 smithi115 ceph-mon[104408]: pgmap v758: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 297 op/s 2023-10-08T14:06:18.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:18 smithi099 ceph-mon[103849]: pgmap v759: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 547 KiB/s wr, 307 op/s 2023-10-08T14:06:18.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:18 smithi115 ceph-mon[104408]: pgmap v759: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 547 KiB/s wr, 307 op/s 2023-10-08T14:06:18.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:18 smithi057 ceph-mon[99476]: pgmap v759: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 547 KiB/s wr, 307 op/s 2023-10-08T14:06:19.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:19 smithi115 ceph-mon[104408]: osdmap e111: 12 total, 12 up, 12 in 2023-10-08T14:06:19.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:19 smithi115 ceph-mon[104408]: pgmap v761: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 591 KiB/s wr, 236 op/s 2023-10-08T14:06:19.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:19 smithi057 ceph-mon[99476]: osdmap e111: 12 total, 12 up, 12 in 2023-10-08T14:06:19.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:19 smithi057 ceph-mon[99476]: pgmap v761: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 591 KiB/s wr, 236 op/s 2023-10-08T14:06:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:19 smithi099 ceph-mon[103849]: osdmap e111: 12 total, 12 up, 12 in 2023-10-08T14:06:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:19 smithi099 ceph-mon[103849]: pgmap v761: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 591 KiB/s wr, 236 op/s 2023-10-08T14:06:22.237 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:21 smithi057 ceph-mon[99476]: pgmap v762: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 842 KiB/s wr, 262 op/s 2023-10-08T14:06:22.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:21 smithi099 ceph-mon[103849]: pgmap v762: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 842 KiB/s wr, 262 op/s 2023-10-08T14:06:22.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:21 smithi115 ceph-mon[104408]: pgmap v762: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 842 KiB/s wr, 262 op/s 2023-10-08T14:06:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:24 smithi099 ceph-mon[103849]: pgmap v763: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 740 KiB/s wr, 243 op/s 2023-10-08T14:06:24.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:24 smithi115 ceph-mon[104408]: pgmap v763: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 740 KiB/s wr, 243 op/s 2023-10-08T14:06:24.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:24 smithi057 ceph-mon[99476]: pgmap v763: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 740 KiB/s wr, 243 op/s 2023-10-08T14:06:26.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:26 smithi115 ceph-mon[104408]: pgmap v764: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 414 KiB/s wr, 162 op/s 2023-10-08T14:06:26.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:26 smithi057 ceph-mon[99476]: pgmap v764: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 414 KiB/s wr, 162 op/s 2023-10-08T14:06:26.526 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:26 smithi099 ceph-mon[103849]: pgmap v764: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 414 KiB/s wr, 162 op/s 2023-10-08T14:06:27.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:27 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:06:27.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:27 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:06:27.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:27 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:06:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.880+0000 7fd93e646700 1 -- 172.21.15.57:0/3342936732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd938069a90 msgr2=0x7fd938103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.880+0000 7fd93e646700 1 --2- 172.21.15.57:0/3342936732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd938069a90 0x7fd938103180 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0098f0 tx=0x7fd92c009c00 comp rx=0 tx=0).stop 2023-10-08T14:06:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.887+0000 7fd93e646700 1 -- 172.21.15.57:0/3342936732 shutdown_connections 2023-10-08T14:06:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.887+0000 7fd93e646700 1 --2- 172.21.15.57:0/3342936732 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd938069a90 0x7fd938103180 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.887+0000 7fd93e646700 1 --2- 172.21.15.57:0/3342936732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd938105620 0x7fd938068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.887+0000 7fd93e646700 1 --2- 172.21.15.57:0/3342936732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd938105a90 0x7fd9380693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.887+0000 7fd93e646700 1 -- 172.21.15.57:0/3342936732 >> 172.21.15.57:0/3342936732 conn(0x7fd9380764c0 msgr2=0x7fd9380768d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.893+0000 7fd93e646700 1 -- 172.21.15.57:0/3342936732 shutdown_connections 2023-10-08T14:06:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.893+0000 7fd93e646700 1 -- 172.21.15.57:0/3342936732 wait complete. 2023-10-08T14:06:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.894+0000 7fd93e646700 1 Processor -- start 2023-10-08T14:06:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.894+0000 7fd93e646700 1 -- start start 2023-10-08T14:06:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd93e646700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd938105620 0x7fd938071e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd93e646700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd938105a90 0x7fd9380723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd93e646700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd93806af50 0x7fd93806b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd93e646700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93806bbb0 con 0x7fd938105a90 2023-10-08T14:06:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd93e646700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93806bd20 con 0x7fd938105620 2023-10-08T14:06:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd93e646700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93806f9c0 con 0x7fd93806af50 2023-10-08T14:06:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd937fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd938105620 0x7fd938071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd937fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd938105620 0x7fd938071e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33702/0 (socket says 172.21.15.57:33702) 2023-10-08T14:06:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd937fff700 1 -- 172.21.15.57:0/415871975 learned_addr learned my addr 172.21.15.57:0/415871975 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd937fff700 1 -- 172.21.15.57:0/415871975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd93806af50 msgr2=0x7fd93806b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd937fff700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd93806af50 0x7fd93806b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd937fff700 1 -- 172.21.15.57:0/415871975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd938105a90 msgr2=0x7fd9380723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd937fff700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd938105a90 0x7fd9380723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.895+0000 7fd937fff700 1 -- 172.21.15.57:0/415871975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c009520 con 0x7fd938105620 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.896+0000 7fd937fff700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd938105620 0x7fd938071e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd93000ba50 tx=0x7fd93000be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.896+0000 7fd9357fa700 1 -- 172.21.15.57:0/415871975 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93000cab0 con 0x7fd938105620 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.896+0000 7fd9357fa700 1 -- 172.21.15.57:0/415871975 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd93000cc10 con 0x7fd938105620 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.896+0000 7fd9357fa700 1 -- 172.21.15.57:0/415871975 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930007660 con 0x7fd938105620 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.896+0000 7fd93e646700 1 -- 172.21.15.57:0/415871975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93806fc70 con 0x7fd938105620 2023-10-08T14:06:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.896+0000 7fd93e646700 1 -- 172.21.15.57:0/415871975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd938070130 con 0x7fd938105620 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.899+0000 7fd9357fa700 1 -- 172.21.15.57:0/415871975 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd9300077c0 con 0x7fd938105620 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.901+0000 7fd94a82fb80 1 Processor -- start 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.902+0000 7fd94a82fb80 1 -- start start 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.902+0000 7fd9357fa700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9200a1800 0x7fd9200a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.903+0000 7fd9357fa700 1 -- 172.21.15.57:0/415871975 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd930006e40 con 0x7fd938105620 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.904+0000 7fd93cbe3700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9200a1800 0x7fd9200a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.904+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d9880 0x560fc99d9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.904+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.904+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d40b0 0x560fc99d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.904+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc993c400 con 0x560fc99da1a0 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.904+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc992dbc0 con 0x560fc99d9880 2023-10-08T14:06:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.904+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc992b4e0 con 0x560fc99d40b0 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.905+0000 7fd9377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.905+0000 7fd9377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60518/0 (socket says 172.21.15.57:60518) 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.905+0000 7fd9377fe700 1 -- 172.21.15.57:0/3813981043 learned_addr learned my addr 172.21.15.57:0/3813981043 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.905+0000 7fd9377fe700 1 -- 172.21.15.57:0/3813981043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d40b0 msgr2=0x560fc99d4510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.905+0000 7fd9377fe700 1 --2- 172.21.15.57:0/3813981043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d40b0 0x560fc99d4510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.905+0000 7fd9377fe700 1 -- 172.21.15.57:0/3813981043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d9880 msgr2=0x560fc99d9c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.905+0000 7fd9377fe700 1 --2- 172.21.15.57:0/3813981043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d9880 0x560fc99d9c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.905+0000 7fd9377fe700 1 -- 172.21.15.57:0/3813981043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fc9751d80 con 0x560fc99da1a0 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.906+0000 7fd9377fe700 1 --2- 172.21.15.57:0/3813981043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99d3b70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd9280046e0 tx=0x7fd928009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5091828dc006bae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.906+0000 7fd91e7fc700 1 -- 172.21.15.57:0/3813981043 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92800eb60 con 0x560fc99da1a0 2023-10-08T14:06:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.906+0000 7fd91e7fc700 1 -- 172.21.15.57:0/3813981043 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd928004cf0 con 0x560fc99da1a0 2023-10-08T14:06:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.906+0000 7fd91e7fc700 1 -- 172.21.15.57:0/3813981043 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928005700 con 0x560fc99da1a0 2023-10-08T14:06:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.907+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3813981043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 msgr2=0x560fc99d3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.907+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/3813981043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99d3b70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd9280046e0 tx=0x7fd928009ea0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.920+0000 7fd93cbe3700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9200a1800 0x7fd9200a3cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009810 tx=0x7fd92c00bef0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.920+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3813981043 shutdown_connections 2023-10-08T14:06:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.920+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/3813981043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d40b0 0x560fc99d4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.920+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/3813981043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d9880 0x560fc99d9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.920+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/3813981043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99d3b70 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.920+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3813981043 >> 172.21.15.57:0/3813981043 conn(0x560fc9948200 msgr2=0x560fc99ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.922+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3813981043 shutdown_connections 2023-10-08T14:06:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.929+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3813981043 wait complete. 2023-10-08T14:06:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.930+0000 7fd94a82fb80 1 -- start start 2023-10-08T14:06:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.930+0000 7fd94a82fb80 1 Processor -- start 2023-10-08T14:06:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d40b0 0x560fc99dd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d9880 0x560fc99ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99de040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc9926dd0 con 0x560fc99da1a0 2023-10-08T14:06:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc98c36b0 con 0x560fc99d40b0 2023-10-08T14:06:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc98c3350 con 0x560fc99d9880 2023-10-08T14:06:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd94a82fb80 10 client.0 _ll_register_callbacks cb 0x560fc97207a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:06:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd937fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d40b0 0x560fc99dd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd937fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d40b0 0x560fc99dd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33714/0 (socket says 172.21.15.57:33714) 2023-10-08T14:06:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd937fff700 1 -- 172.21.15.57:0/1186541895 learned_addr learned my addr 172.21.15.57:0/1186541895 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd9377fe700 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d9880 0x560fc99ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.931+0000 7fd93cbe3700 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.932+0000 7fd93cbe3700 1 -- 172.21.15.57:0/1186541895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d9880 msgr2=0x560fc99ddb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.932+0000 7fd93cbe3700 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d9880 0x560fc99ddb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.932+0000 7fd93cbe3700 1 -- 172.21.15.57:0/1186541895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d40b0 msgr2=0x560fc99dd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.932+0000 7fd93cbe3700 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d40b0 0x560fc99dd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.932+0000 7fd93cbe3700 1 -- 172.21.15.57:0/1186541895 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9280098a0 con 0x560fc99da1a0 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.932+0000 7fd93cbe3700 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99de040 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009fd0 tx=0x7fd92c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b248dbaed30d61 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.932+0000 7fd9177fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.932+0000 7fd9177fe700 1 -- 172.21.15.57:0/1186541895 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c005240 con 0x560fc99da1a0 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.932+0000 7fd94a82fb80 1 -- 172.21.15.57:0/1186541895 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fc98eee50 con 0x560fc99da1a0 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd9177fe700 1 -- 172.21.15.57:0/1186541895 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd92c0053a0 con 0x560fc99da1a0 2023-10-08T14:06:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd9177fe700 1 -- 172.21.15.57:0/1186541895 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd92c005560 con 0x560fc99da1a0 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd94a82fb80 10 client.17394 fetch_fsmap learned FSMap version 51 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd94a82fb80 1 -- 172.21.15.57:0/1186541895 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x560fc973ac00 con 0x560fc99da1a0 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd9177fe700 1 -- 172.21.15.57:0/1186541895 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd92c035080 con 0x560fc99da1a0 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd9177fe700 1 -- 172.21.15.57:0/1186541895 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 51) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fd92c021580 con 0x560fc99da1a0 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd94a82fb80 10 client.17394 fetch_fsmap finished waiting for FSMap version 51 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd94a82fb80 10 client.17394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd94a82fb80 20 client.17394 populate_metadata read hostname 'smithi057' 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x560fc99edf30 0x560fc99f0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd94a82fb80 4 client.17394 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.933+0000 7fd94a82fb80 1 -- 172.21.15.57:0/1186541895 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fc991fdb0 con 0x560fc99edf30 2023-10-08T14:06:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.934+0000 7fd937fff700 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x560fc99edf30 0x560fc99f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.935+0000 7fd937fff700 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x560fc99edf30 0x560fc99f0350 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2394d077efb210d6 server_cookie=bfe2a7f073793aa in_seq=0 out_seq=0 2023-10-08T14:06:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.935+0000 7fd9177fe700 10 client.17394 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.940+0000 7fd9177fe700 1 -- 172.21.15.57:0/1186541895 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560fc991fdb0 con 0x560fc99edf30 2023-10-08T14:06:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.940+0000 7fd9177fe700 10 client.17394 handle_command_reply: tid=0 2023-10-08T14:06:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.940+0000 7fd94a82fb80 1 client.17394 shutdown 2023-10-08T14:06:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.940+0000 7fd94a82fb80 1 -- 172.21.15.57:0/1186541895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 msgr2=0x560fc99de040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.940+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99de040 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009fd0 tx=0x7fd92c009260 comp rx=0 tx=0).stop 2023-10-08T14:06:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.941+0000 7fd94a82fb80 1 -- 172.21.15.57:0/1186541895 shutdown_connections 2023-10-08T14:06:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.941+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x560fc99edf30 0x560fc99f0350 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.941+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99da1a0 0x560fc99de040 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.941+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d40b0 0x560fc99dd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.941+0000 7fd9177fe700 0 client.17394 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.941+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/1186541895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d9880 0x560fc99ddb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.941+0000 7fd94a82fb80 1 -- 172.21.15.57:0/1186541895 >> 172.21.15.57:0/1186541895 conn(0x560fc9948200 msgr2=0x560fc99ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.941+0000 7fd94a82fb80 1 -- 172.21.15.57:0/1186541895 shutdown_connections 2023-10-08T14:06:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.941+0000 7fd94a82fb80 1 -- 172.21.15.57:0/1186541895 wait complete. 2023-10-08T14:06:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.941+0000 7fd94a82fb80 20 client.17394 trim_cache size 0 max 16384 2023-10-08T14:06:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.988+0000 7fd94a82fb80 1 Processor -- start 2023-10-08T14:06:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.988+0000 7fd94a82fb80 1 -- start start 2023-10-08T14:06:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.988+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d40b0 0x560fc99d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.988+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d9880 0x560fc99d8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.988+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99da1a0 0x560fc99dc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.988+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc97a78c0 con 0x560fc99d9880 2023-10-08T14:06:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.988+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc99d72d0 con 0x560fc99d40b0 2023-10-08T14:06:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.988+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc99c92b0 con 0x560fc99da1a0 2023-10-08T14:06:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.989+0000 7fd9377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d9880 0x560fc99d8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.989+0000 7fd9377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d9880 0x560fc99d8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60532/0 (socket says 172.21.15.57:60532) 2023-10-08T14:06:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.989+0000 7fd9377fe700 1 -- 172.21.15.57:0/2820348406 learned_addr learned my addr 172.21.15.57:0/2820348406 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.989+0000 7fd9377fe700 1 -- 172.21.15.57:0/2820348406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99da1a0 msgr2=0x560fc99dc5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:06:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.989+0000 7fd9377fe700 1 --2- 172.21.15.57:0/2820348406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99da1a0 0x560fc99dc5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.989+0000 7fd9377fe700 1 -- 172.21.15.57:0/2820348406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d40b0 msgr2=0x560fc99d3b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:06:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.989+0000 7fd9377fe700 1 --2- 172.21.15.57:0/2820348406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d40b0 0x560fc99d3b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.989+0000 7fd9377fe700 1 -- 172.21.15.57:0/2820348406 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fc97781f0 con 0x560fc99d9880 2023-10-08T14:06:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.990+0000 7fd9377fe700 1 --2- 172.21.15.57:0/2820348406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d9880 0x560fc99d8f20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd928000c00 tx=0x7fd928005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce60e5093d3933c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.991+0000 7fd91effd700 1 -- 172.21.15.57:0/2820348406 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928005060 con 0x560fc99d9880 2023-10-08T14:06:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.991+0000 7fd91effd700 1 -- 172.21.15.57:0/2820348406 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd928005350 con 0x560fc99d9880 2023-10-08T14:06:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.991+0000 7fd91effd700 1 -- 172.21.15.57:0/2820348406 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928023480 con 0x560fc99d9880 2023-10-08T14:06:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.992+0000 7fd94a82fb80 1 -- 172.21.15.57:0/2820348406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d9880 msgr2=0x560fc99d8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:27.992+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/2820348406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d9880 0x560fc99d8f20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd928000c00 tx=0x7fd928005920 comp rx=0 tx=0).stop 2023-10-08T14:06:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.002+0000 7fd94a82fb80 1 -- 172.21.15.57:0/2820348406 shutdown_connections 2023-10-08T14:06:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.002+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/2820348406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99da1a0 0x560fc99dc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.002+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/2820348406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99d40b0 0x560fc99d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.002+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/2820348406 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d9880 0x560fc99d8f20 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.002+0000 7fd94a82fb80 1 -- 172.21.15.57:0/2820348406 >> 172.21.15.57:0/2820348406 conn(0x560fc9948200 msgr2=0x560fc99dbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.004+0000 7fd94a82fb80 1 -- 172.21.15.57:0/2820348406 shutdown_connections 2023-10-08T14:06:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.005+0000 7fd94a82fb80 1 -- 172.21.15.57:0/2820348406 wait complete. 2023-10-08T14:06:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.005+0000 7fd94a82fb80 1 -- start start 2023-10-08T14:06:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.006+0000 7fd94a82fb80 1 Processor -- start 2023-10-08T14:06:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.006+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d40b0 0x560fc9a3d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.006+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d9880 0x560fc9a3d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.006+0000 7fd94a82fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99da1a0 0x560fc99e2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.006+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc99e3410 con 0x560fc99d40b0 2023-10-08T14:06:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.006+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc99e3580 con 0x560fc99da1a0 2023-10-08T14:06:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.006+0000 7fd94a82fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fc99e3740 con 0x560fc99d9880 2023-10-08T14:06:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.006+0000 7fd94a82fb80 10 client.0 _ll_register_callbacks cb 0x7fd92c00f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:06:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd9377fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d9880 0x560fc9a3d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd937fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d40b0 0x560fc9a3d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd937fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d40b0 0x560fc9a3d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60534/0 (socket says 172.21.15.57:60534) 2023-10-08T14:06:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd937fff700 1 -- 172.21.15.57:0/3615929863 learned_addr learned my addr 172.21.15.57:0/3615929863 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd937fff700 1 -- 172.21.15.57:0/3615929863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d9880 msgr2=0x560fc9a3d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd937fff700 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d9880 0x560fc9a3d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd937fff700 1 -- 172.21.15.57:0/3615929863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99da1a0 msgr2=0x560fc99e2d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd937fff700 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99da1a0 0x560fc99e2d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd937fff700 1 -- 172.21.15.57:0/3615929863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92801c040 con 0x560fc99d40b0 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd937fff700 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d40b0 0x560fc9a3d0f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd930096da0 tx=0x7fd93001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de66f08276d7c1d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.007+0000 7fd9177fe700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.008+0000 7fd9177fe700 1 -- 172.21.15.57:0/3615929863 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9300b2070 con 0x560fc99d40b0 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.008+0000 7fd9177fe700 1 -- 172.21.15.57:0/3615929863 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd930098c40 con 0x560fc99d40b0 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.008+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3615929863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd928004cf0 con 0x560fc99d40b0 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.008+0000 7fd9177fe700 1 -- 172.21.15.57:0/3615929863 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd93000cd80 con 0x560fc99d40b0 2023-10-08T14:06:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.008+0000 7fd94a82fb80 10 client.17400 fetch_fsmap learned FSMap version 51 2023-10-08T14:06:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.008+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3615929863 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x560fc9750250 con 0x560fc99d40b0 2023-10-08T14:06:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.008+0000 7fd9177fe700 1 -- 172.21.15.57:0/3615929863 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9300a6020 con 0x560fc99d40b0 2023-10-08T14:06:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.009+0000 7fd9177fe700 1 -- 172.21.15.57:0/3615929863 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 51) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fd930099d90 con 0x560fc99d40b0 2023-10-08T14:06:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.009+0000 7fd94a82fb80 10 client.17400 fetch_fsmap finished waiting for FSMap version 51 2023-10-08T14:06:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.009+0000 7fd94a82fb80 10 client.17400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:06:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.009+0000 7fd94a82fb80 20 client.17400 populate_metadata read hostname 'smithi057' 2023-10-08T14:06:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.009+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x560fc9a540b0 0x560fc9a56570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.009+0000 7fd94a82fb80 4 client.17400 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:06:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.009+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3615929863 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560fc991fad0 con 0x560fc9a540b0 2023-10-08T14:06:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.014+0000 7fd9377fe700 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x560fc9a540b0 0x560fc9a56570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.015+0000 7fd9177fe700 10 client.17400 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.015+0000 7fd9377fe700 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x560fc9a540b0 0x560fc9a56570 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2fa61da3c7691f9 server_cookie=a934058087dd1365 in_seq=0 out_seq=0 2023-10-08T14:06:28.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:28 smithi115 ceph-mon[104408]: pgmap v765: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 616 KiB/s wr, 150 op/s 2023-10-08T14:06:28.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:28 smithi057 ceph-mon[99476]: pgmap v765: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 616 KiB/s wr, 150 op/s 2023-10-08T14:06:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:28 smithi099 ceph-mon[103849]: pgmap v765: 129 pgs: 70 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 616 KiB/s wr, 150 op/s 2023-10-08T14:06:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.964+0000 7fd9177fe700 1 -- 172.21.15.57:0/3615929863 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560fc991fad0 con 0x560fc9a540b0 2023-10-08T14:06:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.964+0000 7fd9177fe700 10 client.17400 handle_command_reply: tid=0 2023-10-08T14:06:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.965+0000 7fd94a82fb80 1 client.17400 shutdown 2023-10-08T14:06:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.965+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3615929863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d40b0 msgr2=0x560fc9a3d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.965+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d40b0 0x560fc9a3d0f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd930096da0 tx=0x7fd93001c000 comp rx=0 tx=0).stop 2023-10-08T14:06:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.966+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3615929863 shutdown_connections 2023-10-08T14:06:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.966+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x560fc9a540b0 0x560fc9a56570 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.966+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fc99da1a0 0x560fc99e2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.966+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560fc99d40b0 0x560fc9a3d0f0 secure :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd930096da0 tx=0x7fd93001c000 comp rx=0 tx=0).stop 2023-10-08T14:06:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.966+0000 7fd94a82fb80 1 --2- 172.21.15.57:0/3615929863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560fc99d9880 0x560fc9a3d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.966+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3615929863 >> 172.21.15.57:0/3615929863 conn(0x560fc9948200 msgr2=0x560fc99d8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.966+0000 7fd9177fe700 0 client.17400 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.970+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3615929863 shutdown_connections 2023-10-08T14:06:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.975+0000 7fd94a82fb80 1 -- 172.21.15.57:0/3615929863 wait complete. 2023-10-08T14:06:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.981+0000 7fd94a82fb80 20 client.17400 trim_cache size 0 max 16384 2023-10-08T14:06:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:06:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:06:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:06:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:06:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.982+0000 7fd93e646700 1 -- 172.21.15.57:0/415871975 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9200a1800 msgr2=0x7fd9200a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.982+0000 7fd93e646700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9200a1800 0x7fd9200a3cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009810 tx=0x7fd92c00bef0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.982+0000 7fd93e646700 1 -- 172.21.15.57:0/415871975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd938105620 msgr2=0x7fd938071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.982+0000 7fd93e646700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd938105620 0x7fd938071e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd93000ba50 tx=0x7fd93000be10 comp rx=0 tx=0).stop 2023-10-08T14:06:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.983+0000 7fd93e646700 1 -- 172.21.15.57:0/415871975 shutdown_connections 2023-10-08T14:06:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.983+0000 7fd93e646700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd93806af50 0x7fd93806b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.983+0000 7fd93e646700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd9200a1800 0x7fd9200a3cc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.983+0000 7fd93e646700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd938105620 0x7fd938071e30 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd93000ba50 tx=0x7fd93000be10 comp rx=0 tx=0).stop 2023-10-08T14:06:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.983+0000 7fd93e646700 1 --2- 172.21.15.57:0/415871975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd938105a90 0x7fd9380723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.983+0000 7fd93e646700 1 -- 172.21.15.57:0/415871975 >> 172.21.15.57:0/415871975 conn(0x7fd9380764c0 msgr2=0x7fd938102040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:28.996+0000 7fd93e646700 1 -- 172.21.15.57:0/415871975 shutdown_connections 2023-10-08T14:06:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:29.001+0000 7fd93e646700 1 -- 172.21.15.57:0/415871975 wait complete. 2023-10-08T14:06:29.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:29 smithi115 ceph-mon[104408]: pgmap v766: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 593 KiB/s wr, 141 op/s 2023-10-08T14:06:29.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:29 smithi057 ceph-mon[99476]: pgmap v766: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 593 KiB/s wr, 141 op/s 2023-10-08T14:06:30.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:29 smithi099 ceph-mon[103849]: pgmap v766: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 593 KiB/s wr, 141 op/s 2023-10-08T14:06:32.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:31 smithi057 ceph-mon[99476]: pgmap v767: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 578 KiB/s wr, 169 op/s 2023-10-08T14:06:32.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:31 smithi099 ceph-mon[103849]: pgmap v767: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 578 KiB/s wr, 169 op/s 2023-10-08T14:06:32.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:31 smithi115 ceph-mon[104408]: pgmap v767: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 578 KiB/s wr, 169 op/s 2023-10-08T14:06:33.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:33 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:06:33.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:32 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:06:33.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:32 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:06:34.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:34 smithi115 ceph-mon[104408]: pgmap v768: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 570 KiB/s wr, 160 op/s 2023-10-08T14:06:34.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:34 smithi057 ceph-mon[99476]: pgmap v768: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 570 KiB/s wr, 160 op/s 2023-10-08T14:06:34.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:34 smithi099 ceph-mon[103849]: pgmap v768: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 570 KiB/s wr, 160 op/s 2023-10-08T14:06:35.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:06:35.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:06:35.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:35 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:06:35.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:06:35.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:06:35.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:35 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:06:35.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:06:35.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:06:35.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:35 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:06:36.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:36 smithi115 ceph-mon[104408]: pgmap v769: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 787 KiB/s wr, 177 op/s 2023-10-08T14:06:36.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:36 smithi057 ceph-mon[99476]: pgmap v769: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 787 KiB/s wr, 177 op/s 2023-10-08T14:06:36.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:36 smithi099 ceph-mon[103849]: pgmap v769: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 787 KiB/s wr, 177 op/s 2023-10-08T14:06:38.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:38 smithi115 ceph-mon[104408]: pgmap v770: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 692 KiB/s wr, 165 op/s 2023-10-08T14:06:38.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:38 smithi057 ceph-mon[99476]: pgmap v770: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 692 KiB/s wr, 165 op/s 2023-10-08T14:06:38.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:38 smithi099 ceph-mon[103849]: pgmap v770: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 692 KiB/s wr, 165 op/s 2023-10-08T14:06:39.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:39 smithi115 ceph-mon[104408]: pgmap v771: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 508 KiB/s wr, 153 op/s 2023-10-08T14:06:39.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:39 smithi057 ceph-mon[99476]: pgmap v771: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 508 KiB/s wr, 153 op/s 2023-10-08T14:06:40.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:39 smithi099 ceph-mon[103849]: pgmap v771: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 508 KiB/s wr, 153 op/s 2023-10-08T14:06:41.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:41 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:06:41.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:41 smithi115 ceph-mon[104408]: pgmap v772: 129 pgs: 64 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 43 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 612 KiB/s wr, 170 op/s 2023-10-08T14:06:41.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:41 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:06:41.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:41 smithi057 ceph-mon[99476]: pgmap v772: 129 pgs: 64 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 43 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 612 KiB/s wr, 170 op/s 2023-10-08T14:06:42.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:41 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:06:42.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:41 smithi099 ceph-mon[103849]: pgmap v772: 129 pgs: 64 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 43 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 612 KiB/s wr, 170 op/s 2023-10-08T14:06:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.270+0000 7f7f37082700 1 -- 172.21.15.57:0/3920735641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f30119fd0 msgr2=0x7f7f3011c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.270+0000 7f7f37082700 1 --2- 172.21.15.57:0/3920735641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f30119fd0 0x7f7f3011c3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f200099c0 tx=0x7f7f20009cd0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.271+0000 7f7f37082700 1 -- 172.21.15.57:0/3920735641 shutdown_connections 2023-10-08T14:06:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.271+0000 7f7f37082700 1 --2- 172.21.15.57:0/3920735641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f30119fd0 0x7f7f3011c3c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.271+0000 7f7f37082700 1 --2- 172.21.15.57:0/3920735641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f3010ef10 0x7f7f301077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.271+0000 7f7f37082700 1 --2- 172.21.15.57:0/3920735641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f3010f380 0x7f7f30107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.271+0000 7f7f37082700 1 -- 172.21.15.57:0/3920735641 >> 172.21.15.57:0/3920735641 conn(0x7f7f3006b7c0 msgr2=0x7f7f3006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.285+0000 7f7f37082700 1 -- 172.21.15.57:0/3920735641 shutdown_connections 2023-10-08T14:06:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.285+0000 7f7f37082700 1 -- 172.21.15.57:0/3920735641 wait complete. 2023-10-08T14:06:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.285+0000 7f7f37082700 1 Processor -- start 2023-10-08T14:06:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.287+0000 7f7f37082700 1 -- start start 2023-10-08T14:06:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.287+0000 7f7f37082700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f3010ef10 0x7f7f30118360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.287+0000 7f7f37082700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3010f380 0x7f7f301188e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.287+0000 7f7f37082700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f30119fd0 0x7f7f301114b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.287+0000 7f7f37082700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f30111c80 con 0x7f7f30119fd0 2023-10-08T14:06:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.287+0000 7f7f37082700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f30111df0 con 0x7f7f3010ef10 2023-10-08T14:06:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.287+0000 7f7f37082700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f30111fb0 con 0x7f7f3010f380 2023-10-08T14:06:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.287+0000 7f7f34e1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f3010ef10 0x7f7f30118360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.287+0000 7f7f34e1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f3010ef10 0x7f7f30118360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33744/0 (socket says 172.21.15.57:33744) 2023-10-08T14:06:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.287+0000 7f7f34e1e700 1 -- 172.21.15.57:0/386525461 learned_addr learned my addr 172.21.15.57:0/386525461 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.288+0000 7f7f34e1e700 1 -- 172.21.15.57:0/386525461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3010f380 msgr2=0x7f7f301188e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:06:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.288+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3010f380 0x7f7f301188e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.288+0000 7f7f34e1e700 1 -- 172.21.15.57:0/386525461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f30119fd0 msgr2=0x7f7f301114b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:06:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.288+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f30119fd0 0x7f7f301114b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.288+0000 7f7f34e1e700 1 -- 172.21.15.57:0/386525461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f200095f0 con 0x7f7f3010ef10 2023-10-08T14:06:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.288+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f3010ef10 0x7f7f30118360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400b530 tx=0x7f7f2400b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.291+0000 7f7f2dffb700 1 -- 172.21.15.57:0/386525461 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f24011830 con 0x7f7f3010ef10 2023-10-08T14:06:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.291+0000 7f7f2dffb700 1 -- 172.21.15.57:0/386525461 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f24011e50 con 0x7f7f3010ef10 2023-10-08T14:06:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.291+0000 7f7f2dffb700 1 -- 172.21.15.57:0/386525461 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2400f2e0 con 0x7f7f3010ef10 2023-10-08T14:06:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.291+0000 7f7f37082700 1 -- 172.21.15.57:0/386525461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f30112290 con 0x7f7f3010ef10 2023-10-08T14:06:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.291+0000 7f7f37082700 1 -- 172.21.15.57:0/386525461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f30116220 con 0x7f7f3010ef10 2023-10-08T14:06:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.296+0000 7f7f4326bb80 1 Processor -- start 2023-10-08T14:06:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.303+0000 7f7f2dffb700 1 -- 172.21.15.57:0/386525461 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7f2400f440 con 0x7f7f3010ef10 2023-10-08T14:06:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.311+0000 7f7f4326bb80 1 -- start start 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.312+0000 7f7f2dffb700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7f180a17c0 0x7f7f180a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.312+0000 7f7f2dffb700 1 -- 172.21.15.57:0/386525461 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7f240cde40 con 0x7f7f3010ef10 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.313+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61dec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.313+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61df1a0 0x561bb61d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.313+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb61d9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.313+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6141400 con 0x561bb61d90b0 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.313+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6132bc0 con 0x561bb61de880 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.313+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb61304e0 con 0x561bb61df1a0 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.313+0000 7f7f34e1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61dec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.313+0000 7f7f34e1e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61dec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33752/0 (socket says 172.21.15.57:33752) 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.313+0000 7f7f34e1e700 1 -- 172.21.15.57:0/4246020497 learned_addr learned my addr 172.21.15.57:0/4246020497 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.313+0000 7f7f3561f700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7f180a17c0 0x7f7f180a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.314+0000 7f7f34e1e700 1 -- 172.21.15.57:0/4246020497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61df1a0 msgr2=0x561bb61d8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.314+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/4246020497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61df1a0 0x561bb61d8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.314+0000 7f7f34e1e700 1 -- 172.21.15.57:0/4246020497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 msgr2=0x561bb61d9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.314+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/4246020497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb61d9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.314+0000 7f7f34e1e700 1 -- 172.21.15.57:0/4246020497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bb5f56d80 con 0x561bb61de880 2023-10-08T14:06:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.314+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/4246020497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61dec60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400bcf0 tx=0x7f7f2409b270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d510fd6e489c387c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.314+0000 7f7f16ffd700 1 -- 172.21.15.57:0/4246020497 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2409ec40 con 0x561bb61de880 2023-10-08T14:06:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.315+0000 7f7f16ffd700 1 -- 172.21.15.57:0/4246020497 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f240a2e80 con 0x561bb61de880 2023-10-08T14:06:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.315+0000 7f7f16ffd700 1 -- 172.21.15.57:0/4246020497 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2409a240 con 0x561bb61de880 2023-10-08T14:06:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 -- 172.21.15.57:0/4246020497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 msgr2=0x561bb61dec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/4246020497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61dec60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400bcf0 tx=0x7f7f2409b270 comp rx=0 tx=0).stop 2023-10-08T14:06:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f3561f700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7f180a17c0 0x7f7f180a3c80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7f20006010 tx=0x7f7f20009f10 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 -- 172.21.15.57:0/4246020497 shutdown_connections 2023-10-08T14:06:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/4246020497 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb61d9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/4246020497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61dec60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/4246020497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61df1a0 0x561bb61d8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 -- 172.21.15.57:0/4246020497 >> 172.21.15.57:0/4246020497 conn(0x561bb614d200 msgr2=0x561bb61d3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 -- 172.21.15.57:0/4246020497 shutdown_connections 2023-10-08T14:06:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 -- 172.21.15.57:0/4246020497 wait complete. 2023-10-08T14:06:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 -- start start 2023-10-08T14:06:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.316+0000 7f7f4326bb80 1 Processor -- start 2023-10-08T14:06:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb61e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61de880 0x561bb61e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61df1a0 0x561bb61e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb612bdd0 con 0x561bb61d90b0 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb60c86b0 con 0x561bb61df1a0 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb60c8350 con 0x561bb61de880 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f4326bb80 10 client.0 _ll_register_callbacks cb 0x561bb5f257a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f34e1e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb61e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f34e1e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb61e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60560/0 (socket says 172.21.15.57:60560) 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f34e1e700 1 -- 172.21.15.57:0/892460167 learned_addr learned my addr 172.21.15.57:0/892460167 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61de880 0x561bb61e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.317+0000 7f7f3561f700 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61df1a0 0x561bb61e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f2ffff700 1 -- 172.21.15.57:0/892460167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61df1a0 msgr2=0x561bb61e3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61df1a0 0x561bb61e3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f2ffff700 1 -- 172.21.15.57:0/892460167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 msgr2=0x561bb61e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb61e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f2ffff700 1 -- 172.21.15.57:0/892460167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f30116220 con 0x561bb61de880 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61de880 0x561bb61e2ac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f30112b60 tx=0x7f7f2800bb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c842d7e197ce2997 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f0ffff700 1 -- 172.21.15.57:0/892460167 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28011880 con 0x561bb61de880 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f0ffff700 1 -- 172.21.15.57:0/892460167 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7f28006e40 con 0x561bb61de880 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f4326bb80 1 -- 172.21.15.57:0/892460167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bb60f3e50 con 0x561bb61de880 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f0ffff700 1 -- 172.21.15.57:0/892460167 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f280119e0 con 0x561bb61de880 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f4326bb80 10 client.26959 fetch_fsmap learned FSMap version 52 2023-10-08T14:06:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.318+0000 7f7f4326bb80 1 -- 172.21.15.57:0/892460167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x561bb5f3fc00 con 0x561bb61de880 2023-10-08T14:06:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.319+0000 7f7f0ffff700 1 -- 172.21.15.57:0/892460167 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561bb60f3e50 con 0x561bb61de880 2023-10-08T14:06:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.319+0000 7f7f0ffff700 1 -- 172.21.15.57:0/892460167 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 52) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f7f28008030 con 0x561bb61de880 2023-10-08T14:06:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.319+0000 7f7f4326bb80 10 client.26959 fetch_fsmap finished waiting for FSMap version 52 2023-10-08T14:06:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.319+0000 7f7f4326bb80 10 client.26959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:06:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.319+0000 7f7f4326bb80 20 client.26959 populate_metadata read hostname 'smithi057' 2023-10-08T14:06:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.319+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561bb61f2f60 0x561bb61f5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.319+0000 7f7f4326bb80 4 client.26959 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:06:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.319+0000 7f7f4326bb80 1 -- 172.21.15.57:0/892460167 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bb6124db0 con 0x561bb61f2f60 2023-10-08T14:06:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.323+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561bb61f2f60 0x561bb61f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.324+0000 7f7f0ffff700 10 client.26959 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.324+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561bb61f2f60 0x561bb61f5380 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95bab36651c7b442 server_cookie=425f044f4eacf683 in_seq=0 out_seq=0 2023-10-08T14:06:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.325+0000 7f7f0ffff700 1 -- 172.21.15.57:0/892460167 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561bb6124db0 con 0x561bb61f2f60 2023-10-08T14:06:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.325+0000 7f7f0ffff700 10 client.26959 handle_command_reply: tid=0 2023-10-08T14:06:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.325+0000 7f7f4326bb80 1 client.26959 shutdown 2023-10-08T14:06:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.326+0000 7f7f4326bb80 1 -- 172.21.15.57:0/892460167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61de880 msgr2=0x561bb61e2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.326+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61de880 0x561bb61e2ac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f30112b60 tx=0x7f7f2800bb30 comp rx=0 tx=0).stop 2023-10-08T14:06:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.327+0000 7f7f4326bb80 1 -- 172.21.15.57:0/892460167 shutdown_connections 2023-10-08T14:06:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.327+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561bb61f2f60 0x561bb61f5380 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.327+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61df1a0 0x561bb61e3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.327+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb61e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.327+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/892460167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61de880 0x561bb61e2ac0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.327+0000 7f7f4326bb80 1 -- 172.21.15.57:0/892460167 >> 172.21.15.57:0/892460167 conn(0x561bb614d200 msgr2=0x561bb61d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.329+0000 7f7f0ffff700 0 client.26959 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.329+0000 7f7f4326bb80 1 -- 172.21.15.57:0/892460167 shutdown_connections 2023-10-08T14:06:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.329+0000 7f7f4326bb80 1 -- 172.21.15.57:0/892460167 wait complete. 2023-10-08T14:06:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.331+0000 7f7f4326bb80 20 client.26959 trim_cache size 0 max 16384 2023-10-08T14:06:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f4326bb80 1 Processor -- start 2023-10-08T14:06:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f4326bb80 1 -- start start 2023-10-08T14:06:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61d90b0 0x561bb61d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61e9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61df1a0 0x561bb61ddd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb5fac8c0 con 0x561bb61df1a0 2023-10-08T14:06:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6124db0 con 0x561bb61de880 2023-10-08T14:06:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb60f3e50 con 0x561bb61d90b0 2023-10-08T14:06:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f3561f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61df1a0 0x561bb61ddd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f3561f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61df1a0 0x561bb61ddd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60568/0 (socket says 172.21.15.57:60568) 2023-10-08T14:06:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.365+0000 7f7f3561f700 1 -- 172.21.15.57:0/1070813149 learned_addr learned my addr 172.21.15.57:0/1070813149 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.366+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/1070813149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61e9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.366+0000 7f7f2ffff700 1 -- 172.21.15.57:0/1070813149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61d90b0 msgr2=0x561bb61d8b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:06:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.366+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/1070813149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61d90b0 0x561bb61d8b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.366+0000 7f7f2ffff700 1 -- 172.21.15.57:0/1070813149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61df1a0 msgr2=0x561bb61ddd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.366+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/1070813149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61df1a0 0x561bb61ddd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.366+0000 7f7f2ffff700 1 -- 172.21.15.57:0/1070813149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bb61f9fd0 con 0x561bb61de880 2023-10-08T14:06:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.366+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/1070813149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61e9c50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f28004c20 tx=0x7f7f2800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1534db522db7c86 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.372+0000 7f7f177fe700 1 -- 172.21.15.57:0/1070813149 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f280193a0 con 0x561bb61de880 2023-10-08T14:06:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.372+0000 7f7f177fe700 1 -- 172.21.15.57:0/1070813149 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f28019500 con 0x561bb61de880 2023-10-08T14:06:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.372+0000 7f7f177fe700 1 -- 172.21.15.57:0/1070813149 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28011cf0 con 0x561bb61de880 2023-10-08T14:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.373+0000 7f7f4326bb80 1 -- 172.21.15.57:0/1070813149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 msgr2=0x561bb61e9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.373+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/1070813149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61e9c50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f28004c20 tx=0x7f7f2800bfd0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.373+0000 7f7f4326bb80 1 -- 172.21.15.57:0/1070813149 shutdown_connections 2023-10-08T14:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.373+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/1070813149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61df1a0 0x561bb61ddd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.373+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/1070813149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61d90b0 0x561bb61d8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.373+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/1070813149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb61e9c50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.373+0000 7f7f4326bb80 1 -- 172.21.15.57:0/1070813149 >> 172.21.15.57:0/1070813149 conn(0x561bb614d200 msgr2=0x561bb61dd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.379+0000 7f7f4326bb80 1 -- 172.21.15.57:0/1070813149 shutdown_connections 2023-10-08T14:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.379+0000 7f7f4326bb80 1 -- 172.21.15.57:0/1070813149 wait complete. 2023-10-08T14:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.379+0000 7f7f4326bb80 1 -- start start 2023-10-08T14:06:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.380+0000 7f7f4326bb80 1 Processor -- start 2023-10-08T14:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.380+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb623ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.380+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb623b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.380+0000 7f7f4326bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61df1a0 0x561bb623b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.380+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6245ef0 con 0x561bb61d90b0 2023-10-08T14:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.380+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6246060 con 0x561bb61de880 2023-10-08T14:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.380+0000 7f7f4326bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6246220 con 0x561bb61df1a0 2023-10-08T14:06:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.380+0000 7f7f4326bb80 10 client.0 _ll_register_callbacks cb 0x7f7f28009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.385+0000 7f7f34e1e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb623ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.385+0000 7f7f34e1e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb623ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60576/0 (socket says 172.21.15.57:60576) 2023-10-08T14:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.385+0000 7f7f34e1e700 1 -- 172.21.15.57:0/3613778000 learned_addr learned my addr 172.21.15.57:0/3613778000 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.386+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb623b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.392+0000 7f7f3561f700 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61df1a0 0x561bb623b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.393+0000 7f7f34e1e700 1 -- 172.21.15.57:0/3613778000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61df1a0 msgr2=0x561bb623b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.393+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61df1a0 0x561bb623b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.393+0000 7f7f34e1e700 1 -- 172.21.15.57:0/3613778000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 msgr2=0x561bb623b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.393+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb623b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.393+0000 7f7f34e1e700 1 -- 172.21.15.57:0/3613778000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bb5f3fc00 con 0x561bb61d90b0 2023-10-08T14:06:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.393+0000 7f7f34e1e700 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb623ad40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7f2409acb0 tx=0x7f7f24000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e94ab8c2adb18f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.393+0000 7f7f0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-08T14:06:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.394+0000 7f7f0ffff700 1 -- 172.21.15.57:0/3613778000 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2400bea0 con 0x561bb61d90b0 2023-10-08T14:06:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.394+0000 7f7f0ffff700 1 -- 172.21.15.57:0/3613778000 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7f24018310 con 0x561bb61d90b0 2023-10-08T14:06:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.394+0000 7f7f4326bb80 1 -- 172.21.15.57:0/3613778000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f240a2e80 con 0x561bb61d90b0 2023-10-08T14:06:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.395+0000 7f7f0ffff700 1 -- 172.21.15.57:0/3613778000 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f240184d0 con 0x561bb61d90b0 2023-10-08T14:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.399+0000 7f7f4326bb80 10 client.17409 fetch_fsmap learned FSMap version 52 2023-10-08T14:06:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.399+0000 7f7f4326bb80 1 -- 172.21.15.57:0/3613778000 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x561bb5f7d1f0 con 0x561bb61d90b0 2023-10-08T14:06:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.401+0000 7f7f0ffff700 1 -- 172.21.15.57:0/3613778000 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f240a2e80 con 0x561bb61d90b0 2023-10-08T14:06:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.402+0000 7f7f0ffff700 1 -- 172.21.15.57:0/3613778000 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 52) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f7f2409ae60 con 0x561bb61d90b0 2023-10-08T14:06:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.402+0000 7f7f4326bb80 10 client.17409 fetch_fsmap finished waiting for FSMap version 52 2023-10-08T14:06:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.402+0000 7f7f4326bb80 10 client.17409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:06:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.402+0000 7f7f4326bb80 20 client.17409 populate_metadata read hostname 'smithi057' 2023-10-08T14:06:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.402+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561bb62478b0 0x561bb6249cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.402+0000 7f7f4326bb80 4 client.17409 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:06:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.402+0000 7f7f4326bb80 1 -- 172.21.15.57:0/3613778000 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561bb6124ad0 con 0x561bb62478b0 2023-10-08T14:06:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.404+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561bb62478b0 0x561bb6249cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.412+0000 7f7f2ffff700 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561bb62478b0 0x561bb6249cd0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51768bff35e94ede server_cookie=9b3359ef23dff879 in_seq=0 out_seq=0 2023-10-08T14:06:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:42.412+0000 7f7f0ffff700 10 client.17409 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.243+0000 7f7f0ffff700 1 -- 172.21.15.57:0/3613778000 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561bb6124ad0 con 0x561bb62478b0 2023-10-08T14:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.243+0000 7f7f0ffff700 10 client.17409 handle_command_reply: tid=0 2023-10-08T14:06:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.243+0000 7f7f4326bb80 1 client.17409 shutdown 2023-10-08T14:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.247+0000 7f7f4326bb80 1 -- 172.21.15.57:0/3613778000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 msgr2=0x561bb623ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.247+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb623ad40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7f2409acb0 tx=0x7f7f24000bc0 comp rx=0 tx=0).stop 2023-10-08T14:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.247+0000 7f7f4326bb80 1 -- 172.21.15.57:0/3613778000 shutdown_connections 2023-10-08T14:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.247+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x561bb62478b0 0x561bb6249cd0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.247+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561bb61df1a0 0x561bb623b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.247+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561bb61d90b0 0x561bb623ad40 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.247+0000 7f7f4326bb80 1 --2- 172.21.15.57:0/3613778000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bb61de880 0x561bb623b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.247+0000 7f7f4326bb80 1 -- 172.21.15.57:0/3613778000 >> 172.21.15.57:0/3613778000 conn(0x561bb614d200 msgr2=0x561bb61ddee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.251+0000 7f7f0ffff700 0 client.17409 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.251+0000 7f7f4326bb80 1 -- 172.21.15.57:0/3613778000 shutdown_connections 2023-10-08T14:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.251+0000 7f7f4326bb80 1 -- 172.21.15.57:0/3613778000 wait complete. 2023-10-08T14:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.252+0000 7f7f4326bb80 20 client.17409 trim_cache size 0 max 16384 2023-10-08T14:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:06:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:06:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.252+0000 7f7f37082700 1 -- 172.21.15.57:0/386525461 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7f180a17c0 msgr2=0x7f7f180a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.252+0000 7f7f37082700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7f180a17c0 0x7f7f180a3c80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7f20006010 tx=0x7f7f20009f10 comp rx=0 tx=0).stop 2023-10-08T14:06:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.252+0000 7f7f37082700 1 -- 172.21.15.57:0/386525461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f3010ef10 msgr2=0x7f7f30118360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.252+0000 7f7f37082700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f3010ef10 0x7f7f30118360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400b530 tx=0x7f7f2400b8f0 comp rx=0 tx=0).stop 2023-10-08T14:06:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.253+0000 7f7f37082700 1 -- 172.21.15.57:0/386525461 shutdown_connections 2023-10-08T14:06:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.253+0000 7f7f37082700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f30119fd0 0x7f7f301114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.253+0000 7f7f37082700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7f180a17c0 0x7f7f180a3c80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.253+0000 7f7f37082700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f3010ef10 0x7f7f30118360 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.253+0000 7f7f37082700 1 --2- 172.21.15.57:0/386525461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f3010f380 0x7f7f301188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.253+0000 7f7f37082700 1 -- 172.21.15.57:0/386525461 >> 172.21.15.57:0/386525461 conn(0x7f7f3006b7c0 msgr2=0x7f7f3006f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.262+0000 7f7f37082700 1 -- 172.21.15.57:0/386525461 shutdown_connections 2023-10-08T14:06:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:43.267+0000 7f7f37082700 1 -- 172.21.15.57:0/386525461 wait complete. 2023-10-08T14:06:44.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:44 smithi115 ceph-mon[104408]: pgmap v773: 129 pgs: 64 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 43 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 570 KiB/s wr, 129 op/s 2023-10-08T14:06:44.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:44 smithi115 ceph-mon[104408]: osdmap e112: 12 total, 12 up, 12 in 2023-10-08T14:06:44.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:44 smithi057 ceph-mon[99476]: pgmap v773: 129 pgs: 64 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 43 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 570 KiB/s wr, 129 op/s 2023-10-08T14:06:44.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:44 smithi057 ceph-mon[99476]: osdmap e112: 12 total, 12 up, 12 in 2023-10-08T14:06:44.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:44 smithi099 ceph-mon[103849]: pgmap v773: 129 pgs: 64 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 43 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 570 KiB/s wr, 129 op/s 2023-10-08T14:06:44.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:44 smithi099 ceph-mon[103849]: osdmap e112: 12 total, 12 up, 12 in 2023-10-08T14:06:46.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:46 smithi115 ceph-mon[104408]: pgmap v775: 129 pgs: 63 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 45 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 255 KiB/s wr, 153 op/s 2023-10-08T14:06:46.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:46 smithi057 ceph-mon[99476]: pgmap v775: 129 pgs: 63 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 45 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 255 KiB/s wr, 153 op/s 2023-10-08T14:06:46.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:46 smithi099 ceph-mon[103849]: pgmap v775: 129 pgs: 63 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 45 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 255 KiB/s wr, 153 op/s 2023-10-08T14:06:48.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:48 smithi115 ceph-mon[104408]: pgmap v776: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 566 KiB/s wr, 165 op/s 2023-10-08T14:06:48.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:48 smithi057 ceph-mon[99476]: pgmap v776: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 566 KiB/s wr, 165 op/s 2023-10-08T14:06:48.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:48 smithi099 ceph-mon[103849]: pgmap v776: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 566 KiB/s wr, 165 op/s 2023-10-08T14:06:49.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:49 smithi115 ceph-mon[104408]: pgmap v777: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 572 KiB/s wr, 169 op/s 2023-10-08T14:06:49.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:49 smithi057 ceph-mon[99476]: pgmap v777: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 572 KiB/s wr, 169 op/s 2023-10-08T14:06:50.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:49 smithi099 ceph-mon[103849]: pgmap v777: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 572 KiB/s wr, 169 op/s 2023-10-08T14:06:51.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:51 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:06:51.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:51 smithi115 ceph-mon[104408]: pgmap v778: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 439 KiB/s wr, 172 op/s 2023-10-08T14:06:51.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:51 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:06:51.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:51 smithi057 ceph-mon[99476]: pgmap v778: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 439 KiB/s wr, 172 op/s 2023-10-08T14:06:52.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:51 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:06:52.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:51 smithi099 ceph-mon[103849]: pgmap v778: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 439 KiB/s wr, 172 op/s 2023-10-08T14:06:54.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:53 smithi099 ceph-mon[103849]: pgmap v779: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 568 KiB/s wr, 175 op/s 2023-10-08T14:06:54.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:53 smithi115 ceph-mon[104408]: pgmap v779: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 568 KiB/s wr, 175 op/s 2023-10-08T14:06:54.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:53 smithi057 ceph-mon[99476]: pgmap v779: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 568 KiB/s wr, 175 op/s 2023-10-08T14:06:56.109 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2023-10-08T14:06:56.110 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-08T14:06:56.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:56 smithi099 ceph-mon[103849]: pgmap v780: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 504 KiB/s wr, 177 op/s 2023-10-08T14:06:56.321 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:56 smithi057 ceph-mon[99476]: pgmap v780: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 504 KiB/s wr, 177 op/s 2023-10-08T14:06:56.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:56 smithi115 ceph-mon[104408]: pgmap v780: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 504 KiB/s wr, 177 op/s 2023-10-08T14:06:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.470+0000 7fd186932700 1 -- 172.21.15.57:0/3149425987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd18010ef10 msgr2=0x7fd1801077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.470+0000 7fd186932700 1 --2- 172.21.15.57:0/3149425987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd18010ef10 0x7fd1801077f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd178009fc0 tx=0x7fd17800b330 comp rx=0 tx=0).stop 2023-10-08T14:06:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.473+0000 7fd186932700 1 -- 172.21.15.57:0/3149425987 shutdown_connections 2023-10-08T14:06:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.473+0000 7fd186932700 1 --2- 172.21.15.57:0/3149425987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd180119fd0 0x7fd18011c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.473+0000 7fd186932700 1 --2- 172.21.15.57:0/3149425987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd18010ef10 0x7fd1801077f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.473+0000 7fd186932700 1 --2- 172.21.15.57:0/3149425987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd18010f380 0x7fd180107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.473+0000 7fd186932700 1 -- 172.21.15.57:0/3149425987 >> 172.21.15.57:0/3149425987 conn(0x7fd18006b7c0 msgr2=0x7fd18006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.476+0000 7fd186932700 1 -- 172.21.15.57:0/3149425987 shutdown_connections 2023-10-08T14:06:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.477+0000 7fd186932700 1 -- 172.21.15.57:0/3149425987 wait complete. 2023-10-08T14:06:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.478+0000 7fd186932700 1 Processor -- start 2023-10-08T14:06:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.481+0000 7fd186932700 1 -- start start 2023-10-08T14:06:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.482+0000 7fd186932700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd18010ef10 0x7fd180119380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.482+0000 7fd186932700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd18010f380 0x7fd1801198c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.482+0000 7fd186932700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd180119fd0 0x7fd1801134d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.482+0000 7fd186932700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180113ca0 con 0x7fd180119fd0 2023-10-08T14:06:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.482+0000 7fd186932700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180113e10 con 0x7fd18010ef10 2023-10-08T14:06:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.482+0000 7fd186932700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180113fd0 con 0x7fd18010f380 2023-10-08T14:06:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.482+0000 7fd17f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd18010f380 0x7fd1801198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.483+0000 7fd17f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd18010f380 0x7fd1801198c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33170/0 (socket says 172.21.15.57:33170) 2023-10-08T14:06:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.483+0000 7fd17f7fe700 1 -- 172.21.15.57:0/2368488920 learned_addr learned my addr 172.21.15.57:0/2368488920 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.484+0000 7fd184ecf700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd180119fd0 0x7fd1801134d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.484+0000 7fd17f7fe700 1 -- 172.21.15.57:0/2368488920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd18010ef10 msgr2=0x7fd180119380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.484+0000 7fd17f7fe700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd18010ef10 0x7fd180119380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.484+0000 7fd17f7fe700 1 -- 172.21.15.57:0/2368488920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd180119fd0 msgr2=0x7fd1801134d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.484+0000 7fd17f7fe700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd180119fd0 0x7fd1801134d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.484+0000 7fd17f7fe700 1 -- 172.21.15.57:0/2368488920 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd178009c40 con 0x7fd18010f380 2023-10-08T14:06:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.485+0000 7fd17f7fe700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd18010f380 0x7fd1801198c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd17000b520 tx=0x7fd17000b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.485+0000 7fd17d7fa700 1 -- 172.21.15.57:0/2368488920 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170011830 con 0x7fd18010f380 2023-10-08T14:06:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.485+0000 7fd186932700 1 -- 172.21.15.57:0/2368488920 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1801142b0 con 0x7fd18010f380 2023-10-08T14:06:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.485+0000 7fd186932700 1 -- 172.21.15.57:0/2368488920 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1801bb090 con 0x7fd18010f380 2023-10-08T14:06:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.485+0000 7fd17d7fa700 1 -- 172.21.15.57:0/2368488920 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd170011e50 con 0x7fd18010f380 2023-10-08T14:06:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.485+0000 7fd17d7fa700 1 -- 172.21.15.57:0/2368488920 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170010e50 con 0x7fd18010f380 2023-10-08T14:06:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.491+0000 7fd192b1bb80 1 Processor -- start 2023-10-08T14:06:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.492+0000 7fd17d7fa700 1 -- 172.21.15.57:0/2368488920 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd170011990 con 0x7fd18010f380 2023-10-08T14:06:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.493+0000 7fd17d7fa700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd16c0a1670 0x7fd16c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.493+0000 7fd17d7fa700 1 -- 172.21.15.57:0/2368488920 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7fd1700ccd30 con 0x7fd18010f380 2023-10-08T14:06:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.496+0000 7fd192b1bb80 1 -- start start 2023-10-08T14:06:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.496+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402f880 0x5614c402fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.496+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c40301a0 0x5614c4029b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.496+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c402a0b0 0x5614c402a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.496+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c3f92400 con 0x5614c402f880 2023-10-08T14:06:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.496+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c3f83bc0 con 0x5614c40301a0 2023-10-08T14:06:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.496+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c3f814e0 con 0x5614c402a0b0 2023-10-08T14:06:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.500+0000 7fd17f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c40301a0 0x5614c4029b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.500+0000 7fd17f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c40301a0 0x5614c4029b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33792/0 (socket says 172.21.15.57:33792) 2023-10-08T14:06:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.500+0000 7fd17f7fe700 1 -- 172.21.15.57:0/2267741026 learned_addr learned my addr 172.21.15.57:0/2267741026 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.500+0000 7fd17f7fe700 1 -- 172.21.15.57:0/2267741026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c402a0b0 msgr2=0x5614c402a510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T14:06:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.500+0000 7fd17f7fe700 1 --2- 172.21.15.57:0/2267741026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c402a0b0 0x5614c402a510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.500+0000 7fd17f7fe700 1 -- 172.21.15.57:0/2267741026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402f880 msgr2=0x5614c402fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.500+0000 7fd17f7fe700 1 --2- 172.21.15.57:0/2267741026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402f880 0x5614c402fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.500+0000 7fd17f7fe700 1 -- 172.21.15.57:0/2267741026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614c3da7d80 con 0x5614c40301a0 2023-10-08T14:06:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.500+0000 7fd17f7fe700 1 --2- 172.21.15.57:0/2267741026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c40301a0 0x5614c4029b70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd170099a00 tx=0x7fd170099450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73c5190752cce58 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.500+0000 7fd1767fc700 1 -- 172.21.15.57:0/2267741026 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17009dbc0 con 0x5614c40301a0 2023-10-08T14:06:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.501+0000 7fd1767fc700 1 -- 172.21.15.57:0/2267741026 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1700a1e00 con 0x5614c40301a0 2023-10-08T14:06:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.501+0000 7fd1767fc700 1 -- 172.21.15.57:0/2267741026 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1700991c0 con 0x5614c40301a0 2023-10-08T14:06:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.501+0000 7fd184ecf700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd16c0a1670 0x7fd16c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.501+0000 7fd192b1bb80 1 -- 172.21.15.57:0/2267741026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c40301a0 msgr2=0x5614c4029b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.501+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/2267741026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c40301a0 0x5614c4029b70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd170099a00 tx=0x7fd170099450 comp rx=0 tx=0).stop 2023-10-08T14:06:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.502+0000 7fd192b1bb80 1 -- 172.21.15.57:0/2267741026 shutdown_connections 2023-10-08T14:06:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.502+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/2267741026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c402a0b0 0x5614c402a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.502+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/2267741026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402f880 0x5614c402fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.502+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/2267741026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c40301a0 0x5614c4029b70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.502+0000 7fd192b1bb80 1 -- 172.21.15.57:0/2267741026 >> 172.21.15.57:0/2267741026 conn(0x5614c3f9e200 msgr2=0x5614c4024b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.504+0000 7fd192b1bb80 1 -- 172.21.15.57:0/2267741026 shutdown_connections 2023-10-08T14:06:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.508+0000 7fd192b1bb80 1 -- 172.21.15.57:0/2267741026 wait complete. 2023-10-08T14:06:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.509+0000 7fd192b1bb80 1 -- start start 2023-10-08T14:06:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.509+0000 7fd192b1bb80 1 Processor -- start 2023-10-08T14:06:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 0x5614c40335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4033ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4034020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c3f7cdd0 con 0x5614c402a0b0 2023-10-08T14:06:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c3f196b0 con 0x5614c402f880 2023-10-08T14:06:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c3f19350 con 0x5614c40301a0 2023-10-08T14:06:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd192b1bb80 10 client.0 _ll_register_callbacks cb 0x5614c3d767a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:06:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd17ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 0x5614c40335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd17ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 0x5614c40335a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60602/0 (socket says 172.21.15.57:60602) 2023-10-08T14:06:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd17ffff700 1 -- 172.21.15.57:0/583139294 learned_addr learned my addr 172.21.15.57:0/583139294 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.510+0000 7fd184ecf700 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4034020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.511+0000 7fd184ecf700 1 -- 172.21.15.57:0/583139294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 msgr2=0x5614c4033ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T14:06:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.511+0000 7fd184ecf700 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4033ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.511+0000 7fd184ecf700 1 -- 172.21.15.57:0/583139294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 msgr2=0x5614c40335a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.511+0000 7fd184ecf700 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 0x5614c40335a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.511+0000 7fd184ecf700 1 -- 172.21.15.57:0/583139294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1801bb090 con 0x5614c40301a0 2023-10-08T14:06:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.511+0000 7fd184ecf700 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4034020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd168005b40 tx=0x7fd168013f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de4ac27683224e42 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.515+0000 7fd15f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:06:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.515+0000 7fd15f7fe700 1 -- 172.21.15.57:0/583139294 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16800fe00 con 0x5614c40301a0 2023-10-08T14:06:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.515+0000 7fd184ecf700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd16c0a1670 0x7fd16c0a3b30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd168021000 tx=0x7fd16801c000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.515+0000 7fd15f7fe700 1 -- 172.21.15.57:0/583139294 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7fd1680103d0 con 0x5614c40301a0 2023-10-08T14:06:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.515+0000 7fd192b1bb80 1 -- 172.21.15.57:0/583139294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614c3f44e50 con 0x5614c40301a0 2023-10-08T14:06:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.516+0000 7fd15f7fe700 1 -- 172.21.15.57:0/583139294 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd168013080 con 0x5614c40301a0 2023-10-08T14:06:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.516+0000 7fd192b1bb80 10 client.26971 fetch_fsmap learned FSMap version 53 2023-10-08T14:06:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.516+0000 7fd192b1bb80 1 -- 172.21.15.57:0/583139294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5614c3d90c00 con 0x5614c40301a0 2023-10-08T14:06:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.516+0000 7fd15f7fe700 1 -- 172.21.15.57:0/583139294 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614c3f44e50 con 0x5614c40301a0 2023-10-08T14:06:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.517+0000 7fd15f7fe700 1 -- 172.21.15.57:0/583139294 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 53) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fd168010c20 con 0x5614c40301a0 2023-10-08T14:06:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.517+0000 7fd192b1bb80 10 client.26971 fetch_fsmap finished waiting for FSMap version 53 2023-10-08T14:06:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.517+0000 7fd192b1bb80 10 client.26971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:06:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.517+0000 7fd192b1bb80 20 client.26971 populate_metadata read hostname 'smithi057' 2023-10-08T14:06:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.517+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5614c4043eb0 0x5614c40462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.517+0000 7fd192b1bb80 4 client.26971 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T14:06:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.517+0000 7fd192b1bb80 1 -- 172.21.15.57:0/583139294 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614c3f75db0 con 0x5614c4043eb0 2023-10-08T14:06:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.518+0000 7fd17ffff700 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5614c4043eb0 0x5614c40462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.519+0000 7fd17ffff700 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5614c4043eb0 0x5614c40462d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eef302cd6cb5564 server_cookie=1b92f2582a7db505 in_seq=0 out_seq=0 2023-10-08T14:06:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.521+0000 7fd15f7fe700 10 client.26971 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.521+0000 7fd15f7fe700 1 -- 172.21.15.57:0/583139294 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5614c3f75db0 con 0x5614c4043eb0 2023-10-08T14:06:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.521+0000 7fd15f7fe700 10 client.26971 handle_command_reply: tid=0 2023-10-08T14:06:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.524+0000 7fd192b1bb80 1 client.26971 shutdown 2023-10-08T14:06:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.524+0000 7fd192b1bb80 1 -- 172.21.15.57:0/583139294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 msgr2=0x5614c4034020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.524+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4034020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd168005b40 tx=0x7fd168013f60 comp rx=0 tx=0).stop 2023-10-08T14:06:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.527+0000 7fd192b1bb80 1 -- 172.21.15.57:0/583139294 shutdown_connections 2023-10-08T14:06:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.527+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5614c4043eb0 0x5614c40462d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.527+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4034020 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.527+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 0x5614c40335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.527+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/583139294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4033ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.527+0000 7fd192b1bb80 1 -- 172.21.15.57:0/583139294 >> 172.21.15.57:0/583139294 conn(0x5614c3f9e200 msgr2=0x5614c4024010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.527+0000 7fd15f7fe700 0 client.26971 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.528+0000 7fd192b1bb80 1 -- 172.21.15.57:0/583139294 shutdown_connections 2023-10-08T14:06:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.528+0000 7fd192b1bb80 1 -- 172.21.15.57:0/583139294 wait complete. 2023-10-08T14:06:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.538+0000 7fd192b1bb80 20 client.26971 trim_cache size 0 max 16384 2023-10-08T14:06:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.566+0000 7fd192b1bb80 1 Processor -- start 2023-10-08T14:06:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.566+0000 7fd192b1bb80 1 -- start start 2023-10-08T14:06:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.566+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 0x5614c4029b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.566+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4024790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.566+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4024cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.566+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c3dfd8c0 con 0x5614c402a0b0 2023-10-08T14:06:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.566+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c3f75db0 con 0x5614c402f880 2023-10-08T14:06:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c3f44e50 con 0x5614c40301a0 2023-10-08T14:06:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd184ecf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4024cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd184ecf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4024cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33186/0 (socket says 172.21.15.57:33186) 2023-10-08T14:06:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd184ecf700 1 -- 172.21.15.57:0/2927429038 learned_addr learned my addr 172.21.15.57:0/2927429038 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd17ffff700 1 --2- 172.21.15.57:0/2927429038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 0x5614c4029b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd17f7fe700 1 --2- 172.21.15.57:0/2927429038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4024790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd184ecf700 1 -- 172.21.15.57:0/2927429038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 msgr2=0x5614c4024790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd184ecf700 1 --2- 172.21.15.57:0/2927429038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4024790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd184ecf700 1 -- 172.21.15.57:0/2927429038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 msgr2=0x5614c4029b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd184ecf700 1 --2- 172.21.15.57:0/2927429038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 0x5614c4029b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.567+0000 7fd184ecf700 1 -- 172.21.15.57:0/2927429038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614c402a760 con 0x5614c40301a0 2023-10-08T14:06:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.568+0000 7fd184ecf700 1 --2- 172.21.15.57:0/2927429038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4024cd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd16800b300 tx=0x7fd16800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55965d782f4d0cb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.569+0000 7fd176ffd700 1 -- 172.21.15.57:0/2927429038 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16800b480 con 0x5614c40301a0 2023-10-08T14:06:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.569+0000 7fd176ffd700 1 -- 172.21.15.57:0/2927429038 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1680103d0 con 0x5614c40301a0 2023-10-08T14:06:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.569+0000 7fd176ffd700 1 -- 172.21.15.57:0/2927429038 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16800fe60 con 0x5614c40301a0 2023-10-08T14:06:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.570+0000 7fd192b1bb80 1 -- 172.21.15.57:0/2927429038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 msgr2=0x5614c4024cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.570+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/2927429038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4024cd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd16800b300 tx=0x7fd16800b330 comp rx=0 tx=0).stop 2023-10-08T14:06:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.571+0000 7fd192b1bb80 1 -- 172.21.15.57:0/2927429038 shutdown_connections 2023-10-08T14:06:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.571+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/2927429038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c40301a0 0x5614c4024cd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.571+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/2927429038 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c402a0b0 0x5614c4029b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.571+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/2927429038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4024790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.571+0000 7fd192b1bb80 1 -- 172.21.15.57:0/2927429038 >> 172.21.15.57:0/2927429038 conn(0x5614c3f9e200 msgr2=0x5614c4031e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.571+0000 7fd192b1bb80 1 -- 172.21.15.57:0/2927429038 shutdown_connections 2023-10-08T14:06:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.571+0000 7fd192b1bb80 1 -- 172.21.15.57:0/2927429038 wait complete. 2023-10-08T14:06:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.572+0000 7fd192b1bb80 1 -- start start 2023-10-08T14:06:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.572+0000 7fd192b1bb80 1 Processor -- start 2023-10-08T14:06:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c402a0b0 0x5614c4038c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4039170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd192b1bb80 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c40301a0 0x5614c40396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c4096ce0 con 0x5614c40301a0 2023-10-08T14:06:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c4096e20 con 0x5614c402f880 2023-10-08T14:06:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd192b1bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614c4096f60 con 0x5614c402a0b0 2023-10-08T14:06:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd192b1bb80 10 client.0 _ll_register_callbacks cb 0x7fd1680136e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T14:06:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd184ecf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c40301a0 0x5614c40396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd184ecf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c40301a0 0x5614c40396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60614/0 (socket says 172.21.15.57:60614) 2023-10-08T14:06:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd17ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c402a0b0 0x5614c4038c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd184ecf700 1 -- 172.21.15.57:0/212215900 learned_addr learned my addr 172.21.15.57:0/212215900 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd17f7fe700 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4039170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd17ffff700 1 -- 172.21.15.57:0/212215900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 msgr2=0x5614c4039170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd17ffff700 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4039170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd17ffff700 1 -- 172.21.15.57:0/212215900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c40301a0 msgr2=0x5614c40396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T14:06:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd17ffff700 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c40301a0 0x5614c40396b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.573+0000 7fd17ffff700 1 -- 172.21.15.57:0/212215900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614c3d90c00 con 0x5614c402a0b0 2023-10-08T14:06:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.574+0000 7fd184ecf700 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c40301a0 0x5614c40396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:06:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.574+0000 7fd17ffff700 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c402a0b0 0x5614c4038c30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd178009f90 tx=0x7fd178020b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ba5f045e7ecaad7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.574+0000 7fd15f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-10-08T14:06:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.580+0000 7fd15f7fe700 1 -- 172.21.15.57:0/212215900 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17800e040 con 0x5614c402a0b0 2023-10-08T14:06:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.580+0000 7fd15f7fe700 1 -- 172.21.15.57:0/212215900 <== mon.1 v2:172.21.15.115:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7fd17800baa0 con 0x5614c402a0b0 2023-10-08T14:06:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.580+0000 7fd192b1bb80 1 -- 172.21.15.57:0/212215900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1700a1e00 con 0x5614c402a0b0 2023-10-08T14:06:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.591+0000 7fd15f7fe700 1 -- 172.21.15.57:0/212215900 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd17800bc60 con 0x5614c402a0b0 2023-10-08T14:06:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.592+0000 7fd192b1bb80 10 client.26977 fetch_fsmap learned FSMap version 53 2023-10-08T14:06:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.592+0000 7fd192b1bb80 1 -- 172.21.15.57:0/212215900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5614c3dce1f0 con 0x5614c402a0b0 2023-10-08T14:06:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.596+0000 7fd15f7fe700 1 -- 172.21.15.57:0/212215900 <== mon.1 v2:172.21.15.115:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1700a1e00 con 0x5614c402a0b0 2023-10-08T14:06:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.596+0000 7fd15f7fe700 1 -- 172.21.15.57:0/212215900 <== mon.1 v2:172.21.15.115:3300/0 5 ==== fsmap(e 53) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fd17800b050 con 0x5614c402a0b0 2023-10-08T14:06:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.596+0000 7fd192b1bb80 10 client.26977 fetch_fsmap finished waiting for FSMap version 53 2023-10-08T14:06:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.596+0000 7fd192b1bb80 10 client.26977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T14:06:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.596+0000 7fd192b1bb80 20 client.26977 populate_metadata read hostname 'smithi057' 2023-10-08T14:06:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.596+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5614c4098750 0x5614c409ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.596+0000 7fd192b1bb80 4 client.26977 mds_command: new command op to 24662 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T14:06:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.596+0000 7fd192b1bb80 1 -- 172.21.15.57:0/212215900 --> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5614c3f75ad0 con 0x5614c4098750 2023-10-08T14:06:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.604+0000 7fd17f7fe700 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5614c4098750 0x5614c409ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.604+0000 7fd17f7fe700 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5614c4098750 0x5614c409ab70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=627ae44cda5ad0df server_cookie=ccd800803eb5079b in_seq=0 out_seq=0 2023-10-08T14:06:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.607+0000 7fd15f7fe700 10 client.26977 ms_handle_connect on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.733+0000 7fd15f7fe700 1 -- 172.21.15.57:0/212215900 <== mds.0 v2:172.21.15.99:6838/687823376 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5614c3f75ad0 con 0x5614c4098750 2023-10-08T14:06:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.733+0000 7fd15f7fe700 10 client.26977 handle_command_reply: tid=0 2023-10-08T14:06:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.733+0000 7fd192b1bb80 1 client.26977 shutdown 2023-10-08T14:06:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.748+0000 7fd192b1bb80 1 -- 172.21.15.57:0/212215900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c402a0b0 msgr2=0x5614c4038c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.748+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c402a0b0 0x5614c4038c30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd178009f90 tx=0x7fd178020b00 comp rx=0 tx=0).stop 2023-10-08T14:06:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.755+0000 7fd192b1bb80 1 -- 172.21.15.57:0/212215900 shutdown_connections 2023-10-08T14:06:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.755+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.99:6838/687823376,v1:172.21.15.99:6839/687823376] conn(0x5614c4098750 0x5614c409ab70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.755+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5614c40301a0 0x5614c40396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.755+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5614c402a0b0 0x5614c4038c30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.755+0000 7fd192b1bb80 1 --2- 172.21.15.57:0/212215900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614c402f880 0x5614c4039170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.755+0000 7fd192b1bb80 1 -- 172.21.15.57:0/212215900 >> 172.21.15.57:0/212215900 conn(0x5614c3f9e200 msgr2=0x5614c402e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.755+0000 7fd15f7fe700 0 client.26977 ms_handle_reset on v2:172.21.15.99:6838/687823376 2023-10-08T14:06:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.756+0000 7fd192b1bb80 1 -- 172.21.15.57:0/212215900 shutdown_connections 2023-10-08T14:06:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.756+0000 7fd192b1bb80 1 -- 172.21.15.57:0/212215900 wait complete. 2023-10-08T14:06:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.756+0000 7fd192b1bb80 20 client.26977 trim_cache size 0 max 16384 2023-10-08T14:06:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2023-10-08T14:06:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2023-10-08T14:06:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2023-10-08T14:06:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2023-10-08T14:06:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.767+0000 7fd186932700 1 -- 172.21.15.57:0/2368488920 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd16c0a1670 msgr2=0x7fd16c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.767+0000 7fd186932700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd16c0a1670 0x7fd16c0a3b30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd168021000 tx=0x7fd16801c000 comp rx=0 tx=0).stop 2023-10-08T14:06:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.767+0000 7fd186932700 1 -- 172.21.15.57:0/2368488920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd18010f380 msgr2=0x7fd1801198c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.767+0000 7fd186932700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd18010f380 0x7fd1801198c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd17000b520 tx=0x7fd17000b8e0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.774+0000 7fd186932700 1 -- 172.21.15.57:0/2368488920 shutdown_connections 2023-10-08T14:06:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.774+0000 7fd186932700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd180119fd0 0x7fd1801134d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.774+0000 7fd186932700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fd16c0a1670 0x7fd16c0a3b30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.774+0000 7fd186932700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd18010ef10 0x7fd180119380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.774+0000 7fd186932700 1 --2- 172.21.15.57:0/2368488920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd18010f380 0x7fd1801198c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.774+0000 7fd186932700 1 -- 172.21.15.57:0/2368488920 >> 172.21.15.57:0/2368488920 conn(0x7fd18006b7c0 msgr2=0x7fd18010b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.781+0000 7fd186932700 1 -- 172.21.15.57:0/2368488920 shutdown_connections 2023-10-08T14:06:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2023-10-08T14:06:56.783+0000 7fd186932700 1 -- 172.21.15.57:0/2368488920 wait complete. 2023-10-08T14:06:56.844 DEBUG:teuthology.parallel:result is None 2023-10-08T14:06:56.844 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-10-08T14:06:57.037 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-10-08T14:06:57.037 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-10-08T14:06:57.047 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T14:06:57.062 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-10-08T14:06:57.098 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-10-08T14:06:57.098 INFO:teuthology.task.exec:Executing custom commands... 2023-10-08T14:06:57.099 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T14:06:57.099 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-10-08T14:06:57.223 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-10-08T14:06:57.339 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-08T14:06:57.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.626+0000 7fa565090700 1 -- 172.21.15.57:0/528866926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5600ff2f0 msgr2=0x7fa56010b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:57.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.626+0000 7fa565090700 1 --2- 172.21.15.57:0/528866926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5600ff2f0 0x7fa56010b7b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5580074e0 tx=0x7fa55800af90 comp rx=0 tx=0).stop 2023-10-08T14:06:57.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.626+0000 7fa565090700 1 -- 172.21.15.57:0/528866926 shutdown_connections 2023-10-08T14:06:57.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.626+0000 7fa565090700 1 --2- 172.21.15.57:0/528866926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5600ff2f0 0x7fa56010b7b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:57.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.626+0000 7fa565090700 1 --2- 172.21.15.57:0/528866926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5601048f0 0x7fa560104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:57.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.626+0000 7fa565090700 1 --2- 172.21.15.57:0/528866926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5600fe930 0x7fa5600fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:57.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.626+0000 7fa565090700 1 -- 172.21.15.57:0/528866926 >> 172.21.15.57:0/528866926 conn(0x7fa560075770 msgr2=0x7fa5600fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:57.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.636+0000 7fa565090700 1 -- 172.21.15.57:0/528866926 shutdown_connections 2023-10-08T14:06:57.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.640+0000 7fa565090700 1 -- 172.21.15.57:0/528866926 wait complete. 2023-10-08T14:06:57.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.641+0000 7fa565090700 1 Processor -- start 2023-10-08T14:06:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.642+0000 7fa565090700 1 -- start start 2023-10-08T14:06:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.642+0000 7fa565090700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5600fe930 0x7fa56010b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.642+0000 7fa565090700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5600ff2f0 0x7fa5601087b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.642+0000 7fa565090700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5601048f0 0x7fa560108de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.642+0000 7fa565090700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5601095b0 con 0x7fa5600fe930 2023-10-08T14:06:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.642+0000 7fa565090700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa560071260 con 0x7fa5601048f0 2023-10-08T14:06:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.642+0000 7fa565090700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa560071420 con 0x7fa5600ff2f0 2023-10-08T14:06:57.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.643+0000 7fa55f59e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5601048f0 0x7fa560108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:57.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.643+0000 7fa55ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5600fe930 0x7fa56010b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:57.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.643+0000 7fa55ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5600fe930 0x7fa56010b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60626/0 (socket says 172.21.15.57:60626) 2023-10-08T14:06:57.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.643+0000 7fa55ed9d700 1 -- 172.21.15.57:0/3761770370 learned_addr learned my addr 172.21.15.57:0/3761770370 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:57.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.644+0000 7fa55ed9d700 1 -- 172.21.15.57:0/3761770370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5600ff2f0 msgr2=0x7fa5601087b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:06:57.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.644+0000 7fa55ed9d700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5600ff2f0 0x7fa5601087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:57.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.644+0000 7fa55ed9d700 1 -- 172.21.15.57:0/3761770370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5601048f0 msgr2=0x7fa560108de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:57.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.644+0000 7fa55ed9d700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5601048f0 0x7fa560108de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:57.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.644+0000 7fa55ed9d700 1 -- 172.21.15.57:0/3761770370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5580071c0 con 0x7fa5600fe930 2023-10-08T14:06:57.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.644+0000 7fa55ed9d700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5600fe930 0x7fa56010b6e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa55000ba50 tx=0x7fa55000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:57.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.645+0000 7fa547fff700 1 -- 172.21.15.57:0/3761770370 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55000c9b0 con 0x7fa5600fe930 2023-10-08T14:06:57.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.645+0000 7fa565090700 1 -- 172.21.15.57:0/3761770370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa560071700 con 0x7fa5600fe930 2023-10-08T14:06:57.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.645+0000 7fa565090700 1 -- 172.21.15.57:0/3761770370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa560071c60 con 0x7fa5600fe930 2023-10-08T14:06:57.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.646+0000 7fa547fff700 1 -- 172.21.15.57:0/3761770370 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa550009930 con 0x7fa5600fe930 2023-10-08T14:06:57.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.646+0000 7fa547fff700 1 -- 172.21.15.57:0/3761770370 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550007370 con 0x7fa5600fe930 2023-10-08T14:06:57.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.648+0000 7fa547fff700 1 -- 172.21.15.57:0/3761770370 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa5500074d0 con 0x7fa5600fe930 2023-10-08T14:06:57.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.649+0000 7fa547fff700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5480a16b0 0x7fa5480a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:57.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.649+0000 7fa547fff700 1 -- 172.21.15.57:0/3761770370 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7fa550006e40 con 0x7fa5600fe930 2023-10-08T14:06:57.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.649+0000 7fa565090700 1 -- 172.21.15.57:0/3761770370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54c0052f0 con 0x7fa5600fe930 2023-10-08T14:06:57.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.654+0000 7fa55e59c700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5480a16b0 0x7fa5480a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:57.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.654+0000 7fa55e59c700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5480a16b0 0x7fa5480a3b70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa560109ee0 tx=0x7fa554006c60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:57.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:57.659+0000 7fa547fff700 1 -- 172.21.15.57:0/3761770370 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa55004f100 con 0x7fa5600fe930 2023-10-08T14:06:58.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.079+0000 7fa565090700 1 -- 172.21.15.57:0/3761770370 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fa54c000bc0 con 0x7fa5480a16b0 2023-10-08T14:06:58.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.168+0000 7fa547fff700 1 -- 172.21.15.57:0/3761770370 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7fa54c000bc0 con 0x7fa5480a16b0 2023-10-08T14:06:58.171 INFO:teuthology.orchestra.run.smithi057.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T13:51:30", "first": "2023-10-08T13:52:00", "last": "2023-10-08T14:06:00", "last_pruned": "2023-10-08T14:06:04", "created_count": 15, "pruned_count": 8, "active": true} 2023-10-08T14:06:58.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.178+0000 7fa565090700 1 -- 172.21.15.57:0/3761770370 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5480a16b0 msgr2=0x7fa5480a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:58.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.178+0000 7fa565090700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5480a16b0 0x7fa5480a3b70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa560109ee0 tx=0x7fa554006c60 comp rx=0 tx=0).stop 2023-10-08T14:06:58.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.178+0000 7fa565090700 1 -- 172.21.15.57:0/3761770370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5600fe930 msgr2=0x7fa56010b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:58.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.178+0000 7fa565090700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5600fe930 0x7fa56010b6e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa55000ba50 tx=0x7fa55000bd60 comp rx=0 tx=0).stop 2023-10-08T14:06:58.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.184+0000 7fa565090700 1 -- 172.21.15.57:0/3761770370 shutdown_connections 2023-10-08T14:06:58.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.184+0000 7fa565090700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fa5480a16b0 0x7fa5480a3b70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:58.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.184+0000 7fa565090700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5601048f0 0x7fa560108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:58.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.184+0000 7fa565090700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5600fe930 0x7fa56010b6e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:58.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.184+0000 7fa565090700 1 --2- 172.21.15.57:0/3761770370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5600ff2f0 0x7fa5601087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:58.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.184+0000 7fa565090700 1 -- 172.21.15.57:0/3761770370 >> 172.21.15.57:0/3761770370 conn(0x7fa560075770 msgr2=0x7fa560101780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:58.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.184+0000 7fa565090700 1 -- 172.21.15.57:0/3761770370 shutdown_connections 2023-10-08T14:06:58.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.184+0000 7fa565090700 1 -- 172.21.15.57:0/3761770370 wait complete. 2023-10-08T14:06:58.216 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-08T14:06:58.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:58 smithi057 ceph-mon[99476]: pgmap v781: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 438 KiB/s wr, 137 op/s 2023-10-08T14:06:58.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:58 smithi115 ceph-mon[104408]: pgmap v781: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 438 KiB/s wr, 137 op/s 2023-10-08T14:06:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:58 smithi099 ceph-mon[103849]: pgmap v781: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 438 KiB/s wr, 137 op/s 2023-10-08T14:06:58.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.540+0000 7fe493efd700 1 -- 172.21.15.57:0/3008790805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe48c108380 msgr2=0x7fe48c10b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.540+0000 7fe493efd700 1 --2- 172.21.15.57:0/3008790805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe48c108380 0x7fe48c10b940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe488009990 tx=0x7fe488009ca0 comp rx=0 tx=0).stop 2023-10-08T14:06:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.540+0000 7fe493efd700 1 -- 172.21.15.57:0/3008790805 shutdown_connections 2023-10-08T14:06:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.540+0000 7fe493efd700 1 --2- 172.21.15.57:0/3008790805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c10c070 0x7fe48c078b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.540+0000 7fe493efd700 1 --2- 172.21.15.57:0/3008790805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe48c107a60 0x7fe48c107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.540+0000 7fe493efd700 1 --2- 172.21.15.57:0/3008790805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe48c108380 0x7fe48c10b940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:58.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.540+0000 7fe493efd700 1 -- 172.21.15.57:0/3008790805 >> 172.21.15.57:0/3008790805 conn(0x7fe48c06b7c0 msgr2=0x7fe48c06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:58.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.547+0000 7fe493efd700 1 -- 172.21.15.57:0/3008790805 shutdown_connections 2023-10-08T14:06:58.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.552+0000 7fe493efd700 1 -- 172.21.15.57:0/3008790805 wait complete. 2023-10-08T14:06:58.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.553+0000 7fe493efd700 1 Processor -- start 2023-10-08T14:06:58.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.559+0000 7fe493efd700 1 -- start start 2023-10-08T14:06:58.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.559+0000 7fe493efd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe48c108380 0x7fe48c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:58.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.559+0000 7fe493efd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe48c10c070 0x7fe48c0751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:58.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.559+0000 7fe493efd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0756e0 0x7fe48c1bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:58.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.559+0000 7fe493efd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c078e50 con 0x7fe48c108380 2023-10-08T14:06:58.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.559+0000 7fe493efd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c1bce20 con 0x7fe48c0756e0 2023-10-08T14:06:58.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.559+0000 7fe493efd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c1bcf60 con 0x7fe48c10c070 2023-10-08T14:06:58.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.560+0000 7fe491498700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe48c10c070 0x7fe48c0751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:58.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.560+0000 7fe491498700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe48c10c070 0x7fe48c0751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33214/0 (socket says 172.21.15.57:33214) 2023-10-08T14:06:58.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.560+0000 7fe491498700 1 -- 172.21.15.57:0/682512929 learned_addr learned my addr 172.21.15.57:0/682512929 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.560+0000 7fe491c99700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe48c108380 0x7fe48c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.560+0000 7fe491498700 1 -- 172.21.15.57:0/682512929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0756e0 msgr2=0x7fe48c1bc8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.560+0000 7fe491498700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0756e0 0x7fe48c1bc8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.560+0000 7fe491498700 1 -- 172.21.15.57:0/682512929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe48c108380 msgr2=0x7fe48c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.560+0000 7fe491498700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe48c108380 0x7fe48c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.560+0000 7fe491498700 1 -- 172.21.15.57:0/682512929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4880095f0 con 0x7fe48c10c070 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.561+0000 7fe491498700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe48c10c070 0x7fe48c0751a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe488005b40 tx=0x7fe488003bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.562+0000 7fe482ffd700 1 -- 172.21.15.57:0/682512929 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48801d070 con 0x7fe48c10c070 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.562+0000 7fe482ffd700 1 -- 172.21.15.57:0/682512929 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4880036a0 con 0x7fe48c10c070 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.562+0000 7fe482ffd700 1 -- 172.21.15.57:0/682512929 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48800fdc0 con 0x7fe48c10c070 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.562+0000 7fe493efd700 1 -- 172.21.15.57:0/682512929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48c1bd1e0 con 0x7fe48c10c070 2023-10-08T14:06:58.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.562+0000 7fe493efd700 1 -- 172.21.15.57:0/682512929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48c1bd640 con 0x7fe48c10c070 2023-10-08T14:06:58.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.570+0000 7fe493efd700 1 -- 172.21.15.57:0/682512929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4700052f0 con 0x7fe48c10c070 2023-10-08T14:06:58.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.583+0000 7fe482ffd700 1 -- 172.21.15.57:0/682512929 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe48800f460 con 0x7fe48c10c070 2023-10-08T14:06:58.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.584+0000 7fe482ffd700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4780a1740 0x7fe4780a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:58.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.584+0000 7fe482ffd700 1 -- 172.21.15.57:0/682512929 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7fe4880ce5f0 con 0x7fe48c10c070 2023-10-08T14:06:58.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.584+0000 7fe482ffd700 1 -- 172.21.15.57:0/682512929 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe4880cead0 con 0x7fe48c10c070 2023-10-08T14:06:58.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.592+0000 7fe491c99700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4780a1740 0x7fe4780a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:58.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:58.598+0000 7fe491c99700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4780a1740 0x7fe4780a3c00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe48c06c950 tx=0x7fe48400d040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:59.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.024+0000 7fe493efd700 1 -- 172.21.15.57:0/682512929 --> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe470000bc0 con 0x7fe4780a1740 2023-10-08T14:06:59.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.119+0000 7fe482ffd700 1 -- 172.21.15.57:0/682512929 <== mgr.14217 v2:172.21.15.57:6800/2547096577 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fe470000bc0 con 0x7fe4780a1740 2023-10-08T14:06:59.130 INFO:teuthology.orchestra.run.smithi057.stdout:/ 1m 6m3h 2023-10-08T14:06:59.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 -- 172.21.15.57:0/682512929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4780a1740 msgr2=0x7fe4780a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:59.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4780a1740 0x7fe4780a3c00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe48c06c950 tx=0x7fe48400d040 comp rx=0 tx=0).stop 2023-10-08T14:06:59.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 -- 172.21.15.57:0/682512929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe48c10c070 msgr2=0x7fe48c0751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:59.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe48c10c070 0x7fe48c0751a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe488005b40 tx=0x7fe488003bd0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 -- 172.21.15.57:0/682512929 shutdown_connections 2023-10-08T14:06:59.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe4780a1740 0x7fe4780a3c00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0756e0 0x7fe48c1bc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe48c108380 0x7fe48c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 --2- 172.21.15.57:0/682512929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe48c10c070 0x7fe48c0751a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 -- 172.21.15.57:0/682512929 >> 172.21.15.57:0/682512929 conn(0x7fe48c06b7c0 msgr2=0x7fe48c070650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:59.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.134+0000 7fe493efd700 1 -- 172.21.15.57:0/682512929 shutdown_connections 2023-10-08T14:06:59.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.135+0000 7fe493efd700 1 -- 172.21.15.57:0/682512929 wait complete. 2023-10-08T14:06:59.160 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-08T14:06:59.170 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-08T14:06:59.170 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-10-08T14:06:59.171 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-08T14:06:59.236 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T14:06:59.327 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:06:59 smithi057 ceph-mon[99476]: from='client.17424 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T14:06:59.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:06:59 smithi115 ceph-mon[104408]: from='client.17424 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T14:06:59.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:06:59 smithi099 ceph-mon[103849]: from='client.17424 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T14:06:59.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.536+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1009053992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81077f0 msgr2=0x7f82e8107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:59.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.536+0000 7f82ee2ef700 1 --2- 172.21.15.57:0/1009053992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81077f0 0x7f82e8107c70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f82dc01a9a0 tx=0x7f82dc01acb0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.537+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1009053992 shutdown_connections 2023-10-08T14:06:59.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.537+0000 7f82ee2ef700 1 --2- 172.21.15.57:0/1009053992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82e81083a0 0x7f82e811bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.537+0000 7f82ee2ef700 1 --2- 172.21.15.57:0/1009053992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e810edf0 0x7f82e810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.537+0000 7f82ee2ef700 1 --2- 172.21.15.57:0/1009053992 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81077f0 0x7f82e8107c70 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.537+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1009053992 >> 172.21.15.57:0/1009053992 conn(0x7f82e806b760 msgr2=0x7f82e806bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:59.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.538+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1009053992 shutdown_connections 2023-10-08T14:06:59.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.538+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1009053992 wait complete. 2023-10-08T14:06:59.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.538+0000 7f82ee2ef700 1 Processor -- start 2023-10-08T14:06:59.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82ee2ef700 1 -- start start 2023-10-08T14:06:59.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82ee2ef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81077f0 0x7f82e8117da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:59.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82ee2ef700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e81083a0 0x7f82e81182e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:59.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82ee2ef700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82e810edf0 0x7f82e81a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:59.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82ee2ef700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e81a2160 con 0x7f82e81077f0 2023-10-08T14:06:59.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82ee2ef700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e81a22d0 con 0x7f82e810edf0 2023-10-08T14:06:59.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82ee2ef700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e81a2490 con 0x7f82e81083a0 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82e7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81077f0 0x7f82e8117da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82e7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81077f0 0x7f82e8117da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60644/0 (socket says 172.21.15.57:60644) 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82e7fff700 1 -- 172.21.15.57:0/1963820745 learned_addr learned my addr 172.21.15.57:0/1963820745 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.539+0000 7f82ec88c700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82e810edf0 0x7f82e81a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.540+0000 7f82e7fff700 1 -- 172.21.15.57:0/1963820745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e81083a0 msgr2=0x7f82e81182e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.540+0000 7f82e7fff700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e81083a0 0x7f82e81182e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.540+0000 7f82e7fff700 1 -- 172.21.15.57:0/1963820745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82e810edf0 msgr2=0x7f82e81a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.540+0000 7f82e7fff700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82e810edf0 0x7f82e81a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.540+0000 7f82e7fff700 1 -- 172.21.15.57:0/1963820745 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc01a600 con 0x7f82e81077f0 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.540+0000 7f82ec88c700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82e810edf0 0x7f82e81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:06:59.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.540+0000 7f82e7fff700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81077f0 0x7f82e8117da0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f82e000ba50 tx=0x7f82e000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:59.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.546+0000 7f82e57fa700 1 -- 172.21.15.57:0/1963820745 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e000c9b0 con 0x7f82e81077f0 2023-10-08T14:06:59.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.546+0000 7f82e57fa700 1 -- 172.21.15.57:0/1963820745 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e0014440 con 0x7f82e81077f0 2023-10-08T14:06:59.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.546+0000 7f82e57fa700 1 -- 172.21.15.57:0/1963820745 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0007590 con 0x7f82e81077f0 2023-10-08T14:06:59.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.546+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1963820745 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e81a2770 con 0x7f82e81077f0 2023-10-08T14:06:59.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.546+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1963820745 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e81a9ef0 con 0x7f82e81077f0 2023-10-08T14:06:59.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.551+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1963820745 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e810c050 con 0x7f82e81077f0 2023-10-08T14:06:59.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.553+0000 7f82e57fa700 1 -- 172.21.15.57:0/1963820745 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f82e00076f0 con 0x7f82e81077f0 2023-10-08T14:06:59.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.554+0000 7f82e57fa700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82d00a1730 0x7f82d00a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:06:59.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.554+0000 7f82e57fa700 1 -- 172.21.15.57:0/1963820745 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7f82e00cda90 con 0x7f82e81077f0 2023-10-08T14:06:59.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.561+0000 7f82e57fa700 1 -- 172.21.15.57:0/1963820745 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f82e004f100 con 0x7f82e81077f0 2023-10-08T14:06:59.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.568+0000 7f82e77fe700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82d00a1730 0x7f82d00a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:06:59.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.569+0000 7f82e77fe700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82d00a1730 0x7f82d00a3bf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f82dc01a5d0 tx=0x7f82dc01a540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:06:59.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.954+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1963820745 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82e8118ec0 con 0x7f82e81077f0 2023-10-08T14:06:59.971 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T14:06:59.971 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":112,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T14:06:43.512968+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":24,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":24,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":24,"length":1}]},{"pool":4,"snaps":[{"begin":12,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":1},{"begin":24,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":8,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T14:06:59.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.964+0000 7f82e57fa700 1 -- 172.21.15.57:0/1963820745 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+25150 (secure 0 0 0) 0x7f82e0096ab0 con 0x7f82e81077f0 2023-10-08T14:06:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.980+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1963820745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82d00a1730 msgr2=0x7f82d00a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.980+0000 7f82ee2ef700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82d00a1730 0x7f82d00a3bf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f82dc01a5d0 tx=0x7f82dc01a540 comp rx=0 tx=0).stop 2023-10-08T14:06:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.980+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1963820745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81077f0 msgr2=0x7f82e8117da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:06:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.980+0000 7f82ee2ef700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81077f0 0x7f82e8117da0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f82e000ba50 tx=0x7f82e000bd60 comp rx=0 tx=0).stop 2023-10-08T14:06:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.981+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1963820745 shutdown_connections 2023-10-08T14:06:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.981+0000 7f82ee2ef700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f82d00a1730 0x7f82d00a3bf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.981+0000 7f82ee2ef700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82e810edf0 0x7f82e81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.981+0000 7f82ee2ef700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82e81077f0 0x7f82e8117da0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.981+0000 7f82ee2ef700 1 --2- 172.21.15.57:0/1963820745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e81083a0 0x7f82e81182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:06:59.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.981+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1963820745 >> 172.21.15.57:0/1963820745 conn(0x7f82e806b760 msgr2=0x7f82e806f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:06:59.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:06:59.998+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1963820745 shutdown_connections 2023-10-08T14:07:00.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.002+0000 7f82ee2ef700 1 -- 172.21.15.57:0/1963820745 wait complete. 2023-10-08T14:07:00.036 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T14:07:00.037 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T14:07:00.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.311+0000 7f03df0f4700 1 -- 172.21.15.57:0/4009512080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03d8119fd0 msgr2=0x7f03d811c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:00.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.311+0000 7f03df0f4700 1 --2- 172.21.15.57:0/4009512080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03d8119fd0 0x7f03d811c3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f03c80098f0 tx=0x7f03c8009c00 comp rx=0 tx=0).stop 2023-10-08T14:07:00.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.317+0000 7f03df0f4700 1 -- 172.21.15.57:0/4009512080 shutdown_connections 2023-10-08T14:07:00.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.317+0000 7f03df0f4700 1 --2- 172.21.15.57:0/4009512080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03d8119fd0 0x7f03d811c3c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:00.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.317+0000 7f03df0f4700 1 --2- 172.21.15.57:0/4009512080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03d810ef80 0x7f03d81077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:00.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.317+0000 7f03df0f4700 1 --2- 172.21.15.57:0/4009512080 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03d8107d30 0x7f03d81081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:00.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.317+0000 7f03df0f4700 1 -- 172.21.15.57:0/4009512080 >> 172.21.15.57:0/4009512080 conn(0x7f03d806b830 msgr2=0x7f03d806bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:07:00.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.317+0000 7f03df0f4700 1 -- 172.21.15.57:0/4009512080 shutdown_connections 2023-10-08T14:07:00.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.317+0000 7f03df0f4700 1 -- 172.21.15.57:0/4009512080 wait complete. 2023-10-08T14:07:00.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.318+0000 7f03df0f4700 1 Processor -- start 2023-10-08T14:07:00.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.318+0000 7f03df0f4700 1 -- start start 2023-10-08T14:07:00.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.318+0000 7f03df0f4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03d8107d30 0x7f03d81a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:00.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.318+0000 7f03df0f4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03d810ef80 0x7f03d81a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:00.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.318+0000 7f03df0f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03d8119fd0 0x7f03d81aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:00.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.318+0000 7f03df0f4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d81aa940 con 0x7f03d810ef80 2023-10-08T14:07:00.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.318+0000 7f03df0f4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d81aaab0 con 0x7f03d8119fd0 2023-10-08T14:07:00.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.318+0000 7f03df0f4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d81aac70 con 0x7f03d8107d30 2023-10-08T14:07:00.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.319+0000 7f03dce90700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03d8107d30 0x7f03d81a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:00.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.319+0000 7f03dce90700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03d8107d30 0x7f03d81a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33236/0 (socket says 172.21.15.57:33236) 2023-10-08T14:07:00.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.319+0000 7f03dce90700 1 -- 172.21.15.57:0/4216783192 learned_addr learned my addr 172.21.15.57:0/4216783192 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:07:00.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.319+0000 7f03dd691700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03d8119fd0 0x7f03d81aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:00.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.319+0000 7f03dd691700 1 -- 172.21.15.57:0/4216783192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03d8107d30 msgr2=0x7f03d81a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:00.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.319+0000 7f03dd691700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03d8107d30 0x7f03d81a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:00.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.319+0000 7f03dd691700 1 -- 172.21.15.57:0/4216783192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03d810ef80 msgr2=0x7f03d81a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:00.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.319+0000 7f03dd691700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03d810ef80 0x7f03d81a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:00.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.319+0000 7f03dd691700 1 -- 172.21.15.57:0/4216783192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c8009520 con 0x7f03d8119fd0 2023-10-08T14:07:00.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.319+0000 7f03dce90700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03d8107d30 0x7f03d81a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:07:00.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.321+0000 7f03dd691700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03d8119fd0 0x7f03d81aa370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009230 tx=0x7f03c800fa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:07:00.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.323+0000 7f03d5ffb700 1 -- 172.21.15.57:0/4216783192 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c801c070 con 0x7f03d8119fd0 2023-10-08T14:07:00.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.323+0000 7f03d5ffb700 1 -- 172.21.15.57:0/4216783192 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03c8005500 con 0x7f03d8119fd0 2023-10-08T14:07:00.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.323+0000 7f03d5ffb700 1 -- 172.21.15.57:0/4216783192 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8004b40 con 0x7f03d8119fd0 2023-10-08T14:07:00.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.323+0000 7f03df0f4700 1 -- 172.21.15.57:0/4216783192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d81aaef0 con 0x7f03d8119fd0 2023-10-08T14:07:00.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.323+0000 7f03df0f4700 1 -- 172.21.15.57:0/4216783192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d81b2620 con 0x7f03d8119fd0 2023-10-08T14:07:00.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.326+0000 7f03d5ffb700 1 -- 172.21.15.57:0/4216783192 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f03c800b920 con 0x7f03d8119fd0 2023-10-08T14:07:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.327+0000 7f03d5ffb700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f03c00a1740 0x7f03c00a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.327+0000 7f03d5ffb700 1 -- 172.21.15.57:0/4216783192 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7f03c80ce5e0 con 0x7f03d8119fd0 2023-10-08T14:07:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.328+0000 7f03df0f4700 1 -- 172.21.15.57:0/4216783192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d806ccb0 con 0x7f03d8119fd0 2023-10-08T14:07:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.329+0000 7f03dce90700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f03c00a1740 0x7f03c00a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.330+0000 7f03dce90700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f03c00a1740 0x7f03c00a3c00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f03cc005950 tx=0x7f03cc009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:07:00.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.333+0000 7f03d5ffb700 1 -- 172.21.15.57:0/4216783192 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f03c80975e0 con 0x7f03d8119fd0 2023-10-08T14:07:00.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:00 smithi115 ceph-mon[104408]: pgmap v782: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 275 KiB/s wr, 123 op/s 2023-10-08T14:07:00.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:00 smithi115 ceph-mon[104408]: from='client.26983 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T14:07:00.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:00 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1963820745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T14:07:00.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:00 smithi057 ceph-mon[99476]: pgmap v782: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 275 KiB/s wr, 123 op/s 2023-10-08T14:07:00.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:00 smithi057 ceph-mon[99476]: from='client.26983 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T14:07:00.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:00 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1963820745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T14:07:00.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:00 smithi099 ceph-mon[103849]: pgmap v782: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 275 KiB/s wr, 123 op/s 2023-10-08T14:07:00.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:00 smithi099 ceph-mon[103849]: from='client.26983 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T14:07:00.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:00 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1963820745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T14:07:00.732 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T14:07:00.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.724+0000 7f03df0f4700 1 -- 172.21.15.57:0/4216783192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03d8067530 con 0x7f03d8119fd0 2023-10-08T14:07:00.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.728+0000 7f03d5ffb700 1 -- 172.21.15.57:0/4216783192 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03d8067530 con 0x7f03d8119fd0 2023-10-08T14:07:00.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.733+0000 7f03df0f4700 1 -- 172.21.15.57:0/4216783192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f03c00a1740 msgr2=0x7f03c00a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:00.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.733+0000 7f03df0f4700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f03c00a1740 0x7f03c00a3c00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f03cc005950 tx=0x7f03cc009310 comp rx=0 tx=0).stop 2023-10-08T14:07:00.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.733+0000 7f03df0f4700 1 -- 172.21.15.57:0/4216783192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03d8119fd0 msgr2=0x7f03d81aa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:00.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.733+0000 7f03df0f4700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03d8119fd0 0x7f03d81aa370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009230 tx=0x7f03c800fa90 comp rx=0 tx=0).stop 2023-10-08T14:07:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.740+0000 7f03df0f4700 1 -- 172.21.15.57:0/4216783192 shutdown_connections 2023-10-08T14:07:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.740+0000 7f03df0f4700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f03c00a1740 0x7f03c00a3c00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.740+0000 7f03df0f4700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03d8119fd0 0x7f03d81aa370 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.740+0000 7f03df0f4700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03d8107d30 0x7f03d81a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.740+0000 7f03df0f4700 1 --2- 172.21.15.57:0/4216783192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f03d810ef80 0x7f03d81a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:00.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.740+0000 7f03df0f4700 1 -- 172.21.15.57:0/4216783192 >> 172.21.15.57:0/4216783192 conn(0x7f03d806b830 msgr2=0x7f03d810cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:07:00.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.745+0000 7f03df0f4700 1 -- 172.21.15.57:0/4216783192 shutdown_connections 2023-10-08T14:07:00.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:00.750+0000 7f03df0f4700 1 -- 172.21.15.57:0/4216783192 wait complete. 2023-10-08T14:07:00.771 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T14:07:01.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.061+0000 7f1e258b4700 1 -- 172.21.15.57:0/185651568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e20105640 msgr2=0x7f1e20105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:01.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.061+0000 7f1e258b4700 1 --2- 172.21.15.57:0/185651568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e20105640 0x7f1e20105a20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009990 tx=0x7f1e10009ca0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.066+0000 7f1e258b4700 1 -- 172.21.15.57:0/185651568 shutdown_connections 2023-10-08T14:07:01.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.066+0000 7f1e258b4700 1 --2- 172.21.15.57:0/185651568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e20100000 0x7f1e2010c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.066+0000 7f1e258b4700 1 --2- 172.21.15.57:0/185651568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e20105640 0x7f1e20105a20 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.066+0000 7f1e258b4700 1 --2- 172.21.15.57:0/185651568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e200ff640 0x7f1e200ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.066+0000 7f1e258b4700 1 -- 172.21.15.57:0/185651568 >> 172.21.15.57:0/185651568 conn(0x7f1e200767f0 msgr2=0x7f1e20076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:07:01.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.078+0000 7f1e258b4700 1 -- 172.21.15.57:0/185651568 shutdown_connections 2023-10-08T14:07:01.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.084+0000 7f1e258b4700 1 -- 172.21.15.57:0/185651568 wait complete. 2023-10-08T14:07:01.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.085+0000 7f1e258b4700 1 Processor -- start 2023-10-08T14:07:01.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.085+0000 7f1e258b4700 1 -- start start 2023-10-08T14:07:01.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e258b4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e200ff640 0x7f1e2019d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:01.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e258b4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e20100000 0x7f1e2019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:01.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e258b4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e20105640 0x7f1e201a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:01.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e258b4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201a21e0 con 0x7f1e20100000 2023-10-08T14:07:01.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e258b4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201a2350 con 0x7f1e20105640 2023-10-08T14:07:01.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e258b4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201a2510 con 0x7f1e200ff640 2023-10-08T14:07:01.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e1effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e200ff640 0x7f1e2019d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:01.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e1e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e20100000 0x7f1e2019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:01.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e1effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e200ff640 0x7f1e2019d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33248/0 (socket says 172.21.15.57:33248) 2023-10-08T14:07:01.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e1effd700 1 -- 172.21.15.57:0/2885020759 learned_addr learned my addr 172.21.15.57:0/2885020759 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:07:01.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e1f7fe700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e20105640 0x7f1e201a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:01.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e1e7fc700 1 -- 172.21.15.57:0/2885020759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e200ff640 msgr2=0x7f1e2019d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:01.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e1e7fc700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e200ff640 0x7f1e2019d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e1e7fc700 1 -- 172.21.15.57:0/2885020759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e20105640 msgr2=0x7f1e201a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:01.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e1e7fc700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e20105640 0x7f1e201a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.086+0000 7f1e1e7fc700 1 -- 172.21.15.57:0/2885020759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100095f0 con 0x7f1e20100000 2023-10-08T14:07:01.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.087+0000 7f1e1e7fc700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e20100000 0x7f1e2019d800 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1e14006fd0 tx=0x7f1e1400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:07:01.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.090+0000 7f1e248b2700 1 -- 172.21.15.57:0/2885020759 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14011830 con 0x7f1e20100000 2023-10-08T14:07:01.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.090+0000 7f1e248b2700 1 -- 172.21.15.57:0/2885020759 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e14011e50 con 0x7f1e20100000 2023-10-08T14:07:01.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.090+0000 7f1e248b2700 1 -- 172.21.15.57:0/2885020759 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14010d20 con 0x7f1e20100000 2023-10-08T14:07:01.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.090+0000 7f1e258b4700 1 -- 172.21.15.57:0/2885020759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e201a27f0 con 0x7f1e20100000 2023-10-08T14:07:01.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.090+0000 7f1e258b4700 1 -- 172.21.15.57:0/2885020759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e201949e0 con 0x7f1e20100000 2023-10-08T14:07:01.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.092+0000 7f1e258b4700 1 -- 172.21.15.57:0/2885020759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e201a2a10 con 0x7f1e20100000 2023-10-08T14:07:01.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.104+0000 7f1e248b2700 1 -- 172.21.15.57:0/2885020759 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1e14010380 con 0x7f1e20100000 2023-10-08T14:07:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.105+0000 7f1e248b2700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e080a16a0 0x7f1e080a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.105+0000 7f1e248b2700 1 -- 172.21.15.57:0/2885020759 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7f1e140cd0e0 con 0x7f1e20100000 2023-10-08T14:07:01.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.105+0000 7f1e1effd700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e080a16a0 0x7f1e080a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:01.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.106+0000 7f1e1effd700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e080a16a0 0x7f1e080a3b60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000b2e0 tx=0x7f1e1000b610 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:07:01.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.107+0000 7f1e248b2700 1 -- 172.21.15.57:0/2885020759 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1e14096160 con 0x7f1e20100000 2023-10-08T14:07:01.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:01 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4216783192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T14:07:01.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.473+0000 7f1e258b4700 1 -- 172.21.15.57:0/2885020759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e20067980 con 0x7f1e20100000 2023-10-08T14:07:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.479+0000 7f1e248b2700 1 -- 172.21.15.57:0/2885020759 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e14099780 con 0x7f1e20100000 2023-10-08T14:07:01.480 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T14:07:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.482+0000 7f1e258b4700 1 -- 172.21.15.57:0/2885020759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e080a16a0 msgr2=0x7f1e080a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.482+0000 7f1e258b4700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e080a16a0 0x7f1e080a3b60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000b2e0 tx=0x7f1e1000b610 comp rx=0 tx=0).stop 2023-10-08T14:07:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.482+0000 7f1e258b4700 1 -- 172.21.15.57:0/2885020759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e20100000 msgr2=0x7f1e2019d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.482+0000 7f1e258b4700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e20100000 0x7f1e2019d800 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1e14006fd0 tx=0x7f1e1400b730 comp rx=0 tx=0).stop 2023-10-08T14:07:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.482+0000 7f1e258b4700 1 -- 172.21.15.57:0/2885020759 shutdown_connections 2023-10-08T14:07:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.482+0000 7f1e258b4700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1e080a16a0 0x7f1e080a3b60 secure :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000b2e0 tx=0x7f1e1000b610 comp rx=0 tx=0).stop 2023-10-08T14:07:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.482+0000 7f1e258b4700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e20105640 0x7f1e201a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.482+0000 7f1e258b4700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e200ff640 0x7f1e2019d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.482+0000 7f1e258b4700 1 --2- 172.21.15.57:0/2885020759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1e20100000 0x7f1e2019d800 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.482+0000 7f1e258b4700 1 -- 172.21.15.57:0/2885020759 >> 172.21.15.57:0/2885020759 conn(0x7f1e200767f0 msgr2=0x7f1e200fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:07:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.485+0000 7f1e258b4700 1 -- 172.21.15.57:0/2885020759 shutdown_connections 2023-10-08T14:07:01.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.486+0000 7f1e258b4700 1 -- 172.21.15.57:0/2885020759 wait complete. 2023-10-08T14:07:01.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:01 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4216783192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T14:07:01.512 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T14:07:01.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:01 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4216783192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T14:07:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.807+0000 7fe7770e0700 1 -- 172.21.15.57:0/2046187901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe770108a40 msgr2=0x7fe770114480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.807+0000 7fe7770e0700 1 --2- 172.21.15.57:0/2046187901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe770108a40 0x7fe770114480 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe75800aad0 tx=0x7fe75800ade0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.807+0000 7fe7770e0700 1 -- 172.21.15.57:0/2046187901 shutdown_connections 2023-10-08T14:07:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.807+0000 7fe7770e0700 1 --2- 172.21.15.57:0/2046187901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe770108a40 0x7fe770114480 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.807+0000 7fe7770e0700 1 --2- 172.21.15.57:0/2046187901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe77010c5f0 0x7fe77010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.807+0000 7fe7770e0700 1 --2- 172.21.15.57:0/2046187901 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe770108080 0x7fe770108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.807+0000 7fe7770e0700 1 -- 172.21.15.57:0/2046187901 >> 172.21.15.57:0/2046187901 conn(0x7fe77006b830 msgr2=0x7fe77006bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:07:01.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.810+0000 7fe7770e0700 1 -- 172.21.15.57:0/2046187901 shutdown_connections 2023-10-08T14:07:01.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.810+0000 7fe7770e0700 1 -- 172.21.15.57:0/2046187901 wait complete. 2023-10-08T14:07:01.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.811+0000 7fe7770e0700 1 Processor -- start 2023-10-08T14:07:01.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.811+0000 7fe7770e0700 1 -- start start 2023-10-08T14:07:01.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.811+0000 7fe7770e0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe770108080 0x7fe7701115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:01.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.811+0000 7fe7770e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe770108a40 0x7fe770111af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:01.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.811+0000 7fe7770e0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe770112030 0x7fe7701b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:01.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.811+0000 7fe7770e0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe770114740 con 0x7fe770108080 2023-10-08T14:07:01.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.811+0000 7fe7770e0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7701b2490 con 0x7fe770108a40 2023-10-08T14:07:01.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.811+0000 7fe7770e0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7701b25d0 con 0x7fe770112030 2023-10-08T14:07:01.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.812+0000 7fe77567d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe770112030 0x7fe7701b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:01.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.812+0000 7fe77567d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe770112030 0x7fe7701b1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33258/0 (socket says 172.21.15.57:33258) 2023-10-08T14:07:01.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.812+0000 7fe77567d700 1 -- 172.21.15.57:0/4165834357 learned_addr learned my addr 172.21.15.57:0/4165834357 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:07:01.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.812+0000 7fe76ffff700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe770108a40 0x7fe770111af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:01.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.812+0000 7fe77567d700 1 -- 172.21.15.57:0/4165834357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe770108a40 msgr2=0x7fe770111af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.812+0000 7fe77567d700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe770108a40 0x7fe770111af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.812+0000 7fe77567d700 1 -- 172.21.15.57:0/4165834357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe770108080 msgr2=0x7fe7701115b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:07:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.812+0000 7fe77567d700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe770108080 0x7fe7701115b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.812+0000 7fe77567d700 1 -- 172.21.15.57:0/4165834357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75800a700 con 0x7fe770112030 2023-10-08T14:07:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.812+0000 7fe77567d700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe770112030 0x7fe7701b1f50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe758004230 tx=0x7fe758004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:07:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.813+0000 7fe76dffb700 1 -- 172.21.15.57:0/4165834357 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe758003dd0 con 0x7fe770112030 2023-10-08T14:07:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.813+0000 7fe7770e0700 1 -- 172.21.15.57:0/4165834357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7701b27d0 con 0x7fe770112030 2023-10-08T14:07:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.813+0000 7fe7770e0700 1 -- 172.21.15.57:0/4165834357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7701b2c50 con 0x7fe770112030 2023-10-08T14:07:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.813+0000 7fe76dffb700 1 -- 172.21.15.57:0/4165834357 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe758003890 con 0x7fe770112030 2023-10-08T14:07:01.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.813+0000 7fe76dffb700 1 -- 172.21.15.57:0/4165834357 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe758016680 con 0x7fe770112030 2023-10-08T14:07:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.819+0000 7fe7770e0700 1 -- 172.21.15.57:0/4165834357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7500052f0 con 0x7fe770112030 2023-10-08T14:07:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.821+0000 7fe76dffb700 1 -- 172.21.15.57:0/4165834357 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe7580167e0 con 0x7fe770112030 2023-10-08T14:07:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.822+0000 7fe76dffb700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe75c0a1730 0x7fe75c0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.822+0000 7fe76dffb700 1 -- 172.21.15.57:0/4165834357 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7fe758012070 con 0x7fe770112030 2023-10-08T14:07:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.826+0000 7fe774e7c700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe75c0a1730 0x7fe75c0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.827+0000 7fe774e7c700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe75c0a1730 0x7fe75c0a3bf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe77006ca00 tx=0x7fe768009230 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:07:01.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:01.832+0000 7fe76dffb700 1 -- 172.21.15.57:0/4165834357 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe758097d00 con 0x7fe770112030 2023-10-08T14:07:02.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.213+0000 7fe7770e0700 1 -- 172.21.15.57:0/4165834357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe750005b20 con 0x7fe770112030 2023-10-08T14:07:02.220 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T14:07:02.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.219+0000 7fe76dffb700 1 -- 172.21.15.57:0/4165834357 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe75809b320 con 0x7fe770112030 2023-10-08T14:07:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.231+0000 7fe7770e0700 1 -- 172.21.15.57:0/4165834357 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe75c0a1730 msgr2=0x7fe75c0a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:02.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.231+0000 7fe7770e0700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe75c0a1730 0x7fe75c0a3bf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe77006ca00 tx=0x7fe768009230 comp rx=0 tx=0).stop 2023-10-08T14:07:02.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.231+0000 7fe7770e0700 1 -- 172.21.15.57:0/4165834357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe770112030 msgr2=0x7fe7701b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:02.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.231+0000 7fe7770e0700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe770112030 0x7fe7701b1f50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe758004230 tx=0x7fe758004260 comp rx=0 tx=0).stop 2023-10-08T14:07:02.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.236+0000 7fe7770e0700 1 -- 172.21.15.57:0/4165834357 shutdown_connections 2023-10-08T14:07:02.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.236+0000 7fe7770e0700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe770112030 0x7fe7701b1f50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:02.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.236+0000 7fe7770e0700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe770108080 0x7fe7701115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:02.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.236+0000 7fe7770e0700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe75c0a1730 0x7fe75c0a3bf0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:02.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.236+0000 7fe7770e0700 1 --2- 172.21.15.57:0/4165834357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe770108a40 0x7fe770111af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:02.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.236+0000 7fe7770e0700 1 -- 172.21.15.57:0/4165834357 >> 172.21.15.57:0/4165834357 conn(0x7fe77006b830 msgr2=0x7fe770070150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:07:02.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.243+0000 7fe7770e0700 1 -- 172.21.15.57:0/4165834357 shutdown_connections 2023-10-08T14:07:02.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.245+0000 7fe7770e0700 1 -- 172.21.15.57:0/4165834357 wait complete. 2023-10-08T14:07:02.274 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T14:07:02.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:02 smithi115 ceph-mon[104408]: pgmap v783: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 350 KiB/s wr, 161 op/s 2023-10-08T14:07:02.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:02 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2885020759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T14:07:02.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:02 smithi057 ceph-mon[99476]: pgmap v783: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 350 KiB/s wr, 161 op/s 2023-10-08T14:07:02.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:02 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2885020759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T14:07:02.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:02 smithi099 ceph-mon[103849]: pgmap v783: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 350 KiB/s wr, 161 op/s 2023-10-08T14:07:02.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:02 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2885020759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T14:07:02.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.557+0000 7f1fef34d700 1 -- 172.21.15.57:0/3295562463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fe80ff0b0 msgr2=0x7f1fe80ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:02.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.557+0000 7f1fef34d700 1 --2- 172.21.15.57:0/3295562463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fe80ff0b0 0x7f1fe80ff560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc013f60 tx=0x7f1fdc013f90 comp rx=0 tx=0).stop 2023-10-08T14:07:02.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.564+0000 7f1fef34d700 1 -- 172.21.15.57:0/3295562463 shutdown_connections 2023-10-08T14:07:02.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.564+0000 7f1fef34d700 1 --2- 172.21.15.57:0/3295562463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fe80ff0b0 0x7f1fe80ff560 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:02.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.564+0000 7f1fef34d700 1 --2- 172.21.15.57:0/3295562463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe8068440 0x7f1fe8103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:02.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.564+0000 7f1fef34d700 1 --2- 172.21.15.57:0/3295562463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fe80688b0 0x7f1fe81042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:02.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.564+0000 7f1fef34d700 1 -- 172.21.15.57:0/3295562463 >> 172.21.15.57:0/3295562463 conn(0x7f1fe8074940 msgr2=0x7f1fe8074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:07:02.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.570+0000 7f1fef34d700 1 -- 172.21.15.57:0/3295562463 shutdown_connections 2023-10-08T14:07:02.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.573+0000 7f1fef34d700 1 -- 172.21.15.57:0/3295562463 wait complete. 2023-10-08T14:07:02.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.574+0000 7f1fef34d700 1 Processor -- start 2023-10-08T14:07:02.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.574+0000 7f1fef34d700 1 -- start start 2023-10-08T14:07:02.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.574+0000 7f1fef34d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fe8068440 0x7f1fe8071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:02.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.574+0000 7f1fef34d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fe80688b0 0x7f1fe80722f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:02.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.574+0000 7f1fef34d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe80ff0b0 0x7f1fe806af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:02.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.574+0000 7f1fef34d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe806b6e0 con 0x7f1fe80688b0 2023-10-08T14:07:02.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.574+0000 7f1fef34d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe806b850 con 0x7f1fe80ff0b0 2023-10-08T14:07:02.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.574+0000 7f1fef34d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe806ba10 con 0x7f1fe8068440 2023-10-08T14:07:02.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.579+0000 7f1fed8ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe80ff0b0 0x7f1fe806af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:02.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.579+0000 7f1fed8ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe80ff0b0 0x7f1fe806af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33882/0 (socket says 172.21.15.57:33882) 2023-10-08T14:07:02.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.579+0000 7f1fed8ea700 1 -- 172.21.15.57:0/2438124807 learned_addr learned my addr 172.21.15.57:0/2438124807 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:07:02.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.579+0000 7f1fec8e8700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fe80688b0 0x7f1fe80722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:02.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.581+0000 7f1fed8ea700 1 -- 172.21.15.57:0/2438124807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fe8068440 msgr2=0x7f1fe8071d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T14:07:02.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.581+0000 7f1fed8ea700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fe8068440 0x7f1fe8071d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:02.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.581+0000 7f1fed8ea700 1 -- 172.21.15.57:0/2438124807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fe80688b0 msgr2=0x7f1fe80722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:02.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.581+0000 7f1fed8ea700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fe80688b0 0x7f1fe80722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:02.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.581+0000 7f1fed8ea700 1 -- 172.21.15.57:0/2438124807 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc013b90 con 0x7f1fe80ff0b0 2023-10-08T14:07:02.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.581+0000 7f1fed8ea700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe80ff0b0 0x7f1fe806af10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009af0 tx=0x7f1fdc01a840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:07:02.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.587+0000 7f1fe67fc700 1 -- 172.21.15.57:0/2438124807 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc01aeb0 con 0x7f1fe80ff0b0 2023-10-08T14:07:02.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.587+0000 7f1fe67fc700 1 -- 172.21.15.57:0/2438124807 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fdc005500 con 0x7f1fe80ff0b0 2023-10-08T14:07:02.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.587+0000 7f1fe67fc700 1 -- 172.21.15.57:0/2438124807 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc003b10 con 0x7f1fe80ff0b0 2023-10-08T14:07:02.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.587+0000 7f1fef34d700 1 -- 172.21.15.57:0/2438124807 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe806bc90 con 0x7f1fe80ff0b0 2023-10-08T14:07:02.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.587+0000 7f1fef34d700 1 -- 172.21.15.57:0/2438124807 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe806fb40 con 0x7f1fe80ff0b0 2023-10-08T14:07:02.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.591+0000 7f1fef34d700 1 -- 172.21.15.57:0/2438124807 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd40052f0 con 0x7f1fe80ff0b0 2023-10-08T14:07:02.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.600+0000 7f1fe67fc700 1 -- 172.21.15.57:0/2438124807 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1fdc003c70 con 0x7f1fe80ff0b0 2023-10-08T14:07:02.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.600+0000 7f1fe67fc700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1fd00a1800 0x7f1fd00a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:02.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.601+0000 7f1fe67fc700 1 -- 172.21.15.57:0/2438124807 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7f1fdc0d6210 con 0x7f1fe80ff0b0 2023-10-08T14:07:02.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.602+0000 7f1fed0e9700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1fd00a1800 0x7f1fd00a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:02.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.602+0000 7f1fed0e9700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1fd00a1800 0x7f1fd00a3cc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8077170 tx=0x7f1fd8009230 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:07:02.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:02.605+0000 7f1fe67fc700 1 -- 172.21.15.57:0/2438124807 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1fdc09f1b0 con 0x7f1fe80ff0b0 2023-10-08T14:07:03.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.009+0000 7f1fef34d700 1 -- 172.21.15.57:0/2438124807 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1fd4005b20 con 0x7f1fe80ff0b0 2023-10-08T14:07:03.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.014+0000 7f1fe67fc700 1 -- 172.21.15.57:0/2438124807 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v112) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1fdc0a27d0 con 0x7f1fe80ff0b0 2023-10-08T14:07:03.015 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T14:07:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.018+0000 7f1fef34d700 1 -- 172.21.15.57:0/2438124807 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1fd00a1800 msgr2=0x7f1fd00a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.018+0000 7f1fef34d700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1fd00a1800 0x7f1fd00a3cc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8077170 tx=0x7f1fd8009230 comp rx=0 tx=0).stop 2023-10-08T14:07:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.018+0000 7f1fef34d700 1 -- 172.21.15.57:0/2438124807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe80ff0b0 msgr2=0x7f1fe806af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.018+0000 7f1fef34d700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe80ff0b0 0x7f1fe806af10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009af0 tx=0x7f1fdc01a840 comp rx=0 tx=0).stop 2023-10-08T14:07:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.019+0000 7f1fef34d700 1 -- 172.21.15.57:0/2438124807 shutdown_connections 2023-10-08T14:07:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.019+0000 7f1fef34d700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f1fd00a1800 0x7f1fd00a3cc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.019+0000 7f1fef34d700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe80ff0b0 0x7f1fe806af10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.019+0000 7f1fef34d700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fe8068440 0x7f1fe8071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.019+0000 7f1fef34d700 1 --2- 172.21.15.57:0/2438124807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1fe80688b0 0x7f1fe80722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:03.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.019+0000 7f1fef34d700 1 -- 172.21.15.57:0/2438124807 >> 172.21.15.57:0/2438124807 conn(0x7f1fe8074940 msgr2=0x7f1fe80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:07:03.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.024+0000 7f1fef34d700 1 -- 172.21.15.57:0/2438124807 shutdown_connections 2023-10-08T14:07:03.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.032+0000 7f1fef34d700 1 -- 172.21.15.57:0/2438124807 wait complete. 2023-10-08T14:07:03.083 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-08T14:07:03.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.354+0000 7fc2f6945700 1 -- 172.21.15.57:0/1452164788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f01077f0 msgr2=0x7fc2f0107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:03.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.354+0000 7fc2f6945700 1 --2- 172.21.15.57:0/1452164788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f01077f0 0x7fc2f0107c70 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc2e001a9a0 tx=0x7fc2e001acb0 comp rx=0 tx=0).stop 2023-10-08T14:07:03.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.373+0000 7fc2f6945700 1 -- 172.21.15.57:0/1452164788 shutdown_connections 2023-10-08T14:07:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.373+0000 7fc2f6945700 1 --2- 172.21.15.57:0/1452164788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2f01083a0 0x7fc2f011bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.373+0000 7fc2f6945700 1 --2- 172.21.15.57:0/1452164788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2f010edf0 0x7fc2f010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.373+0000 7fc2f6945700 1 --2- 172.21.15.57:0/1452164788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f01077f0 0x7fc2f0107c70 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.373+0000 7fc2f6945700 1 -- 172.21.15.57:0/1452164788 >> 172.21.15.57:0/1452164788 conn(0x7fc2f006b760 msgr2=0x7fc2f006bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:07:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.374+0000 7fc2f6945700 1 -- 172.21.15.57:0/1452164788 shutdown_connections 2023-10-08T14:07:03.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.380+0000 7fc2f6945700 1 -- 172.21.15.57:0/1452164788 wait complete. 2023-10-08T14:07:03.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.381+0000 7fc2f6945700 1 Processor -- start 2023-10-08T14:07:03.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.388+0000 7fc2f6945700 1 -- start start 2023-10-08T14:07:03.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.389+0000 7fc2f6945700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2f01077f0 0x7fc2f0117da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:03.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.389+0000 7fc2f6945700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f01083a0 0x7fc2f01182e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.389+0000 7fc2f6945700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2f010edf0 0x7fc2f01c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.389+0000 7fc2f6945700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f01c4170 con 0x7fc2f01083a0 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.389+0000 7fc2f6945700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f01c42e0 con 0x7fc2f01077f0 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.389+0000 7fc2f6945700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f01c44a0 con 0x7fc2f010edf0 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.390+0000 7fc2effff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2f01077f0 0x7fc2f0117da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.390+0000 7fc2effff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2f01077f0 0x7fc2f0117da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33894/0 (socket says 172.21.15.57:33894) 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.390+0000 7fc2effff700 1 -- 172.21.15.57:0/953513458 learned_addr learned my addr 172.21.15.57:0/953513458 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.390+0000 7fc2effff700 1 -- 172.21.15.57:0/953513458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2f010edf0 msgr2=0x7fc2f01c3b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.390+0000 7fc2effff700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2f010edf0 0x7fc2f01c3b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.390+0000 7fc2effff700 1 -- 172.21.15.57:0/953513458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f01083a0 msgr2=0x7fc2f01182e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:07:03.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.390+0000 7fc2effff700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f01083a0 0x7fc2f01182e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:03.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.390+0000 7fc2effff700 1 -- 172.21.15.57:0/953513458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e001a600 con 0x7fc2f01077f0 2023-10-08T14:07:03.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.391+0000 7fc2effff700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2f01077f0 0x7fc2f0117da0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400b550 tx=0x7fc2e400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:07:03.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.391+0000 7fc2ed7fa700 1 -- 172.21.15.57:0/953513458 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4011830 con 0x7fc2f01077f0 2023-10-08T14:07:03.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.391+0000 7fc2ed7fa700 1 -- 172.21.15.57:0/953513458 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e4011e50 con 0x7fc2f01077f0 2023-10-08T14:07:03.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.391+0000 7fc2ed7fa700 1 -- 172.21.15.57:0/953513458 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400f2e0 con 0x7fc2f01077f0 2023-10-08T14:07:03.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.391+0000 7fc2f6945700 1 -- 172.21.15.57:0/953513458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f01c4780 con 0x7fc2f01077f0 2023-10-08T14:07:03.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.391+0000 7fc2f6945700 1 -- 172.21.15.57:0/953513458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f01cbf00 con 0x7fc2f01077f0 2023-10-08T14:07:03.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.395+0000 7fc2ed7fa700 1 -- 172.21.15.57:0/953513458 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc2e400f440 con 0x7fc2f01077f0 2023-10-08T14:07:03.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.395+0000 7fc2f6945700 1 -- 172.21.15.57:0/953513458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f010c050 con 0x7fc2f01077f0 2023-10-08T14:07:03.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.395+0000 7fc2ed7fa700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc2d80a1800 0x7fc2d80a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:07:03.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.396+0000 7fc2ed7fa700 1 -- 172.21.15.57:0/953513458 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9457+0+0 (secure 0 0 0) 0x7fc2e40cde60 con 0x7fc2f01077f0 2023-10-08T14:07:03.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.401+0000 7fc2ef7fe700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc2d80a1800 0x7fc2d80a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:07:03.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.407+0000 7fc2ed7fa700 1 -- 172.21.15.57:0/953513458 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc2e4096e80 con 0x7fc2f01077f0 2023-10-08T14:07:03.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.412+0000 7fc2ef7fe700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc2d80a1800 0x7fc2d80a3cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0000c00 tx=0x7fc2e0005a90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:07:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:03 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4165834357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T14:07:03.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:03 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2438124807' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T14:07:03.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:03 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4165834357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T14:07:03.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:03 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2438124807' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T14:07:03.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:03 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4165834357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T14:07:03.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:03 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2438124807' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T14:07:03.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.881+0000 7fc2f6945700 1 -- 172.21.15.57:0/953513458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fc2f0118ec0 con 0x7fc2f01077f0 2023-10-08T14:07:03.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.899+0000 7fc2ed7fa700 1 -- 172.21.15.57:0/953513458 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+734409 (secure 0 0 0) 0x7fc2e4015020 con 0x7fc2f01077f0 2023-10-08T14:07:03.913 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T14:07:03.913 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T14:07:03.913 INFO:teuthology.orchestra.run.smithi057.stdout: "epoch": 24, 2023-10-08T14:07:03.913 INFO:teuthology.orchestra.run.smithi057.stdout: "active_gid": 14217, 2023-10-08T14:07:03.913 INFO:teuthology.orchestra.run.smithi057.stdout: "active_name": "x", 2023-10-08T14:07:03.913 INFO:teuthology.orchestra.run.smithi057.stdout: "active_addrs": { 2023-10-08T14:07:03.913 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-08T14:07:03.913 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:03.913 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-08T14:07:03.914 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:6800", 2023-10-08T14:07:03.914 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2547096577 2023-10-08T14:07:03.914 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.914 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:03.914 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-08T14:07:03.914 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:6801", 2023-10-08T14:07:03.914 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2547096577 2023-10-08T14:07:03.914 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:03.914 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:03.914 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.915 INFO:teuthology.orchestra.run.smithi057.stdout: "active_addr": "172.21.15.57:6801/2547096577", 2023-10-08T14:07:03.915 INFO:teuthology.orchestra.run.smithi057.stdout: "active_change": "2023-10-08T13:44:14.025199+0000", 2023-10-08T14:07:03.915 INFO:teuthology.orchestra.run.smithi057.stdout: "active_mgr_features": 4540138322906710015, 2023-10-08T14:07:03.915 INFO:teuthology.orchestra.run.smithi057.stdout: "available": true, 2023-10-08T14:07:03.915 INFO:teuthology.orchestra.run.smithi057.stdout: "standbys": [ 2023-10-08T14:07:03.915 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:03.915 INFO:teuthology.orchestra.run.smithi057.stdout: "gid": 24148, 2023-10-08T14:07:03.915 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "z", 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr_features": 4540138322906710015, 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:03.916 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.917 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:03.918 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:03.918 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.918 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.918 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.918 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:03.918 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.918 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.918 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:03.918 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:03.919 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:03.919 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:03.919 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:03.919 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:03.919 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:03.919 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:03.919 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:03.919 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.919 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.920 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.920 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.920 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:03.920 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:03.920 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.921 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.921 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.922 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:03.922 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.923 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.923 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.923 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:03.924 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.924 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.925 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.925 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.925 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:03.925 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:03.926 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.926 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.926 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.926 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:03.926 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.926 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.926 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.927 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.928 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.928 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:03.928 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:03.928 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.929 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.929 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.929 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:03.929 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.929 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.930 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2023-10-08T14:07:03.931 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.932 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.933 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.934 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2023-10-08T14:07:03.935 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.936 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.937 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:03.937 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.937 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.937 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.937 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2023-10-08T14:07:03.937 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.937 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.937 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.937 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.937 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2023-10-08T14:07:03.938 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2023-10-08T14:07:03.938 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.938 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.938 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.938 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:03.938 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.938 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.938 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.938 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-08T14:07:03.938 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.939 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:03.940 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2023-10-08T14:07:03.941 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.942 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.943 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:03.944 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.945 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.946 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.947 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2023-10-08T14:07:03.948 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-08T14:07:03.948 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.948 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.948 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.948 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2023-10-08T14:07:03.948 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2023-10-08T14:07:03.948 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.948 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.948 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.948 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:03.949 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.950 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:03.951 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:03.952 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:03.952 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:03.952 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:03.952 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:03.952 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:03.952 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.952 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.952 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.952 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.952 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:03.953 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:03.953 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.953 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.953 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.953 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:03.953 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.953 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.953 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.953 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:03.953 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.954 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.955 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:03.956 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.957 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.958 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap" 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.959 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.960 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:03.961 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.962 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.962 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.962 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-08T14:07:03.962 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.962 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.962 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.962 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.962 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2023-10-08T14:07:03.962 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.963 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-08T14:07:03.964 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:03.965 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.966 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.967 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2023-10-08T14:07:03.968 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.969 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.970 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.970 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:03.970 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.970 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.970 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.970 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-08T14:07:03.970 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-08T14:07:03.970 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.970 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.970 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-08T14:07:03.971 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.972 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.973 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.974 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-08T14:07:03.975 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.976 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.977 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.977 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-08T14:07:03.977 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.977 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.977 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.977 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.977 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2023-10-08T14:07:03.977 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2023-10-08T14:07:03.977 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.977 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.978 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.979 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.980 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-08T14:07:03.981 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.982 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.983 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2023-10-08T14:07:03.984 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2023-10-08T14:07:03.984 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.984 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.984 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.984 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-08T14:07:03.984 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.984 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.984 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.984 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2023-10-08T14:07:03.984 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.985 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.986 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2023-10-08T14:07:03.986 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.986 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.986 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.986 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.986 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2023-10-08T14:07:03.986 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2023-10-08T14:07:03.986 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.986 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.987 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.988 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.989 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.990 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.991 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.992 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2023-10-08T14:07:03.993 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2023-10-08T14:07:03.993 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.993 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.993 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.993 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-08T14:07:03.993 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.993 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.993 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.993 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2023-10-08T14:07:03.993 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.994 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.995 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2023-10-08T14:07:03.996 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.997 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:03.998 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:03.999 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.000 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.000 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.000 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-08T14:07:04.000 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.000 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.000 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.000 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.000 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2023-10-08T14:07:04.000 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2023-10-08T14:07:04.000 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.001 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2023-10-08T14:07:04.002 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2023-10-08T14:07:04.002 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.002 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.002 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.002 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2023-10-08T14:07:04.002 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.002 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.002 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.002 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-08T14:07:04.002 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.003 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.004 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.005 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.006 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.007 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.008 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.009 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.010 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.011 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.012 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.013 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.013 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.013 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.013 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-08T14:07:04.013 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.013 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.013 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.013 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.013 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.013 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.014 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.015 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.016 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-08T14:07:04.017 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.018 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.018 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.018 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.018 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-08T14:07:04.018 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-08T14:07:04.018 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.018 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.018 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.018 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2023-10-08T14:07:04.018 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.019 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.019 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.019 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-08T14:07:04.019 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.019 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.019 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.019 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.019 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2023-10-08T14:07:04.019 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2023-10-08T14:07:04.019 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.020 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.021 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2023-10-08T14:07:04.021 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2023-10-08T14:07:04.021 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.021 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.021 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.021 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.021 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.021 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.021 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.021 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.022 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.023 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.024 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-08T14:07:04.025 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.026 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.027 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2023-10-08T14:07:04.028 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.029 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.030 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.031 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.032 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2023-10-08T14:07:04.033 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.033 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.033 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.033 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2023-10-08T14:07:04.033 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.033 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.033 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.033 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.033 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2023-10-08T14:07:04.033 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.034 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-08T14:07:04.035 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.036 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.036 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.036 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.036 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2023-10-08T14:07:04.036 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2023-10-08T14:07:04.036 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.036 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.036 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.036 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.036 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.037 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.038 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.038 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.038 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.038 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.038 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.038 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2023-10-08T14:07:04.038 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.038 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.038 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.038 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.039 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.040 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.041 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.042 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.043 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.044 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.045 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.046 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.047 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.047 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.047 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.047 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.047 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.047 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.047 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.047 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.047 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.047 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.048 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.049 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.050 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.051 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.052 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.053 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.054 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-08T14:07:04.055 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.056 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.057 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.058 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.059 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-08T14:07:04.060 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.061 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.062 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.063 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.064 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.065 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-08T14:07:04.066 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.067 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.068 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.069 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.070 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2023-10-08T14:07:04.071 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.072 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.073 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.073 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-08T14:07:04.073 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-08T14:07:04.073 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.073 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.073 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.073 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.073 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.073 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.073 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.074 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.075 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.076 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.077 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.078 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.078 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2023-10-08T14:07:04.078 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-08T14:07:04.078 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.078 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.078 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.078 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.078 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.078 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.078 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.079 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.080 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.080 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.080 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.080 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.080 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.080 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.080 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.080 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-08T14:07:04.080 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.081 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.082 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.082 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-08T14:07:04.082 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-08T14:07:04.082 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.082 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.082 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.082 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.082 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.082 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.082 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.083 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.084 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.084 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.084 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.084 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.084 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.084 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.084 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.084 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-08T14:07:04.084 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-08T14:07:04.084 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.085 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.086 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.087 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.088 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.089 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.090 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.091 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.092 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.093 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.094 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.095 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.096 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.096 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.096 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.096 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.096 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.096 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.096 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.096 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.096 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.096 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.097 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.098 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.099 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "UNSAFE_TLS_v1_2": { 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.100 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-08T14:07:04.101 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.102 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.103 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.104 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2023-10-08T14:07:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2023-10-08T14:07:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.105 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.106 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2023-10-08T14:07:04.107 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.108 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.109 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.110 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.111 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.112 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.112 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.112 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.112 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.112 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.112 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.112 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.112 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.112 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.113 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.113 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.113 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.113 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.113 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.113 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.113 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.113 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.113 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.113 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.114 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.115 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.116 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.117 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.118 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.119 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.120 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-08T14:07:04.121 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.122 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.123 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.124 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.125 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-08T14:07:04.126 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.127 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.128 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.129 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.129 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.129 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.129 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.129 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.129 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2023-10-08T14:07:04.129 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.129 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.129 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.129 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.130 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.131 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.132 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.132 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.132 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.132 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.132 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.132 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.132 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.132 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.132 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.132 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.133 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.134 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.134 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.134 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.134 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.134 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.134 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.134 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.134 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.134 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.134 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.135 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.136 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2023-10-08T14:07:04.137 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.138 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.139 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.140 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.141 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.142 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:04.143 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.144 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.145 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.146 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.146 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.146 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.146 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.146 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.146 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2023-10-08T14:07:04.146 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.146 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.146 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.146 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.147 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.147 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.147 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.147 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.147 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.147 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.147 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.147 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.147 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.147 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.148 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.149 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.150 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.151 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.152 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.153 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.154 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:04.155 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.156 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.156 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.156 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:04.156 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.156 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.156 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.156 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.156 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.156 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.156 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.157 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.158 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.159 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.160 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2023-10-08T14:07:04.161 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.162 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.163 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.164 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.165 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.166 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.167 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2023-10-08T14:07:04.168 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.169 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.170 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.171 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.172 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T14:07:04.172 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.172 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.172 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.172 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.172 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2023-10-08T14:07:04.172 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2023-10-08T14:07:04.172 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.172 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.172 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2023-10-08T14:07:04.173 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.174 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.174 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.174 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.174 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.174 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.174 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.174 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2023-10-08T14:07:04.174 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.174 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.174 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.175 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.176 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.177 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.178 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.179 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.179 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.179 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.179 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.179 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.179 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.179 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.179 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.179 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.179 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.180 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.181 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.182 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.183 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.184 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.185 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.186 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.187 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.188 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.189 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.189 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.189 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.189 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.189 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.189 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.189 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.189 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.189 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.189 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.190 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2023-10-08T14:07:04.191 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.192 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.193 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.194 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.195 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.196 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.196 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.196 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.196 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.196 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.196 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.196 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.196 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.196 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.196 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.197 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.198 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.199 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.199 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.199 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.199 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.199 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.199 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.199 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.199 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.199 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.199 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.200 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.201 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.202 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.203 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.204 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.205 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.206 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.206 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.206 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.206 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.206 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.206 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.206 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.206 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.206 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.206 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.207 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.208 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.208 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.208 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.208 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.208 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.208 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2023-10-08T14:07:04.208 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-08T14:07:04.208 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.208 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.208 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2023-10-08T14:07:04.209 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.210 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.211 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2023-10-08T14:07:04.212 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.213 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.213 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.213 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2023-10-08T14:07:04.213 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.213 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.213 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.213 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-08T14:07:04.213 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-08T14:07:04.213 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.213 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.214 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.215 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.216 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.216 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.216 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.216 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.216 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.216 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.216 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.216 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.216 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.216 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.217 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.218 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.219 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.220 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.221 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.222 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.223 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.224 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.225 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.226 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.227 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.228 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2023-10-08T14:07:04.228 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.228 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.228 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.228 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.228 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.228 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.228 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.228 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.228 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.229 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.230 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.231 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.231 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.231 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.231 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.231 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.231 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.231 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.231 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.231 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.231 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.232 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.233 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.233 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.233 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.233 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.233 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.233 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.233 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.233 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.233 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.234 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.234 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.234 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.234 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.234 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.234 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.234 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.234 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.234 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.235 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.235 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.235 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.235 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.235 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.235 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.235 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-08T14:07:04.235 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.235 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.235 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.236 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2023-10-08T14:07:04.236 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2023-10-08T14:07:04.236 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.236 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.236 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.236 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.236 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.236 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.236 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.236 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-08T14:07:04.237 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.237 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.237 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.237 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.237 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.237 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.237 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.237 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.237 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.237 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.238 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.238 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.238 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.238 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.238 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.238 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.238 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.238 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.238 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.238 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.239 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.239 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.239 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.239 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.239 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.239 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.239 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.239 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.239 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.239 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.240 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.241 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.242 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.242 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.242 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.242 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.242 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.242 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.242 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.242 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.243 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.243 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.243 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.243 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.243 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.243 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.243 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.243 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.243 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.243 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.244 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.244 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.244 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.244 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.244 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2023-10-08T14:07:04.244 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-08T14:07:04.244 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.244 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.244 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.244 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.245 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.246 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.247 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.247 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.247 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.247 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.247 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.247 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.247 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.247 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.247 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.247 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.248 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.248 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.248 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.248 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.248 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.248 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.248 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.248 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.248 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.248 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.249 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.250 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.251 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.252 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.253 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.254 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.255 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.256 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.257 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.258 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.259 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.260 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.261 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.262 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.263 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.264 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.264 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.264 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.264 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.264 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.264 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.264 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.264 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.264 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.265 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.265 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.265 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.265 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.265 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.265 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.265 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.265 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.265 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.265 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.266 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.267 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.268 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.268 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.268 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2023-10-08T14:07:04.268 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2023-10-08T14:07:04.268 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.268 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.268 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.268 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-08T14:07:04.268 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2023-10-08T14:07:04.268 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2023-10-08T14:07:04.269 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.270 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.271 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.272 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.273 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.273 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.273 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.273 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.273 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.273 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.273 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.273 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.273 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.273 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.274 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.274 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.274 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.274 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.274 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.274 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.274 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.274 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.274 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.275 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.275 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.275 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.275 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.275 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.275 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.275 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.275 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.275 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.275 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.276 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.277 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.278 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.279 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.280 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.281 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:04.281 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.281 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.281 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.281 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2023-10-08T14:07:04.281 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.281 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.281 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.281 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.281 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.282 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2023-10-08T14:07:04.283 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.284 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.285 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.285 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.285 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.285 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.285 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.285 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.285 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.285 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.285 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.285 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.286 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.287 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.287 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.287 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.287 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.287 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.287 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.287 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.287 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.287 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.287 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.288 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.289 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.290 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.291 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-08T14:07:04.292 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.294 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2023-10-08T14:07:04.294 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2023-10-08T14:07:04.294 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.294 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.294 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.294 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2023-10-08T14:07:04.294 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.294 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.294 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.294 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-08T14:07:04.295 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-08T14:07:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-08T14:07:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-08T14:07:04.296 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.297 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.297 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.297 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2023-10-08T14:07:04.297 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.297 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.297 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.297 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-08T14:07:04.297 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.297 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.297 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.298 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.299 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-08T14:07:04.300 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.301 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.302 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.303 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.304 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.305 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.306 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.307 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.308 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.309 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.310 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.311 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2023-10-08T14:07:04.312 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.313 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.314 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.315 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.316 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.316 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.316 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.316 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.316 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.316 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.316 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.316 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.316 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.316 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.317 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.318 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.319 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.320 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.320 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.320 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.320 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.320 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.320 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.320 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.320 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.321 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.322 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-08T14:07:04.323 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.324 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.325 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.325 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.325 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2023-10-08T14:07:04.325 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.325 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.325 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.325 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.325 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.325 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.325 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.326 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.326 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.326 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.326 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.326 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.326 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.326 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.326 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.326 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.327 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.328 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.329 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.330 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.331 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: "drive_group_interval": { 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "drive_group_interval", 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.332 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300.0", 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.333 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.334 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.335 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.337 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.338 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.338 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.338 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.338 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.338 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.338 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.338 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.338 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.338 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.338 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.339 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.340 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.341 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-08T14:07:04.341 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.341 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.341 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.341 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.341 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.341 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.341 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.341 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2023-10-08T14:07:04.341 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.342 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.343 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.344 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.345 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.346 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.347 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.348 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2023-10-08T14:07:04.349 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.350 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.350 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.350 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2023-10-08T14:07:04.350 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.350 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.350 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.350 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.350 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.350 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.350 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.351 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.352 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.353 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.354 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.354 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.354 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.354 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.354 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.354 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.354 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.354 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.354 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2023-10-08T14:07:04.354 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.355 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.356 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.356 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2023-10-08T14:07:04.356 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2023-10-08T14:07:04.356 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.356 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.356 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.356 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.356 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.356 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.356 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.357 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.358 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.358 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.358 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.358 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.358 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.358 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.358 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.358 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.358 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2023-10-08T14:07:04.358 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2023-10-08T14:07:04.359 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.359 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.359 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.359 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.359 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:04.359 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2023-10-08T14:07:04.359 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.359 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.359 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.359 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.360 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.361 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.362 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.362 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.362 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.362 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.362 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.362 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.362 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.362 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2023-10-08T14:07:04.362 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2023-10-08T14:07:04.362 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.363 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.364 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.365 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.366 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.367 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.368 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.369 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.370 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.371 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.372 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.373 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.373 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.373 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.373 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.373 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.373 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.373 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2023-10-08T14:07:04.373 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.373 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.373 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.374 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.375 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.376 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.377 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.378 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.378 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.378 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.378 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.378 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.378 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.378 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.378 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.378 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.378 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.379 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.379 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.379 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.379 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.379 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.379 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.379 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.379 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.380 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.380 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.380 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.380 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.380 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.380 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.381 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.381 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.381 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.381 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.381 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.381 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2023-10-08T14:07:04.381 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.381 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.381 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.381 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.382 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.383 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.383 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.383 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.383 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.383 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.383 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.383 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.383 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.383 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.383 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.384 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.385 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.386 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.387 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2023-10-08T14:07:04.388 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.389 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.390 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.391 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.392 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.392 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.392 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.392 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.392 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.392 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.392 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.392 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.392 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.392 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.393 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.394 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.395 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.395 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.395 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.395 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.395 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.395 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.395 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.395 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.395 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.395 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.396 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2023-10-08T14:07:04.397 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.398 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.399 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-08T14:07:04.399 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.399 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.399 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.399 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.399 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2023-10-08T14:07:04.399 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2023-10-08T14:07:04.399 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.399 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.399 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2023-10-08T14:07:04.400 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.401 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.401 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.401 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.401 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.401 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.401 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.401 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-08T14:07:04.401 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.401 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.401 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.402 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.403 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.404 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2023-10-08T14:07:04.405 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2023-10-08T14:07:04.406 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.406 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.406 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.406 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.406 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.406 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.406 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.406 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.406 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.406 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.407 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.408 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.409 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.409 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.409 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.409 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.409 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.409 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.409 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.409 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:04.409 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:04.409 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.410 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.410 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.410 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2023-10-08T14:07:04.410 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2023-10-08T14:07:04.410 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.410 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.410 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.410 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.410 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.410 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.411 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.412 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.413 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.414 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.415 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.416 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.417 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.418 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.419 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.420 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.421 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.421 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.421 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.421 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2023-10-08T14:07:04.421 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2023-10-08T14:07:04.421 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.421 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.421 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.421 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.422 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.422 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.422 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.422 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.422 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.422 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.422 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.422 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.422 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2023-10-08T14:07:04.422 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.423 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.424 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2023-10-08T14:07:04.424 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2023-10-08T14:07:04.424 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.424 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.424 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.424 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-08T14:07:04.424 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.424 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.424 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.424 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.425 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.426 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.426 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.426 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.426 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.426 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.426 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.426 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.426 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.426 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.426 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.427 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.428 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.429 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.430 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.431 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.431 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.431 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.431 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.431 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.431 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.431 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.431 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.431 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.431 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.432 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.433 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.434 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.434 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.434 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.434 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.434 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.434 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.434 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.434 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.434 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.434 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.435 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.436 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.436 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.436 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.436 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.436 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.436 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.436 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.436 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.436 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.436 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.437 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.438 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.439 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.440 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.441 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.442 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.443 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2023-10-08T14:07:04.444 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.445 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.445 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.445 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.445 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.445 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.445 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.445 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.445 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2023-10-08T14:07:04.445 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2023-10-08T14:07:04.446 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.446 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.446 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.446 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.446 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.446 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.446 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.446 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.446 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.446 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.447 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.447 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.447 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:04.447 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:04.447 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.447 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.447 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.447 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:04.447 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.447 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.448 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.449 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.449 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.449 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.449 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.449 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.449 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.449 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.449 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.449 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.449 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.450 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.451 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.452 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.453 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.454 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.455 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.456 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.456 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2023-10-08T14:07:04.456 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2023-10-08T14:07:04.456 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.456 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.456 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.456 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2023-10-08T14:07:04.456 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.456 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.456 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.457 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-08T14:07:04.458 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.458 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.458 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.458 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.458 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.458 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.458 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.458 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.458 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.458 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: "gid": 24164, 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "y", 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr_features": 4540138322906710015, 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.459 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.460 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.461 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.462 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.463 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.463 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.463 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.463 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.463 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.463 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.463 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.463 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.463 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.463 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.464 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.465 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.466 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.467 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.468 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.469 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.470 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.471 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.472 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2023-10-08T14:07:04.473 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.474 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.475 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.476 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.477 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2023-10-08T14:07:04.478 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.479 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.480 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2023-10-08T14:07:04.481 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.482 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-08T14:07:04.483 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2023-10-08T14:07:04.484 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2023-10-08T14:07:04.485 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.486 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.487 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.488 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.489 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.490 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.491 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.492 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2023-10-08T14:07:04.493 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.494 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2023-10-08T14:07:04.495 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap" 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.496 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:04.497 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.498 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-08T14:07:04.499 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.500 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.501 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-08T14:07:04.501 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.501 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.501 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.501 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.501 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.501 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.501 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.501 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2023-10-08T14:07:04.501 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.502 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.503 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-08T14:07:04.503 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.503 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.503 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.503 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.503 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2023-10-08T14:07:04.503 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2023-10-08T14:07:04.503 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.503 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.503 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2023-10-08T14:07:04.504 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.505 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-08T14:07:04.506 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.507 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2023-10-08T14:07:04.508 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.509 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2023-10-08T14:07:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2023-10-08T14:07:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2023-10-08T14:07:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-08T14:07:04.510 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.511 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.512 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.513 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2023-10-08T14:07:04.514 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.515 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2023-10-08T14:07:04.516 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.517 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.518 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.519 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2023-10-08T14:07:04.520 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2023-10-08T14:07:04.521 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.521 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.521 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.521 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-08T14:07:04.521 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.521 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.521 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.521 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2023-10-08T14:07:04.521 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.522 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.523 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2023-10-08T14:07:04.524 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.525 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2023-10-08T14:07:04.526 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.527 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.528 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2023-10-08T14:07:04.529 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.530 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.531 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.532 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.533 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.534 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2023-10-08T14:07:04.535 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.536 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-08T14:07:04.537 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.538 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.539 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.540 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.541 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.541 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:04.541 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.541 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.541 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.541 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2023-10-08T14:07:04.541 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.541 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.541 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.541 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.542 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.543 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.544 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.545 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.545 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.545 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.545 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.545 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.545 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.545 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.545 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.545 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.545 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.546 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.547 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.548 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.549 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.550 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.551 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-08T14:07:04.552 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.553 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.554 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.555 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.555 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.555 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2023-10-08T14:07:04.555 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2023-10-08T14:07:04.555 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.555 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.555 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.555 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.555 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.555 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.556 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-08T14:07:04.557 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2023-10-08T14:07:04.558 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.559 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.560 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2023-10-08T14:07:04.561 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.562 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.563 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-08T14:07:04.564 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.565 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.566 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2023-10-08T14:07:04.567 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-08T14:07:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.568 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.569 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-08T14:07:04.570 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.570 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.570 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.570 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.570 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2023-10-08T14:07:04.570 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2023-10-08T14:07:04.570 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.570 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.570 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.570 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2023-10-08T14:07:04.571 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.572 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.573 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.574 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.575 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.576 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.577 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.578 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.579 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.580 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.581 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.582 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.583 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-08T14:07:04.583 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.583 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.583 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.583 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.583 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2023-10-08T14:07:04.583 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2023-10-08T14:07:04.583 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.583 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.583 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.584 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.585 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.586 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.587 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.588 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.589 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.590 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.591 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-08T14:07:04.592 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.593 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.594 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.595 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.596 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.597 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-08T14:07:04.598 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.599 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.600 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.601 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.602 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-08T14:07:04.603 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.604 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.605 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.606 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.607 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.608 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.608 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.608 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.608 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.608 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.608 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.608 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.608 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.608 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-08T14:07:04.608 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.609 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.610 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.611 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.612 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.613 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.614 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.614 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.614 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.614 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.614 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.614 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.614 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.614 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.614 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.614 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.615 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.616 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.617 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-08T14:07:04.618 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.619 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.620 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.621 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.622 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-08T14:07:04.623 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.624 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.625 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.626 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2023-10-08T14:07:04.627 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.628 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2023-10-08T14:07:04.629 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.630 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.631 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.632 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.633 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: "UNSAFE_TLS_v1_2": { 2023-10-08T14:07:04.634 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.635 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.636 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.637 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.638 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2023-10-08T14:07:04.639 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2023-10-08T14:07:04.640 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.640 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.640 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.640 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.640 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.640 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.640 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.640 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.640 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.640 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.641 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.642 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.643 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.644 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.644 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2023-10-08T14:07:04.644 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.644 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.644 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.644 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.644 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.644 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.644 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.644 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.645 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.646 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.647 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.648 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.648 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.648 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.648 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.648 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.648 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.648 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.648 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.648 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.648 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.649 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.650 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.651 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.652 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.652 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.652 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.652 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.652 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.652 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.652 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.652 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.652 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2023-10-08T14:07:04.652 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.653 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.654 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.655 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.656 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.657 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.658 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.659 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.660 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.661 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2023-10-08T14:07:04.662 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.663 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.664 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.665 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.666 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.667 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.668 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.669 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.670 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.671 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.673 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2023-10-08T14:07:04.673 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2023-10-08T14:07:04.673 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.673 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.673 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.673 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2023-10-08T14:07:04.673 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.673 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.673 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.674 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2023-10-08T14:07:04.675 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.676 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.677 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.678 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.679 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.680 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.681 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.682 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.683 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.684 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.685 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.685 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.685 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.685 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.685 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.685 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.685 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.685 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.685 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.685 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.686 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.687 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.688 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2023-10-08T14:07:04.689 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.690 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.691 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.692 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.693 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.694 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.694 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2023-10-08T14:07:04.694 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2023-10-08T14:07:04.694 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.694 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.694 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.694 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2023-10-08T14:07:04.694 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.694 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.694 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.695 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.696 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.697 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.698 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.698 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.698 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.698 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.698 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.698 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.698 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.698 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.699 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.699 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.699 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.699 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.699 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.699 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.699 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.699 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.700 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.700 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.700 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.700 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.700 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.700 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.700 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.700 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.700 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.700 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.701 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.702 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.702 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.702 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.702 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.702 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.702 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.703 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.703 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.703 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.703 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.703 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.703 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.703 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.703 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.703 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.703 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.704 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.705 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.705 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.705 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.705 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2023-10-08T14:07:04.705 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2023-10-08T14:07:04.705 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.705 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.705 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.706 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.706 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.706 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.706 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.706 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2023-10-08T14:07:04.706 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.706 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.706 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.707 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.707 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2023-10-08T14:07:04.707 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2023-10-08T14:07:04.707 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.707 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.707 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.707 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2023-10-08T14:07:04.707 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.707 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.707 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.708 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2023-10-08T14:07:04.708 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.708 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.708 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.708 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.708 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-08T14:07:04.708 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-08T14:07:04.708 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.708 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.709 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.709 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2023-10-08T14:07:04.709 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.709 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.709 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.709 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T14:07:04.709 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.709 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.709 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.709 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-08T14:07:04.710 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.711 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2023-10-08T14:07:04.712 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2023-10-08T14:07:04.713 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.714 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.715 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.716 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.717 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.718 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.718 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.718 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.718 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.718 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.718 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.718 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.718 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.718 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.718 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.719 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.719 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.719 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.719 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.719 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.719 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.719 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.719 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.719 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.720 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.721 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.722 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.723 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.724 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.725 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.726 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.727 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2023-10-08T14:07:04.728 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2023-10-08T14:07:04.729 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.729 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.729 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.729 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.729 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2023-10-08T14:07:04.729 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.729 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.729 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-08T14:07:04.729 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.730 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.731 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.732 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.733 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.734 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.735 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2023-10-08T14:07:04.736 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.737 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.738 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.739 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.740 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.741 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.742 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.743 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-08T14:07:04.744 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2023-10-08T14:07:04.745 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.746 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.747 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.748 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.749 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.750 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.751 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.752 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.753 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.754 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.755 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.755 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.755 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.755 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.755 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.755 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.755 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.755 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.755 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.755 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.756 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.757 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.758 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.759 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.760 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.761 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.762 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.763 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.764 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.765 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.766 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.767 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.768 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.769 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.770 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.771 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.772 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.773 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.774 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.775 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.776 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.777 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.778 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.779 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.780 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.781 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.782 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.783 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.784 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.785 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.786 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.787 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.788 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.789 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.790 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.791 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.792 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.793 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.794 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.795 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.796 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.797 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2023-10-08T14:07:04.798 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2023-10-08T14:07:04.799 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.800 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.801 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.802 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.803 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.803 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.803 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.803 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.803 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.803 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.803 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.803 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.803 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.803 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.804 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.805 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.806 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.808 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.809 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.810 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.811 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-08T14:07:04.812 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.813 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.814 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.815 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.816 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.817 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.819 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.820 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.821 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-08T14:07:04.822 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.823 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.824 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.825 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.826 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-08T14:07:04.827 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.828 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.829 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.830 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.831 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.832 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.833 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.834 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.834 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.834 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.834 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.834 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.834 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.834 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.834 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.834 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.834 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.835 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.836 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.837 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.838 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.838 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.838 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.838 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.838 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.838 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.838 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.838 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2023-10-08T14:07:04.838 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.839 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.839 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.839 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2023-10-08T14:07:04.839 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2023-10-08T14:07:04.839 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.839 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.839 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.839 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.839 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.839 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.840 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.841 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.842 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.843 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.844 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.845 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.846 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.847 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.848 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.849 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.850 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.851 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.852 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.853 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.854 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.855 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: "drive_group_interval": { 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "drive_group_interval", 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.856 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300.0", 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.857 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.858 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.859 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.860 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.861 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.862 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.863 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.864 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.865 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.866 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.867 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.868 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.869 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.870 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.871 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.873 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.874 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.875 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.876 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2023-10-08T14:07:04.877 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2023-10-08T14:07:04.878 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.879 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.880 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.881 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.882 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.883 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2023-10-08T14:07:04.884 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.885 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.886 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.887 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.888 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.889 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.890 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.891 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.892 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.893 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.894 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.895 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.896 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.897 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.898 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2023-10-08T14:07:04.899 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.900 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.901 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.902 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.903 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.904 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.905 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.906 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:04.907 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.908 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.909 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.910 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.911 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.912 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.913 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.914 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.915 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.916 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2023-10-08T14:07:04.917 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.918 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.919 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-08T14:07:04.920 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.921 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.922 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.923 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.924 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:04.925 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.926 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.927 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.928 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.929 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.930 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.931 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.932 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.933 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.934 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.935 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.936 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.937 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.938 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.939 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.940 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.941 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.942 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.943 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.944 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.945 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.946 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.947 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.948 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.949 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.950 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.951 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.952 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.953 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.954 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.955 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.956 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.957 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.958 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.959 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.960 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.961 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.962 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.963 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.964 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.965 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.966 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-08T14:07:04.967 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.968 INFO:teuthology.orchestra.run.smithi057.stdout: "modules": [ 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: "dashboard", 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: "iostat", 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: "nfs", 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: "restful", 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: "snap_schedule" 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.969 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.970 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.971 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.972 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.973 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.974 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.975 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2023-10-08T14:07:04.976 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.977 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.978 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.979 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.980 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.981 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.982 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.983 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2023-10-08T14:07:04.984 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.985 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.986 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.987 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2023-10-08T14:07:04.988 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.989 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.990 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-08T14:07:04.991 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.992 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.993 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.994 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.995 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.996 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.997 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:04.998 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:04.999 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.000 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.001 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2023-10-08T14:07:05.002 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap" 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.003 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.004 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.005 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.006 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.007 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.008 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.009 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-08T14:07:05.010 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.011 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.012 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.013 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2023-10-08T14:07:05.014 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2023-10-08T14:07:05.015 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.016 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.017 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.018 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.019 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2023-10-08T14:07:05.020 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.021 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.022 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.023 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.024 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.025 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-08T14:07:05.026 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.027 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2023-10-08T14:07:05.028 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2023-10-08T14:07:05.029 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.030 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.031 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.032 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.033 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.034 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2023-10-08T14:07:05.035 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.036 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.037 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.038 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.039 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.040 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2023-10-08T14:07:05.041 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.042 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2023-10-08T14:07:05.043 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2023-10-08T14:07:05.044 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.045 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.046 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.047 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.048 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.049 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.050 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.051 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.052 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.053 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.055 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2023-10-08T14:07:05.056 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.057 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2023-10-08T14:07:05.058 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.059 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.060 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2023-10-08T14:07:05.061 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-08T14:07:05.062 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.063 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.064 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.065 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.066 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-08T14:07:05.067 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.068 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.069 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.070 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.071 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2023-10-08T14:07:05.072 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.073 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.074 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-08T14:07:05.075 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.076 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.077 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.078 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.079 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.080 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.081 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.082 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.083 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.083 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.083 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.083 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.083 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.083 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.083 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.083 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.084 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-08T14:07:05.085 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-08T14:07:05.086 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-08T14:07:05.087 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.087 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.087 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.087 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:05.087 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.087 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.087 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.087 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.088 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.089 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.090 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-08T14:07:05.091 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.092 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.093 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.094 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.095 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.096 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.097 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.098 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-08T14:07:05.099 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.100 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.101 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.102 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.102 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-08T14:07:05.102 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-08T14:07:05.102 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.102 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.102 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.102 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.102 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.102 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.103 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.104 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2023-10-08T14:07:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.106 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.107 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.108 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.109 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.110 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.111 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.112 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.113 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-08T14:07:05.114 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.115 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.116 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.117 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.118 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.119 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.120 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.121 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2023-10-08T14:07:05.122 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.123 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.124 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.125 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.126 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2023-10-08T14:07:05.127 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.128 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.129 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2023-10-08T14:07:05.130 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.131 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.132 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "UNSAFE_TLS_v1_2": { 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.133 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.134 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.135 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.136 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.137 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2023-10-08T14:07:05.138 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.139 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.140 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.141 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.142 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.143 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.144 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.145 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.146 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.147 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.148 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.149 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.150 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.151 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.152 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.153 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.154 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-08T14:07:05.155 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.157 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.158 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2023-10-08T14:07:05.159 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2023-10-08T14:07:05.160 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.161 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.162 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.163 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.164 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.165 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.166 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.167 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-08T14:07:05.168 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.169 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.170 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.171 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.172 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.173 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.174 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.175 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.176 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.177 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.178 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.179 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.180 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.181 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.182 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2023-10-08T14:07:05.183 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2023-10-08T14:07:05.184 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-08T14:07:05.185 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.186 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.187 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.188 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.189 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.190 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.191 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.192 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.193 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.194 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.195 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.195 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.195 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.195 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.195 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2023-10-08T14:07:05.195 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2023-10-08T14:07:05.195 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.195 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.195 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.195 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.196 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2023-10-08T14:07:05.197 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.198 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.199 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2023-10-08T14:07:05.200 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.201 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.202 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.203 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.204 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.205 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.206 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.207 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2023-10-08T14:07:05.208 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.209 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.210 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.211 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.212 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.213 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.214 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2023-10-08T14:07:05.215 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.216 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.217 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.218 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.219 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.220 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.221 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.222 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.223 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-08T14:07:05.224 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.225 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.226 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.227 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.228 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.229 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.230 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.231 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.232 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.233 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.234 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.235 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.236 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.236 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2023-10-08T14:07:05.236 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.236 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.236 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.236 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.237 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.238 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.239 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.240 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.241 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.242 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.243 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.244 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.245 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.246 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.247 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.248 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.249 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.250 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.251 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.252 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.253 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.254 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.255 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.256 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.257 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.258 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.259 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.260 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.261 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.262 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.263 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.264 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.265 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.266 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.267 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.268 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.269 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.270 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.271 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.272 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.273 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.274 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.275 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.276 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.277 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.278 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.279 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.280 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.281 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.282 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.283 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.284 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.285 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2023-10-08T14:07:05.286 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.287 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.288 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.289 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.290 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.291 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.292 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.293 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.294 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2023-10-08T14:07:05.295 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.296 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.297 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2023-10-08T14:07:05.298 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.299 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.300 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.301 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.302 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.303 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.304 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.305 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-08T14:07:05.306 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.307 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.308 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.309 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.310 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2023-10-08T14:07:05.311 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.312 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.313 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.314 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.315 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.316 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.317 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.318 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.319 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.320 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.321 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2023-10-08T14:07:05.322 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.323 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.324 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.325 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2023-10-08T14:07:05.326 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.327 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.328 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.329 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.330 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.331 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.332 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.333 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2023-10-08T14:07:05.334 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.335 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.336 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.337 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.338 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.339 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.340 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.341 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: "drive_group_interval": { 2023-10-08T14:07:05.342 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "drive_group_interval", 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300.0", 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.343 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.344 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.345 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.346 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.347 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.348 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.349 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.350 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.351 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.352 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.353 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.354 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.355 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.356 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.357 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.358 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.359 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2023-10-08T14:07:05.360 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2023-10-08T14:07:05.361 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.362 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.363 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.364 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.365 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2023-10-08T14:07:05.366 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.367 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.368 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2023-10-08T14:07:05.369 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.370 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.371 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.372 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.373 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.374 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.375 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.376 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.377 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.378 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.379 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.380 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.381 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.382 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.383 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.384 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.385 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.386 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.387 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.388 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.389 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.390 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.391 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.392 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:05.393 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.394 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.395 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.396 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.397 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.398 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.399 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.401 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.402 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.403 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.404 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2023-10-08T14:07:05.405 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.406 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.407 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.408 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.409 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.410 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:05.411 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.412 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.413 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.414 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.415 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.416 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.417 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.418 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.419 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.420 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.421 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.422 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.423 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.424 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.425 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.426 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.427 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.428 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.429 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.430 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.431 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.432 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.433 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.434 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.435 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.436 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.437 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.438 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.439 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2023-10-08T14:07:05.440 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.441 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2023-10-08T14:07:05.442 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2023-10-08T14:07:05.444 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.445 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.446 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.447 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2023-10-08T14:07:05.448 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2023-10-08T14:07:05.449 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2023-10-08T14:07:05.450 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.451 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2023-10-08T14:07:05.452 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.453 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2023-10-08T14:07:05.454 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "services": { 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "dashboard": "https://172.21.15.57:8443/" 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "always_on_modules": { 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "octopus": [ 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2023-10-08T14:07:05.455 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "pacific": [ 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2023-10-08T14:07:05.456 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "quincy": [ 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2023-10-08T14:07:05.457 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "reef": [ 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2023-10-08T14:07:05.458 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "last_failure_osd_epoch": 5, 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "active_clients": [ 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2023-10-08T14:07:05.459 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1007973081 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "libcephsqlite", 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.460 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1204879539 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-08T14:07:05.461 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2777153216 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2649037112 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.462 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3991171620 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.463 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2023-10-08T14:07:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-08T14:07:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-08T14:07:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-08T14:07:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-08T14:07:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3719711945 2023-10-08T14:07:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:05.464 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-08T14:07:05.464 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T14:07:05.465 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:04 smithi099 ceph-mon[103849]: pgmap v784: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 380 KiB/s wr, 160 op/s 2023-10-08T14:07:05.465 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:04 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/953513458' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-08T14:07:05.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:04 smithi115 ceph-mon[104408]: pgmap v784: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 380 KiB/s wr, 160 op/s 2023-10-08T14:07:05.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:04 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/953513458' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-08T14:07:05.466 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:05 smithi115 ceph-mon[104408]: pgmap v785: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 248 KiB/s wr, 176 op/s 2023-10-08T14:07:05.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.926+0000 7fc2f6945700 1 -- 172.21.15.57:0/953513458 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc2d80a1800 msgr2=0x7fc2d80a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:05.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.926+0000 7fc2f6945700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc2d80a1800 0x7fc2d80a3cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0000c00 tx=0x7fc2e0005a90 comp rx=0 tx=0).stop 2023-10-08T14:07:05.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.926+0000 7fc2f6945700 1 -- 172.21.15.57:0/953513458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2f01077f0 msgr2=0x7fc2f0117da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:07:05.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.926+0000 7fc2f6945700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2f01077f0 0x7fc2f0117da0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400b550 tx=0x7fc2e400b910 comp rx=0 tx=0).stop 2023-10-08T14:07:05.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.932+0000 7fc2f6945700 1 -- 172.21.15.57:0/953513458 shutdown_connections 2023-10-08T14:07:05.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.932+0000 7fc2f6945700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2f010edf0 0x7fc2f01c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:05.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.932+0000 7fc2f6945700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc2d80a1800 0x7fc2d80a3cc0 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0000c00 tx=0x7fc2e0005a90 comp rx=0 tx=0).stop 2023-10-08T14:07:05.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.932+0000 7fc2f6945700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2f01077f0 0x7fc2f0117da0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:05.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.932+0000 7fc2f6945700 1 --2- 172.21.15.57:0/953513458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f01083a0 0x7fc2f01182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:07:05.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.932+0000 7fc2f6945700 1 -- 172.21.15.57:0/953513458 >> 172.21.15.57:0/953513458 conn(0x7fc2f006b760 msgr2=0x7fc2f006f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:07:05.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.938+0000 7fc2f6945700 1 -- 172.21.15.57:0/953513458 shutdown_connections 2023-10-08T14:07:05.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:07:03.944+0000 7fc2f6945700 1 -- 172.21.15.57:0/953513458 wait complete. 2023-10-08T14:07:05.519 DEBUG:tasks.check_counter:Getting stats from a 2023-10-08T14:07:05.519 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-08T14:07:05.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:04 smithi057 ceph-mon[99476]: pgmap v784: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 380 KiB/s wr, 160 op/s 2023-10-08T14:07:05.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:04 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/953513458' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-08T14:07:05.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:05 smithi057 ceph-mon[99476]: pgmap v785: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 248 KiB/s wr, 176 op/s 2023-10-08T14:07:05.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:05 smithi099 ceph-mon[103849]: pgmap v785: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 248 KiB/s wr, 176 op/s 2023-10-08T14:07:07.128 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T14:07:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:07 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:07.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:07 smithi057 ceph-mon[99476]: pgmap v786: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 249 KiB/s wr, 152 op/s 2023-10-08T14:07:07.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:07 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:07.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:07 smithi099 ceph-mon[103849]: pgmap v786: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 249 KiB/s wr, 152 op/s 2023-10-08T14:07:07.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:07 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:07.686 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:07 smithi115 ceph-mon[104408]: pgmap v786: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 249 KiB/s wr, 152 op/s 2023-10-08T14:07:09.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:09 smithi115 ceph-mon[104408]: pgmap v787: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 243 KiB/s wr, 137 op/s 2023-10-08T14:07:10.003 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:09 smithi057 ceph-mon[99476]: pgmap v787: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 243 KiB/s wr, 137 op/s 2023-10-08T14:07:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:09 smithi099 ceph-mon[103849]: pgmap v787: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 243 KiB/s wr, 137 op/s 2023-10-08T14:07:10.048 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T14:07:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 1769, 2023-10-08T14:07:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 1769, 2023-10-08T14:07:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 109598321, 2023-10-08T14:07:10.048 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 549364, 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 8, 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 4, 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.249781736, 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.058263608, 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.108044738, 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.067769104, 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 1768, 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.608956875, 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000344432 2023-10-08T14:07:10.049 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 1016, 2023-10-08T14:07:10.050 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 1229, 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 88354719, 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 762026, 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 7, 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 6, 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.168187770, 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.033543613, 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.085440245, 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.036278341, 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 1227, 2023-10-08T14:07:10.051 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.396510982, 2023-10-08T14:07:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000323154 2023-10-08T14:07:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T14:07:10.052 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 446304 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 942, 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 910, 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 67043120, 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 360629, 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 5, 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 4, 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.160385840, 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.034664199, 2023-10-08T14:07:10.053 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.088603039, 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.034125373, 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 909, 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.426964346, 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000469707 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.054 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 980832, 2023-10-08T14:07:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 174640 2023-10-08T14:07:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2023-10-08T14:07:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 1, 2023-10-08T14:07:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2023-10-08T14:07:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "finisher-MDSRank": { 2023-10-08T14:07:10.055 INFO:teuthology.orchestra.run.smithi057.stdout: "queue_len": 0, 2023-10-08T14:07:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "complete_latency": { 2023-10-08T14:07:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 5619, 2023-10-08T14:07:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.174838257, 2023-10-08T14:07:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000031115 2023-10-08T14:07:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "finisher-PurgeQueue": { 2023-10-08T14:07:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "queue_len": 0, 2023-10-08T14:07:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "complete_latency": { 2023-10-08T14:07:10.056 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2023-10-08T14:07:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "request": 0, 2023-10-08T14:07:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "reply": 0, 2023-10-08T14:07:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "reply_latency": { 2023-10-08T14:07:10.057 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "slow_reply": 0, 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "forward": 0, 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_fetch_complete": 0, 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_fetch_keys": 0, 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_commit": 0, 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_split": 0, 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_merge": 0, 2023-10-08T14:07:10.058 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes": 35659, 2023-10-08T14:07:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_top": 241, 2023-10-08T14:07:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_bottom": 19369, 2023-10-08T14:07:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_pin_tail": 16049, 2023-10-08T14:07:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_pinned": 35314, 2023-10-08T14:07:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_expired": 556, 2023-10-08T14:07:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_with_caps": 0, 2023-10-08T14:07:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": 0, 2023-10-08T14:07:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "subtrees": 1, 2023-10-08T14:07:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse": 0, 2023-10-08T14:07:10.059 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_hit": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_forward": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_discover": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_dir_fetch": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_remote_ino": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_lock": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "load_cent": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "q": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "exported": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "exported_inodes": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "imported": 0, 2023-10-08T14:07:10.060 INFO:teuthology.orchestra.run.smithi057.stdout: "imported_inodes": 0, 2023-10-08T14:07:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_dir_fetch": 0, 2023-10-08T14:07:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_backtrace_fetch": 0, 2023-10-08T14:07:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_peer_discover": 0, 2023-10-08T14:07:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rfiles": 0, 2023-10-08T14:07:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rbytes": 0, 2023-10-08T14:07:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rsnaps": 0, 2023-10-08T14:07:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T14:07:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_set_tag": 0, 2023-10-08T14:07:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T14:07:10.061 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_inotable_repaired": 0, 2023-10-08T14:07:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dir_inodes": 0, 2023-10-08T14:07:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T14:07:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T14:07:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_file_inodes": 0, 2023-10-08T14:07:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_inode_file_caps": 0, 2023-10-08T14:07:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T14:07:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_grant": 0, 2023-10-08T14:07:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T14:07:10.062 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T14:07:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T14:07:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_caps": 0, 2023-10-08T14:07:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_caps_dirty": 0, 2023-10-08T14:07:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_cap_release": 0, 2023-10-08T14:07:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "process_request_cap_release": 0 2023-10-08T14:07:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_cache": { 2023-10-08T14:07:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_update": 0, 2023-10-08T14:07:10.063 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_update_receipt": 0, 2023-10-08T14:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_try_discover": 0, 2023-10-08T14:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_send_discover": 0, 2023-10-08T14:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_handle_discover": 0, 2023-10-08T14:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays": 0, 2023-10-08T14:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays_delayed": 0, 2023-10-08T14:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays_enqueuing": 0, 2023-10-08T14:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_created": 0, 2023-10-08T14:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_enqueued": 0, 2023-10-08T14:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_reintegrated": 0, 2023-10-08T14:07:10.064 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_migrated": 0, 2023-10-08T14:07:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_processing": 0, 2023-10-08T14:07:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_enqueued": 0, 2023-10-08T14:07:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_prioritized": 0, 2023-10-08T14:07:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "recovery_started": 0, 2023-10-08T14:07:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "recovery_completed": 0, 2023-10-08T14:07:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T14:07:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_exportdir": 0, 2023-10-08T14:07:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_flush": 0, 2023-10-08T14:07:10.065 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_fragmentdir": 0, 2023-10-08T14:07:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_fragstats": 0, 2023-10-08T14:07:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_inodestats": 0 2023-10-08T14:07:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_log": { 2023-10-08T14:07:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "evlrg": 0, 2023-10-08T14:07:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "evadd": 0, 2023-10-08T14:07:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "evex": 0, 2023-10-08T14:07:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "evtrm": 0, 2023-10-08T14:07:10.066 INFO:teuthology.orchestra.run.smithi057.stdout: "ev": 74883, 2023-10-08T14:07:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "evexg": 0, 2023-10-08T14:07:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "evexd": 0, 2023-10-08T14:07:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "segadd": 0, 2023-10-08T14:07:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "segex": 0, 2023-10-08T14:07:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "segtrm": 0, 2023-10-08T14:07:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "seg": 106, 2023-10-08T14:07:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "segmjr": 8, 2023-10-08T14:07:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "segexg": 0, 2023-10-08T14:07:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "segexd": 0, 2023-10-08T14:07:10.067 INFO:teuthology.orchestra.run.smithi057.stdout: "expos": 4194304, 2023-10-08T14:07:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "wrpos": 267213478, 2023-10-08T14:07:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "rdpos": 267204768, 2023-10-08T14:07:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "jlat": { 2023-10-08T14:07:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.068 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "replayed": 74883 2023-10-08T14:07:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_mem": { 2023-10-08T14:07:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 11791, 2023-10-08T14:07:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "ino+": 41826, 2023-10-08T14:07:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "ino-": 30035, 2023-10-08T14:07:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "dir": 437, 2023-10-08T14:07:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "dir+": 1082, 2023-10-08T14:07:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "dir-": 645, 2023-10-08T14:07:10.069 INFO:teuthology.orchestra.run.smithi057.stdout: "dn": 35659, 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "dn+": 69236, 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "dn-": 33577, 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "cap": 0, 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "cap+": 0, 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "cap-": 0, 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "rss": 129356, 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "heap": 207152 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_server": { 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "dispatch_client_request": 0, 2023-10-08T14:07:10.070 INFO:teuthology.orchestra.run.smithi057.stdout: "dispatch_server_request": 0, 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_request": 0, 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_session": 0, 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_peer_request": 0, 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "req_create_latency": { 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getattr_latency": { 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.071 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getfilelock_latency": { 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "req_link_latency": { 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.072 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookup_latency": { 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookuphash_latency": { 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupino_latency": { 2023-10-08T14:07:10.073 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupname_latency": { 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupparent_latency": { 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.074 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupsnap_latency": { 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lssnap_latency": { 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.075 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mkdir_latency": { 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mknod_latency": { 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.076 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mksnap_latency": { 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "req_open_latency": { 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "req_readdir_latency": { 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.077 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rename_latency": { 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "req_renamesnap_latency": { 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.078 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "req_snapdiff_latency": { 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmdir_latency": { 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.079 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmsnap_latency": { 2023-10-08T14:07:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmxattr_latency": { 2023-10-08T14:07:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.080 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setattr_latency": { 2023-10-08T14:07:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setdirlayout_latency": { 2023-10-08T14:07:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.081 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setfilelock_latency": { 2023-10-08T14:07:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setlayout_latency": { 2023-10-08T14:07:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.082 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setxattr_latency": { 2023-10-08T14:07:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "req_symlink_latency": { 2023-10-08T14:07:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.083 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "req_unlink_latency": { 2023-10-08T14:07:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.084 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_revoke_eviction": 0, 2023-10-08T14:07:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_acquisition_throttle": 0, 2023-10-08T14:07:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getvxattr_latency": { 2023-10-08T14:07:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:10.085 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_sessions": { 2023-10-08T14:07:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "session_count": 1, 2023-10-08T14:07:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "session_add": 1, 2023-10-08T14:07:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "session_remove": 0, 2023-10-08T14:07:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions_open": 1, 2023-10-08T14:07:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions_stale": 0, 2023-10-08T14:07:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "total_load": 0, 2023-10-08T14:07:10.086 INFO:teuthology.orchestra.run.smithi057.stdout: "average_load": 0, 2023-10-08T14:07:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "avg_session_uptime": 2344, 2023-10-08T14:07:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "mdthresh_evicted": 0 2023-10-08T14:07:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2023-10-08T14:07:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:10.087 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:10.088 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:10.089 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:10.090 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2023-10-08T14:07:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 649541, 2023-10-08T14:07:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 12119, 2023-10-08T14:07:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:07:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2023-10-08T14:07:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2023-10-08T14:07:10.091 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2023-10-08T14:07:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 9680, 2023-10-08T14:07:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 191, 2023-10-08T14:07:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:10.092 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2023-10-08T14:07:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 52782475, 2023-10-08T14:07:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 545220, 2023-10-08T14:07:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2023-10-08T14:07:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2023-10-08T14:07:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.093 INFO:teuthology.orchestra.run.smithi057.stdout: "objecter": { 2023-10-08T14:07:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "op_active": 0, 2023-10-08T14:07:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "op_laggy": 0, 2023-10-08T14:07:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "op_send": 3378, 2023-10-08T14:07:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "op_send_bytes": 21, 2023-10-08T14:07:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "op_resend": 3, 2023-10-08T14:07:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "op_reply": 3378, 2023-10-08T14:07:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "op_latency": { 2023-10-08T14:07:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 3378, 2023-10-08T14:07:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 38.519426184, 2023-10-08T14:07:10.094 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.011403027 2023-10-08T14:07:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "op_inflight": 0, 2023-10-08T14:07:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "oplen_avg": { 2023-10-08T14:07:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 3378, 2023-10-08T14:07:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 3379 2023-10-08T14:07:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "op": 3378, 2023-10-08T14:07:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "op_r": 3378, 2023-10-08T14:07:10.095 INFO:teuthology.orchestra.run.smithi057.stdout: "op_w": 0, 2023-10-08T14:07:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "op_rmw": 0, 2023-10-08T14:07:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "op_pg": 0, 2023-10-08T14:07:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_stat": 1964, 2023-10-08T14:07:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_create": 0, 2023-10-08T14:07:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_read": 1412, 2023-10-08T14:07:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_write": 0, 2023-10-08T14:07:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_writefull": 0, 2023-10-08T14:07:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_writesame": 0, 2023-10-08T14:07:10.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_append": 0, 2023-10-08T14:07:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_zero": 0, 2023-10-08T14:07:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_truncate": 0, 2023-10-08T14:07:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_delete": 0, 2023-10-08T14:07:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_mapext": 0, 2023-10-08T14:07:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_sparse_read": 0, 2023-10-08T14:07:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_clonerange": 0, 2023-10-08T14:07:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_getxattr": 1, 2023-10-08T14:07:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_setxattr": 0, 2023-10-08T14:07:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_cmpxattr": 0, 2023-10-08T14:07:10.097 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_rmxattr": 0, 2023-10-08T14:07:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_resetxattrs": 0, 2023-10-08T14:07:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_call": 0, 2023-10-08T14:07:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_watch": 0, 2023-10-08T14:07:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_notify": 0, 2023-10-08T14:07:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T14:07:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_pgls": 0, 2023-10-08T14:07:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_pgls_filter": 0, 2023-10-08T14:07:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_other": 0, 2023-10-08T14:07:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_active": 0, 2023-10-08T14:07:10.098 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_send": 0, 2023-10-08T14:07:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_resend": 0, 2023-10-08T14:07:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_ping": 0, 2023-10-08T14:07:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_active": 0, 2023-10-08T14:07:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_send": 0, 2023-10-08T14:07:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_resend": 0, 2023-10-08T14:07:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_active": 0, 2023-10-08T14:07:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_send": 0, 2023-10-08T14:07:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_resend": 0, 2023-10-08T14:07:10.099 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_active": 0, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_send": 0, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_resend": 0, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "command_active": 0, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "command_send": 0, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "command_resend": 0, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "map_epoch": 113, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "map_full": 0, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "map_inc": 25, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_sessions": 11, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_session_open": 11, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_session_close": 0, 2023-10-08T14:07:10.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_laggy": 0, 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_wr": 0, 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_rd": 2, 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_del": 0 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "oft": { 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_objs": 0, 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_kv_pairs": 0, 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_updates": 0, 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_removes": 0 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.101 INFO:teuthology.orchestra.run.smithi057.stdout: "osd-slow-ops": { 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "slow_ops_count": 0 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "purge_queue": { 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_ops": 0, 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing": 0, 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_high_water": 0, 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executed": 0, 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_item_in_journal": 0 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:10.102 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 3723, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 264640735, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 3723, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 3723, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 264640735, 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:10.103 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-objecter_bytes": { 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:10.104 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 3378, 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 263070936, 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 3378, 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 263070936, 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-objecter_ops": { 2023-10-08T14:07:10.105 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 1024, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 3378, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 3378, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 3378, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 3378, 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:10.106 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-write_buf_throttle": { 2023-10-08T14:07:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 3758096384, 2023-10-08T14:07:10.107 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 0, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 0, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.108 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-write_buf_throttle-0x5639d1956960": { 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 3758096384, 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-08T14:07:10.109 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 0, 2023-10-08T14:07:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 0, 2023-10-08T14:07:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:10.110 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:10.110 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T14:07:10.514 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-10-08T14:07:10.514 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-10-08T14:07:10.514 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2023-10-08T14:07:10.514 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2023-10-08T14:07:10.514 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2023-10-08T14:07:10.514 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-10-08T14:07:10.514 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-10-08T14:07:10.515 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2023-10-08T14:07:10.515 DEBUG:tasks.check_counter:Getting stats from d 2023-10-08T14:07:10.515 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-08T14:07:10.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:10 smithi115 ceph-mon[104408]: osdmap e113: 12 total, 12 up, 12 in 2023-10-08T14:07:10.993 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:10 smithi057 ceph-mon[99476]: osdmap e113: 12 total, 12 up, 12 in 2023-10-08T14:07:11.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:10 smithi099 ceph-mon[103849]: osdmap e113: 12 total, 12 up, 12 in 2023-10-08T14:07:11.499 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T14:07:11.851 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:11 smithi057 ceph-mon[99476]: pgmap v789: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 90 op/s 2023-10-08T14:07:11.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:11 smithi115 ceph-mon[104408]: pgmap v789: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 90 op/s 2023-10-08T14:07:12.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:11 smithi099 ceph-mon[103849]: pgmap v789: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 90 op/s 2023-10-08T14:07:13.387 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T14:07:13.387 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:13.387 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 1059, 2023-10-08T14:07:13.387 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 1027, 2023-10-08T14:07:13.387 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 64109022, 2023-10-08T14:07:13.387 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 398576, 2023-10-08T14:07:13.387 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 7, 2023-10-08T14:07:13.388 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 3, 2023-10-08T14:07:13.388 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.158594602, 2023-10-08T14:07:13.388 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.032802813, 2023-10-08T14:07:13.388 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.084619797, 2023-10-08T14:07:13.388 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.036743629, 2023-10-08T14:07:13.388 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:13.388 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 1026, 2023-10-08T14:07:13.388 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.339715913, 2023-10-08T14:07:13.388 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000331107 2023-10-08T14:07:13.388 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.389 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:13.389 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.389 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.389 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.389 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.389 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 982192, 2023-10-08T14:07:13.389 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 177760 2023-10-08T14:07:13.389 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.389 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:13.390 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 654, 2023-10-08T14:07:13.390 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 871, 2023-10-08T14:07:13.390 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 55180580, 2023-10-08T14:07:13.390 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 651680, 2023-10-08T14:07:13.390 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 6, 2023-10-08T14:07:13.390 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 5, 2023-10-08T14:07:13.390 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.111588909, 2023-10-08T14:07:13.390 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.029264344, 2023-10-08T14:07:13.390 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.050971802, 2023-10-08T14:07:13.390 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.024420345, 2023-10-08T14:07:13.391 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:13.391 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 869, 2023-10-08T14:07:13.391 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.259453207, 2023-10-08T14:07:13.391 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000298565 2023-10-08T14:07:13.391 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.391 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:13.391 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.391 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.391 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.391 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.392 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T14:07:13.392 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 448592 2023-10-08T14:07:13.392 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.392 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:13.392 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 2036, 2023-10-08T14:07:13.392 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 2036, 2023-10-08T14:07:13.392 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 88690939, 2023-10-08T14:07:13.392 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 632458, 2023-10-08T14:07:13.392 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 6, 2023-10-08T14:07:13.392 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 5, 2023-10-08T14:07:13.393 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.254219036, 2023-10-08T14:07:13.393 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.061314195, 2023-10-08T14:07:13.393 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.101691236, 2023-10-08T14:07:13.393 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.075047948, 2023-10-08T14:07:13.393 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:13.393 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 2035, 2023-10-08T14:07:13.393 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.566130400, 2023-10-08T14:07:13.393 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000278196 2023-10-08T14:07:13.393 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.394 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:13.394 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.394 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.394 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.394 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.394 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T14:07:13.394 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T14:07:13.394 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.394 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2023-10-08T14:07:13.394 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 1, 2023-10-08T14:07:13.395 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2023-10-08T14:07:13.395 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.395 INFO:teuthology.orchestra.run.smithi057.stdout: "finisher-MDSRank": { 2023-10-08T14:07:13.395 INFO:teuthology.orchestra.run.smithi057.stdout: "queue_len": 0, 2023-10-08T14:07:13.395 INFO:teuthology.orchestra.run.smithi057.stdout: "complete_latency": { 2023-10-08T14:07:13.395 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 5556, 2023-10-08T14:07:13.395 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.187337328, 2023-10-08T14:07:13.395 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000033718 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: "finisher-PurgeQueue": { 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: "queue_len": 0, 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: "complete_latency": { 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.396 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2023-10-08T14:07:13.397 INFO:teuthology.orchestra.run.smithi057.stdout: "request": 0, 2023-10-08T14:07:13.397 INFO:teuthology.orchestra.run.smithi057.stdout: "reply": 0, 2023-10-08T14:07:13.397 INFO:teuthology.orchestra.run.smithi057.stdout: "reply_latency": { 2023-10-08T14:07:13.397 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.397 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.397 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.397 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.397 INFO:teuthology.orchestra.run.smithi057.stdout: "slow_reply": 0, 2023-10-08T14:07:13.397 INFO:teuthology.orchestra.run.smithi057.stdout: "forward": 0, 2023-10-08T14:07:13.398 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_fetch_complete": 0, 2023-10-08T14:07:13.398 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_fetch_keys": 0, 2023-10-08T14:07:13.398 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_commit": 0, 2023-10-08T14:07:13.398 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_split": 0, 2023-10-08T14:07:13.398 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_merge": 0, 2023-10-08T14:07:13.398 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes": 31965, 2023-10-08T14:07:13.398 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_top": 596, 2023-10-08T14:07:13.398 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_bottom": 16460, 2023-10-08T14:07:13.398 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_pin_tail": 14909, 2023-10-08T14:07:13.399 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_pinned": 31113, 2023-10-08T14:07:13.399 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_expired": 910, 2023-10-08T14:07:13.399 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_with_caps": 0, 2023-10-08T14:07:13.399 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": 0, 2023-10-08T14:07:13.399 INFO:teuthology.orchestra.run.smithi057.stdout: "subtrees": 1, 2023-10-08T14:07:13.399 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse": 0, 2023-10-08T14:07:13.399 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_hit": 0, 2023-10-08T14:07:13.400 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_forward": 0, 2023-10-08T14:07:13.400 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_discover": 0, 2023-10-08T14:07:13.400 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_dir_fetch": 0, 2023-10-08T14:07:13.400 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_remote_ino": 0, 2023-10-08T14:07:13.400 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_lock": 0, 2023-10-08T14:07:13.400 INFO:teuthology.orchestra.run.smithi057.stdout: "load_cent": 0, 2023-10-08T14:07:13.400 INFO:teuthology.orchestra.run.smithi057.stdout: "q": 0, 2023-10-08T14:07:13.400 INFO:teuthology.orchestra.run.smithi057.stdout: "exported": 0, 2023-10-08T14:07:13.401 INFO:teuthology.orchestra.run.smithi057.stdout: "exported_inodes": 0, 2023-10-08T14:07:13.401 INFO:teuthology.orchestra.run.smithi057.stdout: "imported": 0, 2023-10-08T14:07:13.401 INFO:teuthology.orchestra.run.smithi057.stdout: "imported_inodes": 0, 2023-10-08T14:07:13.401 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_dir_fetch": 0, 2023-10-08T14:07:13.401 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_backtrace_fetch": 0, 2023-10-08T14:07:13.401 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_peer_discover": 0, 2023-10-08T14:07:13.401 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rfiles": 0, 2023-10-08T14:07:13.401 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rbytes": 0, 2023-10-08T14:07:13.401 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rsnaps": 0, 2023-10-08T14:07:13.401 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_set_tag": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_inotable_repaired": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dir_inodes": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_file_inodes": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_inode_file_caps": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_grant": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T14:07:13.402 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_caps": 0, 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_caps_dirty": 0, 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_cap_release": 0, 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "process_request_cap_release": 0 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_cache": { 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_update": 0, 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_update_receipt": 0, 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_try_discover": 0, 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_send_discover": 0, 2023-10-08T14:07:13.403 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_handle_discover": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays_delayed": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays_enqueuing": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_created": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_enqueued": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_reintegrated": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_migrated": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_processing": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_enqueued": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_prioritized": 0, 2023-10-08T14:07:13.404 INFO:teuthology.orchestra.run.smithi057.stdout: "recovery_started": 0, 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "recovery_completed": 0, 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_exportdir": 0, 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_flush": 0, 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_fragmentdir": 0, 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_fragstats": 0, 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_inodestats": 0 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_log": { 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "evlrg": 0, 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "evadd": 0, 2023-10-08T14:07:13.405 INFO:teuthology.orchestra.run.smithi057.stdout: "evex": 0, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "evtrm": 0, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "ev": 70209, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "evexg": 0, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "evexd": 0, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "segadd": 0, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "segex": 0, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "segtrm": 0, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "seg": 100, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "segmjr": 7, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "segexg": 0, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "segexd": 0, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "expos": 4194304, 2023-10-08T14:07:13.406 INFO:teuthology.orchestra.run.smithi057.stdout: "wrpos": 210193000, 2023-10-08T14:07:13.407 INFO:teuthology.orchestra.run.smithi057.stdout: "rdpos": 210187807, 2023-10-08T14:07:13.407 INFO:teuthology.orchestra.run.smithi057.stdout: "jlat": { 2023-10-08T14:07:13.407 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.407 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.407 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.407 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.407 INFO:teuthology.orchestra.run.smithi057.stdout: "replayed": 70209 2023-10-08T14:07:13.407 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.407 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_mem": { 2023-10-08T14:07:13.407 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 9709, 2023-10-08T14:07:13.409 INFO:teuthology.orchestra.run.smithi057.stdout: "ino+": 34268, 2023-10-08T14:07:13.409 INFO:teuthology.orchestra.run.smithi057.stdout: "ino-": 24559, 2023-10-08T14:07:13.409 INFO:teuthology.orchestra.run.smithi057.stdout: "dir": 382, 2023-10-08T14:07:13.409 INFO:teuthology.orchestra.run.smithi057.stdout: "dir+": 1139, 2023-10-08T14:07:13.409 INFO:teuthology.orchestra.run.smithi057.stdout: "dir-": 757, 2023-10-08T14:07:13.409 INFO:teuthology.orchestra.run.smithi057.stdout: "dn": 31965, 2023-10-08T14:07:13.409 INFO:teuthology.orchestra.run.smithi057.stdout: "dn+": 59805, 2023-10-08T14:07:13.409 INFO:teuthology.orchestra.run.smithi057.stdout: "dn-": 27840, 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: "cap": 0, 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: "cap+": 0, 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: "cap-": 0, 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: "rss": 114248, 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: "heap": 207152 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_server": { 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: "dispatch_client_request": 0, 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: "dispatch_server_request": 0, 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_request": 0, 2023-10-08T14:07:13.410 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_session": 0, 2023-10-08T14:07:13.411 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_peer_request": 0, 2023-10-08T14:07:13.411 INFO:teuthology.orchestra.run.smithi057.stdout: "req_create_latency": { 2023-10-08T14:07:13.411 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.411 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.411 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.411 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.411 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getattr_latency": { 2023-10-08T14:07:13.411 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.411 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.411 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getfilelock_latency": { 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: "req_link_latency": { 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.412 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.413 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookup_latency": { 2023-10-08T14:07:13.413 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.413 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.413 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.413 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.413 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookuphash_latency": { 2023-10-08T14:07:13.413 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.413 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.413 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.413 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.414 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupino_latency": { 2023-10-08T14:07:13.414 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.414 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.414 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.414 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.414 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupname_latency": { 2023-10-08T14:07:13.414 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.414 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.414 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.414 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.415 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupparent_latency": { 2023-10-08T14:07:13.415 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.415 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.415 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.415 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.415 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupsnap_latency": { 2023-10-08T14:07:13.415 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.415 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.415 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.416 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.416 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lssnap_latency": { 2023-10-08T14:07:13.416 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.416 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.416 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.416 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.416 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mkdir_latency": { 2023-10-08T14:07:13.416 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.416 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.416 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.417 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.417 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mknod_latency": { 2023-10-08T14:07:13.417 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.417 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.417 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.417 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.417 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mksnap_latency": { 2023-10-08T14:07:13.417 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.417 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.417 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.418 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.418 INFO:teuthology.orchestra.run.smithi057.stdout: "req_open_latency": { 2023-10-08T14:07:13.418 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.418 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.418 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.418 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.418 INFO:teuthology.orchestra.run.smithi057.stdout: "req_readdir_latency": { 2023-10-08T14:07:13.418 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.419 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.419 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.419 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.419 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rename_latency": { 2023-10-08T14:07:13.420 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.420 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.420 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.420 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.420 INFO:teuthology.orchestra.run.smithi057.stdout: "req_renamesnap_latency": { 2023-10-08T14:07:13.420 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: "req_snapdiff_latency": { 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmdir_latency": { 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.421 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmsnap_latency": { 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmxattr_latency": { 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.422 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setattr_latency": { 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setdirlayout_latency": { 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setfilelock_latency": { 2023-10-08T14:07:13.423 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setlayout_latency": { 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setxattr_latency": { 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.424 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: "req_symlink_latency": { 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: "req_unlink_latency": { 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.425 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_revoke_eviction": 0, 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_acquisition_throttle": 0, 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getvxattr_latency": { 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_sessions": { 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: "session_count": 1, 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: "session_add": 2, 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: "session_remove": 1, 2023-10-08T14:07:13.426 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions_open": 1, 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions_stale": 0, 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "total_load": 0, 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "average_load": 0, 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "avg_session_uptime": 2348, 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "mdthresh_evicted": 0 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:13.427 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:13.428 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:13.429 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:13.429 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:13.429 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:13.429 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:13.429 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:13.429 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:13.429 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:13.429 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:13.430 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:13.430 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:13.430 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:13.430 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:13.430 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 569084, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 9978, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2023-10-08T14:07:13.431 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 9816, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 195, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2023-10-08T14:07:13.432 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 45310030, 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 481092, 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "objecter": { 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "op_active": 0, 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "op_laggy": 0, 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "op_send": 3395, 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "op_send_bytes": 21, 2023-10-08T14:07:13.433 INFO:teuthology.orchestra.run.smithi057.stdout: "op_resend": 3, 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: "op_reply": 3395, 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: "op_latency": { 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 3395, 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 39.120413066, 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.011522949 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: "op_inflight": 0, 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: "oplen_avg": { 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 3395, 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 3396 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: "op": 3395, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "op_r": 3395, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "op_w": 0, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "op_rmw": 0, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "op_pg": 0, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_stat": 2044, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_create": 0, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_read": 1349, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_write": 0, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_writefull": 0, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_writesame": 0, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_append": 0, 2023-10-08T14:07:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_zero": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_truncate": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_delete": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_mapext": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_sparse_read": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_clonerange": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_getxattr": 1, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_setxattr": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_cmpxattr": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_rmxattr": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_resetxattrs": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_call": 0, 2023-10-08T14:07:13.436 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_watch": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_notify": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_pgls": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_pgls_filter": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_other": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_active": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_send": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_resend": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_ping": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_active": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_send": 0, 2023-10-08T14:07:13.437 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_resend": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_active": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_send": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_resend": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_active": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_send": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_resend": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "command_active": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "command_send": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "command_resend": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "map_epoch": 113, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "map_full": 0, 2023-10-08T14:07:13.438 INFO:teuthology.orchestra.run.smithi057.stdout: "map_inc": 25, 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_sessions": 10, 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_session_open": 10, 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_session_close": 0, 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_laggy": 0, 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_wr": 0, 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_rd": 2, 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_del": 0 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "oft": { 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_objs": 0, 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_kv_pairs": 0, 2023-10-08T14:07:13.439 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_updates": 0, 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_removes": 0 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: "osd-slow-ops": { 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: "slow_ops_count": 0 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: "purge_queue": { 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_ops": 0, 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing": 0, 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_high_water": 0, 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executed": 0, 2023-10-08T14:07:13.440 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_item_in_journal": 0 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 3745, 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 207622876, 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 3745, 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-08T14:07:13.441 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 3745, 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 207622876, 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-objecter_bytes": { 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-08T14:07:13.442 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 3395, 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 206049497, 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 3395, 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 206049497, 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.443 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-objecter_ops": { 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 1024, 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:13.444 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 3395, 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 3395, 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 3395, 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 3395, 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.445 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-write_buf_throttle": { 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 3758096384, 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 0, 2023-10-08T14:07:13.446 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 0, 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-write_buf_throttle-0x55d123440960": { 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 3758096384, 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2023-10-08T14:07:13.447 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 0, 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 0, 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:13.448 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:13.449 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T14:07:13.848 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2023-10-08T14:07:13.848 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2023-10-08T14:07:13.848 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2023-10-08T14:07:13.848 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2023-10-08T14:07:13.848 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2023-10-08T14:07:13.848 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2023-10-08T14:07:13.848 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2023-10-08T14:07:13.848 INFO:tasks.check_counter:Daemon mds.d mds.dir_split=0 2023-10-08T14:07:13.848 DEBUG:tasks.check_counter:Getting stats from g 2023-10-08T14:07:13.849 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-08T14:07:14.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:14 smithi057 ceph-mon[99476]: pgmap v790: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 56 op/s 2023-10-08T14:07:14.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:14 smithi099 ceph-mon[103849]: pgmap v790: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 56 op/s 2023-10-08T14:07:14.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:14 smithi115 ceph-mon[104408]: pgmap v790: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 56 op/s 2023-10-08T14:07:14.800 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T14:07:15.976 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:07:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T14:07:15.850+0000 7fe4ce904700 -1 calc_pg_upmaps abort due to max <= 0 2023-10-08T14:07:15.976 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:07:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T14:07:15.850+0000 7fe4ce904700 -1 calc_pg_upmaps abort due to max <= 0 2023-10-08T14:07:16.238 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:16 smithi057 ceph-mon[99476]: pgmap v791: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 KiB/s wr, 23 op/s 2023-10-08T14:07:16.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:16 smithi115 ceph-mon[104408]: pgmap v791: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 KiB/s wr, 23 op/s 2023-10-08T14:07:16.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:16 smithi099 ceph-mon[103849]: pgmap v791: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 KiB/s wr, 23 op/s 2023-10-08T14:07:18.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:18 smithi115 ceph-mon[104408]: pgmap v792: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-08T14:07:18.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:18 smithi057 ceph-mon[99476]: pgmap v792: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-08T14:07:18.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:18 smithi099 ceph-mon[103849]: pgmap v792: 129 pgs: 72 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-08T14:07:19.404 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T14:07:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 4, 2023-10-08T14:07:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 4, 2023-10-08T14:07:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 2992, 2023-10-08T14:07:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 448, 2023-10-08T14:07:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2023-10-08T14:07:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.001863094, 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000045098, 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.000684345, 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 3, 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000809513, 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000269837 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 2, 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 217, 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 320, 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 191568, 2023-10-08T14:07:19.406 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 2, 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.015953589, 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.012132884, 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.001771669, 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 215, 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.073564252, 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000342159 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.407 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-08T14:07:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 191568 2023-10-08T14:07:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 290, 2023-10-08T14:07:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 285, 2023-10-08T14:07:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 764800, 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 171280, 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.044796094, 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.016706645, 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.021009936, 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.007906435, 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 284, 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.122195418, 2023-10-08T14:07:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000430265 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 764800, 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 171280 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2023-10-08T14:07:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2023-10-08T14:07:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2023-10-08T14:07:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 147920, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 27, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2023-10-08T14:07:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2023-10-08T14:07:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 292, 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 723815, 2023-10-08T14:07:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 292, 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 292, 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 723815, 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:19.419 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T14:07:19.743 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:19 smithi057 ceph-mon[99476]: pgmap v793: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2023-10-08T14:07:19.847 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-10-08T14:07:19.847 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-10-08T14:07:19.848 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-10-08T14:07:19.848 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-10-08T14:07:19.848 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-10-08T14:07:19.848 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-10-08T14:07:19.848 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-10-08T14:07:19.849 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-10-08T14:07:19.849 DEBUG:tasks.check_counter:Getting stats from j 2023-10-08T14:07:19.849 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-08T14:07:19.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:19 smithi115 ceph-mon[104408]: pgmap v793: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2023-10-08T14:07:20.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:19 smithi099 ceph-mon[103849]: pgmap v793: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2023-10-08T14:07:20.798 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T14:07:22.227 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:22 smithi057 ceph-mon[99476]: pgmap v794: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2023-10-08T14:07:22.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:22 smithi099 ceph-mon[103849]: pgmap v794: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2023-10-08T14:07:22.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:22 smithi115 ceph-mon[104408]: pgmap v794: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2023-10-08T14:07:22.599 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-08T14:07:22.599 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:22.599 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 0, 2023-10-08T14:07:22.599 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 2, 2023-10-08T14:07:22.599 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 0, 2023-10-08T14:07:22.599 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 0, 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.001511812, 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000000000, 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.000417618, 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:22.600 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 2, 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 221, 2023-10-08T14:07:22.601 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 320, 2023-10-08T14:07:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 192272, 2023-10-08T14:07:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2023-10-08T14:07:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 2, 2023-10-08T14:07:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.015955493, 2023-10-08T14:07:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.011833967, 2023-10-08T14:07:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.002013165, 2023-10-08T14:07:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T14:07:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 219, 2023-10-08T14:07:22.602 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.080912906, 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000369465 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 192272 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:22.603 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 299, 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 292, 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 769776, 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 174560, 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.047483565, 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.017900801, 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.022671126, 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.008063965, 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:22.604 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 292, 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.078329254, 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000268250 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 769776, 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 174560 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.605 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:22.608 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:22.608 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:22.608 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:22.608 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:22.608 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:22.608 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:22.608 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:22.608 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:22.608 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:22.608 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 148761, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 31, 2023-10-08T14:07:22.609 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:22.610 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:22.611 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:22.611 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2023-10-08T14:07:22.611 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2023-10-08T14:07:22.611 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2023-10-08T14:07:22.611 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:22.611 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2023-10-08T14:07:22.611 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:22.611 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2023-10-08T14:07:22.611 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-08T14:07:22.611 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:22.612 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2023-10-08T14:07:22.612 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2023-10-08T14:07:22.612 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2023-10-08T14:07:22.612 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 297, 2023-10-08T14:07:22.612 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 725102, 2023-10-08T14:07:22.612 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:22.612 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 297, 2023-10-08T14:07:22.612 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2023-10-08T14:07:22.612 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2023-10-08T14:07:22.612 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 297, 2023-10-08T14:07:22.613 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 725102, 2023-10-08T14:07:22.613 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2023-10-08T14:07:22.613 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2023-10-08T14:07:22.613 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2023-10-08T14:07:22.613 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2023-10-08T14:07:22.613 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:22.613 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-08T14:07:22.613 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-08T14:07:22.976 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-10-08T14:07:22.976 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-10-08T14:07:22.976 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-10-08T14:07:22.976 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-10-08T14:07:22.976 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-10-08T14:07:22.976 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-10-08T14:07:22.976 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-10-08T14:07:22.976 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-10-08T14:07:22.976 DEBUG:tasks.check_counter:Getting stats from b 2023-10-08T14:07:22.977 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-08T14:07:23.863 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T14:07:24.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:24 smithi115 ceph-mon[104408]: pgmap v795: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-10-08T14:07:24.450 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:24 smithi099 ceph-mon[103849]: pgmap v795: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-10-08T14:07:24.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:24 smithi057 ceph-mon[99476]: pgmap v795: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-10-08T14:07:25.668 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T14:07:25.668 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:25.669 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 233673, 2023-10-08T14:07:25.669 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 230575, 2023-10-08T14:07:25.669 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 57636779, 2023-10-08T14:07:25.669 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 170243747, 2023-10-08T14:07:25.669 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 18, 2023-10-08T14:07:25.669 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2023-10-08T14:07:25.669 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 9.507421218, 2023-10-08T14:07:25.669 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 3.828794547, 2023-10-08T14:07:25.669 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 5.792371791, 2023-10-08T14:07:25.670 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.974358487, 2023-10-08T14:07:25.670 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:25.670 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 230573, 2023-10-08T14:07:25.670 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 58.866535616, 2023-10-08T14:07:25.670 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000255305 2023-10-08T14:07:25.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.670 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:25.670 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 418066, 2023-10-08T14:07:25.670 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.038134043, 2023-10-08T14:07:25.670 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000091 2023-10-08T14:07:25.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.671 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 491904, 2023-10-08T14:07:25.671 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 155664 2023-10-08T14:07:25.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.671 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:25.671 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 46294, 2023-10-08T14:07:25.671 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 43231, 2023-10-08T14:07:25.671 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 24112866, 2023-10-08T14:07:25.671 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 147698189, 2023-10-08T14:07:25.672 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 32, 2023-10-08T14:07:25.672 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 6, 2023-10-08T14:07:25.672 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 2.867954286, 2023-10-08T14:07:25.672 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.861537011, 2023-10-08T14:07:25.672 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.997887416, 2023-10-08T14:07:25.672 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 1.116446685, 2023-10-08T14:07:25.672 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:25.672 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 43230, 2023-10-08T14:07:25.672 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 68.766819503, 2023-10-08T14:07:25.673 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001590719 2023-10-08T14:07:25.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.673 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:25.673 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 25948, 2023-10-08T14:07:25.673 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.004232209, 2023-10-08T14:07:25.673 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000163 2023-10-08T14:07:25.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.673 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-10-08T14:07:25.673 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 465792 2023-10-08T14:07:25.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.674 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:25.674 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 223537, 2023-10-08T14:07:25.674 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 300294, 2023-10-08T14:07:25.674 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 114643656, 2023-10-08T14:07:25.674 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 219126524, 2023-10-08T14:07:25.674 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 161, 2023-10-08T14:07:25.674 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2023-10-08T14:07:25.674 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 11.255886746, 2023-10-08T14:07:25.675 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 4.876670128, 2023-10-08T14:07:25.675 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 91.092435289, 2023-10-08T14:07:25.675 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.850734168, 2023-10-08T14:07:25.675 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:25.675 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 300293, 2023-10-08T14:07:25.675 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 125.112253800, 2023-10-08T14:07:25.675 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000416633 2023-10-08T14:07:25.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.675 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:25.675 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 362829, 2023-10-08T14:07:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.060299699, 2023-10-08T14:07:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000166 2023-10-08T14:07:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 64504608, 2023-10-08T14:07:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 152745072 2023-10-08T14:07:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-10-08T14:07:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2023-10-08T14:07:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-10-08T14:07:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.677 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2023-10-08T14:07:25.677 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-10-08T14:07:25.677 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-10-08T14:07:25.677 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 7334, 2023-10-08T14:07:25.677 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 58.962368690, 2023-10-08T14:07:25.677 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.008039592 2023-10-08T14:07:25.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:25.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.677 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2023-10-08T14:07:25.677 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-10-08T14:07:25.678 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-10-08T14:07:25.678 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 20443, 2023-10-08T14:07:25.678 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3.632761507, 2023-10-08T14:07:25.678 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000177701 2023-10-08T14:07:25.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:25.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.678 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2023-10-08T14:07:25.678 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 104174, 2023-10-08T14:07:25.678 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 97019, 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 97019, 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2403.930449876, 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.024777934 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 7155, 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 159, 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 1, 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 4616, 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 69, 2023-10-08T14:07:25.679 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 21124, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 14625, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 6414, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 85, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 183, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 64431, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 1, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 1, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 7, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 183461, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 153259, 2023-10-08T14:07:25.680 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 6998, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 981, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 17, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 223, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 19, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 395, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 48518, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 322, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 53926, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2023-10-08T14:07:25.681 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 604, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 4, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 246, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 7, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 604, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 41, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 4, 2023-10-08T14:07:25.682 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 37, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 563, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 54, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 8772, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 23184, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 5, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 6, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 32085, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 14983, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 330, 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 24990 2023-10-08T14:07:25.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 2864, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 7698, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 4283, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 5285, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 3641, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 75, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 10809, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 10734, 2023-10-08T14:07:25.684 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 4, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 397, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 69, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2023-10-08T14:07:25.685 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 4, 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 70626, 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 70612, 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 70612, 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 14, 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 1339, 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 152, 2023-10-08T14:07:25.686 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 151, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 151, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 1, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 1, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 2, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 240638307, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 240678870, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2375, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 178.719344652, 2023-10-08T14:07:25.687 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.075250250 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 12107, 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 59044, 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 46937, 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 535, 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 4757, 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 4222, 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 21126, 2023-10-08T14:07:25.688 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 102708, 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 81582, 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 1, 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 37995, 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 37994, 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 189328, 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 207152 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 107789, 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 1745, 2023-10-08T14:07:25.689 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 104174, 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 416, 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 2319, 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 11413, 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 58.934566828, 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.005163810 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 588, 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 10.250210995, 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.017432331 2023-10-08T14:07:25.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2023-10-08T14:07:25.691 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 60543, 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2061.952004864, 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.034057645 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1, 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000293306, 2023-10-08T14:07:25.692 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000293306 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.693 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 16, 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.007995647, 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000499727 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2023-10-08T14:07:25.694 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 96, 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.705426679, 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.007348194 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 16, 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 9.549173123, 2023-10-08T14:07:25.695 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.596823320 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2946, 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 12.514008164, 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.004247796 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 547, 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3.751933309, 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.006859110 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.696 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 11008, 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 196.176088821, 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.017821228 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2023-10-08T14:07:25.697 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.698 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.698 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.698 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2023-10-08T14:07:25.698 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 79, 2023-10-08T14:07:25.698 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 14.784922448, 2023-10-08T14:07:25.698 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.187150917 2023-10-08T14:07:25.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.698 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2023-10-08T14:07:25.698 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 9, 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 15.941242648, 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 1.771249183 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 7, 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.029244952, 2023-10-08T14:07:25.699 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.004177850 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 7, 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.020424443, 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002917777 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3, 2023-10-08T14:07:25.701 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.002476162, 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000825387 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 9740, 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 19.310437487, 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001982591 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2023-10-08T14:07:25.702 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 4, 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 2, 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 2, 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 8660, 2023-10-08T14:07:25.703 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 4330, 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 943, 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:25.704 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:25.705 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:25.706 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 1397469, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 12342, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 176, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 2, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:25.707 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9816, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 195, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 43888190, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 440231, 2023-10-08T14:07:25.708 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 73195, 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 305149261, 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 32, 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 73195, 2023-10-08T14:07:25.709 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 73195, 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 35576.398969542, 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.486049579 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 73195, 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 147599 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 73195, 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 1022, 2023-10-08T14:07:25.710 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 72173, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 3916, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 42625, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 254, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 2629, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 391, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 69, 2023-10-08T14:07:25.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 21585, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 619, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 64611, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2023-10-08T14:07:25.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 5, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2023-10-08T14:07:25.713 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 113, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 30, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 12, 2023-10-08T14:07:25.714 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 12, 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 9015, 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 320, 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 1560 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2023-10-08T14:07:25.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 4 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 10, 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 6, 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 10734, 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:25.716 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 503500, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 142919728, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 503500, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 503500, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 142919728, 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:25.717 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:25.718 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 73195, 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 306834775, 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 73195, 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 306834775, 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.719 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 73195, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 73195, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 73195, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 73195, 2023-10-08T14:07:25.720 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 10734, 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1688062, 2023-10-08T14:07:25.721 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 10734, 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 253, 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1688062, 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:25.722 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5620338806e0": { 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 70626, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 236484566, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 70626, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2375, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 236484566, 2023-10-08T14:07:25.723 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:25.724 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:25.724 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:25.724 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:25.724 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:25.724 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:25.724 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T14:07:26.046 INFO:tasks.check_counter:Daemon mds.b mds.exported=395 2023-10-08T14:07:26.046 INFO:tasks.check_counter:Daemon mds.b mds.imported=322 2023-10-08T14:07:26.046 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=2864 2023-10-08T14:07:26.046 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=7698 2023-10-08T14:07:26.046 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2023-10-08T14:07:26.046 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2023-10-08T14:07:26.046 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=9 2023-10-08T14:07:26.047 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=69 2023-10-08T14:07:26.047 DEBUG:tasks.check_counter:Getting stats from e 2023-10-08T14:07:26.047 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-08T14:07:26.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:26 smithi099 ceph-mon[103849]: pgmap v796: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T14:07:26.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:26 smithi115 ceph-mon[104408]: pgmap v796: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T14:07:26.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:26 smithi057 ceph-mon[99476]: pgmap v796: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T14:07:26.784 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T14:07:28.282 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:28 smithi099 ceph-mon[103849]: pgmap v797: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T14:07:28.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:28 smithi115 ceph-mon[104408]: pgmap v797: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T14:07:28.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:28 smithi057 ceph-mon[99476]: pgmap v797: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T14:07:28.596 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T14:07:28.597 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:28.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 2240, 2023-10-08T14:07:28.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2239, 2023-10-08T14:07:28.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 172686151, 2023-10-08T14:07:28.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 695540, 2023-10-08T14:07:28.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 8, 2023-10-08T14:07:28.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2023-10-08T14:07:28.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.263206435, 2023-10-08T14:07:28.597 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.053531861, 2023-10-08T14:07:28.598 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.127231205, 2023-10-08T14:07:28.598 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.066906140, 2023-10-08T14:07:28.598 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:28.598 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2238, 2023-10-08T14:07:28.598 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.279772752, 2023-10-08T14:07:28.598 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000125010 2023-10-08T14:07:28.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.598 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:28.598 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.598 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.599 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.599 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T14:07:28.599 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T14:07:28.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.599 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:28.599 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1056, 2023-10-08T14:07:28.599 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1024, 2023-10-08T14:07:28.599 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 131213149, 2023-10-08T14:07:28.599 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 398663, 2023-10-08T14:07:28.600 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2023-10-08T14:07:28.600 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2023-10-08T14:07:28.600 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.160881023, 2023-10-08T14:07:28.600 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.029677249, 2023-10-08T14:07:28.600 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.103062741, 2023-10-08T14:07:28.600 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.026363962, 2023-10-08T14:07:28.600 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:28.600 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1023, 2023-10-08T14:07:28.601 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.161646189, 2023-10-08T14:07:28.601 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000158011 2023-10-08T14:07:28.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.601 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:28.601 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.601 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.601 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.601 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 983904, 2023-10-08T14:07:28.601 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 179968 2023-10-08T14:07:28.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.602 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:28.602 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 936, 2023-10-08T14:07:28.602 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1155, 2023-10-08T14:07:28.602 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 151290598, 2023-10-08T14:07:28.602 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 744481, 2023-10-08T14:07:28.602 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2023-10-08T14:07:28.602 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2023-10-08T14:07:28.602 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.155987325, 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.025470622, 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.095303050, 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.027079171, 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1153, 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.193111210, 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000167485 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.603 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.604 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.604 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T14:07:28.604 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 453696 2023-10-08T14:07:28.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.604 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-10-08T14:07:28.604 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2023-10-08T14:07:28.604 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-10-08T14:07:28.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.604 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2023-10-08T14:07:28.605 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-10-08T14:07:28.605 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-10-08T14:07:28.605 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 6303, 2023-10-08T14:07:28.605 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.168366842, 2023-10-08T14:07:28.605 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000026712 2023-10-08T14:07:28.605 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:28.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.605 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2023-10-08T14:07:28.606 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-10-08T14:07:28.606 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-10-08T14:07:28.606 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.606 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.606 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:28.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.606 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2023-10-08T14:07:28.607 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2023-10-08T14:07:28.607 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2023-10-08T14:07:28.607 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2023-10-08T14:07:28.607 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.607 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.607 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.607 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2023-10-08T14:07:28.607 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2023-10-08T14:07:28.607 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2023-10-08T14:07:28.608 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2023-10-08T14:07:28.608 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2023-10-08T14:07:28.608 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2023-10-08T14:07:28.608 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2023-10-08T14:07:28.608 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 83929, 2023-10-08T14:07:28.608 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 6759, 2023-10-08T14:07:28.608 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 77015, 2023-10-08T14:07:28.608 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 155, 2023-10-08T14:07:28.608 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 74272, 2023-10-08T14:07:28.608 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 1013, 2023-10-08T14:07:28.609 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2023-10-08T14:07:28.609 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2023-10-08T14:07:28.609 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 3, 2023-10-08T14:07:28.609 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2023-10-08T14:07:28.609 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2023-10-08T14:07:28.609 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2023-10-08T14:07:28.609 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2023-10-08T14:07:28.609 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2023-10-08T14:07:28.609 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2023-10-08T14:07:28.610 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2023-10-08T14:07:28.610 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2023-10-08T14:07:28.610 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2023-10-08T14:07:28.610 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2023-10-08T14:07:28.610 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2023-10-08T14:07:28.610 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2023-10-08T14:07:28.610 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2023-10-08T14:07:28.610 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2023-10-08T14:07:28.610 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T14:07:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T14:07:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2023-10-08T14:07:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2023-10-08T14:07:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T14:07:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2023-10-08T14:07:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T14:07:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T14:07:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T14:07:28.612 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2023-10-08T14:07:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2023-10-08T14:07:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2023-10-08T14:07:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2023-10-08T14:07:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2023-10-08T14:07:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2023-10-08T14:07:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2023-10-08T14:07:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2023-10-08T14:07:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2023-10-08T14:07:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2023-10-08T14:07:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2023-10-08T14:07:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2023-10-08T14:07:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2023-10-08T14:07:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2023-10-08T14:07:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2023-10-08T14:07:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2023-10-08T14:07:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2023-10-08T14:07:28.614 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2023-10-08T14:07:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2023-10-08T14:07:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2023-10-08T14:07:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2023-10-08T14:07:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2023-10-08T14:07:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T14:07:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2023-10-08T14:07:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2023-10-08T14:07:28.615 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2023-10-08T14:07:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2023-10-08T14:07:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2023-10-08T14:07:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2023-10-08T14:07:28.616 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2023-10-08T14:07:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2023-10-08T14:07:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2023-10-08T14:07:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2023-10-08T14:07:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 119791, 2023-10-08T14:07:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2023-10-08T14:07:28.617 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2023-10-08T14:07:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2023-10-08T14:07:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2023-10-08T14:07:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2023-10-08T14:07:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 136, 2023-10-08T14:07:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 11, 2023-10-08T14:07:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2023-10-08T14:07:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2023-10-08T14:07:28.618 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 67546137, 2023-10-08T14:07:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 457208917, 2023-10-08T14:07:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 457208343, 2023-10-08T14:07:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2023-10-08T14:07:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 119791 2023-10-08T14:07:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2023-10-08T14:07:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 28294, 2023-10-08T14:07:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 96406, 2023-10-08T14:07:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 68112, 2023-10-08T14:07:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 1090, 2023-10-08T14:07:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 2988, 2023-10-08T14:07:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 1898, 2023-10-08T14:07:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 83929, 2023-10-08T14:07:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 168163, 2023-10-08T14:07:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 84234, 2023-10-08T14:07:28.620 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2023-10-08T14:07:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2023-10-08T14:07:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2023-10-08T14:07:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 197176, 2023-10-08T14:07:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 207152 2023-10-08T14:07:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2023-10-08T14:07:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2023-10-08T14:07:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2023-10-08T14:07:28.621 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2023-10-08T14:07:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2023-10-08T14:07:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2023-10-08T14:07:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2023-10-08T14:07:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2023-10-08T14:07:28.622 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2023-10-08T14:07:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2023-10-08T14:07:28.623 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.624 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2023-10-08T14:07:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2023-10-08T14:07:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2023-10-08T14:07:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2023-10-08T14:07:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2023-10-08T14:07:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.627 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2023-10-08T14:07:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2023-10-08T14:07:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.628 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2023-10-08T14:07:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2023-10-08T14:07:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.629 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2023-10-08T14:07:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2023-10-08T14:07:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.630 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2023-10-08T14:07:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2023-10-08T14:07:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2023-10-08T14:07:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2023-10-08T14:07:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2023-10-08T14:07:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2023-10-08T14:07:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2023-10-08T14:07:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2023-10-08T14:07:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2023-10-08T14:07:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2023-10-08T14:07:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2023-10-08T14:07:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2023-10-08T14:07:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2023-10-08T14:07:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2023-10-08T14:07:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2023-10-08T14:07:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.637 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2023-10-08T14:07:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 1, 2023-10-08T14:07:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 1, 2023-10-08T14:07:28.638 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 0, 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 1, 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 2374, 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:28.640 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:28.641 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 1223431, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 29019, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:07:28.642 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-10-08T14:07:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-10-08T14:07:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-10-08T14:07:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9816, 2023-10-08T14:07:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 195, 2023-10-08T14:07:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:28.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-10-08T14:07:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 125504128, 2023-10-08T14:07:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 1176125, 2023-10-08T14:07:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-10-08T14:07:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:28.644 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 3873, 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 21, 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 3, 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 3873, 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3873, 2023-10-08T14:07:28.645 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 39.905087215, 2023-10-08T14:07:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.010303404 2023-10-08T14:07:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2023-10-08T14:07:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2023-10-08T14:07:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3873, 2023-10-08T14:07:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3874 2023-10-08T14:07:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 3873, 2023-10-08T14:07:28.646 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 3873, 2023-10-08T14:07:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2023-10-08T14:07:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2023-10-08T14:07:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2023-10-08T14:07:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 2264, 2023-10-08T14:07:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2023-10-08T14:07:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 1607, 2023-10-08T14:07:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2023-10-08T14:07:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2023-10-08T14:07:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2023-10-08T14:07:28.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2023-10-08T14:07:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2023-10-08T14:07:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2023-10-08T14:07:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2023-10-08T14:07:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2023-10-08T14:07:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2023-10-08T14:07:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2023-10-08T14:07:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 1, 2023-10-08T14:07:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2023-10-08T14:07:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2023-10-08T14:07:28.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2023-10-08T14:07:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2023-10-08T14:07:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2023-10-08T14:07:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2023-10-08T14:07:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2023-10-08T14:07:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T14:07:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2023-10-08T14:07:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2023-10-08T14:07:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2023-10-08T14:07:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2023-10-08T14:07:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2023-10-08T14:07:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2023-10-08T14:07:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2023-10-08T14:07:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2023-10-08T14:07:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2023-10-08T14:07:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2023-10-08T14:07:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2023-10-08T14:07:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2023-10-08T14:07:28.650 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2023-10-08T14:07:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2023-10-08T14:07:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2023-10-08T14:07:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2023-10-08T14:07:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2023-10-08T14:07:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2023-10-08T14:07:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2023-10-08T14:07:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 113, 2023-10-08T14:07:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2023-10-08T14:07:28.651 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 25, 2023-10-08T14:07:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 11, 2023-10-08T14:07:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 11, 2023-10-08T14:07:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2023-10-08T14:07:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2023-10-08T14:07:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2023-10-08T14:07:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2023-10-08T14:07:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2023-10-08T14:07:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2023-10-08T14:07:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2023-10-08T14:07:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2023-10-08T14:07:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2023-10-08T14:07:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2023-10-08T14:07:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2023-10-08T14:07:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2023-10-08T14:07:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2023-10-08T14:07:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2023-10-08T14:07:28.653 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T14:07:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2023-10-08T14:07:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2023-10-08T14:07:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2023-10-08T14:07:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2023-10-08T14:07:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.654 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-10-08T14:07:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 4228, 2023-10-08T14:07:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 454788568, 2023-10-08T14:07:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 4228, 2023-10-08T14:07:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:28.655 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 4228, 2023-10-08T14:07:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 454788568, 2023-10-08T14:07:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2023-10-08T14:07:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:28.656 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-10-08T14:07:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3873, 2023-10-08T14:07:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 453131528, 2023-10-08T14:07:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3873, 2023-10-08T14:07:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 453131528, 2023-10-08T14:07:28.657 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2023-10-08T14:07:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2023-10-08T14:07:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:28.658 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3873, 2023-10-08T14:07:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 3873, 2023-10-08T14:07:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3873, 2023-10-08T14:07:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 3873, 2023-10-08T14:07:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:28.659 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:28.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:28.661 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x56281b4b6960": { 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2023-10-08T14:07:28.662 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2023-10-08T14:07:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:28.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:28.663 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T14:07:29.772 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:29 smithi099 ceph-mon[103849]: pgmap v798: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-10-08T14:07:29.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:29 smithi115 ceph-mon[104408]: pgmap v798: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-10-08T14:07:29.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:29 smithi057 ceph-mon[99476]: pgmap v798: 129 pgs: 70 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 37 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-10-08T14:07:31.239 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-10-08T14:07:31.239 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-10-08T14:07:31.240 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-10-08T14:07:31.240 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-10-08T14:07:31.240 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-10-08T14:07:31.240 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-10-08T14:07:31.240 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-10-08T14:07:31.240 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-10-08T14:07:31.240 DEBUG:tasks.check_counter:Getting stats from h 2023-10-08T14:07:31.240 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-08T14:07:32.269 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T14:07:32.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:32 smithi099 ceph-mon[103849]: pgmap v799: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2023-10-08T14:07:32.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:31 smithi115 ceph-mon[104408]: pgmap v799: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2023-10-08T14:07:32.493 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:31 smithi057 ceph-mon[99476]: pgmap v799: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2023-10-08T14:07:34.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:34 smithi099 ceph-mon[103849]: pgmap v800: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2023-10-08T14:07:34.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:34 smithi115 ceph-mon[104408]: pgmap v800: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2023-10-08T14:07:34.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:34 smithi057 ceph-mon[99476]: pgmap v800: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2023-10-08T14:07:36.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:36 smithi099 ceph-mon[103849]: pgmap v801: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T14:07:36.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:36 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:07:36.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:36 smithi115 ceph-mon[104408]: pgmap v801: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T14:07:36.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:36 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:07:36.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:36 smithi057 ceph-mon[99476]: pgmap v801: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T14:07:36.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:36 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T14:07:37.182 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T14:07:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 713430, 2023-10-08T14:07:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 762898, 2023-10-08T14:07:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 167210193, 2023-10-08T14:07:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 305838450, 2023-10-08T14:07:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 21, 2023-10-08T14:07:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2023-10-08T14:07:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 22.921194772, 2023-10-08T14:07:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 9.318847023, 2023-10-08T14:07:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 296.309541753, 2023-10-08T14:07:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.422509069, 2023-10-08T14:07:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 762892, 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 68.526827310, 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000089825 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1356044, 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.144922334, 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000106 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 49720720, 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 101112688 2023-10-08T14:07:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 217558, 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 219392, 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 48765302, 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 118450245, 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 8, 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 8.651055856, 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 3.641066253, 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 5.844491823, 2023-10-08T14:07:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.357508717, 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 219396, 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 11.282963614, 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000051427 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 417076, 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.041640724, 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000099 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 996208, 2023-10-08T14:07:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 189040 2023-10-08T14:07:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 21248, 2023-10-08T14:07:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 20784, 2023-10-08T14:07:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 9729225, 2023-10-08T14:07:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 83220191, 2023-10-08T14:07:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 14, 2023-10-08T14:07:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 8, 2023-10-08T14:07:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 1.534738071, 2023-10-08T14:07:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.547060027, 2023-10-08T14:07:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.690980766, 2023-10-08T14:07:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.389725734, 2023-10-08T14:07:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 20782, 2023-10-08T14:07:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 5.444936746, 2023-10-08T14:07:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000262002 2023-10-08T14:07:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 24553, 2023-10-08T14:07:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.004049816, 2023-10-08T14:07:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000164 2023-10-08T14:07:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-08T14:07:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 544576 2023-10-08T14:07:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-10-08T14:07:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2023-10-08T14:07:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-10-08T14:07:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2023-10-08T14:07:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-10-08T14:07:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-10-08T14:07:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 6170, 2023-10-08T14:07:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 47.238434258, 2023-10-08T14:07:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.007656148 2023-10-08T14:07:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2023-10-08T14:07:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-10-08T14:07:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-10-08T14:07:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 17901, 2023-10-08T14:07:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2.563658361, 2023-10-08T14:07:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000143213 2023-10-08T14:07:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2023-10-08T14:07:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 63544, 2023-10-08T14:07:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 62340, 2023-10-08T14:07:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2023-10-08T14:07:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 62340, 2023-10-08T14:07:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2051.339631123, 2023-10-08T14:07:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.032905672 2023-10-08T14:07:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2023-10-08T14:07:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 1203, 2023-10-08T14:07:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 89, 2023-10-08T14:07:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2023-10-08T14:07:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 515, 2023-10-08T14:07:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 56, 2023-10-08T14:07:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2023-10-08T14:07:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 21949, 2023-10-08T14:07:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 2, 2023-10-08T14:07:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 1, 2023-10-08T14:07:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 21946, 2023-10-08T14:07:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 21946, 2023-10-08T14:07:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 41630, 2023-10-08T14:07:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2023-10-08T14:07:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2023-10-08T14:07:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 5, 2023-10-08T14:07:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 114729, 2023-10-08T14:07:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 94130, 2023-10-08T14:07:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 1020, 2023-10-08T14:07:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 1112, 2023-10-08T14:07:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 7, 2023-10-08T14:07:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2023-10-08T14:07:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 764, 2023-10-08T14:07:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 2000, 2023-10-08T14:07:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 2, 2023-10-08T14:07:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 332, 2023-10-08T14:07:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 32564, 2023-10-08T14:07:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 438, 2023-10-08T14:07:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 26926, 2023-10-08T14:07:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2023-10-08T14:07:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 286, 2023-10-08T14:07:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2023-10-08T14:07:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 2768, 2023-10-08T14:07:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 32974639, 2023-10-08T14:07:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 2, 2023-10-08T14:07:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 286, 2023-10-08T14:07:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2023-10-08T14:07:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T14:07:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2023-10-08T14:07:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2023-10-08T14:07:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T14:07:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T14:07:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 286, 2023-10-08T14:07:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 200, 2023-10-08T14:07:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 10291, 2023-10-08T14:07:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 20790, 2023-10-08T14:07:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T14:07:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 2, 2023-10-08T14:07:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T14:07:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 34365, 2023-10-08T14:07:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 13685, 2023-10-08T14:07:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 345, 2023-10-08T14:07:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 23614 2023-10-08T14:07:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2023-10-08T14:07:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 2298, 2023-10-08T14:07:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 7723, 2023-10-08T14:07:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 4402, 2023-10-08T14:07:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 5458, 2023-10-08T14:07:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 4243, 2023-10-08T14:07:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 69, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 9281, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 9212, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2023-10-08T14:07:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 332, 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 56, 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 2, 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 74883, 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2023-10-08T14:07:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 74883, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 106, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 106, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 8, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2023-10-08T14:07:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2023-10-08T14:07:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 267213478, 2023-10-08T14:07:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2023-10-08T14:07:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2023-10-08T14:07:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3041, 2023-10-08T14:07:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 134.957545554, 2023-10-08T14:07:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.044379330 2023-10-08T14:07:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2023-10-08T14:07:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 11363, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 42610, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 31247, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 446, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 4316, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 3870, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 21949, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 72833, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 50884, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 22396, 2023-10-08T14:07:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 22396, 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 179908, 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 198960 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 69681, 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 4162, 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 63544, 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 501, 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 5939, 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2023-10-08T14:07:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 10377, 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 29.141409543, 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002808269 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 50, 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 10.478817425, 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.209576348 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2023-10-08T14:07:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2023-10-08T14:07:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 29760, 2023-10-08T14:07:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1535.120146819, 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.051583338 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2023-10-08T14:07:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2023-10-08T14:07:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2023-10-08T14:07:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 79, 2023-10-08T14:07:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 6.082326654, 2023-10-08T14:07:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.076991476 2023-10-08T14:07:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2023-10-08T14:07:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2023-10-08T14:07:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2023-10-08T14:07:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2929, 2023-10-08T14:07:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 6.257891939, 2023-10-08T14:07:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002136528 2023-10-08T14:07:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2023-10-08T14:07:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 422, 2023-10-08T14:07:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.749120380, 2023-10-08T14:07:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.004144835 2023-10-08T14:07:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2023-10-08T14:07:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 10380, 2023-10-08T14:07:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 429.410731585, 2023-10-08T14:07:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.041369049 2023-10-08T14:07:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 69, 2023-10-08T14:07:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 14.771554791, 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.214080504 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 8274, 2023-10-08T14:07:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 18.327631987, 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002215087 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2023-10-08T14:07:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 1, 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 1, 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 0, 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 1, 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 962, 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 962, 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 2381, 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-10-08T14:07:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 1204432, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 9763, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9816, 2023-10-08T14:07:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 195, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 42328184, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 449712, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 23390, 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 275074033, 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 33, 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 23390, 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 23390, 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1019.644167818, 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.043593166 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 23390, 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 24283 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 23390, 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 671, 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 22719, 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2023-10-08T14:07:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 296, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 3337, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 174, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 56, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 18498, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 287, 2023-10-08T14:07:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2023-10-08T14:07:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2023-10-08T14:07:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 113, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 25, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 12, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 12, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 1260, 2023-10-08T14:07:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 178, 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 167 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 19 2023-10-08T14:07:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 10, 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 5, 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 9212, 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 952232, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 133879347, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 952232, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 952232, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 133879347, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 23390, 2023-10-08T14:07:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 276480581, 2023-10-08T14:07:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 23390, 2023-10-08T14:07:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 276480581, 2023-10-08T14:07:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 23390, 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 23390, 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 23390, 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 23390, 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 9212, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1407972, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 9212, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 296, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1407972, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5612b2fc26e0": { 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 74883, 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 263019174, 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 74883, 2023-10-08T14:07:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3041, 2023-10-08T14:07:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 263019174, 2023-10-08T14:07:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:37.242 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T14:07:37.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:07:37.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:07:37.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:37 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:07:37.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:07:37.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:07:37.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:37 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:07:37.500 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T14:07:37.501 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T14:07:37.501 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:37 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' 2023-10-08T14:07:37.519 INFO:tasks.check_counter:Daemon mds.h mds.exported=332 2023-10-08T14:07:37.520 INFO:tasks.check_counter:Daemon mds.h mds.imported=438 2023-10-08T14:07:37.520 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=2298 2023-10-08T14:07:37.520 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=7723 2023-10-08T14:07:37.520 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=2 2023-10-08T14:07:37.520 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-10-08T14:07:37.520 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-10-08T14:07:37.520 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=56 2023-10-08T14:07:37.520 DEBUG:tasks.check_counter:Getting stats from k 2023-10-08T14:07:37.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-08T14:07:38.185 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T14:07:38.420 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:38 smithi099 ceph-mon[103849]: pgmap v802: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-08T14:07:38.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:38 smithi115 ceph-mon[104408]: pgmap v802: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-08T14:07:38.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:38 smithi057 ceph-mon[99476]: pgmap v802: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-08T14:07:39.916 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-08T14:07:39.916 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:39.916 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 2179, 2023-10-08T14:07:39.916 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2181, 2023-10-08T14:07:39.916 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 102197321, 2023-10-08T14:07:39.916 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 677670, 2023-10-08T14:07:39.916 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 8, 2023-10-08T14:07:39.916 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.228247772, 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.054143199, 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.096053387, 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.062575385, 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2179, 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.272611590, 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000125108 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 932, 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1153, 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 100911857, 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 746381, 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 7, 2023-10-08T14:07:39.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 6, 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.138193722, 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.023783569, 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.074775117, 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.031528936, 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1152, 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.203649350, 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000176778 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.919 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 458016 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 900, 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 868, 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 80828616, 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 351337, 2023-10-08T14:07:39.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.119876411, 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.026060543, 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.071479799, 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.022291734, 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 867, 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.103609823, 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000119503 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 984992, 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 182464 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2023-10-08T14:07:39.922 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5952, 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.151992338, 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000025536 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2023-10-08T14:07:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 58995, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 1062, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 50707, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 7226, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 57477, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 340, 2023-10-08T14:07:39.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 1, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2023-10-08T14:07:39.926 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2023-10-08T14:07:39.927 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T14:07:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2023-10-08T14:07:39.929 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2023-10-08T14:07:39.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 82032, 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2023-10-08T14:07:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 122, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 8, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 286072183, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 286064419, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.932 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 82032 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 20751, 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 42720, 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 21969, 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 702, 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 996, 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 294, 2023-10-08T14:07:39.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 58995, 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 72024, 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 13029, 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 161988, 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 207152 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2023-10-08T14:07:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2023-10-08T14:07:39.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.937 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2023-10-08T14:07:39.939 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.941 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2023-10-08T14:07:39.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.943 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2023-10-08T14:07:39.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.946 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.947 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 2, 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 0, 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 2, 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2023-10-08T14:07:39.948 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 2382, 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:39.950 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2023-10-08T14:07:39.951 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 958975, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 21220, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9816, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 195, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 90086003, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 927461, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2023-10-08T14:07:39.953 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 1, 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 3649, 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 21, 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 3, 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 3648, 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3648, 2023-10-08T14:07:39.954 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 30.476305763, 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.008354250 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 1, 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3649, 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3650 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 3649, 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 3649, 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2023-10-08T14:07:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 2216, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 1431, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2023-10-08T14:07:39.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 1, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2023-10-08T14:07:39.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2023-10-08T14:07:39.958 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 113, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 25, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 11, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 11, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2023-10-08T14:07:39.959 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2023-10-08T14:07:39.960 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-10-08T14:07:39.961 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 4007, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 283556146, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 4007, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 4007, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 283556146, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.962 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3649, 2023-10-08T14:07:39.963 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 281934606, 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3648, 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 281934606, 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 1, 2023-10-08T14:07:39.964 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3649, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 3649, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3648, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 3648, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.965 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:39.966 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5597ff9a6960": { 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2023-10-08T14:07:39.967 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2023-10-08T14:07:39.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:39.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-08T14:07:39.969 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-08T14:07:39.969 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:39 smithi115 ceph-mon[104408]: pgmap v803: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-08T14:07:39.969 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:39 smithi099 ceph-mon[103849]: pgmap v803: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-08T14:07:39.993 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:39 smithi057 ceph-mon[99476]: pgmap v803: 129 pgs: 58 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 51 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-08T14:07:40.211 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-10-08T14:07:40.211 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-10-08T14:07:40.211 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2023-10-08T14:07:40.211 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2023-10-08T14:07:40.211 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2023-10-08T14:07:40.211 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-10-08T14:07:40.211 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-10-08T14:07:40.211 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2023-10-08T14:07:40.211 DEBUG:tasks.check_counter:Getting stats from c 2023-10-08T14:07:40.212 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-08T14:07:41.139 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T14:07:42.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:42 smithi115 ceph-mon[104408]: pgmap v804: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-08T14:07:42.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:42 smithi057 ceph-mon[99476]: pgmap v804: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-08T14:07:42.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:42 smithi099 ceph-mon[103849]: pgmap v804: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-08T14:07:43.606 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-10-08T14:07:43.606 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:43.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 54359, 2023-10-08T14:07:43.606 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 41794, 2023-10-08T14:07:43.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 39298774, 2023-10-08T14:07:43.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 78494433, 2023-10-08T14:07:43.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 11, 2023-10-08T14:07:43.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2023-10-08T14:07:43.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 3.049967768, 2023-10-08T14:07:43.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 1.154938901, 2023-10-08T14:07:43.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 2.061853613, 2023-10-08T14:07:43.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.456172778, 2023-10-08T14:07:43.607 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:43.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 41794, 2023-10-08T14:07:43.608 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 6.562356881, 2023-10-08T14:07:43.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000157016 2023-10-08T14:07:43.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.608 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:43.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 77340, 2023-10-08T14:07:43.608 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.011307301, 2023-10-08T14:07:43.608 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000146 2023-10-08T14:07:43.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.609 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1920, 2023-10-08T14:07:43.609 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 1568 2023-10-08T14:07:43.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.609 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:43.609 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 26824, 2023-10-08T14:07:43.609 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 21351, 2023-10-08T14:07:43.609 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 18743590, 2023-10-08T14:07:43.609 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 91253717, 2023-10-08T14:07:43.609 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 16, 2023-10-08T14:07:43.610 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 8, 2023-10-08T14:07:43.610 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 1.959126804, 2023-10-08T14:07:43.610 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.755422050, 2023-10-08T14:07:43.610 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.896327423, 2023-10-08T14:07:43.610 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.498584796, 2023-10-08T14:07:43.610 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:43.610 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 21350, 2023-10-08T14:07:43.610 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.537154318, 2023-10-08T14:07:43.611 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000165674 2023-10-08T14:07:43.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.611 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:43.611 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 28751, 2023-10-08T14:07:43.611 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.004236623, 2023-10-08T14:07:43.611 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000147 2023-10-08T14:07:43.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.611 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-10-08T14:07:43.611 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 462480 2023-10-08T14:07:43.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 165465, 2023-10-08T14:07:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 230743, 2023-10-08T14:07:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 67308287, 2023-10-08T14:07:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 173871626, 2023-10-08T14:07:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 15, 2023-10-08T14:07:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 8, 2023-10-08T14:07:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 7.759123586, 2023-10-08T14:07:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 3.609857149, 2023-10-08T14:07:43.612 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 56.465878702, 2023-10-08T14:07:43.613 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.320526488, 2023-10-08T14:07:43.613 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:43.613 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 230740, 2023-10-08T14:07:43.613 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 69.759124404, 2023-10-08T14:07:43.613 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000302327 2023-10-08T14:07:43.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.613 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:43.613 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 259083, 2023-10-08T14:07:43.613 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.046243745, 2023-10-08T14:07:43.613 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000178 2023-10-08T14:07:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 58506224, 2023-10-08T14:07:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 100504032 2023-10-08T14:07:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2023-10-08T14:07:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2023-10-08T14:07:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2023-10-08T14:07:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2023-10-08T14:07:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-10-08T14:07:43.615 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-10-08T14:07:43.615 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4793, 2023-10-08T14:07:43.615 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 46.196464516, 2023-10-08T14:07:43.615 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.009638319 2023-10-08T14:07:43.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:43.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.615 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2023-10-08T14:07:43.615 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-10-08T14:07:43.615 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-10-08T14:07:43.616 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 16901, 2023-10-08T14:07:43.616 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.792546079, 2023-10-08T14:07:43.616 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000224397 2023-10-08T14:07:43.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:43.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.616 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2023-10-08T14:07:43.616 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 63372, 2023-10-08T14:07:43.616 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 54626, 2023-10-08T14:07:43.616 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2023-10-08T14:07:43.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 54626, 2023-10-08T14:07:43.617 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 408.048934522, 2023-10-08T14:07:43.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.007469866 2023-10-08T14:07:43.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.617 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2023-10-08T14:07:43.617 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 8746, 2023-10-08T14:07:43.617 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 66, 2023-10-08T14:07:43.617 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2023-10-08T14:07:43.617 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 664, 2023-10-08T14:07:43.617 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 73, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 17663, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 3, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 74, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 17586, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 17656, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 37780, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 1, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 1, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 5, 2023-10-08T14:07:43.618 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 102168, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 81165, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 8616, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 1061, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 33, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 382, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 265, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 25516, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 328, 2023-10-08T14:07:43.619 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 16033, 2023-10-08T14:07:43.620 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2023-10-08T14:07:43.620 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 267, 2023-10-08T14:07:43.620 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2023-10-08T14:07:43.620 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 2768, 2023-10-08T14:07:43.620 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 32974639, 2023-10-08T14:07:43.620 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 2, 2023-10-08T14:07:43.620 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 267, 2023-10-08T14:07:43.620 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2023-10-08T14:07:43.620 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T14:07:43.620 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 267, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 156, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 11774, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 22508, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 4, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 32223, 2023-10-08T14:07:43.621 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 15300, 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 285, 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 25712 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 2355, 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 7353, 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 4180, 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 5324, 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 3343, 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 63, 2023-10-08T14:07:43.622 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 9039, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 8976, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2023-10-08T14:07:43.623 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 265, 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 73, 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 70209, 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2023-10-08T14:07:43.624 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 70209, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 100, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 100, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 7, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2023-10-08T14:07:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 210193000, 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2202, 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 148.223912653, 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.067313311 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2023-10-08T14:07:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 8861, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 35147, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 26286, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 395, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 4286, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 3891, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 17663, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 66018, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 48355, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 1, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 21414, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 21413, 2023-10-08T14:07:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 158096, 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 198960 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 68019, 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 2169, 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 63372, 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 256, 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 3813, 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 11578, 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 24.502672140, 2023-10-08T14:07:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002116313 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 172, 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 5.011626663, 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.029137364 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 18834, 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 169.630043693, 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.009006586 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2023-10-08T14:07:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000200560, 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000200560 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 91, 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4.762019879, 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.052329888 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2023-10-08T14:07:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3517, 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 12.963523292, 2023-10-08T14:07:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.003685960 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 441, 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.029052178, 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002333451 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 11595, 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 156.624724050, 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.013507953 2023-10-08T14:07:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2023-10-08T14:07:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 63, 2023-10-08T14:07:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 18.608732921, 2023-10-08T14:07:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.295376713 2023-10-08T14:07:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2023-10-08T14:07:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2023-10-08T14:07:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2023-10-08T14:07:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2023-10-08T14:07:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2023-10-08T14:07:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 8334, 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 14.916339146, 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001789817 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2023-10-08T14:07:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2023-10-08T14:07:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2023-10-08T14:07:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2023-10-08T14:07:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 1, 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 2, 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 1, 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 1, 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 867, 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 867, 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 970, 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2023-10-08T14:07:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2023-10-08T14:07:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 1102532, 2023-10-08T14:07:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 7721, 2023-10-08T14:07:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:07:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2023-10-08T14:07:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2023-10-08T14:07:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2023-10-08T14:07:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9816, 2023-10-08T14:07:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 195, 2023-10-08T14:07:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2023-10-08T14:07:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 33464492, 2023-10-08T14:07:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 355939, 2023-10-08T14:07:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2023-10-08T14:07:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2023-10-08T14:07:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2023-10-08T14:07:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2023-10-08T14:07:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2023-10-08T14:07:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 22177, 2023-10-08T14:07:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 220127576, 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 33, 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 22177, 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 22177, 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 519.757415786, 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.023436777 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 22177, 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 23193 2023-10-08T14:07:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 22177, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 617, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 21560, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 10, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 284, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 2487, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 191, 2023-10-08T14:07:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 73, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 18013, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 267, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 20, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2023-10-08T14:07:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2023-10-08T14:07:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2023-10-08T14:07:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 113, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 25, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 1542, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 132, 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 174 2023-10-08T14:07:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 2 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2023-10-08T14:07:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 10, 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 5, 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 8976, 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 246644, 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 96195438, 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 246644, 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 246644, 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 96195438, 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 22177, 2023-10-08T14:07:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 221397688, 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 22177, 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 221397688, 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 22177, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 22177, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 22177, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 22177, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 8976, 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 1271168, 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 8976, 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 285, 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 1271168, 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x560da5e346e0": { 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 70209, 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 205998696, 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 70209, 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 2202, 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 205998696, 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:43.666 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-10-08T14:07:44.067 INFO:tasks.check_counter:Daemon mds.c mds.exported=265 2023-10-08T14:07:44.067 INFO:tasks.check_counter:Daemon mds.c mds.imported=328 2023-10-08T14:07:44.067 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=2355 2023-10-08T14:07:44.067 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=7353 2023-10-08T14:07:44.067 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=2 2023-10-08T14:07:44.067 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-10-08T14:07:44.067 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-10-08T14:07:44.067 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=73 2023-10-08T14:07:44.068 DEBUG:tasks.check_counter:Getting stats from f 2023-10-08T14:07:44.068 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-08T14:07:44.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:44 smithi115 ceph-mon[104408]: pgmap v805: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-08T14:07:44.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:44 smithi057 ceph-mon[99476]: pgmap v805: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-08T14:07:44.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:44 smithi099 ceph-mon[103849]: pgmap v805: 129 pgs: 58 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 52 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-08T14:07:44.837 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T14:07:45.343 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:45 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:45.496 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:45 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:45.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:45 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:46.412 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-10-08T14:07:46.412 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:46.413 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 159052, 2023-10-08T14:07:46.413 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 222089, 2023-10-08T14:07:46.413 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 60684626, 2023-10-08T14:07:46.413 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 227840856, 2023-10-08T14:07:46.413 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 22, 2023-10-08T14:07:46.413 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2023-10-08T14:07:46.413 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 8.348975547, 2023-10-08T14:07:46.413 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 3.833917463, 2023-10-08T14:07:46.413 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 98.800920284, 2023-10-08T14:07:46.413 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.577495961, 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 222085, 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 73.155106931, 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000329401 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 252630, 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.044747125, 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000177 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.414 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 55866672, 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 116165856 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 26463, 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 32377, 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 10872109, 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 108342390, 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 13, 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 2.422156860, 2023-10-08T14:07:46.415 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.845450159, 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.805064611, 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.803134199, 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 32375, 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 7.640858298, 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000236011 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 27571, 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.004976878, 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000180 2023-10-08T14:07:46.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 548240 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 53353, 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 54721, 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 33734437, 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 97373092, 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 8, 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2023-10-08T14:07:46.417 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 3.522343509, 2023-10-08T14:07:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 1.314741643, 2023-10-08T14:07:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 2.070886663, 2023-10-08T14:07:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.578651591, 2023-10-08T14:07:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 54723, 2023-10-08T14:07:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 11.061732726, 2023-10-08T14:07:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000202140 2023-10-08T14:07:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 85422, 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.014232715, 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000166 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1008512, 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 189472 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.419 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4673, 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 52.995552470, 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.011340798 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 26498, 2023-10-08T14:07:46.420 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4.938892235, 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000186387 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 80149, 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 72816, 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 72816, 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1841.408159944, 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.025288510 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.421 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 7333, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 87, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 584, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 64, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 37432, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 382, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 28787, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 8263, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 36883, 2023-10-08T14:07:46.422 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 18677, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 1, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 1, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 7, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 135198, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 115102, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 7279, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 890, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 14, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 427, 2023-10-08T14:07:46.423 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 1000, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 1, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 216, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 12443, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 254, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 17984, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 81, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 2768, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 32974639, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 2, 2023-10-08T14:07:46.424 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 81, 2023-10-08T14:07:46.425 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2023-10-08T14:07:46.425 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T14:07:46.425 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2023-10-08T14:07:46.425 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2023-10-08T14:07:46.425 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T14:07:46.425 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T14:07:46.425 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 81, 2023-10-08T14:07:46.425 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 189, 2023-10-08T14:07:46.425 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 9382, 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 25028, 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 29180, 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 13270, 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 309, 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 31909 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2023-10-08T14:07:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 3426, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 7247, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 4187, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 5169, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 4985, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 110, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 14168, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 14058, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2023-10-08T14:07:46.427 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2023-10-08T14:07:46.428 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2023-10-08T14:07:46.428 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2023-10-08T14:07:46.428 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2023-10-08T14:07:46.428 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2023-10-08T14:07:46.428 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T14:07:46.428 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 221, 2023-10-08T14:07:46.428 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2023-10-08T14:07:46.428 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 64, 2023-10-08T14:07:46.428 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2023-10-08T14:07:46.428 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2023-10-08T14:07:46.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.429 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2023-10-08T14:07:46.429 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 1, 2023-10-08T14:07:46.429 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 82032, 2023-10-08T14:07:46.429 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2023-10-08T14:07:46.429 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2023-10-08T14:07:46.429 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 82032, 2023-10-08T14:07:46.429 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 122, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 122, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 8, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 286072183, 2023-10-08T14:07:46.430 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2216, 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 150.331035056, 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.067838914 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 20208, 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 43082, 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 22874, 2023-10-08T14:07:46.431 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 747, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 4004, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 3257, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 37432, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 74956, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 37524, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 1, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 24602, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 24601, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 202120, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 198960 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.432 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 87272, 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 2930, 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 80149, 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 317, 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 3918, 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 9792, 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 39.212056302, 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004004499 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.433 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2023-10-08T14:07:46.434 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 84, 2023-10-08T14:07:46.434 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.405410195, 2023-10-08T14:07:46.434 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.028635835 2023-10-08T14:07:46.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.434 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2023-10-08T14:07:46.434 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.435 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.435 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.435 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2023-10-08T14:07:46.435 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.435 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.435 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.435 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 36033, 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1488.210938687, 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.041301333 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.436 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.437 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.438 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2023-10-08T14:07:46.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.438 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.438 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2023-10-08T14:07:46.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.439 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.439 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2023-10-08T14:07:46.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 72, 2023-10-08T14:07:46.439 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 5.803980848, 2023-10-08T14:07:46.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.080610845 2023-10-08T14:07:46.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.439 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2796, 2023-10-08T14:07:46.440 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 23.970860929, 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.008573269 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 693, 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.727734867, 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002493123 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 9765, 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 237.573471969, 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.024329080 2023-10-08T14:07:46.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.442 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 110, 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 15.706466412, 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.142786058 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.443 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.445 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2023-10-08T14:07:46.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 13471, 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 26.797239735, 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001989253 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.447 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 2, 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 0, 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 2770, 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 1385, 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 927, 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2023-10-08T14:07:46.449 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:46.449 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:46.449 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:46.449 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:46.449 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:46.449 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:46.449 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:46.449 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:46.449 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:46.449 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:46.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2023-10-08T14:07:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 1745709, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 15273, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:46.452 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9816, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 195, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 73712196, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 744383, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2023-10-08T14:07:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:46.454 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2023-10-08T14:07:46.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.454 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2023-10-08T14:07:46.454 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2023-10-08T14:07:46.454 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2023-10-08T14:07:46.454 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 32315, 2023-10-08T14:07:46.454 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 295736338, 2023-10-08T14:07:46.454 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 29, 2023-10-08T14:07:46.454 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 32315, 2023-10-08T14:07:46.454 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 32315, 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 805.502278260, 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.024926575 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 32315, 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 33277 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 32315, 2023-10-08T14:07:46.455 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 546, 2023-10-08T14:07:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 31769, 2023-10-08T14:07:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2023-10-08T14:07:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2023-10-08T14:07:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2023-10-08T14:07:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 10, 2023-10-08T14:07:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 378, 2023-10-08T14:07:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 2594, 2023-10-08T14:07:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 193, 2023-10-08T14:07:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2023-10-08T14:07:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2023-10-08T14:07:46.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2023-10-08T14:07:46.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 64, 2023-10-08T14:07:46.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 28195, 2023-10-08T14:07:46.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2023-10-08T14:07:46.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2023-10-08T14:07:46.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2023-10-08T14:07:46.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 84, 2023-10-08T14:07:46.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 20, 2023-10-08T14:07:46.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2023-10-08T14:07:46.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2023-10-08T14:07:46.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2023-10-08T14:07:46.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2023-10-08T14:07:46.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2023-10-08T14:07:46.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2023-10-08T14:07:46.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T14:07:46.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2023-10-08T14:07:46.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2023-10-08T14:07:46.459 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 113, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 25, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2023-10-08T14:07:46.460 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 1399, 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 174, 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 166 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2023-10-08T14:07:46.461 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 24 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 10, 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 5, 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 14058, 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:46.462 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 238864, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 76860386, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 238864, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 238864, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 76860386, 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:46.463 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:46.464 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 32315, 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 297844772, 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 32315, 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 297844772, 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2023-10-08T14:07:46.465 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 32315, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 32315, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 32315, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 32315, 2023-10-08T14:07:46.466 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:46.467 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.467 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.467 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.467 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:46.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.467 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2023-10-08T14:07:46.467 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:46.467 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-10-08T14:07:46.468 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:46.468 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 14058, 2023-10-08T14:07:46.468 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 2109826, 2023-10-08T14:07:46.468 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:46.468 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 14058, 2023-10-08T14:07:46.468 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:07:46.468 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:07:46.468 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 378, 2023-10-08T14:07:46.468 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 2109826, 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55e76f1846e0": { 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 82032, 2023-10-08T14:07:46.469 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 281877879, 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 82032, 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 2216, 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 281877879, 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:46.470 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:46.471 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:46.471 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-10-08T14:07:46.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:46 smithi115 ceph-mon[104408]: pgmap v806: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-08T14:07:46.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:46 smithi057 ceph-mon[99476]: pgmap v806: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-08T14:07:46.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:46 smithi099 ceph-mon[103849]: pgmap v806: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-10-08T14:07:47.512 INFO:tasks.check_counter:Daemon mds.f mds.exported=216 2023-10-08T14:07:47.512 INFO:tasks.check_counter:Daemon mds.f mds.imported=254 2023-10-08T14:07:47.512 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=3426 2023-10-08T14:07:47.513 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=7247 2023-10-08T14:07:47.513 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=2 2023-10-08T14:07:47.513 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-10-08T14:07:47.513 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-10-08T14:07:47.513 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=64 2023-10-08T14:07:47.513 DEBUG:tasks.check_counter:Getting stats from i 2023-10-08T14:07:47.513 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-08T14:07:48.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:48 smithi115 ceph-mon[104408]: pgmap v807: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-10-08T14:07:48.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:48 smithi057 ceph-mon[99476]: pgmap v807: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-10-08T14:07:48.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:48 smithi099 ceph-mon[103849]: pgmap v807: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-10-08T14:07:49.098 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T14:07:49.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:49 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:49.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:49 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:49.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:49 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:50.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:50 smithi115 ceph-mon[104408]: pgmap v808: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-08T14:07:50.439 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:50 smithi115 ceph-mon[104408]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:50.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:50 smithi057 ceph-mon[99476]: pgmap v808: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-08T14:07:50.490 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:50 smithi057 ceph-mon[99476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:50.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:50 smithi099 ceph-mon[103849]: pgmap v808: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-08T14:07:50.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:50 smithi099 ceph-mon[103849]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T14:07:52.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:52 smithi115 ceph-mon[104408]: pgmap v809: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T14:07:52.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:52 smithi057 ceph-mon[99476]: pgmap v809: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T14:07:52.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:52 smithi099 ceph-mon[103849]: pgmap v809: 129 pgs: 58 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 53 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T14:07:53.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:53 smithi057 ceph-mon[99476]: pgmap v810: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-10-08T14:07:53.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:53 smithi099 ceph-mon[103849]: pgmap v810: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-10-08T14:07:53.689 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:53 smithi115 ceph-mon[104408]: pgmap v810: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-10-08T14:07:56.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:56 smithi115 ceph-mon[104408]: pgmap v811: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-10-08T14:07:56.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:56 smithi057 ceph-mon[99476]: pgmap v811: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-10-08T14:07:56.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:56 smithi099 ceph-mon[103849]: pgmap v811: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-10-08T14:07:57.588 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-10-08T14:07:57.588 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:07:57.588 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 605119, 2023-10-08T14:07:57.588 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 623497, 2023-10-08T14:07:57.588 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 133339487, 2023-10-08T14:07:57.588 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 345352016, 2023-10-08T14:07:57.589 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 12, 2023-10-08T14:07:57.589 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2023-10-08T14:07:57.589 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 19.803610832, 2023-10-08T14:07:57.589 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 7.858014219, 2023-10-08T14:07:57.589 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 12.926908737, 2023-10-08T14:07:57.589 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.662647191, 2023-10-08T14:07:57.589 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:57.589 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 623494, 2023-10-08T14:07:57.589 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 32.896944937, 2023-10-08T14:07:57.590 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000052762 2023-10-08T14:07:57.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.590 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:57.590 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1189134, 2023-10-08T14:07:57.590 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.115153039, 2023-10-08T14:07:57.590 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000096 2023-10-08T14:07:57.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.590 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T14:07:57.590 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T14:07:57.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.591 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:07:57.591 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 279254, 2023-10-08T14:07:57.591 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 458618, 2023-10-08T14:07:57.591 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 123022744, 2023-10-08T14:07:57.591 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 362789571, 2023-10-08T14:07:57.591 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 19, 2023-10-08T14:07:57.591 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2023-10-08T14:07:57.591 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 15.386721918, 2023-10-08T14:07:57.591 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 7.123103050, 2023-10-08T14:07:57.591 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 24.977868971, 2023-10-08T14:07:57.592 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.701374376, 2023-10-08T14:07:57.592 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:57.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 458618, 2023-10-08T14:07:57.592 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 113.023250714, 2023-10-08T14:07:57.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000246443 2023-10-08T14:07:57.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.592 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:57.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 528272, 2023-10-08T14:07:57.592 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.086064565, 2023-10-08T14:07:57.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000162 2023-10-08T14:07:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 88541776, 2023-10-08T14:07:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 243075776 2023-10-08T14:07:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:07:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 39620, 2023-10-08T14:07:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 42144, 2023-10-08T14:07:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 19569267, 2023-10-08T14:07:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 174529278, 2023-10-08T14:07:57.593 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 12, 2023-10-08T14:07:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 8, 2023-10-08T14:07:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 3.299626384, 2023-10-08T14:07:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 1.250618541, 2023-10-08T14:07:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 1.189874527, 2023-10-08T14:07:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.946672849, 2023-10-08T14:07:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:07:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 42143, 2023-10-08T14:07:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 9.720229405, 2023-10-08T14:07:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000230648 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 34969, 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.010641233, 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000304 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1027664, 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 195600 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2023-10-08T14:07:57.595 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7550, 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 93.415116878, 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.012372863 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2023-10-08T14:07:57.596 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 40495, 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 9.344171706, 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000230748 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 164034, 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 155880, 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 155880, 2023-10-08T14:07:57.597 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4541.615728137, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.029135333 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 8145, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 355, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 893, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 98, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 52989, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 337, 2023-10-08T14:07:57.598 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 24753, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 27899, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 52502, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 95507, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 2, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 2, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 6, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 278405, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 226244, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 8037, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 964, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 40, 2023-10-08T14:07:57.599 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 497, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 506, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 73987, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 372, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 78159, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 2482, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2023-10-08T14:07:57.600 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 2768, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 32974639, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 2, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 2482, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 28, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 28, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 2454, 2023-10-08T14:07:57.601 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 225, 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 16524, 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 45922, 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 39, 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 39380, 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 22585, 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 739, 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 48051 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.602 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 4158, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 4708, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 2577, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 3455, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 9093, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 192, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 21793, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 21601, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2023-10-08T14:07:57.603 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 513, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 99, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2023-10-08T14:07:57.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 5, 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 119791, 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 31661, 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 24581, 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 95210, 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 7080, 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 169, 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 41, 2023-10-08T14:07:57.605 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 33, 2023-10-08T14:07:57.606 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 136, 2023-10-08T14:07:57.606 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 8, 2023-10-08T14:07:57.606 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2023-10-08T14:07:57.606 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 8, 2023-10-08T14:07:57.606 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 67546137, 2023-10-08T14:07:57.606 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 457208917, 2023-10-08T14:07:57.606 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2023-10-08T14:07:57.606 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2023-10-08T14:07:57.606 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3557, 2023-10-08T14:07:57.606 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 354.852662200, 2023-10-08T14:07:57.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.099761783 2023-10-08T14:07:57.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.607 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2023-10-08T14:07:57.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.607 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2023-10-08T14:07:57.607 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 25996, 2023-10-08T14:07:57.607 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 98515, 2023-10-08T14:07:57.607 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 72519, 2023-10-08T14:07:57.607 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 1123, 2023-10-08T14:07:57.607 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 5197, 2023-10-08T14:07:57.608 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 4074, 2023-10-08T14:07:57.608 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 52989, 2023-10-08T14:07:57.608 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 174784, 2023-10-08T14:07:57.608 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 121795, 2023-10-08T14:07:57.608 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 2, 2023-10-08T14:07:57.608 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 62812, 2023-10-08T14:07:57.608 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 62810, 2023-10-08T14:07:57.608 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 228668, 2023-10-08T14:07:57.608 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 198960 2023-10-08T14:07:57.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.609 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2023-10-08T14:07:57.609 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 178992, 2023-10-08T14:07:57.609 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 1847, 2023-10-08T14:07:57.609 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 164034, 2023-10-08T14:07:57.609 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 394, 2023-10-08T14:07:57.609 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 4067, 2023-10-08T14:07:57.609 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2023-10-08T14:07:57.609 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 21419, 2023-10-08T14:07:57.609 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 135.423408337, 2023-10-08T14:07:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.006322583 2023-10-08T14:07:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2023-10-08T14:07:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2336, 2023-10-08T14:07:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 104.349137670, 2023-10-08T14:07:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.044670007 2023-10-08T14:07:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2023-10-08T14:07:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.611 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.611 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2023-10-08T14:07:57.611 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.611 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.611 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.611 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2023-10-08T14:07:57.611 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 82852, 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3829.737736534, 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.046223841 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.612 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.614 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 181, 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.728655504, 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.009550582 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.615 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5977, 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 62.335889316, 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.010429293 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1007, 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 5.351905442, 2023-10-08T14:07:57.616 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.005314702 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 21438, 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 322.653669401, 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.015050548 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.618 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2023-10-08T14:07:57.618 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.618 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.618 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.618 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2023-10-08T14:07:57.618 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 192, 2023-10-08T14:07:57.618 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 31.298722060, 2023-10-08T14:07:57.618 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.163014177 2023-10-08T14:07:57.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.619 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2023-10-08T14:07:57.619 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.619 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.619 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.619 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2023-10-08T14:07:57.619 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.619 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.619 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.620 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2023-10-08T14:07:57.620 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.620 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.620 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.620 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2023-10-08T14:07:57.620 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.620 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.620 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.621 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2023-10-08T14:07:57.621 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.621 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.621 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.621 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2023-10-08T14:07:57.621 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.621 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.622 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.622 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2023-10-08T14:07:57.622 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.622 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.622 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.622 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2023-10-08T14:07:57.622 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.622 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 20478, 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 48.736603873, 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002379949 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.623 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 1, 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 1, 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 0, 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 1, 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 2890, 2023-10-08T14:07:57.624 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 2890, 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 920, 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:07:57.625 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2023-10-08T14:07:57.626 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:07:57.627 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 2799210, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 26669, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2023-10-08T14:07:57.628 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9816, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 195, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2023-10-08T14:07:57.629 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 99920187, 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 1088715, 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 52160, 2023-10-08T14:07:57.630 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 475733292, 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 27, 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 52160, 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 52160, 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1532.230053272, 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.029375576 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 52160, 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 53812 2023-10-08T14:07:57.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 52160, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 3338, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 48822, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 1, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 28, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 501, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 4060, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 261, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2023-10-08T14:07:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 98, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 43337, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 2490, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 53, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2023-10-08T14:07:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2023-10-08T14:07:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2023-10-08T14:07:57.635 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 113, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 25, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 2052, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 710, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 221 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.636 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 24 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 10, 2023-10-08T14:07:57.637 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 5, 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 21601, 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 923989, 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 181394532, 2023-10-08T14:07:57.638 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 923989, 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 923989, 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 181394532, 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:57.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 52160, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 478875969, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 52160, 2023-10-08T14:07:57.640 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 478875969, 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:57.641 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 52160, 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 52160, 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 52160, 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 52160, 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 21601, 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 3148357, 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 21601, 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:07:57.643 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 501, 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 3148357, 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x559f1535e6e0": { 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-10-08T14:07:57.644 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 119791, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 453014613, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 119791, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3557, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 453014613, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:07:57.645 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:07:57.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:57.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:07:57.646 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-10-08T14:07:57.994 INFO:tasks.check_counter:Daemon mds.i mds.exported=506 2023-10-08T14:07:57.994 INFO:tasks.check_counter:Daemon mds.i mds.imported=372 2023-10-08T14:07:57.994 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=4158 2023-10-08T14:07:57.995 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=4708 2023-10-08T14:07:57.995 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=2 2023-10-08T14:07:57.995 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-10-08T14:07:57.995 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-10-08T14:07:57.995 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=98 2023-10-08T14:07:57.995 DEBUG:tasks.check_counter:Getting stats from l 2023-10-08T14:07:57.995 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-08T14:07:58.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:58 smithi115 ceph-mon[104408]: pgmap v812: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2023-10-08T14:07:58.489 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:58 smithi057 ceph-mon[99476]: pgmap v812: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2023-10-08T14:07:58.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:58 smithi099 ceph-mon[103849]: pgmap v812: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2023-10-08T14:07:58.695 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T14:07:59.600 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:07:59 smithi115 ceph-mon[104408]: pgmap v813: 129 pgs: 54 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 59 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2023-10-08T14:07:59.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:07:59 smithi057 ceph-mon[99476]: pgmap v813: 129 pgs: 54 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 59 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2023-10-08T14:08:00.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:07:59 smithi099 ceph-mon[103849]: pgmap v813: 129 pgs: 54 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 59 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2023-10-08T14:08:01.480 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-10-08T14:08:01.480 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T14:08:01.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2219, 2023-10-08T14:08:01.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 2182, 2023-10-08T14:08:01.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 78675340, 2023-10-08T14:08:01.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 761548, 2023-10-08T14:08:01.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 7, 2023-10-08T14:08:01.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2023-10-08T14:08:01.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.241760371, 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.055388509, 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.109823224, 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.065608159, 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2182, 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.359801912, 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000164895 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.481 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1008976, 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 186816 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 805, 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1033, 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 71518295, 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 802199, 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 8, 2023-10-08T14:08:01.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.107307903, 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.025262559, 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.053614380, 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.022000461, 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1031, 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.150671642, 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000146141 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.483 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 552400 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1115, 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1116, 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 88380779, 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 346461, 2023-10-08T14:08:01.484 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 6, 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.131772629, 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.025854068, 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.063292299, 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.035848134, 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1114, 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.201552687, 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000180927 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-10-08T14:08:01.485 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.486 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6218, 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.793320637, 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000127584 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.487 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 39159, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 27351, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 11807, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 1, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 86, 2023-10-08T14:08:01.489 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 1343, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 3, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2023-10-08T14:08:01.490 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 4, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 246, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 7, 2023-10-08T14:08:01.491 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T14:08:01.492 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2023-10-08T14:08:01.493 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2023-10-08T14:08:01.494 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2023-10-08T14:08:01.495 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 70628, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 2, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 66, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2023-10-08T14:08:01.496 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 240638307, 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 240683813, 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 240679011, 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 70628 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2023-10-08T14:08:01.497 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 13075, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 58013, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 44938, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 480, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 1733, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 1253, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 39159, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 98030, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 58871, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2023-10-08T14:08:01.498 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 125036, 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 207152 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.499 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2023-10-08T14:08:01.501 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.502 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.503 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.505 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2023-10-08T14:08:01.506 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.507 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.508 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.509 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.510 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.511 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.512 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 4, 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 2, 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 2390, 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2023-10-08T14:08:01.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T14:08:01.514 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T14:08:01.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T14:08:01.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 693523, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 13391, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2023-10-08T14:08:01.517 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9816, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 195, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 58012750, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 572007, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2023-10-08T14:08:01.518 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 3765, 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 26, 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 4, 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 3765, 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2023-10-08T14:08:01.519 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3765, 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 33.253382149, 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.008832239 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3765, 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3766 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 3765, 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 3765, 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2023-10-08T14:08:01.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 2179, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1583, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2023-10-08T14:08:01.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 2, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2023-10-08T14:08:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2023-10-08T14:08:01.523 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 113, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 25, 2023-10-08T14:08:01.524 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 11, 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 11, 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2023-10-08T14:08:01.525 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2023-10-08T14:08:01.526 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 4135, 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 238180716, 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 4135, 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:08:01.527 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4135, 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 238180716, 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:08:01.528 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3765, 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 236522398, 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3765, 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 236522398, 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3765, 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 3765, 2023-10-08T14:08:01.530 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3765, 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 3765, 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-10-08T14:08:01.531 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.532 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x5645187da960": { 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-10-08T14:08:01.533 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-10-08T14:08:01.534 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2023-10-08T14:08:01.534 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2023-10-08T14:08:01.534 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-10-08T14:08:01.534 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-10-08T14:08:01.534 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-10-08T14:08:01.534 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-10-08T14:08:01.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:08:01.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-10-08T14:08:01.534 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-10-08T14:08:01.947 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-10-08T14:08:01.948 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-10-08T14:08:01.948 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2023-10-08T14:08:01.948 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2023-10-08T14:08:01.948 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=7 2023-10-08T14:08:01.948 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2023-10-08T14:08:01.948 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2023-10-08T14:08:01.948 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2023-10-08T14:08:01.948 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-08T14:08:01.959 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-08T14:08:01.959 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T14:08:01.959 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T14:08:02.050 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T14:08:02.051 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T14:08:02.121 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T14:08:02.188 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:02 smithi115 ceph-mon[104408]: pgmap v814: 129 pgs: 46 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2023-10-08T14:08:02.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.478+0000 7fed8b3f1700 1 -- 172.21.15.57:0/2340452397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed84111410 msgr2=0x7fed84113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:02.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.478+0000 7fed8b3f1700 1 --2- 172.21.15.57:0/2340452397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed84111410 0x7fed84113800 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00b0a0 tx=0x7fed7c00b3b0 comp rx=0 tx=0).stop 2023-10-08T14:08:02.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.481+0000 7fed8b3f1700 1 -- 172.21.15.57:0/2340452397 shutdown_connections 2023-10-08T14:08:02.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.481+0000 7fed8b3f1700 1 --2- 172.21.15.57:0/2340452397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed84111410 0x7fed84113800 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:02.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.481+0000 7fed8b3f1700 1 --2- 172.21.15.57:0/2340452397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed840fef80 0x7fed840ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:02.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.481+0000 7fed8b3f1700 1 --2- 172.21.15.57:0/2340452397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed840ff8a0 0x7fed84110ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:02.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.481+0000 7fed8b3f1700 1 -- 172.21.15.57:0/2340452397 >> 172.21.15.57:0/2340452397 conn(0x7fed84074860 msgr2=0x7fed84074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:02.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.482+0000 7fed8b3f1700 1 -- 172.21.15.57:0/2340452397 shutdown_connections 2023-10-08T14:08:02.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.484+0000 7fed8b3f1700 1 -- 172.21.15.57:0/2340452397 wait complete. 2023-10-08T14:08:02.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.484+0000 7fed8b3f1700 1 Processor -- start 2023-10-08T14:08:02.515 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:02 smithi057 ceph-mon[99476]: pgmap v814: 129 pgs: 46 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2023-10-08T14:08:02.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.489+0000 7fed8b3f1700 1 -- start start 2023-10-08T14:08:02.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.489+0000 7fed8b3f1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed840fef80 0x7fed84071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:02.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.489+0000 7fed8b3f1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed84111410 0x7fed8406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:02.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.489+0000 7fed8b3f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed8406b600 0x7fed8406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:02.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.489+0000 7fed8b3f1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed840728a0 con 0x7fed84111410 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.489+0000 7fed8b3f1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8406fb80 con 0x7fed8406b600 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.489+0000 7fed8b3f1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8406fd40 con 0x7fed840fef80 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.489+0000 7fed8998e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed8406b600 0x7fed8406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.489+0000 7fed8998e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed8406b600 0x7fed8406bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33904/0 (socket says 172.21.15.57:33904) 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.489+0000 7fed8998e700 1 -- 172.21.15.57:0/441660671 learned_addr learned my addr 172.21.15.57:0/441660671 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.490+0000 7fed8898c700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed84111410 0x7fed8406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.490+0000 7fed8998e700 1 -- 172.21.15.57:0/441660671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed840fef80 msgr2=0x7fed84071ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.490+0000 7fed8998e700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed840fef80 0x7fed84071ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.490+0000 7fed8998e700 1 -- 172.21.15.57:0/441660671 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed84111410 msgr2=0x7fed8406b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.490+0000 7fed8998e700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed84111410 0x7fed8406b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:02.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.490+0000 7fed8998e700 1 -- 172.21.15.57:0/441660671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c009d00 con 0x7fed8406b600 2023-10-08T14:08:02.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.490+0000 7fed8998e700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed8406b600 0x7fed8406bab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00b070 tx=0x7fed7c005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:02.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.490+0000 7fed7a7fc700 1 -- 172.21.15.57:0/441660671 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c00e040 con 0x7fed8406b600 2023-10-08T14:08:02.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.490+0000 7fed7a7fc700 1 -- 172.21.15.57:0/441660671 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed7c01b450 con 0x7fed8406b600 2023-10-08T14:08:02.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.490+0000 7fed7a7fc700 1 -- 172.21.15.57:0/441660671 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c01c660 con 0x7fed8406b600 2023-10-08T14:08:02.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.491+0000 7fed8b3f1700 1 -- 172.21.15.57:0/441660671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8406ffc0 con 0x7fed8406b600 2023-10-08T14:08:02.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.491+0000 7fed8b3f1700 1 -- 172.21.15.57:0/441660671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed84070450 con 0x7fed8406b600 2023-10-08T14:08:02.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.494+0000 7fed7a7fc700 1 -- 172.21.15.57:0/441660671 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fed7c01b5c0 con 0x7fed8406b600 2023-10-08T14:08:02.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.495+0000 7fed8b3f1700 1 -- 172.21.15.57:0/441660671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed840687f0 con 0x7fed8406b600 2023-10-08T14:08:02.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.495+0000 7fed7a7fc700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fed700a1670 0x7fed700a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:02.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.496+0000 7fed7a7fc700 1 -- 172.21.15.57:0/441660671 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9581+0+0 (secure 0 0 0) 0x7fed7c0ce3e0 con 0x7fed8406b600 2023-10-08T14:08:02.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.496+0000 7fed8918d700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fed700a1670 0x7fed700a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:02.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.497+0000 7fed8918d700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fed700a1670 0x7fed700a3b30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fed80009710 tx=0x7fed80006c60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:02.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.503+0000 7fed7a7fc700 1 -- 172.21.15.57:0/441660671 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fed7c056100 con 0x7fed8406b600 2023-10-08T14:08:02.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:02 smithi099 ceph-mon[103849]: pgmap v814: 129 pgs: 46 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2023-10-08T14:08:02.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.851+0000 7fed8b3f1700 1 -- 172.21.15.57:0/441660671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed8406c5a0 con 0x7fed8406b600 2023-10-08T14:08:02.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.857+0000 7fed7a7fc700 1 -- 172.21.15.57:0/441660671 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+25340 (secure 0 0 0) 0x7fed7c097370 con 0x7fed8406b600 2023-10-08T14:08:02.864 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T14:08:02.864 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":113,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T14:07:09.533272+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":14,"length":1},{"begin":24,"length":1},{"begin":26,"length":1}]},{"pool":4,"snaps":[{"begin":12,"length":1},{"begin":14,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":1},{"begin":24,"length":1},{"begin":26,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":26,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":26,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T14:08:02.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.869+0000 7fed8b3f1700 1 -- 172.21.15.57:0/441660671 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fed700a1670 msgr2=0x7fed700a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:02.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.869+0000 7fed8b3f1700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fed700a1670 0x7fed700a3b30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fed80009710 tx=0x7fed80006c60 comp rx=0 tx=0).stop 2023-10-08T14:08:02.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.869+0000 7fed8b3f1700 1 -- 172.21.15.57:0/441660671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed8406b600 msgr2=0x7fed8406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:02.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.869+0000 7fed8b3f1700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed8406b600 0x7fed8406bab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00b070 tx=0x7fed7c005af0 comp rx=0 tx=0).stop 2023-10-08T14:08:02.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.869+0000 7fed8b3f1700 1 -- 172.21.15.57:0/441660671 shutdown_connections 2023-10-08T14:08:02.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.869+0000 7fed8b3f1700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fed700a1670 0x7fed700a3b30 secure :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fed80009710 tx=0x7fed80006c60 comp rx=0 tx=0).stop 2023-10-08T14:08:02.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.869+0000 7fed8b3f1700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed8406b600 0x7fed8406bab0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:02.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.869+0000 7fed8b3f1700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed840fef80 0x7fed84071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:02.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.869+0000 7fed8b3f1700 1 --2- 172.21.15.57:0/441660671 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed84111410 0x7fed8406b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:02.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.869+0000 7fed8b3f1700 1 -- 172.21.15.57:0/441660671 >> 172.21.15.57:0/441660671 conn(0x7fed84074860 msgr2=0x7fed8410ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:02.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.870+0000 7fed8b3f1700 1 -- 172.21.15.57:0/441660671 shutdown_connections 2023-10-08T14:08:02.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:02.870+0000 7fed8b3f1700 1 -- 172.21.15.57:0/441660671 wait complete. 2023-10-08T14:08:02.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T13:45:50.544240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T13:49:42.356778+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T13:49:43.577547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T13:49:47.570705+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T14:08:02.896 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T14:08:03.158 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.157+0000 7f072d28a700 1 -- 172.21.15.57:0/2049976526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0728107a60 msgr2=0x7f0728107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:03.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.157+0000 7f072d28a700 1 --2- 172.21.15.57:0/2049976526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0728107a60 0x7f0728107e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f07100099c0 tx=0x7f0710009cd0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.157+0000 7f072d28a700 1 -- 172.21.15.57:0/2049976526 shutdown_connections 2023-10-08T14:08:03.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.157+0000 7f072d28a700 1 --2- 172.21.15.57:0/2049976526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f072810c500 0x7f0728114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.157+0000 7f072d28a700 1 --2- 172.21.15.57:0/2049976526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0728107a60 0x7f0728107e40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.157+0000 7f072d28a700 1 --2- 172.21.15.57:0/2049976526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0728108410 0x7f072810bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.157+0000 7f072d28a700 1 -- 172.21.15.57:0/2049976526 >> 172.21.15.57:0/2049976526 conn(0x7f072806bc80 msgr2=0x7f072806c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:03.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.157+0000 7f072d28a700 1 -- 172.21.15.57:0/2049976526 shutdown_connections 2023-10-08T14:08:03.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.157+0000 7f072d28a700 1 -- 172.21.15.57:0/2049976526 wait complete. 2023-10-08T14:08:03.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.158+0000 7f072d28a700 1 Processor -- start 2023-10-08T14:08:03.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.164+0000 7f072d28a700 1 -- start start 2023-10-08T14:08:03.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.164+0000 7f072d28a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f072810c500 0x7f072810e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:03.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.164+0000 7f072d28a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f072810e590 0x7f072810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:03.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.164+0000 7f072d28a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728080680 0x7f0728082af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:03.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.164+0000 7f072d28a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0728083030 con 0x7f072810c500 2023-10-08T14:08:03.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.164+0000 7f072d28a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0728083170 con 0x7f0728080680 2023-10-08T14:08:03.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.164+0000 7f072d28a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0728083330 con 0x7f072810e590 2023-10-08T14:08:03.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.170+0000 7f07277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728080680 0x7f0728082af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:03.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.170+0000 7f07277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728080680 0x7f0728082af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33918/0 (socket says 172.21.15.57:33918) 2023-10-08T14:08:03.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.170+0000 7f07277fe700 1 -- 172.21.15.57:0/2429335840 learned_addr learned my addr 172.21.15.57:0/2429335840 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:03.173 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.170+0000 7f07267fc700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f072810e590 0x7f072810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:03.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.170+0000 7f07277fe700 1 -- 172.21.15.57:0/2429335840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f072810e590 msgr2=0x7f072810ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:03.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.170+0000 7f07277fe700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f072810e590 0x7f072810ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.170+0000 7f07277fe700 1 -- 172.21.15.57:0/2429335840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f072810c500 msgr2=0x7f072810e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:03.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.170+0000 7f07277fe700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f072810c500 0x7f072810e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.170+0000 7f07277fe700 1 -- 172.21.15.57:0/2429335840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100095f0 con 0x7f0728080680 2023-10-08T14:08:03.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.171+0000 7f07277fe700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728080680 0x7f0728082af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f072000ef30 tx=0x7f072000c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:03.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.171+0000 7f071ffff700 1 -- 172.21.15.57:0/2429335840 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720018070 con 0x7f0728080680 2023-10-08T14:08:03.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.171+0000 7f072d28a700 1 -- 172.21.15.57:0/2429335840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07280835b0 con 0x7f0728080680 2023-10-08T14:08:03.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.172+0000 7f071ffff700 1 -- 172.21.15.57:0/2429335840 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f072000f040 con 0x7f0728080680 2023-10-08T14:08:03.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.172+0000 7f071ffff700 1 -- 172.21.15.57:0/2429335840 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720004020 con 0x7f0728080680 2023-10-08T14:08:03.175 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.172+0000 7f072d28a700 1 -- 172.21.15.57:0/2429335840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07281bba70 con 0x7f0728080680 2023-10-08T14:08:03.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.173+0000 7f072d28a700 1 -- 172.21.15.57:0/2429335840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0728108a10 con 0x7f0728080680 2023-10-08T14:08:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.177+0000 7f071ffff700 1 -- 172.21.15.57:0/2429335840 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0720004180 con 0x7f0728080680 2023-10-08T14:08:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.177+0000 7f071ffff700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f07140a16f0 0x7f07140a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.177+0000 7f071ffff700 1 -- 172.21.15.57:0/2429335840 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9581+0+0 (secure 0 0 0) 0x7f07200cdec0 con 0x7f0728080680 2023-10-08T14:08:03.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.183+0000 7f0726ffd700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f07140a16f0 0x7f07140a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:03.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.184+0000 7f071ffff700 1 -- 172.21.15.57:0/2429335840 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0720096df0 con 0x7f0728080680 2023-10-08T14:08:03.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.185+0000 7f0726ffd700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f07140a16f0 0x7f07140a3bb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0710009fd0 tx=0x7f0710000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:03.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:03 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/441660671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T14:08:03.438 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:03 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/441660671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T14:08:03.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:03 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/441660671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T14:08:03.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.540+0000 7f072d28a700 1 -- 172.21.15.57:0/2429335840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f072810f490 con 0x7f0728080680 2023-10-08T14:08:03.553 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-08T14:08:03.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.547+0000 7f071ffff700 1 -- 172.21.15.57:0/2429335840 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f072009a410 con 0x7f0728080680 2023-10-08T14:08:03.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.565+0000 7f072d28a700 1 -- 172.21.15.57:0/2429335840 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f07140a16f0 msgr2=0x7f07140a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:03.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.565+0000 7f072d28a700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f07140a16f0 0x7f07140a3bb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0710009fd0 tx=0x7f0710000bc0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.565+0000 7f072d28a700 1 -- 172.21.15.57:0/2429335840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728080680 msgr2=0x7f0728082af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.565+0000 7f072d28a700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728080680 0x7f0728082af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f072000ef30 tx=0x7f072000c450 comp rx=0 tx=0).stop 2023-10-08T14:08:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.565+0000 7f072d28a700 1 -- 172.21.15.57:0/2429335840 shutdown_connections 2023-10-08T14:08:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.565+0000 7f072d28a700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f07140a16f0 0x7f07140a3bb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.565+0000 7f072d28a700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728080680 0x7f0728082af0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.565+0000 7f072d28a700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f072810c500 0x7f072810e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.565+0000 7f072d28a700 1 --2- 172.21.15.57:0/2429335840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f072810e590 0x7f072810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.565+0000 7f072d28a700 1 -- 172.21.15.57:0/2429335840 >> 172.21.15.57:0/2429335840 conn(0x7f072806bc80 msgr2=0x7f072806fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:03.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.568+0000 7f072d28a700 1 -- 172.21.15.57:0/2429335840 shutdown_connections 2023-10-08T14:08:03.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.569+0000 7f072d28a700 1 -- 172.21.15.57:0/2429335840 wait complete. 2023-10-08T14:08:03.590 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T14:08:03.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.869+0000 7fde2ba1a700 1 -- 172.21.15.57:0/1972238258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde2410ef80 msgr2=0x7fde241077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:03.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.869+0000 7fde2ba1a700 1 --2- 172.21.15.57:0/1972238258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde2410ef80 0x7fde241077f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fde1801a9a0 tx=0x7fde1801acb0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.874+0000 7fde2ba1a700 1 -- 172.21.15.57:0/1972238258 shutdown_connections 2023-10-08T14:08:03.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.874+0000 7fde2ba1a700 1 --2- 172.21.15.57:0/1972238258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24119fd0 0x7fde2411c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.874+0000 7fde2ba1a700 1 --2- 172.21.15.57:0/1972238258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde2410ef80 0x7fde241077f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.874+0000 7fde2ba1a700 1 --2- 172.21.15.57:0/1972238258 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde24107d30 0x7fde241081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.874+0000 7fde2ba1a700 1 -- 172.21.15.57:0/1972238258 >> 172.21.15.57:0/1972238258 conn(0x7fde2406b830 msgr2=0x7fde2406bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:03.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.882+0000 7fde2ba1a700 1 -- 172.21.15.57:0/1972238258 shutdown_connections 2023-10-08T14:08:03.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.889+0000 7fde2ba1a700 1 -- 172.21.15.57:0/1972238258 wait complete. 2023-10-08T14:08:03.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.890+0000 7fde2ba1a700 1 Processor -- start 2023-10-08T14:08:03.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.893+0000 7fde2ba1a700 1 -- start start 2023-10-08T14:08:03.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde2ba1a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24107d30 0x7fde24115800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:03.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde2ba1a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde2410ef80 0x7fde24115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:03.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde2ba1a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde24119fd0 0x7fde241a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:03.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde2ba1a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde241a23f0 con 0x7fde2410ef80 2023-10-08T14:08:03.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde2ba1a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde241a2560 con 0x7fde24119fd0 2023-10-08T14:08:03.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde2ba1a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde241a2720 con 0x7fde24107d30 2023-10-08T14:08:03.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde28fb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde2410ef80 0x7fde24115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:03.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde28fb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde2410ef80 0x7fde24115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60734/0 (socket says 172.21.15.57:60734) 2023-10-08T14:08:03.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde28fb5700 1 -- 172.21.15.57:0/3968182853 learned_addr learned my addr 172.21.15.57:0/3968182853 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:03.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde29fb7700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde24119fd0 0x7fde241a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:03.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde29fb7700 1 -- 172.21.15.57:0/3968182853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24107d30 msgr2=0x7fde24115800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T14:08:03.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde29fb7700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24107d30 0x7fde24115800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde29fb7700 1 -- 172.21.15.57:0/3968182853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde2410ef80 msgr2=0x7fde24115d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:03.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde29fb7700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde2410ef80 0x7fde24115d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:03.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde29fb7700 1 -- 172.21.15.57:0/3968182853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1801a600 con 0x7fde24119fd0 2023-10-08T14:08:03.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.894+0000 7fde28fb5700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde2410ef80 0x7fde24115d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T14:08:03.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.895+0000 7fde29fb7700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde24119fd0 0x7fde241a1c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00be00 tx=0x7fde1c009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:03.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.895+0000 7fde167fc700 1 -- 172.21.15.57:0/3968182853 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00cc10 con 0x7fde24119fd0 2023-10-08T14:08:03.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.895+0000 7fde167fc700 1 -- 172.21.15.57:0/3968182853 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde1c00cd70 con 0x7fde24119fd0 2023-10-08T14:08:03.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.895+0000 7fde167fc700 1 -- 172.21.15.57:0/3968182853 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c012720 con 0x7fde24119fd0 2023-10-08T14:08:03.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.899+0000 7fde2ba1a700 1 -- 172.21.15.57:0/3968182853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde241a2a00 con 0x7fde24119fd0 2023-10-08T14:08:03.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.899+0000 7fde2ba1a700 1 -- 172.21.15.57:0/3968182853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde241a9fe0 con 0x7fde24119fd0 2023-10-08T14:08:03.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.902+0000 7fde2ba1a700 1 -- 172.21.15.57:0/3968182853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2406ccb0 con 0x7fde24119fd0 2023-10-08T14:08:03.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.902+0000 7fde167fc700 1 -- 172.21.15.57:0/3968182853 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fde1c014a00 con 0x7fde24119fd0 2023-10-08T14:08:03.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.903+0000 7fde167fc700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde100a1630 0x7fde100a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:03.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.903+0000 7fde167fc700 1 -- 172.21.15.57:0/3968182853 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9581+0+0 (secure 0 0 0) 0x7fde1c0cc8f0 con 0x7fde24119fd0 2023-10-08T14:08:03.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.908+0000 7fde297b6700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde100a1630 0x7fde100a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:03.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.908+0000 7fde297b6700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde100a1630 0x7fde100a3af0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fde18005e50 tx=0x7fde1802a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:03.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:03.915+0000 7fde167fc700 1 -- 172.21.15.57:0/3968182853 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fde1c04e100 con 0x7fde24119fd0 2023-10-08T14:08:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.274+0000 7fde2ba1a700 1 -- 172.21.15.57:0/3968182853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde24116a00 con 0x7fde24119fd0 2023-10-08T14:08:04.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.275+0000 7fde167fc700 1 -- 172.21.15.57:0/3968182853 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde1c095900 con 0x7fde24119fd0 2023-10-08T14:08:04.276 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T14:08:04.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.279+0000 7fde2ba1a700 1 -- 172.21.15.57:0/3968182853 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde100a1630 msgr2=0x7fde100a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:04.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.279+0000 7fde2ba1a700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde100a1630 0x7fde100a3af0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fde18005e50 tx=0x7fde1802a040 comp rx=0 tx=0).stop 2023-10-08T14:08:04.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.279+0000 7fde2ba1a700 1 -- 172.21.15.57:0/3968182853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde24119fd0 msgr2=0x7fde241a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:04.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.279+0000 7fde2ba1a700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde24119fd0 0x7fde241a1c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00be00 tx=0x7fde1c009ed0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.282+0000 7fde2ba1a700 1 -- 172.21.15.57:0/3968182853 shutdown_connections 2023-10-08T14:08:04.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.282+0000 7fde2ba1a700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fde100a1630 0x7fde100a3af0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.282+0000 7fde2ba1a700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde24119fd0 0x7fde241a1c20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.282+0000 7fde2ba1a700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24107d30 0x7fde24115800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.282+0000 7fde2ba1a700 1 --2- 172.21.15.57:0/3968182853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fde2410ef80 0x7fde24115d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.282+0000 7fde2ba1a700 1 -- 172.21.15.57:0/3968182853 >> 172.21.15.57:0/3968182853 conn(0x7fde2406b830 msgr2=0x7fde2411a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:04.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.286+0000 7fde2ba1a700 1 -- 172.21.15.57:0/3968182853 shutdown_connections 2023-10-08T14:08:04.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.286+0000 7fde2ba1a700 1 -- 172.21.15.57:0/3968182853 wait complete. 2023-10-08T14:08:04.308 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T14:08:04.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:04 smithi057 ceph-mon[99476]: pgmap v815: 129 pgs: 46 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 KiB/s wr, 24 op/s 2023-10-08T14:08:04.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:04 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2429335840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T14:08:04.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:04 smithi115 ceph-mon[104408]: pgmap v815: 129 pgs: 46 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 KiB/s wr, 24 op/s 2023-10-08T14:08:04.442 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:04 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2429335840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T14:08:04.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:04 smithi099 ceph-mon[103849]: pgmap v815: 129 pgs: 46 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 KiB/s wr, 24 op/s 2023-10-08T14:08:04.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:04 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2429335840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T14:08:04.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.547+0000 7fe61de8a700 1 -- 172.21.15.57:0/2845772419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618105b30 msgr2=0x7fe61810d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:04.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.547+0000 7fe61de8a700 1 --2- 172.21.15.57:0/2845772419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618105b30 0x7fe61810d6d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe610014a30 tx=0x7fe610014d40 comp rx=0 tx=0).stop 2023-10-08T14:08:04.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.547+0000 7fe61de8a700 1 -- 172.21.15.57:0/2845772419 shutdown_connections 2023-10-08T14:08:04.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.547+0000 7fe61de8a700 1 --2- 172.21.15.57:0/2845772419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618105b30 0x7fe61810d6d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.547+0000 7fe61de8a700 1 --2- 172.21.15.57:0/2845772419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6180ff320 0x7fe6180ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.547+0000 7fe61de8a700 1 --2- 172.21.15.57:0/2845772419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6180ffcd0 0x7fe618105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.548 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.547+0000 7fe61de8a700 1 -- 172.21.15.57:0/2845772419 >> 172.21.15.57:0/2845772419 conn(0x7fe618074940 msgr2=0x7fe618074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:04.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.548+0000 7fe61de8a700 1 -- 172.21.15.57:0/2845772419 shutdown_connections 2023-10-08T14:08:04.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.548+0000 7fe61de8a700 1 -- 172.21.15.57:0/2845772419 wait complete. 2023-10-08T14:08:04.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.549+0000 7fe61de8a700 1 Processor -- start 2023-10-08T14:08:04.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.549+0000 7fe61de8a700 1 -- start start 2023-10-08T14:08:04.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.549+0000 7fe61de8a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6180ff320 0x7fe618194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:04.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.549+0000 7fe61de8a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618105b30 0x7fe618194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:04.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.549+0000 7fe61de8a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe618199340 0x7fe6181997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:04.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.549+0000 7fe61de8a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618199d30 con 0x7fe6180ff320 2023-10-08T14:08:04.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.549+0000 7fe61de8a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618199e70 con 0x7fe618105b30 2023-10-08T14:08:04.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.550+0000 7fe61de8a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61819a030 con 0x7fe618199340 2023-10-08T14:08:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.550+0000 7fe617fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe618199340 0x7fe6181997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.550+0000 7fe617fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe618199340 0x7fe6181997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33328/0 (socket says 172.21.15.57:33328) 2023-10-08T14:08:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.550+0000 7fe617fff700 1 -- 172.21.15.57:0/2091132801 learned_addr learned my addr 172.21.15.57:0/2091132801 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.550+0000 7fe616ffd700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618105b30 0x7fe618194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.550+0000 7fe617fff700 1 -- 172.21.15.57:0/2091132801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618105b30 msgr2=0x7fe618194f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.550+0000 7fe617fff700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618105b30 0x7fe618194f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.550+0000 7fe617fff700 1 -- 172.21.15.57:0/2091132801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6180ff320 msgr2=0x7fe618194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.550+0000 7fe617fff700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6180ff320 0x7fe618194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.550+0000 7fe617fff700 1 -- 172.21.15.57:0/2091132801 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610014660 con 0x7fe618199340 2023-10-08T14:08:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.551+0000 7fe617fff700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe618199340 0x7fe6181997f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6100121f0 tx=0x7fe61001cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.552+0000 7fe614ff9700 1 -- 172.21.15.57:0/2091132801 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610013af0 con 0x7fe618199340 2023-10-08T14:08:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.552+0000 7fe61de8a700 1 -- 172.21.15.57:0/2091132801 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61806a210 con 0x7fe618199340 2023-10-08T14:08:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.552+0000 7fe61de8a700 1 -- 172.21.15.57:0/2091132801 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61806a770 con 0x7fe618199340 2023-10-08T14:08:04.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.552+0000 7fe614ff9700 1 -- 172.21.15.57:0/2091132801 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe610013c50 con 0x7fe618199340 2023-10-08T14:08:04.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.552+0000 7fe614ff9700 1 -- 172.21.15.57:0/2091132801 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610008910 con 0x7fe618199340 2023-10-08T14:08:04.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.555+0000 7fe614ff9700 1 -- 172.21.15.57:0/2091132801 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe61000f260 con 0x7fe618199340 2023-10-08T14:08:04.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.556+0000 7fe614ff9700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe6040b2680 0x7fe6040b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:04.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.556+0000 7fe614ff9700 1 -- 172.21.15.57:0/2091132801 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9581+0+0 (secure 0 0 0) 0x7fe6100cdf00 con 0x7fe618199340 2023-10-08T14:08:04.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.556+0000 7fe6177fe700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe6040b2680 0x7fe6040b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:04.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.557+0000 7fe6177fe700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe6040b2680 0x7fe6040b4b40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe608005950 tx=0x7fe6080058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:04.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.560+0000 7fe61de8a700 1 -- 172.21.15.57:0/2091132801 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f80052f0 con 0x7fe618199340 2023-10-08T14:08:04.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.572+0000 7fe614ff9700 1 -- 172.21.15.57:0/2091132801 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe610096e90 con 0x7fe618199340 2023-10-08T14:08:04.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.920+0000 7fe61de8a700 1 -- 172.21.15.57:0/2091132801 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5f8005b20 con 0x7fe618199340 2023-10-08T14:08:04.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.920+0000 7fe614ff9700 1 -- 172.21.15.57:0/2091132801 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe61002a020 con 0x7fe618199340 2023-10-08T14:08:04.922 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-08T14:08:04.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.927+0000 7fe61de8a700 1 -- 172.21.15.57:0/2091132801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe6040b2680 msgr2=0x7fe6040b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:04.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.927+0000 7fe61de8a700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe6040b2680 0x7fe6040b4b40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe608005950 tx=0x7fe6080058e0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.927+0000 7fe61de8a700 1 -- 172.21.15.57:0/2091132801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe618199340 msgr2=0x7fe6181997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:04.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.927+0000 7fe61de8a700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe618199340 0x7fe6181997f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6100121f0 tx=0x7fe61001cc50 comp rx=0 tx=0).stop 2023-10-08T14:08:04.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.927+0000 7fe61de8a700 1 -- 172.21.15.57:0/2091132801 shutdown_connections 2023-10-08T14:08:04.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.927+0000 7fe61de8a700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe618199340 0x7fe6181997f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.927+0000 7fe61de8a700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6180ff320 0x7fe618194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.927+0000 7fe61de8a700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe6040b2680 0x7fe6040b4b40 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe608005950 tx=0x7fe6080058e0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.927+0000 7fe61de8a700 1 --2- 172.21.15.57:0/2091132801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe618105b30 0x7fe618194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:04.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.927+0000 7fe61de8a700 1 -- 172.21.15.57:0/2091132801 >> 172.21.15.57:0/2091132801 conn(0x7fe618074940 msgr2=0x7fe6180fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:04.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.929+0000 7fe61de8a700 1 -- 172.21.15.57:0/2091132801 shutdown_connections 2023-10-08T14:08:04.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:04.931+0000 7fe61de8a700 1 -- 172.21.15.57:0/2091132801 wait complete. 2023-10-08T14:08:04.958 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T14:08:05.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:05 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3968182853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T14:08:05.240 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:05 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2091132801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T14:08:05.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.256+0000 7f13b46f7700 1 -- 172.21.15.57:0/1945010543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13ac068fb0 msgr2=0x7f13ac069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:05.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.256+0000 7f13b46f7700 1 --2- 172.21.15.57:0/1945010543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13ac068fb0 0x7f13ac069430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f139c0098f0 tx=0x7f139c009c00 comp rx=0 tx=0).stop 2023-10-08T14:08:05.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.258+0000 7f13b46f7700 1 -- 172.21.15.57:0/1945010543 shutdown_connections 2023-10-08T14:08:05.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.258+0000 7f13b46f7700 1 --2- 172.21.15.57:0/1945010543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13ac103190 0x7f13ac1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.258+0000 7f13b46f7700 1 --2- 172.21.15.57:0/1945010543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13ac105690 0x7f13ac068a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.258+0000 7f13b46f7700 1 --2- 172.21.15.57:0/1945010543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13ac068fb0 0x7f13ac069430 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.258+0000 7f13b46f7700 1 -- 172.21.15.57:0/1945010543 >> 172.21.15.57:0/1945010543 conn(0x7f13ac0767c0 msgr2=0x7f13ac076bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:05.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.259+0000 7f13b46f7700 1 -- 172.21.15.57:0/1945010543 shutdown_connections 2023-10-08T14:08:05.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.266+0000 7f13b46f7700 1 -- 172.21.15.57:0/1945010543 wait complete. 2023-10-08T14:08:05.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.266+0000 7f13b46f7700 1 Processor -- start 2023-10-08T14:08:05.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.267+0000 7f13b46f7700 1 -- start start 2023-10-08T14:08:05.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.267+0000 7f13b46f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13ac068fb0 0x7f13ac19d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:05.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.267+0000 7f13b46f7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13ac103190 0x7f13ac19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:05.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.267+0000 7f13b46f7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13ac105690 0x7f13ac1a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:05.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.267+0000 7f13b46f7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac1a21a0 con 0x7f13ac068fb0 2023-10-08T14:08:05.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.267+0000 7f13b46f7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac1a2310 con 0x7f13ac103190 2023-10-08T14:08:05.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.267+0000 7f13b46f7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac1a24d0 con 0x7f13ac105690 2023-10-08T14:08:05.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.267+0000 7f13b1c92700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13ac103190 0x7f13ac19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:05.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.267+0000 7f13b1c92700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13ac103190 0x7f13ac19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33954/0 (socket says 172.21.15.57:33954) 2023-10-08T14:08:05.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.267+0000 7f13b1c92700 1 -- 172.21.15.57:0/3135751997 learned_addr learned my addr 172.21.15.57:0/3135751997 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:05.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.268+0000 7f13b2493700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13ac068fb0 0x7f13ac19d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:05.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.268+0000 7f13b2493700 1 -- 172.21.15.57:0/3135751997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13ac105690 msgr2=0x7f13ac1a1b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T14:08:05.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.268+0000 7f13b2493700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13ac105690 0x7f13ac1a1b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.268+0000 7f13b2493700 1 -- 172.21.15.57:0/3135751997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13ac103190 msgr2=0x7f13ac19d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:05.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.268+0000 7f13b2493700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13ac103190 0x7f13ac19d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.268+0000 7f13b2493700 1 -- 172.21.15.57:0/3135751997 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c009520 con 0x7f13ac068fb0 2023-10-08T14:08:05.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.268+0000 7f13b2493700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13ac068fb0 0x7f13ac19d280 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f13a400ef10 tx=0x7f13a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:05.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.273+0000 7f13a37fe700 1 -- 172.21.15.57:0/3135751997 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400ca70 con 0x7f13ac068fb0 2023-10-08T14:08:05.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.273+0000 7f13a37fe700 1 -- 172.21.15.57:0/3135751997 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13a400f040 con 0x7f13ac068fb0 2023-10-08T14:08:05.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.273+0000 7f13a37fe700 1 -- 172.21.15.57:0/3135751997 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40146e0 con 0x7f13ac068fb0 2023-10-08T14:08:05.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.274+0000 7f13b46f7700 1 -- 172.21.15.57:0/3135751997 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac1a2750 con 0x7f13ac068fb0 2023-10-08T14:08:05.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.274+0000 7f13b46f7700 1 -- 172.21.15.57:0/3135751997 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac105de0 con 0x7f13ac068fb0 2023-10-08T14:08:05.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.277+0000 7f13b46f7700 1 -- 172.21.15.57:0/3135751997 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac0ffc70 con 0x7f13ac068fb0 2023-10-08T14:08:05.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.285+0000 7f13a37fe700 1 -- 172.21.15.57:0/3135751997 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f13a4004ad0 con 0x7f13ac068fb0 2023-10-08T14:08:05.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.286+0000 7f13a37fe700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f13980a1800 0x7f13980a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:05.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.287+0000 7f13a37fe700 1 -- 172.21.15.57:0/3135751997 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9581+0+0 (secure 0 0 0) 0x7f13a40cd120 con 0x7f13ac068fb0 2023-10-08T14:08:05.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.287+0000 7f13b1c92700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f13980a1800 0x7f13980a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:05.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.287+0000 7f13a37fe700 1 -- 172.21.15.57:0/3135751997 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f13a4099ad0 con 0x7f13ac068fb0 2023-10-08T14:08:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.293+0000 7f13b1c92700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f13980a1800 0x7f13980a3cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f13ac19e7d0 tx=0x7f139c019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:05.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:05 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3968182853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T14:08:05.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:05 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2091132801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T14:08:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:05 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3968182853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T14:08:05.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:05 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2091132801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T14:08:05.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.629+0000 7f13b46f7700 1 -- 172.21.15.57:0/3135751997 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f13ac1a28e0 con 0x7f13ac068fb0 2023-10-08T14:08:05.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.634+0000 7f13a37fe700 1 -- 172.21.15.57:0/3135751997 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v113) v1 ==== 103+0+11 (secure 0 0 0) 0x7f13a4099660 con 0x7f13ac068fb0 2023-10-08T14:08:05.639 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-08T14:08:05.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 -- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f13980a1800 msgr2=0x7f13980a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:05.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f13980a1800 0x7f13980a3cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f13ac19e7d0 tx=0x7f139c019040 comp rx=0 tx=0).stop 2023-10-08T14:08:05.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 -- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13ac068fb0 msgr2=0x7f13ac19d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:05.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13ac068fb0 0x7f13ac19d280 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f13a400ef10 tx=0x7f13a400c3a0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 -- 172.21.15.57:0/3135751997 shutdown_connections 2023-10-08T14:08:05.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13ac105690 0x7f13ac1a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13ac068fb0 0x7f13ac19d280 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f13980a1800 0x7f13980a3cc0 secure :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f13ac19e7d0 tx=0x7f139c019040 comp rx=0 tx=0).stop 2023-10-08T14:08:05.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 --2- 172.21.15.57:0/3135751997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13ac103190 0x7f13ac19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 -- 172.21.15.57:0/3135751997 >> 172.21.15.57:0/3135751997 conn(0x7f13ac0767c0 msgr2=0x7f13ac0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:05.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 -- 172.21.15.57:0/3135751997 shutdown_connections 2023-10-08T14:08:05.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.640+0000 7f13b46f7700 1 -- 172.21.15.57:0/3135751997 wait complete. 2023-10-08T14:08:05.683 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T14:08:05.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.962+0000 7f13a7a5a700 1 -- 172.21.15.57:0/3671432929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13a010ef80 msgr2=0x7f13a01077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:05.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.962+0000 7f13a7a5a700 1 --2- 172.21.15.57:0/3671432929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13a010ef80 0x7f13a01077f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f139c009990 tx=0x7f139c009ca0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.963+0000 7f13a7a5a700 1 -- 172.21.15.57:0/3671432929 shutdown_connections 2023-10-08T14:08:05.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.963+0000 7f13a7a5a700 1 --2- 172.21.15.57:0/3671432929 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13a0119fd0 0x7f13a011c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.963+0000 7f13a7a5a700 1 --2- 172.21.15.57:0/3671432929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13a010ef80 0x7f13a01077f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.963+0000 7f13a7a5a700 1 --2- 172.21.15.57:0/3671432929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a0107d30 0x7f13a01081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.963+0000 7f13a7a5a700 1 -- 172.21.15.57:0/3671432929 >> 172.21.15.57:0/3671432929 conn(0x7f13a006b830 msgr2=0x7f13a006bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.963+0000 7f13a7a5a700 1 -- 172.21.15.57:0/3671432929 shutdown_connections 2023-10-08T14:08:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.964+0000 7f13a7a5a700 1 -- 172.21.15.57:0/3671432929 wait complete. 2023-10-08T14:08:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.964+0000 7f13a7a5a700 1 Processor -- start 2023-10-08T14:08:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.964+0000 7f13a7a5a700 1 -- start start 2023-10-08T14:08:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a7a5a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13a010ef80 0x7f13a01ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a7a5a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13a0119fd0 0x7f13a01abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a7a5a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a01a5a50 0x7f13a01a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a7a5a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a01a64d0 con 0x7f13a0119fd0 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a7a5a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a01a6640 con 0x7f13a010ef80 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a7a5a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a01a6800 con 0x7f13a01a5a50 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a57f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13a010ef80 0x7f13a01ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a57f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13a010ef80 0x7f13a01ab920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:33968/0 (socket says 172.21.15.57:33968) 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a57f6700 1 -- 172.21.15.57:0/1553251111 learned_addr learned my addr 172.21.15.57:0/1553251111 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a4ff5700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13a0119fd0 0x7f13a01abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a5ff7700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a01a5a50 0x7f13a01a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a4ff5700 1 -- 172.21.15.57:0/1553251111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a01a5a50 msgr2=0x7f13a01a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a4ff5700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a01a5a50 0x7f13a01a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a4ff5700 1 -- 172.21.15.57:0/1553251111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13a010ef80 msgr2=0x7f13a01ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a4ff5700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13a010ef80 0x7f13a01ab920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:05.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.965+0000 7f13a4ff5700 1 -- 172.21.15.57:0/1553251111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c0095f0 con 0x7f13a0119fd0 2023-10-08T14:08:05.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.966+0000 7f13a4ff5700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13a0119fd0 0x7f13a01abe60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f13a006c6e0 tx=0x7f139000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:05.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.966+0000 7f13967fc700 1 -- 172.21.15.57:0/1553251111 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390011620 con 0x7f13a0119fd0 2023-10-08T14:08:05.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.966+0000 7f13a7a5a700 1 -- 172.21.15.57:0/1553251111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a0078b10 con 0x7f13a0119fd0 2023-10-08T14:08:05.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.966+0000 7f13967fc700 1 -- 172.21.15.57:0/1553251111 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1390011c40 con 0x7f13a0119fd0 2023-10-08T14:08:05.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.966+0000 7f13a7a5a700 1 -- 172.21.15.57:0/1553251111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a0078fd0 con 0x7f13a0119fd0 2023-10-08T14:08:05.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.966+0000 7f13967fc700 1 -- 172.21.15.57:0/1553251111 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390010c70 con 0x7f13a0119fd0 2023-10-08T14:08:05.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.967+0000 7f13a7a5a700 1 -- 172.21.15.57:0/1553251111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13840052f0 con 0x7f13a0119fd0 2023-10-08T14:08:05.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.970+0000 7f13967fc700 1 -- 172.21.15.57:0/1553251111 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1390010380 con 0x7f13a0119fd0 2023-10-08T14:08:05.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.971+0000 7f13967fc700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f138c0e5600 0x7f138c0e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:05.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.971+0000 7f13967fc700 1 -- 172.21.15.57:0/1553251111 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9581+0+0 (secure 0 0 0) 0x7f13900cd230 con 0x7f13a0119fd0 2023-10-08T14:08:05.981 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.974+0000 7f13a57f6700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f138c0e5600 0x7f138c0e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:05.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.981+0000 7f13a57f6700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f138c0e5600 0x7f138c0e7ac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f139c006010 tx=0x7f139c00b560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:05.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:05.983+0000 7f13967fc700 1 -- 172.21.15.57:0/1553251111 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1390096240 con 0x7f13a0119fd0 2023-10-08T14:08:06.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.374+0000 7f13a7a5a700 1 -- 172.21.15.57:0/1553251111 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1384005870 con 0x7f13a0119fd0 2023-10-08T14:08:06.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.380+0000 7f13967fc700 1 -- 172.21.15.57:0/1553251111 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v113) v1 ==== 81+0+882 (secure 0 0 0) 0x7f1390015080 con 0x7f13a0119fd0 2023-10-08T14:08:06.386 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/466175178 2023-10-09T13:44:14.025065+0000 2023-10-08T14:08:06.386 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1455537828 2023-10-09T13:44:14.025065+0000 2023-10-08T14:08:06.386 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/2534706720 2023-10-09T13:44:14.025065+0000 2023-10-08T14:08:06.386 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/2534706720 2023-10-09T13:44:14.025065+0000 2023-10-08T14:08:06.386 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1379042066 2023-10-09T13:44:14.025065+0000 2023-10-08T14:08:06.386 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/19108071 2023-10-09T13:43:06.093214+0000 2023-10-08T14:08:06.386 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3114287016 2023-10-09T13:43:06.093214+0000 2023-10-08T14:08:06.387 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3318653521 2023-10-09T13:43:06.093214+0000 2023-10-08T14:08:06.387 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2210095932 2023-10-09T13:42:39.884205+0000 2023-10-08T14:08:06.387 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/955690770 2023-10-09T13:42:39.884205+0000 2023-10-08T14:08:06.387 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/140726743 2023-10-09T13:42:39.884205+0000 2023-10-08T14:08:06.387 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/2406655290 2023-10-09T13:43:06.093214+0000 2023-10-08T14:08:06.387 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2229924806 2023-10-09T13:42:39.884205+0000 2023-10-08T14:08:06.387 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/140726743 2023-10-09T13:42:39.884205+0000 2023-10-08T14:08:06.387 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/2406655290 2023-10-09T13:43:06.093214+0000 2023-10-08T14:08:06.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.391+0000 7f13a7a5a700 1 -- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f138c0e5600 msgr2=0x7f138c0e7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:06.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.391+0000 7f13a7a5a700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f138c0e5600 0x7f138c0e7ac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f139c006010 tx=0x7f139c00b560 comp rx=0 tx=0).stop 2023-10-08T14:08:06.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.391+0000 7f13a7a5a700 1 -- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13a0119fd0 msgr2=0x7f13a01abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:06.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.391+0000 7f13a7a5a700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13a0119fd0 0x7f13a01abe60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f13a006c6e0 tx=0x7f139000b700 comp rx=0 tx=0).stop 2023-10-08T14:08:06.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.398+0000 7f13a7a5a700 1 -- 172.21.15.57:0/1553251111 shutdown_connections 2023-10-08T14:08:06.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.398+0000 7f13a7a5a700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a01a5a50 0x7f13a01a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:06.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.398+0000 7f13a7a5a700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f138c0e5600 0x7f138c0e7ac0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:06.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.398+0000 7f13a7a5a700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13a010ef80 0x7f13a01ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:06.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.398+0000 7f13a7a5a700 1 --2- 172.21.15.57:0/1553251111 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13a0119fd0 0x7f13a01abe60 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:06.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.398+0000 7f13a7a5a700 1 -- 172.21.15.57:0/1553251111 >> 172.21.15.57:0/1553251111 conn(0x7f13a006b830 msgr2=0x7f13a0117fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:06.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.405+0000 7f13a7a5a700 1 -- 172.21.15.57:0/1553251111 shutdown_connections 2023-10-08T14:08:06.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:06.406+0000 7f13a7a5a700 1 -- 172.21.15.57:0/1553251111 wait complete. 2023-10-08T14:08:06.409 INFO:teuthology.orchestra.run.smithi057.stderr:listed 15 entries 2023-10-08T14:08:06.428 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T14:08:06.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T14:08:06.429 DEBUG:teuthology.orchestra.run.smithi057:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T14:08:06.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:06 smithi057 ceph-mon[99476]: pgmap v816: 129 pgs: 46 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 KiB/s wr, 22 op/s 2023-10-08T14:08:06.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:06 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3135751997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T14:08:06.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:06 smithi115 ceph-mon[104408]: pgmap v816: 129 pgs: 46 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 KiB/s wr, 22 op/s 2023-10-08T14:08:06.437 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:06 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3135751997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T14:08:06.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:06 smithi099 ceph-mon[103849]: pgmap v816: 129 pgs: 46 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 68 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 KiB/s wr, 22 op/s 2023-10-08T14:08:06.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:06 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3135751997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T14:08:06.552 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T14:08:06.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T14:08:06.552 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T14:08:06.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T14:08:06.644 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-08T14:08:06.722 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T14:08:06.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T14:08:06.722 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T14:08:06.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T14:08:06.818 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2023-10-08T14:08:06.980 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-08T14:08:06.980 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-08T14:08:06.980 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T14:08:07.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18873 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/8d21e02541d23be47e93e5b6edd1a62b90df520b8640b4a5970704c3c5f93d25/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J5YLDYHY6SGWXK325GU3YYITWT:/var/lib/containers/storage/overlay/l/7645AR265XDBGZOSKODMDJOQYH,upperdir=/var/lib/containers/storage/overlay/8d21e02541d23be47e93e5b6edd1a62b90df520b8640b4a5970704c3c5f93d25/diff,workdir=/var/lib/containers/storage/overlay/8d21e02541d23be47e93e5b6edd1a62b90df520b8640b4a5970704c3c5f93d25/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e81c9eb28775e4b2a9c2900ade28fb0fe3edc3be6a611d9cee0350c8c0a34332/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J5YLDYHY6SGWXK325GU3YYITWT:/var/lib/containers/storage/overlay/l/7645AR265XDBGZOSKODMDJOQYH,upperdir=/var/lib/containers/storage/overlay/e81c9eb28775e4b2a9c2900ade28fb0fe3edc3be6a611d9cee0350c8c0a34332/diff,workdir=/var/lib/containers/storage/overlay/e81c9eb28775e4b2a9c2900ade28fb0fe3edc3be6a611d9cee0350c8c0a34332/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/f5c61220aad320541da1e44047ca4be361f0c87acd7dc83c83c59ebe40bd1293/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J5YLDYHY6SGWXK325GU3YYITWT:/var/lib/containers/storage/overlay/l/7645AR265XDBGZOSKODMDJOQYH,upperdir=/var/lib/containers/storage/overlay/f5c61220aad320541da1e44047ca4be361f0c87acd7dc83c83c59ebe40bd1293/diff,workdir=/var/lib/containers/storage/overlay/f5c61220aad320541da1e44047ca4be361f0c87acd7dc83c83c59ebe40bd1293/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a3c341dc7ea0d7e0126249450d39311f778d372a13671f341994ade570b010c4/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J5YLDYHY6SGWXK325GU3YYITWT:/var/lib/containers/storage/overlay/l/7645AR265XDBGZOSKODMDJOQYH,upperdir=/var/lib/containers/storage/overlay/a3c341dc7ea0d7e0126249450d39311f778d372a13671f341994ade570b010c4/diff,workdir=/var/lib/containers/storage/overlay/a3c341dc7ea0d7e0126249450d39311f778d372a13671f341994ade570b010c4/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0443ae2a350ba0f0e3a7062c7d6c43087d9714ad1d023d2edd4f9fb9fedd7e8a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J5YLDYHY6SGWXK325GU3YYITWT:/var/lib/containers/storage/overlay/l/7645AR265XDBGZOSKODMDJOQYH,upperdir=/var/lib/containers/storage/overlay/0443ae2a350ba0f0e3a7062c7d6c43087d9714ad1d023d2edd4f9fb9fedd7e8a/diff,workdir=/var/lib/containers/storage/overlay/0443ae2a350ba0f0e3a7062c7d6c43087d9714ad1d023d2edd4f9fb9fedd7e8a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a199bb960f20d37a9fafc283dd284267a31fc1f8929a7e7c3ce14217de86fc46/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J5YLDYHY6SGWXK325GU3YYITWT:/var/lib/containers/storage/overlay/l/7645AR265XDBGZOSKODMDJOQYH,upperdir=/var/lib/containers/storage/overlay/a199bb960f20d37a9fafc283dd284267a31fc1f8929a7e7c3ce14217de86fc46/diff,workdir=/var/lib/containers/storage/overlay/a199bb960f20d37a9fafc283dd284267a31fc1f8929a7e7c3ce14217de86fc46/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/25632f389c7f43016ca942979a2378e3ba779cc4075d3cd392d99ff30376649e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J5YLDYHY6SGWXK325GU3YYITWT:/var/lib/containers/storage/overlay/l/7645AR265XDBGZOSKODMDJOQYH,upperdir=/var/lib/containers/storage/overlay/25632f389c7f43016ca942979a2378e3ba779cc4075d3cd392d99ff30376649e/diff,workdir=/var/lib/containers/storage/overlay/25632f389c7f43016ca942979a2378e3ba779cc4075d3cd392d99ff30376649e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/23416b84fdeb905a93b7089631a0e61bbd88847cd99d2ba4d3c04a898f4263e6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J5YLDYHY6SGWXK325GU3YYITWT:/var/lib/containers/storage/overlay/l/7645AR265XDBGZOSKODMDJOQYH,upperdir=/var/lib/containers/storage/overlay/23416b84fdeb905a93b7089631a0e61bbd88847cd99d2ba4d3c04a898f4263e6/diff,workdir=/var/lib/containers/storage/overlay/23416b84fdeb905a93b7089631a0e61bbd88847cd99d2ba4d3c04a898f4263e6/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2e01f65ad907ed13b484411791536a52cd121d70f5010364cab9cac508503e15/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J5YLDYHY6SGWXK325GU3YYITWT:/var/lib/containers/storage/overlay/l/7645AR265XDBGZOSKODMDJOQYH,upperdir=/var/lib/containers/storage/overlay/2e01f65ad907ed13b484411791536a52cd121d70f5010364cab9cac508503e15/diff,workdir=/var/lib/containers/storage/overlay/2e01f65ad907ed13b484411791536a52cd121d70f5010364cab9cac508503e15/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/61af0144ab97eb053693f4295bd7c98e9dd86bc5ded21451becf6734b0371b58/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J5YLDYHY6SGWXK325GU3YYITWT:/var/lib/containers/storage/overlay/l/7645AR265XDBGZOSKODMDJOQYH,upperdir=/var/lib/containers/storage/overlay/61af0144ab97eb053693f4295bd7c98e9dd86bc5ded21451becf6734b0371b58/diff,workdir=/var/lib/containers/storage/overlay/61af0144ab97eb053693f4295bd7c98e9dd86bc5ded21451becf6734b0371b58/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-10-08T14:08:07.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T14:08:07.017 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-08T14:08:07.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T14:08:07.121 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2023-10-08T14:08:07.296 INFO:teuthology.orchestra.run.smithi057.stderr:Cannot find device "ceph-brx" 2023-10-08T14:08:07.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T14:08:07.301 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-08T14:08:07.312 INFO:tasks.cephadm:Teardown begin 2023-10-08T14:08:07.313 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T14:08:07.313 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T14:08:07.418 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:07 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1553251111' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T14:08:07.440 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:07 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1553251111' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T14:08:07.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:07 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1553251111' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T14:08:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.599+0000 7f7b6c48b700 1 -- 172.21.15.57:0/3089348378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b64101010 msgr2=0x7f7b6410d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:07.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.599+0000 7f7b6c48b700 1 --2- 172.21.15.57:0/3089348378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b64101010 0x7f7b6410d520 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f7b580098f0 tx=0x7f7b58009c00 comp rx=0 tx=0).stop 2023-10-08T14:08:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.601+0000 7f7b6c48b700 1 -- 172.21.15.57:0/3089348378 shutdown_connections 2023-10-08T14:08:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.601+0000 7f7b6c48b700 1 --2- 172.21.15.57:0/3089348378 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b64101010 0x7f7b6410d520 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.601+0000 7f7b6c48b700 1 --2- 172.21.15.57:0/3089348378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b64106660 0x7f7b64106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.601+0000 7f7b6c48b700 1 --2- 172.21.15.57:0/3089348378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b64100650 0x7f7b64100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.601+0000 7f7b6c48b700 1 -- 172.21.15.57:0/3089348378 >> 172.21.15.57:0/3089348378 conn(0x7f7b640fc180 msgr2=0x7f7b640fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.601+0000 7f7b6c48b700 1 -- 172.21.15.57:0/3089348378 shutdown_connections 2023-10-08T14:08:07.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.601+0000 7f7b6c48b700 1 -- 172.21.15.57:0/3089348378 wait complete. 2023-10-08T14:08:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.602+0000 7f7b6c48b700 1 Processor -- start 2023-10-08T14:08:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.602+0000 7f7b6c48b700 1 -- start start 2023-10-08T14:08:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.602+0000 7f7b6c48b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b64100650 0x7f7b6419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.602+0000 7f7b6c48b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b64101010 0x7f7b6419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.602+0000 7f7b6c48b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b64106660 0x7f7b64196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.602+0000 7f7b6c48b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64197160 con 0x7f7b64101010 2023-10-08T14:08:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.602+0000 7f7b6c48b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641972a0 con 0x7f7b64100650 2023-10-08T14:08:07.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.602+0000 7f7b6c48b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64197460 con 0x7f7b64106660 2023-10-08T14:08:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.603+0000 7f7b69a26700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b64101010 0x7f7b6419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.603+0000 7f7b69a26700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b64101010 0x7f7b6419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60786/0 (socket says 172.21.15.57:60786) 2023-10-08T14:08:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.603+0000 7f7b69a26700 1 -- 172.21.15.57:0/4242010518 learned_addr learned my addr 172.21.15.57:0/4242010518 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.603+0000 7f7b69a26700 1 -- 172.21.15.57:0/4242010518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b64106660 msgr2=0x7f7b64196990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T14:08:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.603+0000 7f7b69a26700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b64106660 0x7f7b64196990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.603+0000 7f7b69a26700 1 -- 172.21.15.57:0/4242010518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b64100650 msgr2=0x7f7b6419c8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T14:08:07.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.603+0000 7f7b69a26700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b64100650 0x7f7b6419c8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.603+0000 7f7b69a26700 1 -- 172.21.15.57:0/4242010518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b58009520 con 0x7f7b64101010 2023-10-08T14:08:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.603+0000 7f7b69a26700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b64101010 0x7f7b6419cde0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400c0c0 tx=0x7f7b5400c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.604+0000 7f7b637fe700 1 -- 172.21.15.57:0/4242010518 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5400e020 con 0x7f7b64101010 2023-10-08T14:08:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.604+0000 7f7b637fe700 1 -- 172.21.15.57:0/4242010518 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b5400f040 con 0x7f7b64101010 2023-10-08T14:08:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.604+0000 7f7b6c48b700 1 -- 172.21.15.57:0/4242010518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b641976e0 con 0x7f7b64101010 2023-10-08T14:08:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.604+0000 7f7b637fe700 1 -- 172.21.15.57:0/4242010518 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54013820 con 0x7f7b64101010 2023-10-08T14:08:07.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.604+0000 7f7b6c48b700 1 -- 172.21.15.57:0/4242010518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6406a440 con 0x7f7b64101010 2023-10-08T14:08:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.608+0000 7f7b6c48b700 1 -- 172.21.15.57:0/4242010518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b500052f0 con 0x7f7b64101010 2023-10-08T14:08:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.608+0000 7f7b637fe700 1 -- 172.21.15.57:0/4242010518 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7b540075c0 con 0x7f7b64101010 2023-10-08T14:08:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.609+0000 7f7b637fe700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b4c0a1740 0x7f7b4c0a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.609+0000 7f7b637fe700 1 -- 172.21.15.57:0/4242010518 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9581+0+0 (secure 0 0 0) 0x7f7b540d1140 con 0x7f7b64101010 2023-10-08T14:08:07.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.610+0000 7f7b6a227700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b4c0a1740 0x7f7b4c0a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:07.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.611+0000 7f7b6a227700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b4c0a1740 0x7f7b4c0a3c00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009710 tx=0x7f7b5c006c60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:07.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:07.622+0000 7f7b637fe700 1 -- 172.21.15.57:0/4242010518 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7b5409a150 con 0x7f7b64101010 2023-10-08T14:08:08.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.083+0000 7f7b6c48b700 1 -- 172.21.15.57:0/4242010518 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b500055a0 con 0x7f7b64101010 2023-10-08T14:08:08.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.092+0000 7f7b637fe700 1 -- 172.21.15.57:0/4242010518 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+11378 (secure 0 0 0) 0x7f7b5409d770 con 0x7f7b64101010 2023-10-08T14:08:08.097 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T14:08:08.097 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":57,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T14:07:48.262546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":112,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T14:08:08.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.105+0000 7f7b6c48b700 1 -- 172.21.15.57:0/4242010518 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b4c0a1740 msgr2=0x7f7b4c0a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:08.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.105+0000 7f7b6c48b700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b4c0a1740 0x7f7b4c0a3c00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009710 tx=0x7f7b5c006c60 comp rx=0 tx=0).stop 2023-10-08T14:08:08.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.105+0000 7f7b6c48b700 1 -- 172.21.15.57:0/4242010518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b64101010 msgr2=0x7f7b6419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:08.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.105+0000 7f7b6c48b700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b64101010 0x7f7b6419cde0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400c0c0 tx=0x7f7b5400c480 comp rx=0 tx=0).stop 2023-10-08T14:08:08.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.108+0000 7f7b6c48b700 1 -- 172.21.15.57:0/4242010518 shutdown_connections 2023-10-08T14:08:08.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.108+0000 7f7b6c48b700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b64106660 0x7f7b64196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.108+0000 7f7b6c48b700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f7b4c0a1740 0x7f7b4c0a3c00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.108+0000 7f7b6c48b700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b64100650 0x7f7b6419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.108+0000 7f7b6c48b700 1 --2- 172.21.15.57:0/4242010518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b64101010 0x7f7b6419cde0 secure :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400c0c0 tx=0x7f7b5400c480 comp rx=0 tx=0).stop 2023-10-08T14:08:08.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.108+0000 7f7b6c48b700 1 -- 172.21.15.57:0/4242010518 >> 172.21.15.57:0/4242010518 conn(0x7f7b640fc180 msgr2=0x7f7b640fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:08.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.116+0000 7f7b6c48b700 1 -- 172.21.15.57:0/4242010518 shutdown_connections 2023-10-08T14:08:08.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.116+0000 7f7b6c48b700 1 -- 172.21.15.57:0/4242010518 wait complete. 2023-10-08T14:08:08.133 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 57 2023-10-08T14:08:08.148 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T14:08:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:08 smithi115 ceph-mon[104408]: pgmap v817: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-10-08T14:08:08.436 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:08 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/4242010518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T14:08:08.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.450+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2085068447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a441082f0 msgr2=0x7f2a441086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:08.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.450+0000 7f2a4bf7c700 1 --2- 172.21.15.57:0/2085068447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a441082f0 0x7f2a441086d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009990 tx=0x7f2a34009ca0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.459+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2085068447 shutdown_connections 2023-10-08T14:08:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.459+0000 7f2a4bf7c700 1 --2- 172.21.15.57:0/2085068447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4410c890 0x7f2a441143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.459+0000 7f2a4bf7c700 1 --2- 172.21.15.57:0/2085068447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a441082f0 0x7f2a441086d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.459+0000 7f2a4bf7c700 1 --2- 172.21.15.57:0/2085068447 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a44108ca0 0x7f2a4410c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.459+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2085068447 >> 172.21.15.57:0/2085068447 conn(0x7f2a4406b760 msgr2=0x7f2a4406bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.466+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2085068447 shutdown_connections 2023-10-08T14:08:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.468+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2085068447 wait complete. 2023-10-08T14:08:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.469+0000 7f2a4bf7c700 1 Processor -- start 2023-10-08T14:08:08.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a4bf7c700 1 -- start start 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a4bf7c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a441082f0 0x7f2a4411d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a4bf7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4410c890 0x7f2a4411d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a4bf7c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a441183c0 0x7f2a44118870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a4bf7c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44118f80 con 0x7f2a441082f0 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a4bf7c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a441190f0 con 0x7f2a441183c0 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a4bf7c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44122f90 con 0x7f2a4410c890 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a49517700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4410c890 0x7f2a4411d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a49d18700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a441082f0 0x7f2a4411d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a49517700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4410c890 0x7f2a4411d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33378/0 (socket says 172.21.15.57:33378) 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a49d18700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a441082f0 0x7f2a4411d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60798/0 (socket says 172.21.15.57:60798) 2023-10-08T14:08:08.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.470+0000 7f2a49517700 1 -- 172.21.15.57:0/2492156129 learned_addr learned my addr 172.21.15.57:0/2492156129 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.471+0000 7f2a49d18700 1 -- 172.21.15.57:0/2492156129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4410c890 msgr2=0x7f2a4411d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.471+0000 7f2a49d18700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4410c890 0x7f2a4411d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.471+0000 7f2a49d18700 1 -- 172.21.15.57:0/2492156129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a441183c0 msgr2=0x7f2a44118870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.471+0000 7f2a49d18700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a441183c0 0x7f2a44118870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.471+0000 7f2a49d18700 1 -- 172.21.15.57:0/2492156129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340095f0 con 0x7f2a441082f0 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.471+0000 7f2a49d18700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a441082f0 0x7f2a4411d210 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009990 tx=0x7f2a340037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.472+0000 7f2a3affd700 1 -- 172.21.15.57:0/2492156129 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3401d070 con 0x7f2a441082f0 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.472+0000 7f2a3affd700 1 -- 172.21.15.57:0/2492156129 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a34005580 con 0x7f2a441082f0 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.472+0000 7f2a3affd700 1 -- 172.21.15.57:0/2492156129 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400fd00 con 0x7f2a441082f0 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.472+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2492156129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a44123210 con 0x7f2a441082f0 2023-10-08T14:08:08.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.472+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2492156129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a44123670 con 0x7f2a441082f0 2023-10-08T14:08:08.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.473+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2492156129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4406cab0 con 0x7f2a441082f0 2023-10-08T14:08:08.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.476+0000 7f2a3affd700 1 -- 172.21.15.57:0/2492156129 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2a3400f460 con 0x7f2a441082f0 2023-10-08T14:08:08.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.477+0000 7f2a3affd700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a300a1740 0x7f2a300a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:08.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.477+0000 7f2a3affd700 1 -- 172.21.15.57:0/2492156129 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9581+0+0 (secure 0 0 0) 0x7f2a340ce490 con 0x7f2a441082f0 2023-10-08T14:08:08.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.478+0000 7f2a49517700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a300a1740 0x7f2a300a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:08.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.478+0000 7f2a49517700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a300a1740 0x7f2a300a3c00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2a44119aa0 tx=0x7f2a40009240 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:08 smithi057 ceph-mon[99476]: pgmap v817: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-10-08T14:08:08.488 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:08 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/4242010518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T14:08:08.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.501+0000 7f2a3affd700 1 -- 172.21.15.57:0/2492156129 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2a34097420 con 0x7f2a441082f0 2023-10-08T14:08:08.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:08 smithi099 ceph-mon[103849]: pgmap v817: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-10-08T14:08:08.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:08 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/4242010518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T14:08:08.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.922+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2492156129 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a44119850 con 0x7f2a441082f0 2023-10-08T14:08:08.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.929+0000 7f2a3affd700 1 -- 172.21.15.57:0/2492156129 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+11378 (secure 0 0 0) 0x7f2a3400fe60 con 0x7f2a441082f0 2023-10-08T14:08:08.931 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T14:08:08.932 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14718,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/2801838933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2801838933},{"type":"v1","addr":"172.21.15.57:6835","nonce":2801838933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14778,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1513163924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1513163924},{"type":"v1","addr":"172.21.15.57:6839","nonce":1513163924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":57,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T13:49:44.186896+0000","modified":"2023-10-08T14:07:48.262546+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24662,"mds_1":24652,"mds_2":24634,"mds_3":24622,"mds_4":24617},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14790":{"gid":14790,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6841/1847373212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1847373212},{"type":"v1","addr":"172.21.15.57:6841","nonce":1847373212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2949059139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2949059139},{"type":"v1","addr":"172.21.15.99:6833","nonce":2949059139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.57:6837/4079715365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":4079715365},{"type":"v1","addr":"172.21.15.57:6837","nonce":4079715365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2732351731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2732351731},{"type":"v1","addr":"172.21.15.99:6835","nonce":2732351731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6833/3794414355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3794414355},{"type":"v1","addr":"172.21.15.115:6833","nonce":3794414355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24617":{"gid":24617,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.99:6837/822922696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":822922696},{"type":"v1","addr":"172.21.15.99:6837","nonce":822922696}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6835/389121345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":389121345},{"type":"v1","addr":"172.21.15.115:6835","nonce":389121345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.115:6837/383062444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":383062444},{"type":"v1","addr":"172.21.15.115:6837","nonce":383062444}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24652":{"gid":24652,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":112,"addr":"172.21.15.115:6839/2892511272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2892511272},{"type":"v1","addr":"172.21.15.115:6839","nonce":2892511272}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24662":{"gid":24662,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6839/687823376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":687823376},{"type":"v1","addr":"172.21.15.99:6839","nonce":687823376}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T14:08:08.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.939+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2492156129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a300a1740 msgr2=0x7f2a300a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:08.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.939+0000 7f2a4bf7c700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a300a1740 0x7f2a300a3c00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2a44119aa0 tx=0x7f2a40009240 comp rx=0 tx=0).stop 2023-10-08T14:08:08.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.940+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2492156129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a441082f0 msgr2=0x7f2a4411d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:08.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.940+0000 7f2a4bf7c700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a441082f0 0x7f2a4411d210 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009990 tx=0x7f2a340037c0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.946+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2492156129 shutdown_connections 2023-10-08T14:08:08.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.947+0000 7f2a4bf7c700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f2a300a1740 0x7f2a300a3c00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.947+0000 7f2a4bf7c700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a441183c0 0x7f2a44118870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.947+0000 7f2a4bf7c700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a441082f0 0x7f2a4411d210 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.947+0000 7f2a4bf7c700 1 --2- 172.21.15.57:0/2492156129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4410c890 0x7f2a4411d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:08.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.947+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2492156129 >> 172.21.15.57:0/2492156129 conn(0x7f2a4406b760 msgr2=0x7f2a4410b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:08.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.952+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2492156129 shutdown_connections 2023-10-08T14:08:08.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:08.956+0000 7f2a4bf7c700 1 -- 172.21.15.57:0/2492156129 wait complete. 2023-10-08T14:08:08.976 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 57 2023-10-08T14:08:08.993 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T14:08:09.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.236+0000 7f82ba760700 1 -- 172.21.15.57:0/3750671398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82b410edf0 msgr2=0x7f82b410f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:09.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.236+0000 7f82ba760700 1 --2- 172.21.15.57:0/3750671398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82b410edf0 0x7f82b410f1d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f82a401a9a0 tx=0x7f82a401acb0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.236+0000 7f82ba760700 1 -- 172.21.15.57:0/3750671398 shutdown_connections 2023-10-08T14:08:09.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.236+0000 7f82ba760700 1 --2- 172.21.15.57:0/3750671398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b41083a0 0x7f82b411bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.236+0000 7f82ba760700 1 --2- 172.21.15.57:0/3750671398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82b410edf0 0x7f82b410f1d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.236+0000 7f82ba760700 1 --2- 172.21.15.57:0/3750671398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82b41077f0 0x7f82b4107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.236+0000 7f82ba760700 1 -- 172.21.15.57:0/3750671398 >> 172.21.15.57:0/3750671398 conn(0x7f82b406b760 msgr2=0x7f82b406bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:09.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.239+0000 7f82ba760700 1 -- 172.21.15.57:0/3750671398 shutdown_connections 2023-10-08T14:08:09.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.239+0000 7f82ba760700 1 -- 172.21.15.57:0/3750671398 wait complete. 2023-10-08T14:08:09.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.240+0000 7f82ba760700 1 Processor -- start 2023-10-08T14:08:09.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.240+0000 7f82ba760700 1 -- start start 2023-10-08T14:08:09.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.240+0000 7f82ba760700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82b41077f0 0x7f82b4117cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:09.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.240+0000 7f82ba760700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82b41083a0 0x7f82b4118210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:09.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.240+0000 7f82ba760700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b410edf0 0x7f82b41c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:09.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.240+0000 7f82ba760700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b41c4160 con 0x7f82b41083a0 2023-10-08T14:08:09.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.240+0000 7f82ba760700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b41c42a0 con 0x7f82b41077f0 2023-10-08T14:08:09.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.240+0000 7f82ba760700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b41c4460 con 0x7f82b410edf0 2023-10-08T14:08:09.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.241+0000 7f82b37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82b41083a0 0x7f82b4118210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:09.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.241+0000 7f82b37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82b41083a0 0x7f82b4118210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60814/0 (socket says 172.21.15.57:60814) 2023-10-08T14:08:09.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.241+0000 7f82b37fe700 1 -- 172.21.15.57:0/142210317 learned_addr learned my addr 172.21.15.57:0/142210317 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:09.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.241+0000 7f82b8cfd700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b410edf0 0x7f82b41c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:09.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.241+0000 7f82b8cfd700 1 -- 172.21.15.57:0/142210317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82b41077f0 msgr2=0x7f82b4117cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:09.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.241+0000 7f82b8cfd700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82b41077f0 0x7f82b4117cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.241+0000 7f82b8cfd700 1 -- 172.21.15.57:0/142210317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82b41083a0 msgr2=0x7f82b4118210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:09.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.241+0000 7f82b8cfd700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82b41083a0 0x7f82b4118210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.241+0000 7f82b8cfd700 1 -- 172.21.15.57:0/142210317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a401a600 con 0x7f82b410edf0 2023-10-08T14:08:09.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.242+0000 7f82b8cfd700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b410edf0 0x7f82b41c3b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82a800e940 tx=0x7f82a800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:09.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.244+0000 7f82b17fa700 1 -- 172.21.15.57:0/142210317 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8004220 con 0x7f82b410edf0 2023-10-08T14:08:09.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.244+0000 7f82b17fa700 1 -- 172.21.15.57:0/142210317 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a8004cf0 con 0x7f82b410edf0 2023-10-08T14:08:09.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.244+0000 7f82b17fa700 1 -- 172.21.15.57:0/142210317 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8005020 con 0x7f82b410edf0 2023-10-08T14:08:09.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.244+0000 7f82ba760700 1 -- 172.21.15.57:0/142210317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b41c4740 con 0x7f82b410edf0 2023-10-08T14:08:09.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.244+0000 7f82ba760700 1 -- 172.21.15.57:0/142210317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b41a1cd0 con 0x7f82b410edf0 2023-10-08T14:08:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.248+0000 7f82ba760700 1 -- 172.21.15.57:0/142210317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a00052f0 con 0x7f82b410edf0 2023-10-08T14:08:09.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.250+0000 7f82b17fa700 1 -- 172.21.15.57:0/142210317 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f82a8004830 con 0x7f82b410edf0 2023-10-08T14:08:09.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.250+0000 7f82b17fa700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f829c0a1780 0x7f829c0a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:09.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.251+0000 7f82b17fa700 1 -- 172.21.15.57:0/142210317 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9581+0+0 (secure 0 0 0) 0x7f82a8014070 con 0x7f82b410edf0 2023-10-08T14:08:09.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.251+0000 7f82b3fff700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f829c0a1780 0x7f829c0a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:09.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.251+0000 7f82b3fff700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f829c0a1780 0x7f829c0a3c40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f82a4003800 tx=0x7f82a4005be0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:09.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.256+0000 7f82b17fa700 1 -- 172.21.15.57:0/142210317 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f82a80960e0 con 0x7f82b410edf0 2023-10-08T14:08:09.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.611+0000 7f82ba760700 1 -- 172.21.15.57:0/142210317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82a0005160 con 0x7f82b410edf0 2023-10-08T14:08:09.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.617+0000 7f82b17fa700 1 -- 172.21.15.57:0/142210317 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+25412 (secure 0 0 0) 0x7f82a8004e60 con 0x7f82b410edf0 2023-10-08T14:08:09.618 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-08T14:08:09.619 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":114,"fsid":"5f25708e-65e0-11ee-8db5-212e2dc638e7","created":"2023-10-08T13:42:06.689196+0000","modified":"2023-10-08T14:08:09.585800+0000","last_up_change":"2023-10-08T13:49:02.955307+0000","last_in_change":"2023-10-08T13:48:45.491579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T13:45:50.544240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T13:49:42.356778+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T13:49:43.577547+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":28,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T13:49:47.570705+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":28,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"82271e50-2757-4c42-a3ca-a105fbaa1391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6803","nonce":1388588372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6805","nonce":1388588372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6809","nonce":1388588372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1388588372},{"type":"v1","addr":"172.21.15.57:6807","nonce":1388588372}]},"public_addr":"172.21.15.57:6803/1388588372","cluster_addr":"172.21.15.57:6805/1388588372","heartbeat_back_addr":"172.21.15.57:6809/1388588372","heartbeat_front_addr":"172.21.15.57:6807/1388588372","state":["exists","up"]},{"osd":1,"uuid":"ef769aca-5124-4f13-931f-2520cbbe2144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6801","nonce":4047402511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6803","nonce":4047402511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6807","nonce":4047402511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4047402511},{"type":"v1","addr":"172.21.15.99:6805","nonce":4047402511}]},"public_addr":"172.21.15.99:6801/4047402511","cluster_addr":"172.21.15.99:6803/4047402511","heartbeat_back_addr":"172.21.15.99:6807/4047402511","heartbeat_front_addr":"172.21.15.99:6805/4047402511","state":["exists","up"]},{"osd":2,"uuid":"d8c72122-dbc2-4a3b-9ee3-6c83f989107a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6801","nonce":3640540626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6803","nonce":3640540626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6807","nonce":3640540626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3640540626},{"type":"v1","addr":"172.21.15.115:6805","nonce":3640540626}]},"public_addr":"172.21.15.115:6801/3640540626","cluster_addr":"172.21.15.115:6803/3640540626","heartbeat_back_addr":"172.21.15.115:6807/3640540626","heartbeat_front_addr":"172.21.15.115:6805/3640540626","state":["exists","up"]},{"osd":3,"uuid":"c969ed29-1e01-479b-ab6f-00f8b0dde361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6811","nonce":966286851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6813","nonce":966286851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6817","nonce":966286851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":966286851},{"type":"v1","addr":"172.21.15.57:6815","nonce":966286851}]},"public_addr":"172.21.15.57:6811/966286851","cluster_addr":"172.21.15.57:6813/966286851","heartbeat_back_addr":"172.21.15.57:6817/966286851","heartbeat_front_addr":"172.21.15.57:6815/966286851","state":["exists","up"]},{"osd":4,"uuid":"842feff0-faab-4c13-88b1-91355f53e0e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6809","nonce":1502873104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6811","nonce":1502873104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6815","nonce":1502873104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1502873104},{"type":"v1","addr":"172.21.15.99:6813","nonce":1502873104}]},"public_addr":"172.21.15.99:6809/1502873104","cluster_addr":"172.21.15.99:6811/1502873104","heartbeat_back_addr":"172.21.15.99:6815/1502873104","heartbeat_front_addr":"172.21.15.99:6813/1502873104","state":["exists","up"]},{"osd":5,"uuid":"97f91a58-677f-4005-8a2d-66113c9cdda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6809","nonce":2071180529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6811","nonce":2071180529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6815","nonce":2071180529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":2071180529},{"type":"v1","addr":"172.21.15.115:6813","nonce":2071180529}]},"public_addr":"172.21.15.115:6809/2071180529","cluster_addr":"172.21.15.115:6811/2071180529","heartbeat_back_addr":"172.21.15.115:6815/2071180529","heartbeat_front_addr":"172.21.15.115:6813/2071180529","state":["exists","up"]},{"osd":6,"uuid":"fd2240ad-a8f2-49ed-bf62-8fe8eff60156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6819","nonce":3636365210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6821","nonce":3636365210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6825","nonce":3636365210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":3636365210},{"type":"v1","addr":"172.21.15.57:6823","nonce":3636365210}]},"public_addr":"172.21.15.57:6819/3636365210","cluster_addr":"172.21.15.57:6821/3636365210","heartbeat_back_addr":"172.21.15.57:6825/3636365210","heartbeat_front_addr":"172.21.15.57:6823/3636365210","state":["exists","up"]},{"osd":7,"uuid":"8bc31070-8d5a-4a0c-b14f-5fa855e951de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6817","nonce":4070534570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6819","nonce":4070534570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6823","nonce":4070534570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4070534570},{"type":"v1","addr":"172.21.15.99:6821","nonce":4070534570}]},"public_addr":"172.21.15.99:6817/4070534570","cluster_addr":"172.21.15.99:6819/4070534570","heartbeat_back_addr":"172.21.15.99:6823/4070534570","heartbeat_front_addr":"172.21.15.99:6821/4070534570","state":["exists","up"]},{"osd":8,"uuid":"61534853-3ba8-40cc-b62e-7b1c2efdb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6817","nonce":3107603962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6819","nonce":3107603962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6823","nonce":3107603962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3107603962},{"type":"v1","addr":"172.21.15.115:6821","nonce":3107603962}]},"public_addr":"172.21.15.115:6817/3107603962","cluster_addr":"172.21.15.115:6819/3107603962","heartbeat_back_addr":"172.21.15.115:6823/3107603962","heartbeat_front_addr":"172.21.15.115:6821/3107603962","state":["exists","up"]},{"osd":9,"uuid":"28533ac1-c3f9-4587-93d0-752be200eeda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6827","nonce":1074674274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6829","nonce":1074674274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6833","nonce":1074674274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":1074674274},{"type":"v1","addr":"172.21.15.57:6831","nonce":1074674274}]},"public_addr":"172.21.15.57:6827/1074674274","cluster_addr":"172.21.15.57:6829/1074674274","heartbeat_back_addr":"172.21.15.57:6833/1074674274","heartbeat_front_addr":"172.21.15.57:6831/1074674274","state":["exists","up"]},{"osd":10,"uuid":"15226d7f-8a7d-45da-8750-adaa98217259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6825","nonce":317517288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6827","nonce":317517288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6831","nonce":317517288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":317517288},{"type":"v1","addr":"172.21.15.99:6829","nonce":317517288}]},"public_addr":"172.21.15.99:6825/317517288","cluster_addr":"172.21.15.99:6827/317517288","heartbeat_back_addr":"172.21.15.99:6831/317517288","heartbeat_front_addr":"172.21.15.99:6829/317517288","state":["exists","up"]},{"osd":11,"uuid":"bd75deff-374f-4629-ba93-d7b9e20882a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6825","nonce":901134235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6827","nonce":901134235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6831","nonce":901134235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":901134235},{"type":"v1","addr":"172.21.15.115:6829","nonce":901134235}]},"public_addr":"172.21.15.115:6825/901134235","cluster_addr":"172.21.15.115:6827/901134235","heartbeat_back_addr":"172.21.15.115:6831/901134235","heartbeat_front_addr":"172.21.15.115:6829/901134235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:04.293250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:26.305252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:45:47.043741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:08.846644+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:31.146755+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:46:52.295983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:13.572196+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:35.915062+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:47:57.064479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:17.920659+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:48:39.848958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T13:49:01.214756+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/466175178":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1455537828":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6801/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:6800/2534706720":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/1379042066":"2023-10-09T13:44:14.025065+0000","172.21.15.57:0/19108071":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3114287016":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/3318653521":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2210095932":"2023-10-09T13:42:39.884205+0000","172.21.15.57:0/955690770":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6801/2406655290":"2023-10-09T13:43:06.093214+0000","172.21.15.57:0/2229924806":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/140726743":"2023-10-09T13:42:39.884205+0000","172.21.15.57:6800/2406655290":"2023-10-09T13:43:06.093214+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":14,"length":1},{"begin":16,"length":1},{"begin":24,"length":1},{"begin":26,"length":1},{"begin":28,"length":1}]},{"pool":4,"snaps":[{"begin":12,"length":1},{"begin":14,"length":1},{"begin":16,"length":2},{"begin":19,"length":1},{"begin":21,"length":1},{"begin":24,"length":1},{"begin":26,"length":1},{"begin":28,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":28,"length":1}]},{"pool":4,"snaps":[{"begin":16,"length":1},{"begin":28,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T14:08:09.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.625+0000 7f82ba760700 1 -- 172.21.15.57:0/142210317 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f829c0a1780 msgr2=0x7f829c0a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:09.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.625+0000 7f82ba760700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f829c0a1780 0x7f829c0a3c40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f82a4003800 tx=0x7f82a4005be0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.625+0000 7f82ba760700 1 -- 172.21.15.57:0/142210317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b410edf0 msgr2=0x7f82b41c3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:09.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.625+0000 7f82ba760700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b410edf0 0x7f82b41c3b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82a800e940 tx=0x7f82a800ec50 comp rx=0 tx=0).stop 2023-10-08T14:08:09.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.626+0000 7f82ba760700 1 -- 172.21.15.57:0/142210317 shutdown_connections 2023-10-08T14:08:09.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.626+0000 7f82ba760700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82b410edf0 0x7f82b41c3b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.626+0000 7f82ba760700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f829c0a1780 0x7f829c0a3c40 secure :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f82a4003800 tx=0x7f82a4005be0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.626+0000 7f82ba760700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82b41077f0 0x7f82b4117cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.626+0000 7f82ba760700 1 --2- 172.21.15.57:0/142210317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f82b41083a0 0x7f82b4118210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.626+0000 7f82ba760700 1 -- 172.21.15.57:0/142210317 >> 172.21.15.57:0/142210317 conn(0x7f82b406b760 msgr2=0x7f82b4070740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:09.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.633+0000 7f82ba760700 1 -- 172.21.15.57:0/142210317 shutdown_connections 2023-10-08T14:08:09.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.639+0000 7f82ba760700 1 -- 172.21.15.57:0/142210317 wait complete. 2023-10-08T14:08:09.695 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T14:08:09.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:09 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/2492156129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T14:08:09.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:09 smithi115 ceph-mon[104408]: pgmap v818: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2023-10-08T14:08:09.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:09 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/2492156129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T14:08:09.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:09 smithi057 ceph-mon[99476]: pgmap v818: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2023-10-08T14:08:09.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.982+0000 7fc52d3c0700 1 -- 172.21.15.57:0/46314592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5280ff0b0 msgr2=0x7fc5280ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:09.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.982+0000 7fc52d3c0700 1 --2- 172.21.15.57:0/46314592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5280ff0b0 0x7fc5280ff530 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c240 tx=0x7fc52000c550 comp rx=0 tx=0).stop 2023-10-08T14:08:09.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.986+0000 7fc52d3c0700 1 -- 172.21.15.57:0/46314592 shutdown_connections 2023-10-08T14:08:09.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.986+0000 7fc52d3c0700 1 --2- 172.21.15.57:0/46314592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5280ffa70 0x7fc52810bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.986+0000 7fc52d3c0700 1 --2- 172.21.15.57:0/46314592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc528103df0 0x7fc5281041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.986+0000 7fc52d3c0700 1 --2- 172.21.15.57:0/46314592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5280ff0b0 0x7fc5280ff530 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.986+0000 7fc52d3c0700 1 -- 172.21.15.57:0/46314592 >> 172.21.15.57:0/46314592 conn(0x7fc528074940 msgr2=0x7fc528074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:09.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.986+0000 7fc52d3c0700 1 -- 172.21.15.57:0/46314592 shutdown_connections 2023-10-08T14:08:09.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.986+0000 7fc52d3c0700 1 -- 172.21.15.57:0/46314592 wait complete. 2023-10-08T14:08:09.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.987+0000 7fc52d3c0700 1 Processor -- start 2023-10-08T14:08:09.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.988+0000 7fc52d3c0700 1 -- start start 2023-10-08T14:08:09.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.988+0000 7fc52d3c0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5280ff0b0 0x7fc528108570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:09.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.988+0000 7fc52d3c0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5280ffa70 0x7fc528108ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:09.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.988+0000 7fc52d3c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc528103df0 0x7fc528072260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:09.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.988+0000 7fc52d3c0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc528109060 con 0x7fc528103df0 2023-10-08T14:08:09.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.988+0000 7fc52d3c0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5281091d0 con 0x7fc5280ffa70 2023-10-08T14:08:09.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.988+0000 7fc52d3c0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5281b2030 con 0x7fc5280ff0b0 2023-10-08T14:08:09.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.991+0000 7fc5267fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5280ffa70 0x7fc528108ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:09.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.991+0000 7fc5267fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5280ffa70 0x7fc528108ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:34022/0 (socket says 172.21.15.57:34022) 2023-10-08T14:08:09.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.991+0000 7fc5267fc700 1 -- 172.21.15.57:0/3504729817 learned_addr learned my addr 172.21.15.57:0/3504729817 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:09.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.991+0000 7fc5277fe700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc528103df0 0x7fc528072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:09.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.992+0000 7fc5267fc700 1 -- 172.21.15.57:0/3504729817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5280ff0b0 msgr2=0x7fc528108570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:09.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.992+0000 7fc5267fc700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5280ff0b0 0x7fc528108570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.992+0000 7fc5267fc700 1 -- 172.21.15.57:0/3504729817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc528103df0 msgr2=0x7fc528072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:09.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.992+0000 7fc5267fc700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc528103df0 0x7fc528072260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:09.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.992+0000 7fc5267fc700 1 -- 172.21.15.57:0/3504729817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52000ae10 con 0x7fc5280ffa70 2023-10-08T14:08:10.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.994+0000 7fc5267fc700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5280ffa70 0x7fc528108ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc520008560 tx=0x7fc52000a7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:10.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.994+0000 7fc50ffff700 1 -- 172.21.15.57:0/3504729817 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52000f040 con 0x7fc5280ffa70 2023-10-08T14:08:10.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.994+0000 7fc50ffff700 1 -- 172.21.15.57:0/3504729817 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc52000a590 con 0x7fc5280ffa70 2023-10-08T14:08:10.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.994+0000 7fc50ffff700 1 -- 172.21.15.57:0/3504729817 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52001e480 con 0x7fc5280ffa70 2023-10-08T14:08:10.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.994+0000 7fc52d3c0700 1 -- 172.21.15.57:0/3504729817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5281b22b0 con 0x7fc5280ffa70 2023-10-08T14:08:10.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.995+0000 7fc52d3c0700 1 -- 172.21.15.57:0/3504729817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5281b2770 con 0x7fc5280ffa70 2023-10-08T14:08:10.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.998+0000 7fc52d3c0700 1 -- 172.21.15.57:0/3504729817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc528100520 con 0x7fc5280ffa70 2023-10-08T14:08:10.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:09.999+0000 7fc50ffff700 1 -- 172.21.15.57:0/3504729817 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc5200074a0 con 0x7fc5280ffa70 2023-10-08T14:08:10.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.000+0000 7fc50ffff700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc5100a16b0 0x7fc5100a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:10.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.000+0000 7fc50ffff700 1 -- 172.21.15.57:0/3504729817 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9629+0+0 (secure 0 0 0) 0x7fc5200ce720 con 0x7fc5280ffa70 2023-10-08T14:08:10.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.000+0000 7fc526ffd700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc5100a16b0 0x7fc5100a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:10.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.001+0000 7fc526ffd700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc5100a16b0 0x7fc5100a3b70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc51c017fd0 tx=0x7fc51c017ce0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:10.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.011+0000 7fc50ffff700 1 -- 172.21.15.57:0/3504729817 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc520097710 con 0x7fc5280ffa70 2023-10-08T14:08:10.021 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:09 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/2492156129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T14:08:10.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:09 smithi099 ceph-mon[103849]: pgmap v818: 129 pgs: 39 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 78 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2023-10-08T14:08:10.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.479+0000 7fc52d3c0700 1 -- 172.21.15.57:0/3504729817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc5281097b0 con 0x7fc5280ffa70 2023-10-08T14:08:10.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.512+0000 7fc50ffff700 1 -- 172.21.15.57:0/3504729817 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc52009ad30 con 0x7fc5280ffa70 2023-10-08T14:08:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.534+0000 7fc52d3c0700 1 -- 172.21.15.57:0/3504729817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc5100a16b0 msgr2=0x7fc5100a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.534+0000 7fc52d3c0700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc5100a16b0 0x7fc5100a3b70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc51c017fd0 tx=0x7fc51c017ce0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.534+0000 7fc52d3c0700 1 -- 172.21.15.57:0/3504729817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5280ffa70 msgr2=0x7fc528108ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.534+0000 7fc52d3c0700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5280ffa70 0x7fc528108ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc520008560 tx=0x7fc52000a7f0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.537+0000 7fc52d3c0700 1 -- 172.21.15.57:0/3504729817 shutdown_connections 2023-10-08T14:08:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.537+0000 7fc52d3c0700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc528103df0 0x7fc528072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.537+0000 7fc52d3c0700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5280ff0b0 0x7fc528108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.537+0000 7fc52d3c0700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fc5100a16b0 0x7fc5100a3b70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.537+0000 7fc52d3c0700 1 --2- 172.21.15.57:0/3504729817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5280ffa70 0x7fc528108ab0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc520008560 tx=0x7fc52000a7f0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.537+0000 7fc52d3c0700 1 -- 172.21.15.57:0/3504729817 >> 172.21.15.57:0/3504729817 conn(0x7fc528074940 msgr2=0x7fc528102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:10.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.541+0000 7fc52d3c0700 1 -- 172.21.15.57:0/3504729817 shutdown_connections 2023-10-08T14:08:10.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.541+0000 7fc52d3c0700 1 -- 172.21.15.57:0/3504729817 wait complete. 2023-10-08T14:08:10.550 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T14:08:10.580 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T14:08:10.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a[99452]: 2023-10-08T14:08:10.481+0000 7f3dcc0bf700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T14:08:10.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-mon[99476]: osdmap e114: 12 total, 12 up, 12 in 2023-10-08T14:08:10.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/142210317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T14:08:10.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/3504729817' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T14:08:10.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T14:08:10.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-mon[99476]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-08T14:08:10.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-mon[99476]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T14:08:10.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-mon[99476]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-10-08T14:08:10.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-08T14:08:10.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-mon[99476]: osdmap e115: 12 total, 12 up, 12 in 2023-10-08T14:08:10.738 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-mon[99476]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-08T14:08:10.739 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-d[156645]: ignoring --setuser ceph since I am not root 2023-10-08T14:08:10.739 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-d[156645]: ignoring --setgroup ceph since I am not root 2023-10-08T14:08:10.739 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-d[156645]: starting mds.d at 2023-10-08T14:08:10.741 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-a[157643]: ignoring --setuser ceph since I am not root 2023-10-08T14:08:10.742 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-a[157643]: ignoring --setgroup ceph since I am not root 2023-10-08T14:08:10.742 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 14:08:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-a[157643]: starting mds.a at 2023-10-08T14:08:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.845+0000 7fe1e87a0700 1 -- 172.21.15.57:0/1633581327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e010c500 msgr2=0x7fe1e0114080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.845+0000 7fe1e87a0700 1 --2- 172.21.15.57:0/1633581327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e010c500 0x7fe1e0114080 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009f90 tx=0x7fe1d800b330 comp rx=0 tx=0).stop 2023-10-08T14:08:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.845+0000 7fe1e87a0700 1 -- 172.21.15.57:0/1633581327 shutdown_connections 2023-10-08T14:08:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.845+0000 7fe1e87a0700 1 --2- 172.21.15.57:0/1633581327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e010c500 0x7fe1e0114080 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.846+0000 7fe1e87a0700 1 --2- 172.21.15.57:0/1633581327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1e0107a60 0x7fe1e0107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.846+0000 7fe1e87a0700 1 --2- 172.21.15.57:0/1633581327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0108410 0x7fe1e010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.846+0000 7fe1e87a0700 1 -- 172.21.15.57:0/1633581327 >> 172.21.15.57:0/1633581327 conn(0x7fe1e006bc80 msgr2=0x7fe1e006c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:10.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.854+0000 7fe1e87a0700 1 -- 172.21.15.57:0/1633581327 shutdown_connections 2023-10-08T14:08:10.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.859+0000 7fe1e87a0700 1 -- 172.21.15.57:0/1633581327 wait complete. 2023-10-08T14:08:10.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.860+0000 7fe1e87a0700 1 Processor -- start 2023-10-08T14:08:10.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.861+0000 7fe1e87a0700 1 -- start start 2023-10-08T14:08:10.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.861+0000 7fe1e87a0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1e0107a60 0x7fe1e010dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:10.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.861+0000 7fe1e87a0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e010c500 0x7fe1e010e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:10.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.861+0000 7fe1e87a0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010ea40 0x7fe1e01c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:10.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.861+0000 7fe1e87a0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e01c2f30 con 0x7fe1e0107a60 2023-10-08T14:08:10.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.861+0000 7fe1e87a0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e007a4f0 con 0x7fe1e010c500 2023-10-08T14:08:10.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.861+0000 7fe1e87a0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e007a6b0 con 0x7fe1e010ea40 2023-10-08T14:08:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.862+0000 7fe1e6d3d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010ea40 0x7fe1e01c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.862+0000 7fe1e6d3d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010ea40 0x7fe1e01c28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.57:33442/0 (socket says 172.21.15.57:33442) 2023-10-08T14:08:10.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.862+0000 7fe1e6d3d700 1 -- 172.21.15.57:0/877063011 learned_addr learned my addr 172.21.15.57:0/877063011 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.862+0000 7fe1e653c700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1e0107a60 0x7fe1e010dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.863+0000 7fe1e6d3d700 1 -- 172.21.15.57:0/877063011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e010c500 msgr2=0x7fe1e010e500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:08:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.863+0000 7fe1e6d3d700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e010c500 0x7fe1e010e500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.863+0000 7fe1e6d3d700 1 -- 172.21.15.57:0/877063011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1e0107a60 msgr2=0x7fe1e010dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:10.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.863+0000 7fe1e6d3d700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1e0107a60 0x7fe1e010dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:10.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.863+0000 7fe1e6d3d700 1 -- 172.21.15.57:0/877063011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8009c40 con 0x7fe1e010ea40 2023-10-08T14:08:10.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.863+0000 7fe1e6d3d700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010ea40 0x7fe1e01c28d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006130 tx=0x7fe1d8009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:10.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.864+0000 7fe1d77fe700 1 -- 172.21.15.57:0/877063011 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800e040 con 0x7fe1e010ea40 2023-10-08T14:08:10.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.864+0000 7fe1d77fe700 1 -- 172.21.15.57:0/877063011 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1d8007700 con 0x7fe1e010ea40 2023-10-08T14:08:10.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.864+0000 7fe1d77fe700 1 -- 172.21.15.57:0/877063011 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80052a0 con 0x7fe1e010ea40 2023-10-08T14:08:10.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.864+0000 7fe1e87a0700 1 -- 172.21.15.57:0/877063011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e007a930 con 0x7fe1e010ea40 2023-10-08T14:08:10.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.865+0000 7fe1e87a0700 1 -- 172.21.15.57:0/877063011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e007adf0 con 0x7fe1e010ea40 2023-10-08T14:08:10.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.868+0000 7fe1d77fe700 1 -- 172.21.15.57:0/877063011 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe1d8005400 con 0x7fe1e010ea40 2023-10-08T14:08:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.869+0000 7fe1d77fe700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1cc0a17c0 0x7fe1cc0a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:10.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.869+0000 7fe1d77fe700 1 -- 172.21.15.57:0/877063011 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9971+0+0 (secure 0 0 0) 0x7fe1d80cec80 con 0x7fe1e010ea40 2023-10-08T14:08:10.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.873+0000 7fe1e87a0700 1 -- 172.21.15.57:0/877063011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c40052f0 con 0x7fe1e010ea40 2023-10-08T14:08:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.874+0000 7fe1e653c700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1cc0a17c0 0x7fe1cc0a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:10.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.874+0000 7fe1e653c700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1cc0a17c0 0x7fe1cc0a3c80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0005950 tx=0x7fe1d00058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:10.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:10.881+0000 7fe1d77fe700 1 -- 172.21.15.57:0/877063011 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe1d8097a20 con 0x7fe1e010ea40 2023-10-08T14:08:10.936 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-i[141877]: ignoring --setuser ceph since I am not root 2023-10-08T14:08:10.936 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-i[141877]: ignoring --setgroup ceph since I am not root 2023-10-08T14:08:10.936 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-i[141877]: starting mds.i at 2023-10-08T14:08:10.937 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-l[140851]: ignoring --setuser ceph since I am not root 2023-10-08T14:08:10.937 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-l[140851]: ignoring --setgroup ceph since I am not root 2023-10-08T14:08:10.937 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-l[140851]: starting mds.l at 2023-10-08T14:08:10.937 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-c[141540]: ignoring --setuser ceph since I am not root 2023-10-08T14:08:10.937 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-c[141540]: ignoring --setgroup ceph since I am not root 2023-10-08T14:08:10.937 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mds[141564]: -- 172.21.15.115:0/2444058568 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f602f9f1e0 con 0x55f602f74c00 2023-10-08T14:08:10.938 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-c[141540]: starting mds.c at 2023-10-08T14:08:10.938 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-f[141199]: ignoring --setuser ceph since I am not root 2023-10-08T14:08:10.938 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-f[141199]: ignoring --setgroup ceph since I am not root 2023-10-08T14:08:10.938 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-f[141199]: starting mds.f at 2023-10-08T14:08:10.941 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mon[104408]: osdmap e114: 12 total, 12 up, 12 in 2023-10-08T14:08:10.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/142210317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T14:08:10.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/3504729817' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T14:08:10.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T14:08:10.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mon[104408]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-08T14:08:10.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mon[104408]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T14:08:10.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mon[104408]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-10-08T14:08:10.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-08T14:08:10.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mon[104408]: osdmap e115: 12 total, 12 up, 12 in 2023-10-08T14:08:10.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:10 smithi115 ceph-mon[104408]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-08T14:08:11.022 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-k[139865]: ignoring --setuser ceph since I am not root 2023-10-08T14:08:11.022 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-k[139865]: ignoring --setgroup ceph since I am not root 2023-10-08T14:08:11.022 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-k[139865]: starting mds.k at 2023-10-08T14:08:11.023 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-h[140545]: ignoring --setuser ceph since I am not root 2023-10-08T14:08:11.023 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-h[140545]: ignoring --setgroup ceph since I am not root 2023-10-08T14:08:11.023 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mds[140569]: -- 172.21.15.99:0/962213699 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55db30d711e0 con 0x55db30d47000 2023-10-08T14:08:11.023 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-h[140545]: starting mds.h at 2023-10-08T14:08:11.023 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-e[140207]: ignoring --setuser ceph since I am not root 2023-10-08T14:08:11.023 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-e[140207]: ignoring --setgroup ceph since I am not root 2023-10-08T14:08:11.023 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-e[140207]: starting mds.e at 2023-10-08T14:08:11.024 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b[140884]: ignoring --setuser ceph since I am not root 2023-10-08T14:08:11.024 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b[140884]: ignoring --setgroup ceph since I am not root 2023-10-08T14:08:11.024 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b[140884]: starting mds.b at 2023-10-08T14:08:11.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mon[103849]: osdmap e114: 12 total, 12 up, 12 in 2023-10-08T14:08:11.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/142210317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T14:08:11.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/3504729817' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T14:08:11.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T14:08:11.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mon[103849]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-08T14:08:11.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mon[103849]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T14:08:11.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mon[103849]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-10-08T14:08:11.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-08T14:08:11.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mon[103849]: osdmap e115: 12 total, 12 up, 12 in 2023-10-08T14:08:11.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:10 smithi099 ceph-mon[103849]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-08T14:08:11.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.356+0000 7fe1e87a0700 1 -- 172.21.15.57:0/877063011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe1c4005b20 con 0x7fe1e010ea40 2023-10-08T14:08:11.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.654+0000 7fe1d77fe700 1 -- 172.21.15.57:0/877063011 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1d809b040 con 0x7fe1e010ea40 2023-10-08T14:08:11.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.663+0000 7fe1e87a0700 1 -- 172.21.15.57:0/877063011 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1cc0a17c0 msgr2=0x7fe1cc0a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:11.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.663+0000 7fe1e87a0700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1cc0a17c0 0x7fe1cc0a3c80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0005950 tx=0x7fe1d00058e0 comp rx=0 tx=0).stop 2023-10-08T14:08:11.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.663+0000 7fe1e87a0700 1 -- 172.21.15.57:0/877063011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010ea40 msgr2=0x7fe1e01c28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:11.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.663+0000 7fe1e87a0700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010ea40 0x7fe1e01c28d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006130 tx=0x7fe1d8009750 comp rx=0 tx=0).stop 2023-10-08T14:08:11.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.666+0000 7fe1e87a0700 1 -- 172.21.15.57:0/877063011 shutdown_connections 2023-10-08T14:08:11.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.666+0000 7fe1e87a0700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e010ea40 0x7fe1e01c28d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:11.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.666+0000 7fe1e87a0700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1e0107a60 0x7fe1e010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:11.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.666+0000 7fe1e87a0700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7fe1cc0a17c0 0x7fe1cc0a3c80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:11.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.666+0000 7fe1e87a0700 1 --2- 172.21.15.57:0/877063011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e010c500 0x7fe1e010e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:11.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.666+0000 7fe1e87a0700 1 -- 172.21.15.57:0/877063011 >> 172.21.15.57:0/877063011 conn(0x7fe1e006bc80 msgr2=0x7fe1e00700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:11.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.673+0000 7fe1e87a0700 1 -- 172.21.15.57:0/877063011 shutdown_connections 2023-10-08T14:08:11.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.674+0000 7fe1e87a0700 1 -- 172.21.15.57:0/877063011 wait complete. 2023-10-08T14:08:11.712 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T14:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:11 smithi115 ceph-mon[104408]: pgmap v821: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-10-08T14:08:11.936 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:11 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/877063011' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T14:08:11.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:11 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T14:08:11.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.975+0000 7f5308907700 1 -- 172.21.15.57:0/1761810109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f530410c0e0 msgr2=0x7f5304078b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:11.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.975+0000 7f5308907700 1 --2- 172.21.15.57:0/1761810109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f530410c0e0 0x7f5304078b00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0090c0 tx=0x7f52fc00e850 comp rx=0 tx=0).stop 2023-10-08T14:08:11.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.975+0000 7f5308907700 1 -- 172.21.15.57:0/1761810109 shutdown_connections 2023-10-08T14:08:11.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.975+0000 7f5308907700 1 --2- 172.21.15.57:0/1761810109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f530410c0e0 0x7f5304078b00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:11.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.975+0000 7f5308907700 1 --2- 172.21.15.57:0/1761810109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5304107a60 0x7f5304107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:11.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.975+0000 7f5308907700 1 --2- 172.21.15.57:0/1761810109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5304108380 0x7f530410b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:11.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.975+0000 7f5308907700 1 -- 172.21.15.57:0/1761810109 >> 172.21.15.57:0/1761810109 conn(0x7f530406b830 msgr2=0x7f530406bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:11.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.975+0000 7f5308907700 1 -- 172.21.15.57:0/1761810109 shutdown_connections 2023-10-08T14:08:11.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.976+0000 7f5308907700 1 -- 172.21.15.57:0/1761810109 wait complete. 2023-10-08T14:08:11.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.976+0000 7f5308907700 1 Processor -- start 2023-10-08T14:08:11.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.977+0000 7f5308907700 1 -- start start 2023-10-08T14:08:11.987 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.983+0000 7f5308907700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5304107a60 0x7f5304083190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:11.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.983+0000 7f5308907700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5304108380 0x7f530407d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:11.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.983+0000 7f5308907700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f530410c0e0 0x7f530407d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:11.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.983+0000 7f5308907700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530407de10 con 0x7f5304107a60 2023-10-08T14:08:11.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.983+0000 7f5308907700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530407df80 con 0x7f5304108380 2023-10-08T14:08:11.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.983+0000 7f5308907700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53041c3140 con 0x7f530410c0e0 2023-10-08T14:08:11.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.983+0000 7f530259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5304107a60 0x7f5304083190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:11.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.983+0000 7f530259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5304107a60 0x7f5304083190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60874/0 (socket says 172.21.15.57:60874) 2023-10-08T14:08:11.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.983+0000 7f530259c700 1 -- 172.21.15.57:0/599974724 learned_addr learned my addr 172.21.15.57:0/599974724 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:11.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.984+0000 7f5301d9b700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5304108380 0x7f530407d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:11.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.984+0000 7f5301d9b700 1 -- 172.21.15.57:0/599974724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f530410c0e0 msgr2=0x7f530407d770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:08:11.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.984+0000 7f5301d9b700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f530410c0e0 0x7f530407d770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:11.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.984+0000 7f5301d9b700 1 -- 172.21.15.57:0/599974724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5304107a60 msgr2=0x7f5304083190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:11.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.984+0000 7f5301d9b700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5304107a60 0x7f5304083190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:11.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.984+0000 7f5301d9b700 1 -- 172.21.15.57:0/599974724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc008cf0 con 0x7f5304108380 2023-10-08T14:08:11.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.984+0000 7f5301d9b700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5304108380 0x7f530407d230 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52f8006fd0 tx=0x7f52f800b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:11.993 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:11 smithi057 ceph-mon[99476]: pgmap v821: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-10-08T14:08:11.994 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:11 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/877063011' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T14:08:11.994 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:11 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T14:08:11.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.993+0000 7f52f37fe700 1 -- 172.21.15.57:0/599974724 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8011830 con 0x7f5304108380 2023-10-08T14:08:11.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.993+0000 7f52f37fe700 1 -- 172.21.15.57:0/599974724 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52f8011e50 con 0x7f5304108380 2023-10-08T14:08:11.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.993+0000 7f5308907700 1 -- 172.21.15.57:0/599974724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53041c3340 con 0x7f5304108380 2023-10-08T14:08:11.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.993+0000 7f52f37fe700 1 -- 172.21.15.57:0/599974724 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f800f2e0 con 0x7f5304108380 2023-10-08T14:08:11.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.993+0000 7f5308907700 1 -- 172.21.15.57:0/599974724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53041c3850 con 0x7f5304108380 2023-10-08T14:08:11.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:11.994+0000 7f5308907700 1 -- 172.21.15.57:0/599974724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5304074250 con 0x7f5304108380 2023-10-08T14:08:12.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.006+0000 7f52f37fe700 1 -- 172.21.15.57:0/599974724 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f52f8010510 con 0x7f5304108380 2023-10-08T14:08:12.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.007+0000 7f52f37fe700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a1780 0x7f52ec0a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:12.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.007+0000 7f52f37fe700 1 -- 172.21.15.57:0/599974724 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9971+0+0 (secure 0 0 0) 0x7f52f80cd1a0 con 0x7f5304108380 2023-10-08T14:08:12.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.009+0000 7f530259c700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a1780 0x7f52ec0a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:12.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.010+0000 7f530259c700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a1780 0x7f52ec0a3c40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f52f40099d0 tx=0x7f52f4008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:11 smithi099 ceph-mon[103849]: pgmap v821: 129 pgs: 54 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 58 active+clean; 2.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-10-08T14:08:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:11 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/877063011' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T14:08:12.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:11 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T14:08:12.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.018+0000 7f52f37fe700 1 -- 172.21.15.57:0/599974724 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f52f80960a0 con 0x7f5304108380 2023-10-08T14:08:12.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.418+0000 7f5308907700 1 -- 172.21.15.57:0/599974724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f530407e710 con 0x7f5304108380 2023-10-08T14:08:12.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.657+0000 7f52f37fe700 1 -- 172.21.15.57:0/599974724 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 181+0+0 (secure 0 0 0) 0x7f52f8015070 con 0x7f5304108380 2023-10-08T14:08:12.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.664+0000 7f5308907700 1 -- 172.21.15.57:0/599974724 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a1780 msgr2=0x7f52ec0a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:12.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.664+0000 7f5308907700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a1780 0x7f52ec0a3c40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f52f40099d0 tx=0x7f52f4008040 comp rx=0 tx=0).stop 2023-10-08T14:08:12.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.664+0000 7f5308907700 1 -- 172.21.15.57:0/599974724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5304108380 msgr2=0x7f530407d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:12.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.664+0000 7f5308907700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5304108380 0x7f530407d230 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52f8006fd0 tx=0x7f52f800b7e0 comp rx=0 tx=0).stop 2023-10-08T14:08:12.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.668+0000 7f5308907700 1 -- 172.21.15.57:0/599974724 shutdown_connections 2023-10-08T14:08:12.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.668+0000 7f5308907700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f530410c0e0 0x7f530407d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:12.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.668+0000 7f5308907700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5304107a60 0x7f5304083190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:12.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.668+0000 7f5308907700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f52ec0a1780 0x7f52ec0a3c40 secure :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f52f40099d0 tx=0x7f52f4008040 comp rx=0 tx=0).stop 2023-10-08T14:08:12.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.668+0000 7f5308907700 1 --2- 172.21.15.57:0/599974724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5304108380 0x7f530407d230 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:12.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.668+0000 7f5308907700 1 -- 172.21.15.57:0/599974724 >> 172.21.15.57:0/599974724 conn(0x7f530406b830 msgr2=0x7f53040704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:12.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.671+0000 7f5308907700 1 -- 172.21.15.57:0/599974724 shutdown_connections 2023-10-08T14:08:12.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:12.674+0000 7f5308907700 1 -- 172.21.15.57:0/599974724 wait complete. 2023-10-08T14:08:12.693 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2023-10-08T14:08:12.719 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T14:08:12.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-08T14:08:12.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-08T14:08:12.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: Cluster is now healthy 2023-10-08T14:08:12.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-08T14:08:12.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.57:6836/2921049017,v1:172.21.15.57:6837/2921049017] up:boot 2023-10-08T14:08:12.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.57:6840/3970825978,v1:172.21.15.57:6841/3970825978] up:boot 2023-10-08T14:08:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6838/2040651681,v1:172.21.15.115:6839/2040651681] up:boot 2023-10-08T14:08:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6834/3061265823,v1:172.21.15.115:6835/3061265823] up:boot 2023-10-08T14:08:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6832/3661234838,v1:172.21.15.115:6833/3661234838] up:boot 2023-10-08T14:08:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.115:6836/1863828175,v1:172.21.15.115:6837/1863828175] up:boot 2023-10-08T14:08:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.99:6834/2690629133,v1:172.21.15.99:6836/2690629133] up:boot 2023-10-08T14:08:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.99:6835/3672736004,v1:172.21.15.99:6839/3672736004] up:boot 2023-10-08T14:08:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.99:6837/70019781,v1:172.21.15.99:6838/70019781] up:boot 2023-10-08T14:08:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: mds.? [v2:172.21.15.99:6832/1622303057,v1:172.21.15.99:6833/1622303057] up:boot 2023-10-08T14:08:12.938 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: fsmap 12 up:standby 2023-10-08T14:08:12.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T14:08:12.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T14:08:12.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T14:08:12.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T14:08:12.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T14:08:12.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T14:08:12.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T14:08:12.939 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T14:08:12.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T14:08:12.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T14:08:12.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/599974724' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:12.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:12 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:12.988 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-08T14:08:12.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-08T14:08:12.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: Cluster is now healthy 2023-10-08T14:08:12.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-08T14:08:12.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.57:6836/2921049017,v1:172.21.15.57:6837/2921049017] up:boot 2023-10-08T14:08:12.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.57:6840/3970825978,v1:172.21.15.57:6841/3970825978] up:boot 2023-10-08T14:08:12.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6838/2040651681,v1:172.21.15.115:6839/2040651681] up:boot 2023-10-08T14:08:12.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6834/3061265823,v1:172.21.15.115:6835/3061265823] up:boot 2023-10-08T14:08:12.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6832/3661234838,v1:172.21.15.115:6833/3661234838] up:boot 2023-10-08T14:08:12.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.115:6836/1863828175,v1:172.21.15.115:6837/1863828175] up:boot 2023-10-08T14:08:12.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.99:6834/2690629133,v1:172.21.15.99:6836/2690629133] up:boot 2023-10-08T14:08:12.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.99:6835/3672736004,v1:172.21.15.99:6839/3672736004] up:boot 2023-10-08T14:08:12.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.99:6837/70019781,v1:172.21.15.99:6838/70019781] up:boot 2023-10-08T14:08:12.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: mds.? [v2:172.21.15.99:6832/1622303057,v1:172.21.15.99:6833/1622303057] up:boot 2023-10-08T14:08:12.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: fsmap 12 up:standby 2023-10-08T14:08:12.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T14:08:12.990 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T14:08:12.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T14:08:12.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T14:08:12.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T14:08:12.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T14:08:12.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T14:08:12.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T14:08:12.991 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T14:08:12.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T14:08:12.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/599974724' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:12.992 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:12 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-08T14:08:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-08T14:08:13.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: Cluster is now healthy 2023-10-08T14:08:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-08T14:08:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.57:6836/2921049017,v1:172.21.15.57:6837/2921049017] up:boot 2023-10-08T14:08:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.57:6840/3970825978,v1:172.21.15.57:6841/3970825978] up:boot 2023-10-08T14:08:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6838/2040651681,v1:172.21.15.115:6839/2040651681] up:boot 2023-10-08T14:08:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6834/3061265823,v1:172.21.15.115:6835/3061265823] up:boot 2023-10-08T14:08:13.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6832/3661234838,v1:172.21.15.115:6833/3661234838] up:boot 2023-10-08T14:08:13.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.115:6836/1863828175,v1:172.21.15.115:6837/1863828175] up:boot 2023-10-08T14:08:13.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.99:6834/2690629133,v1:172.21.15.99:6836/2690629133] up:boot 2023-10-08T14:08:13.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.99:6835/3672736004,v1:172.21.15.99:6839/3672736004] up:boot 2023-10-08T14:08:13.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.99:6837/70019781,v1:172.21.15.99:6838/70019781] up:boot 2023-10-08T14:08:13.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: mds.? [v2:172.21.15.99:6832/1622303057,v1:172.21.15.99:6833/1622303057] up:boot 2023-10-08T14:08:13.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: fsmap 12 up:standby 2023-10-08T14:08:13.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T14:08:13.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T14:08:13.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T14:08:13.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T14:08:13.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T14:08:13.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T14:08:13.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T14:08:13.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T14:08:13.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T14:08:13.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='mgr.14217 172.21.15.57:0/2905222608' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T14:08:13.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/599974724' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:13.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:12 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:13.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.027+0000 7f087db12700 1 -- 172.21.15.57:0/881711977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0878119fd0 msgr2=0x7f087811c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:13.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.027+0000 7f087db12700 1 --2- 172.21.15.57:0/881711977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0878119fd0 0x7f087811c3c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f086c009990 tx=0x7f086c009ca0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.031+0000 7f087db12700 1 -- 172.21.15.57:0/881711977 shutdown_connections 2023-10-08T14:08:13.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.031+0000 7f087db12700 1 --2- 172.21.15.57:0/881711977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0878119fd0 0x7f087811c3c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.031+0000 7f087db12700 1 --2- 172.21.15.57:0/881711977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f087810ef80 0x7f08781077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.031+0000 7f087db12700 1 --2- 172.21.15.57:0/881711977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0878107d30 0x7f08781081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.031+0000 7f087db12700 1 -- 172.21.15.57:0/881711977 >> 172.21.15.57:0/881711977 conn(0x7f087806b830 msgr2=0x7f087806bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:13.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.034+0000 7f087db12700 1 -- 172.21.15.57:0/881711977 shutdown_connections 2023-10-08T14:08:13.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.034+0000 7f087db12700 1 -- 172.21.15.57:0/881711977 wait complete. 2023-10-08T14:08:13.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.035+0000 7f087db12700 1 Processor -- start 2023-10-08T14:08:13.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.035+0000 7f087db12700 1 -- start start 2023-10-08T14:08:13.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.036+0000 7f087db12700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0878107d30 0x7f08781ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:13.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.036+0000 7f087db12700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f087810ef80 0x7f08781abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:13.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.036+0000 7f087db12700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0878119fd0 0x7f08781a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:13.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.036+0000 7f087db12700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781a5f40 con 0x7f0878119fd0 2023-10-08T14:08:13.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.036+0000 7f087db12700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781a60b0 con 0x7f0878107d30 2023-10-08T14:08:13.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.036+0000 7f087db12700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781a6270 con 0x7f087810ef80 2023-10-08T14:08:13.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.036+0000 7f0877fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0878119fd0 0x7f08781a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:13.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.036+0000 7f0877fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0878119fd0 0x7f08781a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60888/0 (socket says 172.21.15.57:60888) 2023-10-08T14:08:13.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.036+0000 7f0877fff700 1 -- 172.21.15.57:0/1902305917 learned_addr learned my addr 172.21.15.57:0/1902305917 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:13.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.037+0000 7f08777fe700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0878107d30 0x7f08781ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:13.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.037+0000 7f0877fff700 1 -- 172.21.15.57:0/1902305917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f087810ef80 msgr2=0x7f08781abcf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T14:08:13.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.037+0000 7f0877fff700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f087810ef80 0x7f08781abcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.037+0000 7f0877fff700 1 -- 172.21.15.57:0/1902305917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0878107d30 msgr2=0x7f08781ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:13.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.037+0000 7f0877fff700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0878107d30 0x7f08781ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.037+0000 7f0877fff700 1 -- 172.21.15.57:0/1902305917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c0095f0 con 0x7f0878119fd0 2023-10-08T14:08:13.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.037+0000 7f0877fff700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0878119fd0 0x7f08781a58e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f086c005f50 tx=0x7f086c004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:13.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.037+0000 7f0874ff9700 1 -- 172.21.15.57:0/1902305917 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c01d070 con 0x7f0878119fd0 2023-10-08T14:08:13.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.038+0000 7f0874ff9700 1 -- 172.21.15.57:0/1902305917 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f086c003e90 con 0x7f0878119fd0 2023-10-08T14:08:13.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.038+0000 7f0874ff9700 1 -- 172.21.15.57:0/1902305917 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00fe40 con 0x7f0878119fd0 2023-10-08T14:08:13.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.038+0000 7f087db12700 1 -- 172.21.15.57:0/1902305917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08781a6490 con 0x7f0878119fd0 2023-10-08T14:08:13.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.038+0000 7f087db12700 1 -- 172.21.15.57:0/1902305917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087810f990 con 0x7f0878119fd0 2023-10-08T14:08:13.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.041+0000 7f0874ff9700 1 -- 172.21.15.57:0/1902305917 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f086c022420 con 0x7f0878119fd0 2023-10-08T14:08:13.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.042+0000 7f0874ff9700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08600c3810 0x7f08600c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:13.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.042+0000 7f0874ff9700 1 -- 172.21.15.57:0/1902305917 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9417+0+0 (secure 0 0 0) 0x7f086c0cf620 con 0x7f0878119fd0 2023-10-08T14:08:13.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.043+0000 7f08777fe700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08600c3810 0x7f08600c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:13.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.043+0000 7f087db12700 1 -- 172.21.15.57:0/1902305917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087806ccb0 con 0x7f0878119fd0 2023-10-08T14:08:13.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.052+0000 7f0874ff9700 1 -- 172.21.15.57:0/1902305917 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f086c0985f0 con 0x7f0878119fd0 2023-10-08T14:08:13.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.052+0000 7f08777fe700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08600c3810 0x7f08600c5cd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0870009f40 tx=0x7f08700094d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:13.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.431+0000 7f087db12700 1 -- 172.21.15.57:0/1902305917 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f08781a6da0 con 0x7f0878119fd0 2023-10-08T14:08:13.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.714+0000 7f0874ff9700 1 -- 172.21.15.57:0/1902305917 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) v1 ==== 169+0+0 (secure 0 0 0) 0x7f086c09bc10 con 0x7f0878119fd0 2023-10-08T14:08:13.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.717+0000 7f087db12700 1 -- 172.21.15.57:0/1902305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08600c3810 msgr2=0x7f08600c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:13.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.717+0000 7f087db12700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08600c3810 0x7f08600c5cd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0870009f40 tx=0x7f08700094d0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.717+0000 7f087db12700 1 -- 172.21.15.57:0/1902305917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0878119fd0 msgr2=0x7f08781a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:13.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.717+0000 7f087db12700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0878119fd0 0x7f08781a58e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f086c005f50 tx=0x7f086c004460 comp rx=0 tx=0).stop 2023-10-08T14:08:13.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.722+0000 7f087db12700 1 -- 172.21.15.57:0/1902305917 shutdown_connections 2023-10-08T14:08:13.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.722+0000 7f087db12700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0878119fd0 0x7f08781a58e0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.722+0000 7f087db12700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f08600c3810 0x7f08600c5cd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.722+0000 7f087db12700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0878107d30 0x7f08781ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.722+0000 7f087db12700 1 --2- 172.21.15.57:0/1902305917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f087810ef80 0x7f08781abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:13.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.722+0000 7f087db12700 1 -- 172.21.15.57:0/1902305917 >> 172.21.15.57:0/1902305917 conn(0x7f087806b830 msgr2=0x7f0878117fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:13.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.728+0000 7f087db12700 1 -- 172.21.15.57:0/1902305917 shutdown_connections 2023-10-08T14:08:13.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:13.733+0000 7f087db12700 1 -- 172.21.15.57:0/1902305917 wait complete. 2023-10-08T14:08:13.762 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2023-10-08T14:08:13.783 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-08T14:08:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:13 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T14:08:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:13 smithi057 ceph-mon[99476]: osdmap e116: 12 total, 12 up, 12 in 2023-10-08T14:08:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:13 smithi057 ceph-mon[99476]: pgmap v823: 97 pgs: 67 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 7 active+clean; 1.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T14:08:13.989 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:13 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1902305917' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:14.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:13 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T14:08:14.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:13 smithi099 ceph-mon[103849]: osdmap e116: 12 total, 12 up, 12 in 2023-10-08T14:08:14.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:13 smithi099 ceph-mon[103849]: pgmap v823: 97 pgs: 67 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 7 active+clean; 1.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T14:08:14.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:13 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1902305917' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:14.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.072+0000 7f4ffe547700 1 -- 172.21.15.57:0/3034019532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ff810d6c0 msgr2=0x7f4ff810daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:14.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.072+0000 7f4ffe547700 1 --2- 172.21.15.57:0/3034019532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ff810d6c0 0x7f4ff810daa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009990 tx=0x7f4fec009ca0 comp rx=0 tx=0).stop 2023-10-08T14:08:14.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.072+0000 7f4ffe547700 1 -- 172.21.15.57:0/3034019532 shutdown_connections 2023-10-08T14:08:14.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.072+0000 7f4ffe547700 1 --2- 172.21.15.57:0/3034019532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ff8108a40 0x7f4ff8114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:14.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.072+0000 7f4ffe547700 1 --2- 172.21.15.57:0/3034019532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ff810d6c0 0x7f4ff810daa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:14.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.072+0000 7f4ffe547700 1 --2- 172.21.15.57:0/3034019532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff8108080 0x7f4ff8108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:14.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.072+0000 7f4ffe547700 1 -- 172.21.15.57:0/3034019532 >> 172.21.15.57:0/3034019532 conn(0x7f4ff806bc80 msgr2=0x7f4ff806c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:14.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.073+0000 7f4ffe547700 1 -- 172.21.15.57:0/3034019532 shutdown_connections 2023-10-08T14:08:14.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.073+0000 7f4ffe547700 1 -- 172.21.15.57:0/3034019532 wait complete. 2023-10-08T14:08:14.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.073+0000 7f4ffe547700 1 Processor -- start 2023-10-08T14:08:14.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.073+0000 7f4ffe547700 1 -- start start 2023-10-08T14:08:14.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.074+0000 7f4ffe547700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ff8108080 0x7f4ff8115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:14.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.074+0000 7f4ffe547700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ff8108a40 0x7f4ff81160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:14.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.074+0000 7f4ffe547700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff810d6c0 0x7f4ff81165e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:14.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.074+0000 7f4ffe547700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8117b30 con 0x7f4ff8108a40 2023-10-08T14:08:14.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.074+0000 7f4ffe547700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff81b1d70 con 0x7f4ff810d6c0 2023-10-08T14:08:14.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.074+0000 7f4ffe547700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff81a3530 con 0x7f4ff8108080 2023-10-08T14:08:14.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.074+0000 7f4ffcae4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff810d6c0 0x7f4ff81165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:14.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.074+0000 7f4ffcae4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff810d6c0 0x7f4ff81165e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.57:34092/0 (socket says 172.21.15.57:34092) 2023-10-08T14:08:14.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.074+0000 7f4ffcae4700 1 -- 172.21.15.57:0/1623643577 learned_addr learned my addr 172.21.15.57:0/1623643577 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-08T14:08:14.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.074+0000 7f4ff7fff700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ff8108080 0x7f4ff8115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:14.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.075+0000 7f4ffcae4700 1 -- 172.21.15.57:0/1623643577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ff8108080 msgr2=0x7f4ff8115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:14.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.075+0000 7f4ffcae4700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ff8108080 0x7f4ff8115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:14.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.075+0000 7f4ffcae4700 1 -- 172.21.15.57:0/1623643577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ff8108a40 msgr2=0x7f4ff81160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:14.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.075+0000 7f4ffcae4700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ff8108a40 0x7f4ff81160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:14.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.075+0000 7f4ffcae4700 1 -- 172.21.15.57:0/1623643577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0095f0 con 0x7f4ff810d6c0 2023-10-08T14:08:14.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.085+0000 7f4ffcae4700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff810d6c0 0x7f4ff81165e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000ef30 tx=0x7f4ff000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:14.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.087+0000 7f4ff57fa700 1 -- 172.21.15.57:0/1623643577 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0018070 con 0x7f4ff810d6c0 2023-10-08T14:08:14.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.087+0000 7f4ff57fa700 1 -- 172.21.15.57:0/1623643577 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff000f040 con 0x7f4ff810d6c0 2023-10-08T14:08:14.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.087+0000 7f4ff57fa700 1 -- 172.21.15.57:0/1623643577 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000cd90 con 0x7f4ff810d6c0 2023-10-08T14:08:14.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.087+0000 7f4ffe547700 1 -- 172.21.15.57:0/1623643577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff81a37b0 con 0x7f4ff810d6c0 2023-10-08T14:08:14.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.087+0000 7f4ffe547700 1 -- 172.21.15.57:0/1623643577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff81a3c70 con 0x7f4ff810d6c0 2023-10-08T14:08:14.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.093+0000 7f4ffe547700 1 -- 172.21.15.57:0/1623643577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff810fb70 con 0x7f4ff810d6c0 2023-10-08T14:08:14.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.099+0000 7f4ff57fa700 1 -- 172.21.15.57:0/1623643577 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4ff0004ad0 con 0x7f4ff810d6c0 2023-10-08T14:08:14.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.099+0000 7f4ff57fa700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4fe00a17c0 0x7f4fe00a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T14:08:14.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.100+0000 7f4ff57fa700 1 -- 172.21.15.57:0/1623643577 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8887+0+0 (secure 0 0 0) 0x7f4ff00cd6d0 con 0x7f4ff810d6c0 2023-10-08T14:08:14.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.105+0000 7f4ff7fff700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4fe00a17c0 0x7f4fe00a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T14:08:14.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.105+0000 7f4ff7fff700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4fe00a17c0 0x7f4fe00a3c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4fec006010 tx=0x7f4fec00b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T14:08:14.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.107+0000 7f4ff57fa700 1 -- 172.21.15.57:0/1623643577 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4ff0096930 con 0x7f4ff810d6c0 2023-10-08T14:08:14.186 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:13 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T14:08:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:13 smithi115 ceph-mon[104408]: osdmap e116: 12 total, 12 up, 12 in 2023-10-08T14:08:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:13 smithi115 ceph-mon[104408]: pgmap v823: 97 pgs: 67 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 7 active+clean; 1.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T14:08:14.187 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:13 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1902305917' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:14.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.477+0000 7f4ffe547700 1 -- 172.21.15.57:0/1623643577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4ff806cca0 con 0x7f4ff810d6c0 2023-10-08T14:08:14.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.745+0000 7f4ff57fa700 1 -- 172.21.15.57:0/1623643577 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v118) v1 ==== 178+0+0 (secure 0 0 0) 0x7f4ff0099f50 con 0x7f4ff810d6c0 2023-10-08T14:08:14.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.755+0000 7f4ffe547700 1 -- 172.21.15.57:0/1623643577 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4fe00a17c0 msgr2=0x7f4fe00a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:14.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.755+0000 7f4ffe547700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4fe00a17c0 0x7f4fe00a3c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4fec006010 tx=0x7f4fec00b260 comp rx=0 tx=0).stop 2023-10-08T14:08:14.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.755+0000 7f4ffe547700 1 -- 172.21.15.57:0/1623643577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff810d6c0 msgr2=0x7f4ff81165e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T14:08:14.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.755+0000 7f4ffe547700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff810d6c0 0x7f4ff81165e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000ef30 tx=0x7f4ff000c3a0 comp rx=0 tx=0).stop 2023-10-08T14:08:14.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.756+0000 7f4ffe547700 1 -- 172.21.15.57:0/1623643577 shutdown_connections 2023-10-08T14:08:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.756+0000 7f4ffe547700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.57:6800/2547096577,v1:172.21.15.57:6801/2547096577] conn(0x7f4fe00a17c0 0x7f4fe00a3c80 secure :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4fec006010 tx=0x7f4fec00b260 comp rx=0 tx=0).stop 2023-10-08T14:08:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.756+0000 7f4ffe547700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff810d6c0 0x7f4ff81165e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.756+0000 7f4ffe547700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ff8108080 0x7f4ff8115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.756+0000 7f4ffe547700 1 --2- 172.21.15.57:0/1623643577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ff8108a40 0x7f4ff81160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T14:08:14.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.756+0000 7f4ffe547700 1 -- 172.21.15.57:0/1623643577 >> 172.21.15.57:0/1623643577 conn(0x7f4ff806bc80 msgr2=0x7f4ff8070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T14:08:14.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.767+0000 7f4ffe547700 1 -- 172.21.15.57:0/1623643577 shutdown_connections 2023-10-08T14:08:14.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-08T14:08:14.769+0000 7f4ffe547700 1 -- 172.21.15.57:0/1623643577 wait complete. 2023-10-08T14:08:14.801 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data_ec' removed 2023-10-08T14:08:14.807 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T14:08:14.898 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T14:08:14.964 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T14:08:15.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:14 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1902305917' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T14:08:15.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:14 smithi115 ceph-mon[104408]: osdmap e117: 12 total, 12 up, 12 in 2023-10-08T14:08:15.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:14 smithi115 ceph-mon[104408]: from='client.? 172.21.15.57:0/1623643577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:15.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:14 smithi115 ceph-mon[104408]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:15.015 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-08T14:08:15.015 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-08T14:08:15.021 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:14 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1902305917' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T14:08:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:14 smithi099 ceph-mon[103849]: osdmap e117: 12 total, 12 up, 12 in 2023-10-08T14:08:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:14 smithi099 ceph-mon[103849]: from='client.? 172.21.15.57:0/1623643577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:15.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:14 smithi099 ceph-mon[103849]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:15.034 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-08T14:08:15.055 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-08T14:08:15.082 INFO:tasks.cephadm:Stopping all daemons... 2023-10-08T14:08:15.083 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-08T14:08:15.083 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.a 2023-10-08T14:08:15.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1902305917' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T14:08:15.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:14 smithi057 ceph-mon[99476]: osdmap e117: 12 total, 12 up, 12 in 2023-10-08T14:08:15.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:14 smithi057 ceph-mon[99476]: from='client.? 172.21.15.57:0/1623643577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:15.108 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:14 smithi057 ceph-mon[99476]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T14:08:15.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:15 smithi057 systemd[1]: Stopping Ceph mon.a for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:15.877 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a[99452]: 2023-10-08T14:08:15.509+0000 7f3dd18ca700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:08:15.878 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a[99452]: 2023-10-08T14:08:15.509+0000 7f3dd18ca700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-08T14:08:16.222 INFO:journalctl@ceph.mon.a.smithi057.stdout:Oct 08 14:08:15 smithi057 bash[189632]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-a 2023-10-08T14:08:16.229 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.a.service' 2023-10-08T14:08:16.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:08:16.315 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-08T14:08:16.316 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-08T14:08:16.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.b 2023-10-08T14:08:16.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:16 smithi099 systemd[1]: Stopping Ceph mon.b for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:17.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:16 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-b[103824]: 2023-10-08T14:08:16.769+0000 7f2c1a625700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:08:17.022 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:16 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-b[103824]: 2023-10-08T14:08:16.769+0000 7f2c1a625700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-08T14:08:17.387 INFO:journalctl@ceph.mon.b.smithi099.stdout:Oct 08 14:08:17 smithi099 bash[147160]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-b 2023-10-08T14:08:17.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.b.service' 2023-10-08T14:08:17.445 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:08:17.445 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-08T14:08:17.445 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-08T14:08:17.445 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.c 2023-10-08T14:08:17.937 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:17 smithi115 systemd[1]: Stopping Ceph mon.c for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:18.207 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:17 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-c[104384]: 2023-10-08T14:08:17.935+0000 7f07fb2d1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:08:18.207 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:17 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-c[104384]: 2023-10-08T14:08:17.935+0000 7f07fb2d1700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-08T14:08:18.711 INFO:journalctl@ceph.mon.c.smithi115.stdout:Oct 08 14:08:18 smithi115 bash[147748]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mon-c 2023-10-08T14:08:18.716 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mon.c.service' 2023-10-08T14:08:18.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:08:18.823 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-08T14:08:18.823 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-08T14:08:18.823 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.x 2023-10-08T14:08:19.237 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:08:18 smithi057 systemd[1]: Stopping Ceph mgr.x for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:19.988 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:08:19 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T14:08:19.689+0000 7fe53336d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:08:19.988 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:08:19 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x[99727]: 2023-10-08T14:08:19.689+0000 7fe53336d700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-08T14:08:21.667 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.x.service' 2023-10-08T14:08:21.694 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:08:21 smithi057 bash[189782]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-x 2023-10-08T14:08:21.694 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:08:21 smithi057 systemd[1]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.x.service: Succeeded. 2023-10-08T14:08:21.694 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Oct 08 14:08:21 smithi057 systemd[1]: Stopped Ceph mgr.x for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T14:08:21.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:08:21.707 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-08T14:08:21.707 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-08T14:08:21.707 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.y 2023-10-08T14:08:22.022 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 14:08:21 smithi099 systemd[1]: Stopping Ceph mgr.y for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:22.442 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 14:08:22 smithi099 bash[147308]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-y 2023-10-08T14:08:22.623 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Oct 08 14:08:22 smithi099 systemd[1]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-08T14:08:22.627 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.y.service' 2023-10-08T14:08:22.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:08:22.664 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-08T14:08:22.665 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-08T14:08:22.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.z 2023-10-08T14:08:23.089 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 14:08:22 smithi115 systemd[1]: Stopping Ceph mgr.z for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:23.436 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 14:08:23 smithi115 bash[147897]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mgr-z 2023-10-08T14:08:23.436 INFO:journalctl@ceph.mgr.z.smithi115.stdout:Oct 08 14:08:23 smithi115 systemd[1]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-10-08T14:08:23.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mgr.z.service' 2023-10-08T14:08:23.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:08:23.508 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-08T14:08:23.508 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-08T14:08:23.509 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.0 2023-10-08T14:08:23.859 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 14:08:23 smithi057 systemd[1]: Stopping Ceph osd.0 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:24.237 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 14:08:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-0[124884]: 2023-10-08T14:08:23.858+0000 7f4f5c3ab700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:08:24.237 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 14:08:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-0[124884]: 2023-10-08T14:08:23.858+0000 7f4f5c3ab700 -1 osd.0 118 *** Got signal Terminated *** 2023-10-08T14:08:24.238 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 14:08:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-0[124884]: 2023-10-08T14:08:23.858+0000 7f4f5c3ab700 -1 osd.0 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:08:29.988 INFO:journalctl@ceph.osd.0.smithi057.stdout:Oct 08 14:08:29 smithi057 bash[189923]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-0 2023-10-08T14:08:31.231 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.0.service' 2023-10-08T14:08:31.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:08:31.274 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-08T14:08:31.274 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-08T14:08:31.274 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.1 2023-10-08T14:08:31.698 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 14:08:31 smithi099 systemd[1]: Stopping Ceph osd.1 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:32.022 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 14:08:31 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-1[110129]: 2023-10-08T14:08:31.696+0000 7f6f4ef40700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:08:32.022 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 14:08:31 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-1[110129]: 2023-10-08T14:08:31.696+0000 7f6f4ef40700 -1 osd.1 118 *** Got signal Terminated *** 2023-10-08T14:08:32.022 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 14:08:31 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-1[110129]: 2023-10-08T14:08:31.696+0000 7f6f4ef40700 -1 osd.1 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:08:37.522 INFO:journalctl@ceph.osd.1.smithi099.stdout:Oct 08 14:08:37 smithi099 bash[147456]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-1 2023-10-08T14:08:38.959 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.1.service' 2023-10-08T14:08:39.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:08:39.003 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-08T14:08:39.003 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-08T14:08:39.003 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.2 2023-10-08T14:08:39.436 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 14:08:39 smithi115 systemd[1]: Stopping Ceph osd.2 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:39.936 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 14:08:39 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-2[111508]: 2023-10-08T14:08:39.657+0000 7f8522f2c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:08:39.936 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 14:08:39 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-2[111508]: 2023-10-08T14:08:39.657+0000 7f8522f2c700 -1 osd.2 118 *** Got signal Terminated *** 2023-10-08T14:08:39.937 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 14:08:39 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-2[111508]: 2023-10-08T14:08:39.657+0000 7f8522f2c700 -1 osd.2 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:08:45.537 INFO:journalctl@ceph.osd.2.smithi115.stdout:Oct 08 14:08:45 smithi115 bash[148043]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-2 2023-10-08T14:08:46.996 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.2.service' 2023-10-08T14:08:47.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:08:47.039 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-08T14:08:47.039 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-08T14:08:47.040 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.3 2023-10-08T14:08:47.406 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 08 14:08:47 smithi057 systemd[1]: Stopping Ceph osd.3 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:47.737 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 08 14:08:47 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-3[132958]: 2023-10-08T14:08:47.404+0000 7ffb215d3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:08:47.737 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 08 14:08:47 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-3[132958]: 2023-10-08T14:08:47.404+0000 7ffb215d3700 -1 osd.3 118 *** Got signal Terminated *** 2023-10-08T14:08:47.737 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 08 14:08:47 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-3[132958]: 2023-10-08T14:08:47.404+0000 7ffb215d3700 -1 osd.3 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:08:50.987 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:50 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:50.498+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:08:51.186 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:08:50 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:08:50.910+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:08:51.477 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:08:51 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:08:51.064+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:08:51.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:08:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:08:51.327+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:08:51.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:51 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:51.476+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:08:51.738 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 08 14:08:51 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-3[132958]: 2023-10-08T14:08:51.486+0000 7ffb1bbc5700 -1 osd.3 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.842624+0000 front 2023-10-08T14:08:25.842782+0000 (oldest deadline 2023-10-08T14:08:50.542831+0000) 2023-10-08T14:08:52.186 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:08:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:08:51.922+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:08:52.440 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:08:52 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:08:52.032+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:08:52.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:08:52 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:08:52.320+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:08:52.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:52 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:52.438+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:08:52.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:08:52 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:08:52.499+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:08:53.186 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:08:52 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:08:52.924+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:08:53.313 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:08:53 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:08:53.010+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:08:53.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:08:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:08:53.318+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:08:53.737 INFO:journalctl@ceph.osd.3.smithi057.stdout:Oct 08 14:08:53 smithi057 bash[190197]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-3 2023-10-08T14:08:53.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:53 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:53.458+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:08:53.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:08:53 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:08:53.476+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:08:54.311 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:08:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:08:53.962+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:08:54.470 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:08:54 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:08:54.013+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:08:54.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:08:54 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:08:54.310+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:08:54.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:54 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:54.481+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:08:54.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:08:54 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:08:54.444+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:08:55.187 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:08:54 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:08:54.931+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:08:55.187 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:08:55 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:08:55.129+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:08:55.453 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:08:55 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:08:55.016+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:08:55.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:08:55 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:08:55.292+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:08:55.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:55 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:55.452+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:08:55.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:08:55 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:08:55.450+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:08:56.305 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:08:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:08:56.148+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:08:56.306 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:08:55 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:08:55.947+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:08:56.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:08:56 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:08:56.018+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:08:56.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:08:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:08:56.305+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:08:56.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:08:56 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:08:56.446+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:08:56.806 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:56 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:56.495+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:08:56.815 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.3.service' 2023-10-08T14:08:56.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:08:56.864 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-08T14:08:56.864 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-08T14:08:56.865 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.4 2023-10-08T14:08:57.186 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:08:57 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:08:57.150+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:08:57.187 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:08:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:08:56.917+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:08:57.233 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 14:08:56 smithi099 systemd[1]: Stopping Ceph osd.4 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:08:57.463 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:08:57 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:08:57.060+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:08:57.522 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:08:57 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:08:57.477+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:08:57.522 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 14:08:57 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4[118653]: 2023-10-08T14:08:57.232+0000 7f1d85161700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:08:57.522 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 14:08:57 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4[118653]: 2023-10-08T14:08:57.232+0000 7f1d85161700 -1 osd.4 118 *** Got signal Terminated *** 2023-10-08T14:08:57.523 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 14:08:57 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4[118653]: 2023-10-08T14:08:57.232+0000 7f1d85161700 -1 osd.4 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:08:57.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:08:57 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:08:57.293+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:08:57.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:57 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:57.463+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:08:58.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:08:57 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:08:57.625+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:08:58.173 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:08:57 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:08:57.886+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:08:58.431 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:08:58 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:08:58.060+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:08:58.436 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:08:58 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:08:58.253+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:08:58.436 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:08:58 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:08:58.172+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:08:58.436 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:08:58 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:08:58.172+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:08:58.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:58 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:58.430+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:08:58.747 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:08:58 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:08:58.638+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:08:58.747 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:08:58 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:08:58.476+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:08:59.022 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 14:08:58 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4[118653]: 2023-10-08T14:08:58.746+0000 7f1d7f753700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.036468+0000 front 2023-10-08T14:08:34.036440+0000 (oldest deadline 2023-10-08T14:08:58.136648+0000) 2023-10-08T14:08:59.186 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:08:58 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:08:58.918+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:08:59.187 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:08:59 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:08:59.164+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:08:59.187 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:08:59 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:08:59.164+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:08:59.454 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:08:59 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:08:59.102+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:08:59.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:08:59 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:08:59.246+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:08:59.733 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:08:59 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:08:59.469+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:08:59.734 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:08:59 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:08:59.657+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:08:59.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:59 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:59.453+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:08:59.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:08:59 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:08:59.453+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:00.022 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 14:08:59 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4[118653]: 2023-10-08T14:08:59.732+0000 7f1d7f753700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.036468+0000 front 2023-10-08T14:08:34.036440+0000 (oldest deadline 2023-10-08T14:08:58.136648+0000) 2023-10-08T14:09:00.247 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:00 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:00.146+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:00.248 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:00 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:00.147+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:00.248 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:08:59 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:08:59.958+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:09:00.413 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:00.071+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:00.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:00 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:00.246+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:00.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:00.411+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:00.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:00 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:00.411+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:00.772 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 14:09:00 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4[118653]: 2023-10-08T14:09:00.714+0000 7f1d7f753700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.036468+0000 front 2023-10-08T14:08:34.036440+0000 (oldest deadline 2023-10-08T14:08:58.136648+0000) 2023-10-08T14:09:00.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:00 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:00.676+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:00.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:00 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:00.514+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:01.296 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:01 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:01.121+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:01.296 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:01 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:01.121+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:01.297 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:00 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:00.968+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:09:01.424 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:01 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:01.038+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:01.424 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:01 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:01.038+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:01.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:01 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:01.296+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:01.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:01 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:01.423+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:01.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:01 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:01.423+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:01.772 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 14:09:01 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4[118653]: 2023-10-08T14:09:01.723+0000 7f1d7f753700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.036468+0000 front 2023-10-08T14:08:34.036440+0000 (oldest deadline 2023-10-08T14:08:58.136648+0000) 2023-10-08T14:09:01.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:01 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:01.686+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:01.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:01 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:01.488+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:02.304 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:02 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:02.122+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:02.304 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:02 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:02.122+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:02.305 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:02 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:02.017+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:09:02.305 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:02 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:02.017+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.346402+0000 front 2023-10-08T14:08:36.346481+0000 (oldest deadline 2023-10-08T14:09:01.547147+0000) 2023-10-08T14:09:02.441 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:02 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:02.026+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:02.442 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:02 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:02.026+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:02.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:02 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:02.304+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:02.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:02 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:02.441+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:02.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:02 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:02.441+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:02.924 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:02 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:02.704+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:02.924 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:02 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:02.534+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:02.925 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:02 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:02.534+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:03.208 INFO:journalctl@ceph.osd.4.smithi099.stdout:Oct 08 14:09:03 smithi099 bash[147721]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-4 2023-10-08T14:09:03.290 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:03 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:03.154+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:03.291 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:03 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:03.154+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:03.291 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:02 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:02.982+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:09:03.291 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:02 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:02.982+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.346402+0000 front 2023-10-08T14:08:36.346481+0000 (oldest deadline 2023-10-08T14:09:01.547147+0000) 2023-10-08T14:09:03.422 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:03 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:03.069+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:03.423 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:03 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:03.069+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:03.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:03 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:03.290+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:03.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:03 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:03.422+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:03.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:03 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:03.422+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:03.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:03 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:03.720+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:03.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:03 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:03.509+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:03.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:03 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:03.509+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:04.285 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:04 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:04.135+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:04.285 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:04 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:04.135+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:04.285 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:03 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:03.980+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:09:04.285 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:03 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:03.980+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.346402+0000 front 2023-10-08T14:08:36.346481+0000 (oldest deadline 2023-10-08T14:09:01.547147+0000) 2023-10-08T14:09:04.409 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:04 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:04.114+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:04.409 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:04 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:04.115+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:04.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.4.service' 2023-10-08T14:09:04.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:09:04.508 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-08T14:09:04.508 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-08T14:09:04.508 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.5 2023-10-08T14:09:04.538 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:04 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:04.283+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:04.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:04 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:04.408+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:04.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:04 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:04.408+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:04.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:04 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:04.687+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:04.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:04 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:04.476+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:04.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:04 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:04.476+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:04.861 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:04 smithi115 systemd[1]: Stopping Ceph osd.5 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:09:05.170 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:04 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:04.860+0000 7f5921ac6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:09:05.170 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:04 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:04.860+0000 7f5921ac6700 -1 osd.5 118 *** Got signal Terminated *** 2023-10-08T14:09:05.170 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:04 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:04.860+0000 7f5921ac6700 -1 osd.5 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:09:05.170 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:04 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:04.994+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:09:05.170 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:04 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:04.994+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.346402+0000 front 2023-10-08T14:08:36.346481+0000 (oldest deadline 2023-10-08T14:09:01.547147+0000) 2023-10-08T14:09:05.361 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:05 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:05.107+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:05.361 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:05 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:05.107+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:05.436 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:05 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:05.322+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:05.436 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:05 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:05.169+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:05.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:05 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:05.169+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:05.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:05 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:05.360+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:05.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:05 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:05.360+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:05.773 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:05 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:05.659+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:05.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:05 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:05.520+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:05.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:05 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:05.520+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:06.332 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:06 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:06.216+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:06.332 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:06 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:06.216+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:06.332 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:06 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:06.029+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:09:06.332 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:06 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:06.029+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.346402+0000 front 2023-10-08T14:08:36.346481+0000 (oldest deadline 2023-10-08T14:09:01.547147+0000) 2023-10-08T14:09:06.403 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:06.107+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:06.403 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:06.107+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:06.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:06 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:06.331+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:06.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:06.402+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:06.738 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:06 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:06.402+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:07.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:06 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:06.648+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:07.022 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:06 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:06.526+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:07.023 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:06 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:06.526+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:07.302 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:07 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:07.033+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:09:07.302 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:07 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:07.033+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.346402+0000 front 2023-10-08T14:08:36.346481+0000 (oldest deadline 2023-10-08T14:09:01.547147+0000) 2023-10-08T14:09:07.302 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:07 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:07.179+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:07.302 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:07 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:07.179+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:07.302 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:07 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:07.179+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:07.439 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:07.094+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:07.439 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:07.094+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:07.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:07 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:07.301+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:07.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:07.438+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:07.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:07.438+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:07.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:07 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:07.694+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:07.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:07 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:07.479+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:07.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:07 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:07.479+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:07.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:07 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:07.479+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:08.437 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:08 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:08.277+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:08.437 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:08 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:08.277+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:08.437 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:08 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:08.053+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:09:08.438 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:08 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:08.053+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.346402+0000 front 2023-10-08T14:08:36.346481+0000 (oldest deadline 2023-10-08T14:09:01.547147+0000) 2023-10-08T14:09:08.438 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:08 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:08.053+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.547798+0000 front 2023-10-08T14:08:41.547775+0000 (oldest deadline 2023-10-08T14:09:07.447854+0000) 2023-10-08T14:09:08.438 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:08 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:08.215+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:08.438 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:08 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:08.215+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:08.439 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:08 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:08.215+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:08.478 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:08 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:08.085+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:08.478 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:08 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:08.085+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:08.478 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:08 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:08.085+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:08.705 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:08 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:08.452+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:08.705 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:08 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:08.452+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:08.706 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:08 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:08.452+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:08.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:08 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:08.477+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:08.738 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:08 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:08.477+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:09.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:08 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:08.704+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:09.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:08 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:08.704+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:09.312 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:09 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:09.242+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:09.459 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:09 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:09.242+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:09.459 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:09 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:09.242+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:09.460 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:09 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:09.053+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.843495+0000 front 2023-10-08T14:08:27.843600+0000 (oldest deadline 2023-10-08T14:08:50.143617+0000) 2023-10-08T14:09:09.460 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:09 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:09.053+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.346402+0000 front 2023-10-08T14:08:36.346481+0000 (oldest deadline 2023-10-08T14:09:01.547147+0000) 2023-10-08T14:09:09.460 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:09 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5[119382]: 2023-10-08T14:09:09.053+0000 7f591a8b5700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.547798+0000 front 2023-10-08T14:08:41.547775+0000 (oldest deadline 2023-10-08T14:09:07.447854+0000) 2023-10-08T14:09:09.461 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:09.040+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:09.461 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:09.040+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:09.461 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:09.040+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:09.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:09 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:09.312+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:09.972 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:09 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:09.312+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:09.974 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:09 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:09.677+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:09.974 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:09 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:09.677+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:09.974 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:09.455+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:09.974 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:09.455+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:09.975 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:09 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:09.473+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:09.975 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:09 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:09.473+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:09.975 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:09 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:09.473+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:10.465 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:09.996+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:10.466 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:09.996+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:10.466 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:09 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:09.996+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:10.487 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:10.307+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:10.487 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:10.307+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:10.487 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:10.215+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:10.487 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:10.215+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:10.487 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:10 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:10.215+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:10.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:10.465+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:10.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:10.465+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:10.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:10 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:10.465+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:44.291244+0000 front 2023-10-08T14:08:44.291231+0000 (oldest deadline 2023-10-08T14:09:10.191012+0000) 2023-10-08T14:09:10.743 INFO:journalctl@ceph.osd.5.smithi115.stdout:Oct 08 14:09:10 smithi115 bash[148304]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-5 2023-10-08T14:09:10.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:10.654+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:10.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:10.654+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:10.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:10.436+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:10.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:10.436+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:10.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:10 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:10.436+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:11.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:11.006+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:11.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:11.006+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:11.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:11.006+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:11.679 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:11 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:11.294+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:11.680 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:11 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:11.294+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:11.680 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:11 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:11.199+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:11.680 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:11 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:11.199+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:11.680 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:11 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:11.199+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:11.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:11 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:11.400+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:11.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:11 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:11.400+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:11.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:11 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:11.400+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:11.773 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:11 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:11.624+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:11.773 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:11 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:11.624+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:11.980 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:11.509+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:11.980 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:11.509+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:11.980 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:11.509+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:44.291244+0000 front 2023-10-08T14:08:44.291231+0000 (oldest deadline 2023-10-08T14:09:10.191012+0000) 2023-10-08T14:09:12.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:11.979+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:12.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:11.979+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:12.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:11.979+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:12.437 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:12 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:12.249+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:12.437 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:12 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:12.249+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:12.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:12 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:12.152+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:12.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:12 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:12.153+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:12.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:12 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:12.153+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:12.657 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:12 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:12.359+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:12.657 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:12 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:12.359+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:12.657 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:12 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:12.359+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:12.987 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:12 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:12.498+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:12.987 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:12 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:12.498+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:12.987 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:12 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:12.498+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:44.291244+0000 front 2023-10-08T14:08:44.291231+0000 (oldest deadline 2023-10-08T14:09:10.191012+0000) 2023-10-08T14:09:13.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:12 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:12.655+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:13.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:12 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:12.655+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:13.112 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.5.service' 2023-10-08T14:09:13.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:09:13.201 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-08T14:09:13.201 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-08T14:09:13.201 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.6 2023-10-08T14:09:13.436 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:13 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:13.277+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:13.437 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:13 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:13.277+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:13.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:13 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:13.183+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:13.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:13 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:13.183+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:13.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:13 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:13.183+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:13.487 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:13 smithi057 systemd[1]: Stopping Ceph osd.6 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:09:13.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:13.006+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:13.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:13.006+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:13.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:13.006+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:13.675 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:13 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:13.328+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:13.676 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:13 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:13.328+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:13.676 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:13 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:13.328+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:13.963 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:13.493+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:13.963 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:13.493+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:13.963 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:13.493+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:44.291244+0000 front 2023-10-08T14:08:44.291231+0000 (oldest deadline 2023-10-08T14:09:10.191012+0000) 2023-10-08T14:09:13.963 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:13.550+0000 7f4ff5e95700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:09:13.963 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:13.550+0000 7f4ff5e95700 -1 osd.6 118 *** Got signal Terminated *** 2023-10-08T14:09:13.964 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:13.550+0000 7f4ff5e95700 -1 osd.6 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:09:14.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:13 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:13.674+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:14.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:13 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:13.674+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:14.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:13.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:14.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:13.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:14.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:13.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:14.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:13 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:13.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:14.631 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:14 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:14.302+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:14.631 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:14 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:14.302+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:14.631 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:14 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:14.302+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:14.631 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:14 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:14.302+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:14.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:14 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:14.249+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:14.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:14 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:14.249+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:14.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:14 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:14.229+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:14.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:14 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:14.229+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:14.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:14 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:14.229+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:14.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:14.469+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:14.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:14.469+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:14.738 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:14.469+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:44.291244+0000 front 2023-10-08T14:08:44.291231+0000 (oldest deadline 2023-10-08T14:09:10.191012+0000) 2023-10-08T14:09:15.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:14 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:14.629+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:15.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:14 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:14.629+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:15.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:14.960+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:15.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:14.960+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:15.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:14.960+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:15.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:14 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:14.960+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:15.658 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:15 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:15.325+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:15.659 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:15 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:15.325+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:15.659 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:15 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:15.325+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:15.659 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:15 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:15.325+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:15.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:15 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:15.279+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:15.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:15 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:15.279+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:15.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:15 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:15.279+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:15.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:15 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:15.243+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:15.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:15 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:15.243+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:15.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:15 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:15.244+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:15.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:15.462+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:15.738 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:15.462+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:15.738 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:15.462+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:44.291244+0000 front 2023-10-08T14:08:44.291231+0000 (oldest deadline 2023-10-08T14:09:10.191012+0000) 2023-10-08T14:09:16.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:15 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:15.657+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:16.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:15 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:15.657+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:16.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:15.913+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:16.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:15.913+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:16.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:15.913+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:16.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:15 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:15.913+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:16.657 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:16 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:16.307+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:16.658 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:16 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:16.307+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:16.658 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:16 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:16.307+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:16.658 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:16 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:16.307+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:16.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:16 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:16.311+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:16.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:16 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:16.311+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:16.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:16 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:16.311+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:16.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:16 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:16.268+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:16.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:16 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:16.268+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:16.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:16 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:16.268+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:16.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:16.479+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:16.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:16.479+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:16.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:16.479+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:44.291244+0000 front 2023-10-08T14:08:44.291231+0000 (oldest deadline 2023-10-08T14:09:10.191012+0000) 2023-10-08T14:09:17.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:16 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:16.656+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:17.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:16 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:16.656+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:17.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:16.934+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:17.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:16.934+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:17.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:16.934+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:17.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:16 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:16.934+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:17.611 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:17 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:17.355+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:17.612 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:17 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:17.355+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:17.612 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:17 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:17.355+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:17.612 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:17 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:17.355+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:17.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:17 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:17.294+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:17.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:17 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:17.294+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:17.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:17 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:17.294+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:17.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:17 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:17.322+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:17.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:17 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:17.322+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:17.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:17 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:17.322+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:17.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:17 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:17.431+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:17.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:17 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:17.431+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:17.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:17 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:17.431+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:44.291244+0000 front 2023-10-08T14:08:44.291231+0000 (oldest deadline 2023-10-08T14:09:10.191012+0000) 2023-10-08T14:09:18.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:17 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:17.610+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:18.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:17 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:17.610+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:18.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:17 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:17.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:18.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:17 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:17.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:18.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:17 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:17.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:18.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:17 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:17.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:18.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:18 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:18.313+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:18.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:18 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:18.313+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:18.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:18 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:18.313+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:18.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:18 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:18.314+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:18.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:18 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:18.314+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:18.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:18 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:18.314+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:18.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:18.429+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.187764+0000 front 2023-10-08T14:08:26.187744+0000 (oldest deadline 2023-10-08T14:08:50.287847+0000) 2023-10-08T14:09:18.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:18.429+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.189097+0000 front 2023-10-08T14:08:36.189062+0000 (oldest deadline 2023-10-08T14:08:59.089046+0000) 2023-10-08T14:09:18.738 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:18.429+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:44.291244+0000 front 2023-10-08T14:08:44.291231+0000 (oldest deadline 2023-10-08T14:09:10.191012+0000) 2023-10-08T14:09:18.738 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6[140885]: 2023-10-08T14:09:18.429+0000 7f4fee483700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:52.293472+0000 front 2023-10-08T14:08:52.293496+0000 (oldest deadline 2023-10-08T14:09:18.193647+0000) 2023-10-08T14:09:18.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:18 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:18.631+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:18.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:18 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:18.631+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:18.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:18 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:18.631+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:51.784319+0000 front 2023-10-08T14:08:51.784525+0000 (oldest deadline 2023-10-08T14:09:17.684422+0000) 2023-10-08T14:09:18.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:18 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:18.399+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:18.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:18 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:18.399+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:18.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:18 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:18.399+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:18.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:18 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:18.399+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:19.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:18.946+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:19.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:18.946+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:19.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:18.946+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:19.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:18 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:18.946+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:19.613 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:19 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:19.353+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:19.614 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:19 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:19.353+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:19.614 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:19 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:19.353+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:19.614 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:19 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:19.353+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:19.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:19 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:19.336+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:19.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:19 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:19.336+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:19.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:19 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:19.336+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:19.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:19 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:19.282+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:19.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:19 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:19.282+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:19.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:19 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:19.282+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:19.737 INFO:journalctl@ceph.osd.6.smithi057.stdout:Oct 08 14:09:19 smithi057 bash[190473]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-6 2023-10-08T14:09:20.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:19 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:19.613+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:20.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:19 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:19.613+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:20.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:19 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:19.613+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:51.784319+0000 front 2023-10-08T14:08:51.784525+0000 (oldest deadline 2023-10-08T14:09:17.684422+0000) 2023-10-08T14:09:20.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:19 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:19.937+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:20.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:19 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:19.937+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:20.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:19 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:19.937+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:20.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:19 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:19.937+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:20.615 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:20 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:20.341+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:20.615 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:20 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:20.341+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:20.615 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:20 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:20.341+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:20.615 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:20 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:20.341+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:20.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:20 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:20.300+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:20.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:20 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:20.300+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:20.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:20 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:20.300+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:20.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:20 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:20.259+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:20.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:20 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:20.259+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:20.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:20 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:20.259+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:20.911 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.6.service' 2023-10-08T14:09:20.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:09:20.956 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-08T14:09:20.956 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-08T14:09:20.956 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.7 2023-10-08T14:09:20.987 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:20 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:20.611+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:20.987 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:20 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:20.611+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:20.987 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:20 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:20.611+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:51.784319+0000 front 2023-10-08T14:08:51.784525+0000 (oldest deadline 2023-10-08T14:09:17.684422+0000) 2023-10-08T14:09:21.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:20 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:20.971+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:21.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:20 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:20.971+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:21.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:20 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:20.971+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:21.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:20 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:20.971+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:21.272 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:20 smithi099 systemd[1]: Stopping Ceph osd.7 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:09:21.588 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:21 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:21.327+0000 7f4eaa305700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:09:21.588 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:21 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:21.327+0000 7f4eaa305700 -1 osd.7 118 *** Got signal Terminated *** 2023-10-08T14:09:21.588 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:21 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:21.327+0000 7f4eaa305700 -1 osd.7 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:09:21.588 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:21 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:21.587+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:21.588 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:21 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:21.346+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:21.588 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:21 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:21.346+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:21.588 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:21 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:21.346+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:21.588 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:21 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:21.346+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:21.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:21 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:21.270+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:21.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:21 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:21.270+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:21.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:21 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:21.270+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:21.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:21 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:21.238+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:21.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:21 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:21.238+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:21.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:21 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:21.238+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:22.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:21 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:21.587+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:22.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:21 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:21.587+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:51.784319+0000 front 2023-10-08T14:08:51.784525+0000 (oldest deadline 2023-10-08T14:09:17.684422+0000) 2023-10-08T14:09:22.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:21 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:21.974+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:22.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:21 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:21.974+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:22.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:21 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:21.974+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:22.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:21 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:21.974+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:22.632 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:22 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:22.355+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:22.633 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:22 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:22.355+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:22.633 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:22 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:22.355+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:22.633 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:22 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:22.355+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:22.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:22.313+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:22.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:22.313+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:22.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:22.313+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:22.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:22.261+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:22.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:22.261+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:22.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:22 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:22.261+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:23.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:22 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:22.631+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:23.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:22 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:22.631+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:23.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:22 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:22.631+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:51.784319+0000 front 2023-10-08T14:08:51.784525+0000 (oldest deadline 2023-10-08T14:09:17.684422+0000) 2023-10-08T14:09:23.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:23.009+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:23.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:23.009+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:23.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:23.009+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:23.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:23.009+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:23.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:23 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:23.333+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:23.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:23 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:23.333+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:23.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:23 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:23.333+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:23.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:23 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:23.295+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:23.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:23 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:23.295+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:23.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:23 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:23.295+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:23.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:23.386+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:23.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:23.386+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:23.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:23.386+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:23.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:23.386+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:23.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:23.611+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:23.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:23.611+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:23.773 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:23.611+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:51.784319+0000 front 2023-10-08T14:08:51.784525+0000 (oldest deadline 2023-10-08T14:09:17.684422+0000) 2023-10-08T14:09:23.773 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:23.611+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:57.684882+0000 front 2023-10-08T14:08:57.684833+0000 (oldest deadline 2023-10-08T14:09:22.985102+0000) 2023-10-08T14:09:24.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:23.963+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:24.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:23.963+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:24.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:23.963+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:24.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:23 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:23.963+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:24.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:24 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:24.360+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:24.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:24 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:24.360+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:24.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:24 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:24.360+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:24.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:24 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:24.269+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:24.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:24 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:24.269+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:24.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:24 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:24.269+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:24.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:24.647+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:24.772 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:24.647+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:24.773 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:24.647+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:51.784319+0000 front 2023-10-08T14:08:51.784525+0000 (oldest deadline 2023-10-08T14:09:17.684422+0000) 2023-10-08T14:09:24.773 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:24.647+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:57.684882+0000 front 2023-10-08T14:08:57.684833+0000 (oldest deadline 2023-10-08T14:09:22.985102+0000) 2023-10-08T14:09:24.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:24.406+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:24.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:24.406+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:24.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:24.406+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:24.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:24.406+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:24.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:24 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:24.406+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:25.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:24 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:24.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:25.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:24 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:24.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:25.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:24 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:24.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:25.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:24 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:24.962+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:25.614 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:25.361+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:25.614 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:25.361+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:25.614 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:25.361+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:25.614 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:25.361+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:25.614 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:25.361+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:25.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:25.250+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:25.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:25.250+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:25.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:25.250+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:25.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:25.358+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:25.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:25.358+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:25.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:25.359+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:25.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:25 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:25.359+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:01.071452+0000 front 2023-10-08T14:09:01.071306+0000 (oldest deadline 2023-10-08T14:09:24.571244+0000) 2023-10-08T14:09:26.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:25.613+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.280534+0000 front 2023-10-08T14:08:34.280604+0000 (oldest deadline 2023-10-08T14:08:57.180795+0000) 2023-10-08T14:09:26.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:25.613+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.482274+0000 front 2023-10-08T14:08:42.482119+0000 (oldest deadline 2023-10-08T14:09:08.382370+0000) 2023-10-08T14:09:26.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:25.613+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:51.784319+0000 front 2023-10-08T14:08:51.784525+0000 (oldest deadline 2023-10-08T14:09:17.684422+0000) 2023-10-08T14:09:26.022 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:25 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7[126559]: 2023-10-08T14:09:25.613+0000 7f4ea60fa700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:57.684882+0000 front 2023-10-08T14:08:57.684833+0000 (oldest deadline 2023-10-08T14:09:22.985102+0000) 2023-10-08T14:09:26.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:25 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:25.916+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:26.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:25 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:25.916+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:26.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:25 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:25.916+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:26.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:25 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:25.916+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:26.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:25 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:25.916+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:26.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:26 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:26.372+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:26.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:26 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:26.372+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:26.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:26 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:26.372+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:26.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:26 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:26.372+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:01.071452+0000 front 2023-10-08T14:09:01.071306+0000 (oldest deadline 2023-10-08T14:09:24.571244+0000) 2023-10-08T14:09:26.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:26 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:26.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:26.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:26 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:26.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:26.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:26 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:26.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:26.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:26 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:26.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:26.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:26 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:26.336+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:26.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:26 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:26.336+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:26.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:26 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:26.336+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:26.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:26 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:26.336+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:26.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:26 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:26.336+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:27.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:26 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:26.920+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:27.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:26 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:26.920+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:27.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:26 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:26.920+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:27.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:26 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:26.920+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:27.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:26 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:26.920+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:27.343 INFO:journalctl@ceph.osd.7.smithi099.stdout:Oct 08 14:09:27 smithi099 bash[147997]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-7 2023-10-08T14:09:27.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:27.350+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:27.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:27.350+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:27.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:27.350+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:27.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:27.350+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:01.071452+0000 front 2023-10-08T14:09:01.071306+0000 (oldest deadline 2023-10-08T14:09:24.571244+0000) 2023-10-08T14:09:27.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:27.292+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:27.689 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:27.292+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:27.689 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:27.292+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:27.689 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:27 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:27.292+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:27.771 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:27.342+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:27.771 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:27.342+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:27.771 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:27.342+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:27.771 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:27.342+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:27.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:27.342+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:28.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:27 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:27.951+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:28.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:27 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:27.951+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:28.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:27 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:27.951+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:28.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:27 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:27.951+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:28.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:27 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:27.951+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:28.492 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:28 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:28.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:28.492 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:28 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:28.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:28.492 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:28 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:28.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:28.492 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:28 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:28.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:28.492 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:28 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:28.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:28.498 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.7.service' 2023-10-08T14:09:28.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:09:28.579 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-08T14:09:28.580 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-08T14:09:28.580 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.8 2023-10-08T14:09:28.612 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:28.396+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:28.612 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:28.396+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:28.612 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:28.396+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:28.612 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:28.396+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:01.071452+0000 front 2023-10-08T14:09:01.071306+0000 (oldest deadline 2023-10-08T14:09:24.571244+0000) 2023-10-08T14:09:28.612 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:28.251+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:28.612 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:28.251+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:28.612 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:28.251+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:28.613 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:28.251+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:28.930 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:28 smithi115 systemd[1]: Stopping Ceph osd.8 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:09:28.931 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:28.929+0000 7f61cc02c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:09:29.186 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:28.929+0000 7f61cc02c700 -1 osd.8 118 *** Got signal Terminated *** 2023-10-08T14:09:29.186 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:28 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:28.929+0000 7f61cc02c700 -1 osd.8 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:09:29.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:28 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:28.963+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:29.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:28 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:28.963+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:29.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:28 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:28.963+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:29.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:28 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:28.963+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:29.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:28 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:28.963+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:29.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:29 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:29.426+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:29.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:29 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:29.426+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:29.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:29 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:29.426+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:29.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:29 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:29.426+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:01.071452+0000 front 2023-10-08T14:09:01.071306+0000 (oldest deadline 2023-10-08T14:09:24.571244+0000) 2023-10-08T14:09:29.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:29 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:29.292+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:29.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:29 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:29.292+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:29.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:29 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:29.292+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:29.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:29 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:29.292+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:29.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:29.384+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:29.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:29.384+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:29.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:29.384+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:29.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:29.384+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:29.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:29 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:29.384+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:30.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:29 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:29.981+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:30.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:29 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:29.981+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:30.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:29 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:29.981+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:30.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:29 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:29.981+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:30.238 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:29 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:29.981+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:30.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:30 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:30.427+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:30.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:30 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:30.427+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:30.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:30 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:30.427+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:30.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:30 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:30.427+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:01.071452+0000 front 2023-10-08T14:09:01.071306+0000 (oldest deadline 2023-10-08T14:09:24.571244+0000) 2023-10-08T14:09:30.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:30 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:30.301+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:30.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:30 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:30.302+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:30.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:30 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:30.302+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:30.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:30 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:30.302+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:30.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:30 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:30.432+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:30.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:30 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:30.432+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:30.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:30 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:30.432+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:30.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:30 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:30.432+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:30.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:30 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:30.432+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:31.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:31 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:31.011+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:31.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:31 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:31.011+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:31.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:31 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:31.011+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:31.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:31 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:31.011+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:31.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:31 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:31.011+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:31.686 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:31.426+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:31.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:31.426+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:31.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:31.426+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:31.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:31.426+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:01.071452+0000 front 2023-10-08T14:09:01.071306+0000 (oldest deadline 2023-10-08T14:09:24.571244+0000) 2023-10-08T14:09:31.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:31.278+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:31.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:31.278+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:31.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:31.278+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:31.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:31.278+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:31.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:31.278+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:31.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:31 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:31.412+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:31.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:31 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:31.412+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:31.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:31 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:31.412+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:31.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:31 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:31.412+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:31.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:31 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:31.412+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:32.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:32 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:32.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:32.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:32 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:32.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:32.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:32 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:32.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:32.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:32 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:32.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:32.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:32 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:32.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:32.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:32 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:32.450+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:32.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:32 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:32.450+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:32.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:32 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:32.450+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:32.687 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:32 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:32.450+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:01.071452+0000 front 2023-10-08T14:09:01.071306+0000 (oldest deadline 2023-10-08T14:09:24.571244+0000) 2023-10-08T14:09:32.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:32 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:32.230+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:32.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:32 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:32.230+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:32.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:32 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:32.230+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:32.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:32 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:32.230+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:32.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:32 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:32.230+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:32.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:32 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:32.418+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:32.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:32 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:32.418+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:32.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:32 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:32.418+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:32.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:32 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:32.418+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:32.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:32 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:32.418+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:33.470 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:33 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:33.196+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:33.471 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:33 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:33.196+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:33.471 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:33 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:33.196+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:33.471 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:33 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:33.196+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:33.471 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:33 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:33.196+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:33.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:33 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:33.003+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:33.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:33 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:33.003+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:33.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:33 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:33.003+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:33.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:33 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:33.003+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:33.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:33 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:33.003+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:33.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:33 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:33.003+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:05.293240+0000 front 2023-10-08T14:09:05.293300+0000 (oldest deadline 2023-10-08T14:09:32.294044+0000) 2023-10-08T14:09:33.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:33 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:33.442+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:33.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:33 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:33.442+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:33.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:33 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:33.442+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:33.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:33 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:33.442+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:33.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:33 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:33.442+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:33.936 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:33 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:33.469+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:28.366316+0000 front 2023-10-08T14:08:28.366241+0000 (oldest deadline 2023-10-08T14:08:51.266107+0000) 2023-10-08T14:09:33.936 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:33 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:33.469+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.667936+0000 front 2023-10-08T14:08:40.667955+0000 (oldest deadline 2023-10-08T14:09:08.268637+0000) 2023-10-08T14:09:33.937 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:33 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:33.469+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.269220+0000 front 2023-10-08T14:08:48.269232+0000 (oldest deadline 2023-10-08T14:09:15.269552+0000) 2023-10-08T14:09:33.937 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:33 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8[127328]: 2023-10-08T14:09:33.469+0000 7f61c8622700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:01.071452+0000 front 2023-10-08T14:09:01.071306+0000 (oldest deadline 2023-10-08T14:09:24.571244+0000) 2023-10-08T14:09:34.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:34 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:34.001+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:34.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:34 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:34.001+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:34.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:34 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:34.001+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:34.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:34 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:34.001+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:34.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:34 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:34.001+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:34.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:34 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:34.001+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:05.293240+0000 front 2023-10-08T14:09:05.293300+0000 (oldest deadline 2023-10-08T14:09:32.294044+0000) 2023-10-08T14:09:34.616 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:34 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:34.230+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:34.616 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:34 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:34.230+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:34.616 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:34 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:34.230+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:34.616 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:34 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:34.230+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:34.616 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:34 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:34.230+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:34.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:34 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:34.462+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:34.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:34 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:34.462+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:34.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:34 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:34.462+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:34.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:34 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:34.462+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:34.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:34 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:34.462+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:34.936 INFO:journalctl@ceph.osd.8.smithi115.stdout:Oct 08 14:09:34 smithi115 bash[148577]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-8 2023-10-08T14:09:35.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:35 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:35.042+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:35.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:35 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:35.042+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:35.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:35 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:35.042+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:35.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:35 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:35.042+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:35.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:35 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:35.042+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:35.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:35 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:35.042+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:05.293240+0000 front 2023-10-08T14:09:05.293300+0000 (oldest deadline 2023-10-08T14:09:32.294044+0000) 2023-10-08T14:09:35.655 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:35 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:35.209+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:35.655 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:35 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:35.209+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:35.656 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:35 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:35.209+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:35.656 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:35 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:35.209+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:35.656 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:35 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:35.209+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:35.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:35 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:35.435+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:35.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:35 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:35.435+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:35.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:35 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:35.435+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:35.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:35 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:35.435+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:35.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:35 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:35.435+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:35.915 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.8.service' 2023-10-08T14:09:35.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:09:35.996 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-08T14:09:35.996 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-08T14:09:35.996 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.9 2023-10-08T14:09:36.357 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:36 smithi057 systemd[1]: Stopping Ceph osd.9 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:09:36.357 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:36.061+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:36.357 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:36.061+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:36.357 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:36.061+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:36.357 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:36.061+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:36.358 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:36.061+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:36.358 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:36.061+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:05.293240+0000 front 2023-10-08T14:09:05.293300+0000 (oldest deadline 2023-10-08T14:09:32.294044+0000) 2023-10-08T14:09:36.358 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:36.355+0000 7fd2d06fa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:09:36.358 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:36.355+0000 7fd2d06fa700 -1 osd.9 118 *** Got signal Terminated *** 2023-10-08T14:09:36.358 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:36 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:36.355+0000 7fd2d06fa700 -1 osd.9 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:09:36.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:36 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:36.224+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:36.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:36 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:36.224+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:36.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:36 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:36.224+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:36.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:36 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:36.224+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:36.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:36 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:36.224+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:36.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:36 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:36.401+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:36.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:36 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:36.401+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:36.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:36 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:36.401+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:36.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:36 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:36.401+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:36.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:36 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:36.401+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:36.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:36 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:36.401+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:37.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:37 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:37.013+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:37.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:37 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:37.013+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:37.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:37 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:37.013+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:37.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:37 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:37.013+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:37.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:37 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:37.013+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:37.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:37 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:37.013+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:05.293240+0000 front 2023-10-08T14:09:05.293300+0000 (oldest deadline 2023-10-08T14:09:32.294044+0000) 2023-10-08T14:09:37.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:37 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:37.214+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:37.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:37 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:37.214+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:37.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:37 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:37.214+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:37.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:37 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:37.214+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:37.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:37 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:37.214+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:37.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:37 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:37.422+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:37.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:37 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:37.422+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:37.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:37 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:37.422+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:37.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:37 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:37.422+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:37.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:37 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:37.422+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:37.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:37 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:37.422+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:38.436 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:38 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:38.182+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:38.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:38 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:38.182+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:38.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:38 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:38.182+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:38.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:38 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:38.182+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:38.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:38 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:38.182+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:38.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:38.055+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:38.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:38.055+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:38.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:38.055+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:38.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:38.055+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:38.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:38.055+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:38.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:38 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:38.055+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:05.293240+0000 front 2023-10-08T14:09:05.293300+0000 (oldest deadline 2023-10-08T14:09:32.294044+0000) 2023-10-08T14:09:38.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:38 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:38.390+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:38.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:38 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:38.390+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:38.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:38 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:38.390+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:38.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:38 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:38.390+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:38.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:38 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:38.390+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:38.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:38 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:38.390+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:39.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:39 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:39.163+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:39.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:39 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:39.163+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:39.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:39 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:39.163+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:39.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:39 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:39.163+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:39.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:39 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:39.163+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:39.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:39 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:39.032+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:39.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:39 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:39.032+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:39.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:39 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:39.032+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:39.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:39 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:39.032+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:39.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:39 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:39.032+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:39.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:39 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:39.032+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:05.293240+0000 front 2023-10-08T14:09:05.293300+0000 (oldest deadline 2023-10-08T14:09:32.294044+0000) 2023-10-08T14:09:39.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:39 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:39.411+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:39.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:39 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:39.411+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:39.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:39 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:39.411+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:39.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:39 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:39.411+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:39.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:39 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:39.411+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:39.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:39 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:39.411+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:40.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:40 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:40.159+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:40.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:40 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:40.159+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:40.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:40 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:40.159+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:40.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:40 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:40.159+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:40.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:40 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:40.159+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:40.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:40 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:40.159+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:40.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:40 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:40.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:40.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:40 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:40.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:40.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:40 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:40.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:40.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:40 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:40.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:40.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:40 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:40.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:40.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:40 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:40.050+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:05.293240+0000 front 2023-10-08T14:09:05.293300+0000 (oldest deadline 2023-10-08T14:09:32.294044+0000) 2023-10-08T14:09:40.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:40 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:40.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:40.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:40 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:40.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:40.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:40 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:40.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:40.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:40 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:40.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:40.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:40 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:40.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:40.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:40 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:40.375+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:41.436 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:41 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:41.170+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:41.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:41 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:41.170+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:41.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:41 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:41.170+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:41.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:41 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:41.170+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:41.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:41 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:41.170+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:41.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:41 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:41.170+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:41.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:41 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:41.098+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:26.186582+0000 front 2023-10-08T14:08:26.186603+0000 (oldest deadline 2023-10-08T14:08:50.286814+0000) 2023-10-08T14:09:41.487 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:41 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:41.098+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.688571+0000 front 2023-10-08T14:08:36.688390+0000 (oldest deadline 2023-10-08T14:09:00.788643+0000) 2023-10-08T14:09:41.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:41 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:41.098+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:40.789363+0000 front 2023-10-08T14:08:40.789222+0000 (oldest deadline 2023-10-08T14:09:07.789823+0000) 2023-10-08T14:09:41.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:41 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:41.098+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:49.490730+0000 front 2023-10-08T14:08:49.490859+0000 (oldest deadline 2023-10-08T14:09:13.590924+0000) 2023-10-08T14:09:41.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:41 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:41.098+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:08:59.392437+0000 front 2023-10-08T14:08:59.392368+0000 (oldest deadline 2023-10-08T14:09:25.292598+0000) 2023-10-08T14:09:41.488 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:41 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9[149024]: 2023-10-08T14:09:41.098+0000 7fd2cc4ef700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:05.293240+0000 front 2023-10-08T14:09:05.293300+0000 (oldest deadline 2023-10-08T14:09:32.294044+0000) 2023-10-08T14:09:41.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:41 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:41.356+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:41.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:41 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:41.356+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:41.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:41 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:41.356+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:41.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:41 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:41.356+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:41.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:41 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:41.356+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:41.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:41 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:41.356+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:42.237 INFO:journalctl@ceph.osd.9.smithi057.stdout:Oct 08 14:09:42 smithi057 bash[190749]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-9 2023-10-08T14:09:42.436 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:42 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:42.182+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:42.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:42 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:42.182+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:42.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:42 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:42.182+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:42.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:42 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:42.183+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:42.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:42 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:42.183+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:42.437 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:42 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:42.183+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:42.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:42 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:42.400+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:42.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:42 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:42.400+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:42.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:42 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:42.400+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:42.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:42 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:42.400+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:42.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:42 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:42.400+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:42.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:42 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:42.400+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:42.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:42 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:42.400+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:15.624127+0000 front 2023-10-08T14:09:15.624142+0000 (oldest deadline 2023-10-08T14:09:41.523831+0000) 2023-10-08T14:09:43.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:43 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:43.211+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:43.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:43 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:43.211+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:43.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:43 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:43.211+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:43.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:43 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:43.211+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:43.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:43 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:43.211+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:43.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:43 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:43.211+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:43.708 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.9.service' 2023-10-08T14:09:43.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:09:43.750 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-08T14:09:43.750 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-08T14:09:43.750 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.10 2023-10-08T14:09:43.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:43 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:43.376+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:43.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:43 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:43.376+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:43.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:43 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:43.376+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:43.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:43 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:43.376+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:43.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:43 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:43.376+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:43.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:43 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:43.376+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:43.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:43 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:43.376+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:15.624127+0000 front 2023-10-08T14:09:15.624142+0000 (oldest deadline 2023-10-08T14:09:41.523831+0000) 2023-10-08T14:09:44.097 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:43 smithi099 systemd[1]: Stopping Ceph osd.10 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:09:44.390 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:44 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:44.096+0000 7f8235157700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:09:44.390 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:44 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:44.096+0000 7f8235157700 -1 osd.10 118 *** Got signal Terminated *** 2023-10-08T14:09:44.391 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:44 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:44.096+0000 7f8235157700 -1 osd.10 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:09:44.391 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:44 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:44.389+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:44.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:44 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:44.205+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:44.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:44 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:44.205+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:44.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:44 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:44.205+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:44.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:44 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:44.205+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:44.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:44 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:44.205+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:44.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:44 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:44.205+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:44.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:44 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:44.389+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:44.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:44 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:44.389+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:44.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:44 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:44.389+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:44.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:44 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:44.389+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:44.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:44 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:44.389+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:44.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:44 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:44.389+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:15.624127+0000 front 2023-10-08T14:09:15.624142+0000 (oldest deadline 2023-10-08T14:09:41.523831+0000) 2023-10-08T14:09:45.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:45 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:45.252+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:45.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:45 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:45.252+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:45.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:45 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:45.252+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:45.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:45 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:45.252+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:45.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:45 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:45.252+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:45.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:45 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:45.252+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:45.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:45 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:45.379+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:45.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:45 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:45.379+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:45.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:45 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:45.379+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:45.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:45 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:45.379+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:45.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:45 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:45.379+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:45.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:45 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:45.379+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:45.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:45 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:45.379+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:15.624127+0000 front 2023-10-08T14:09:15.624142+0000 (oldest deadline 2023-10-08T14:09:41.523831+0000) 2023-10-08T14:09:46.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:46 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:46.239+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:46.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:46 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:46.239+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:46.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:46 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:46.239+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:46.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:46 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:46.239+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:46.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:46 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:46.239+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:46.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:46 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:46.239+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:46.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:46 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:46.385+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:46.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:46 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:46.385+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:46.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:46 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:46.385+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:46.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:46 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:46.385+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:46.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:46 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:46.385+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:46.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:46 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:46.385+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:46.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:46 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:46.385+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:15.624127+0000 front 2023-10-08T14:09:15.624142+0000 (oldest deadline 2023-10-08T14:09:41.523831+0000) 2023-10-08T14:09:47.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:47 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:47.251+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:47.867 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:47 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:47.251+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:47.867 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:47 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:47.251+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:47.867 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:47 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:47.251+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:47.867 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:47 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:47.251+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:47.867 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:47 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:47.251+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:47.868 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:47 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:47.431+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:47.868 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:47 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:47.431+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:47.869 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:47 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:47.431+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:47.869 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:47 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:47.431+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:47.869 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:47 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:47.431+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:47.869 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:47 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:47.431+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:47.869 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:47 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:47.431+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:15.624127+0000 front 2023-10-08T14:09:15.624142+0000 (oldest deadline 2023-10-08T14:09:41.523831+0000) 2023-10-08T14:09:48.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:48 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:48.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:48.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:48 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:48.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:48.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:48 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:48.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:48.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:48 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:48.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:48.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:48 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:48.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:48.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:48 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:48.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:48.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:48 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:48.285+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-10-08T14:09:25.940785+0000 front 2023-10-08T14:09:25.940838+0000 (oldest deadline 2023-10-08T14:09:47.640797+0000) 2023-10-08T14:09:48.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:48 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:48.440+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:27.618204+0000 front 2023-10-08T14:08:27.618184+0000 (oldest deadline 2023-10-08T14:08:51.718248+0000) 2023-10-08T14:09:48.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:48 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:48.440+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:36.419127+0000 front 2023-10-08T14:08:36.419218+0000 (oldest deadline 2023-10-08T14:09:01.719343+0000) 2023-10-08T14:09:48.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:48 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:48.440+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:41.719798+0000 front 2023-10-08T14:08:41.720023+0000 (oldest deadline 2023-10-08T14:09:07.019763+0000) 2023-10-08T14:09:48.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:48 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:48.440+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2023-10-08T14:08:48.720926+0000 front 2023-10-08T14:08:48.721028+0000 (oldest deadline 2023-10-08T14:09:13.420928+0000) 2023-10-08T14:09:48.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:48 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:48.440+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:02.122746+0000 front 2023-10-08T14:09:02.122766+0000 (oldest deadline 2023-10-08T14:09:23.822741+0000) 2023-10-08T14:09:48.772 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:48 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:48.440+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:09.723545+0000 front 2023-10-08T14:09:09.723674+0000 (oldest deadline 2023-10-08T14:09:35.623475+0000) 2023-10-08T14:09:48.773 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:48 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10[134546]: 2023-10-08T14:09:48.440+0000 7f822df46700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:15.624127+0000 front 2023-10-08T14:09:15.624142+0000 (oldest deadline 2023-10-08T14:09:41.523831+0000) 2023-10-08T14:09:49.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:49 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:49.326+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:49.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:49 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:49.326+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:49.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:49 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:49.326+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:49.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:49 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:49.326+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:49.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:49 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:49.326+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:49.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:49 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:49.326+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:49.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:49 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:49.326+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-10-08T14:09:25.940785+0000 front 2023-10-08T14:09:25.940838+0000 (oldest deadline 2023-10-08T14:09:47.640797+0000) 2023-10-08T14:09:50.022 INFO:journalctl@ceph.osd.10.smithi099.stdout:Oct 08 14:09:49 smithi099 bash[148263]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-10 2023-10-08T14:09:50.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:50 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:50.349+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:50.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:50 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:50.349+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:50.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:50 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:50.349+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:50.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:50 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:50.349+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:50.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:50 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:50.349+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:50.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:50 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:50.349+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:50.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:50 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:50.349+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-10-08T14:09:25.940785+0000 front 2023-10-08T14:09:25.940838+0000 (oldest deadline 2023-10-08T14:09:47.640797+0000) 2023-10-08T14:09:51.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.10.service' 2023-10-08T14:09:51.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:09:51.230 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-08T14:09:51.230 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-08T14:09:51.230 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.11 2023-10-08T14:09:51.575 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 systemd[1]: Stopping Ceph osd.11 for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:09:51.576 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:51.305+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:51.576 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:51.305+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:51.576 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:51.305+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:51.576 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:51.305+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:51.576 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:51.305+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:51.576 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:51.305+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:51.576 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:51.305+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-10-08T14:09:25.940785+0000 front 2023-10-08T14:09:25.940838+0000 (oldest deadline 2023-10-08T14:09:47.640797+0000) 2023-10-08T14:09:51.936 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:51.574+0000 7f6f78be9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:09:51.936 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:51.574+0000 7f6f78be9700 -1 osd.11 118 *** Got signal Terminated *** 2023-10-08T14:09:51.936 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:51 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:51.574+0000 7f6f78be9700 -1 osd.11 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T14:09:52.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:52 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:52.310+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:52.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:52 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:52.310+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:52.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:52 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:52.310+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:52.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:52 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:52.310+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:52.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:52 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:52.310+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:52.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:52 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:52.310+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:52.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:52 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:52.310+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-10-08T14:09:25.940785+0000 front 2023-10-08T14:09:25.940838+0000 (oldest deadline 2023-10-08T14:09:47.640797+0000) 2023-10-08T14:09:53.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:53.332+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:53.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:53.332+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:53.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:53.332+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:53.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:53.332+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:53.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:53.332+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:53.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:53.332+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:53.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:53 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:53.332+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-10-08T14:09:25.940785+0000 front 2023-10-08T14:09:25.940838+0000 (oldest deadline 2023-10-08T14:09:47.640797+0000) 2023-10-08T14:09:54.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:54 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:54.340+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:54.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:54 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:54.340+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:54.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:54 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:54.340+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:54.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:54 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:54.340+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:54.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:54 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:54.340+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:54.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:54 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:54.340+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:54.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:54 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:54.340+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-10-08T14:09:25.940785+0000 front 2023-10-08T14:09:25.940838+0000 (oldest deadline 2023-10-08T14:09:47.640797+0000) 2023-10-08T14:09:55.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:55 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:55.347+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:55.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:55 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:55.347+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:55.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:55 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:55.347+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:55.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:55 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:55.347+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:55.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:55 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:55.347+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:55.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:55 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:55.347+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:55.688 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:55 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:55.347+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-10-08T14:09:25.940785+0000 front 2023-10-08T14:09:25.940838+0000 (oldest deadline 2023-10-08T14:09:47.640797+0000) 2023-10-08T14:09:56.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:56.383+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2023-10-08T14:08:25.128426+0000 front 2023-10-08T14:08:25.128465+0000 (oldest deadline 2023-10-08T14:08:54.528959+0000) 2023-10-08T14:09:56.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:56.383+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2023-10-08T14:08:34.529631+0000 front 2023-10-08T14:08:34.529620+0000 (oldest deadline 2023-10-08T14:08:58.029621+0000) 2023-10-08T14:09:56.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:56.383+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6804 osd.2 since back 2023-10-08T14:08:42.130660+0000 front 2023-10-08T14:08:42.130745+0000 (oldest deadline 2023-10-08T14:09:06.230863+0000) 2023-10-08T14:09:56.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:56.383+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2023-10-08T14:09:00.734071+0000 front 2023-10-08T14:09:00.734047+0000 (oldest deadline 2023-10-08T14:09:25.434083+0000) 2023-10-08T14:09:56.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:56.383+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6812 osd.5 since back 2023-10-08T14:09:07.735076+0000 front 2023-10-08T14:09:07.735151+0000 (oldest deadline 2023-10-08T14:09:30.635250+0000) 2023-10-08T14:09:56.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:56.383+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2023-10-08T14:09:17.537424+0000 front 2023-10-08T14:09:17.537478+0000 (oldest deadline 2023-10-08T14:09:39.237445+0000) 2023-10-08T14:09:56.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:56.383+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2023-10-08T14:09:25.940785+0000 front 2023-10-08T14:09:25.940838+0000 (oldest deadline 2023-10-08T14:09:47.640797+0000) 2023-10-08T14:09:56.687 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:56 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11[135313]: 2023-10-08T14:09:56.383+0000 7f6f731db700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.115:6820 osd.8 since back 2023-10-08T14:09:31.741832+0000 front 2023-10-08T14:09:31.741864+0000 (oldest deadline 2023-10-08T14:09:55.841911+0000) 2023-10-08T14:09:57.686 INFO:journalctl@ceph.osd.11.smithi115.stdout:Oct 08 14:09:57 smithi115 bash[148850]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-osd-11 2023-10-08T14:09:58.627 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@osd.11.service' 2023-10-08T14:09:58.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:09:58.668 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-08T14:09:58.668 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-08T14:09:58.668 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.a 2023-10-08T14:09:58.987 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 14:09:58 smithi057 systemd[1]: Stopping Ceph mds.a for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:09:59.487 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 14:09:59 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-a[157643]: 2023-10-08T14:09:59.035+0000 7ff227c2b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:09:59.487 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 14:09:59 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-a[157643]: 2023-10-08T14:09:59.035+0000 7ff227c2b700 -1 mds.a *** got signal Terminated *** 2023-10-08T14:10:02.987 INFO:journalctl@ceph.mds.a.smithi057.stdout:Oct 08 14:10:02 smithi057 bash[191009]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-a 2023-10-08T14:10:03.176 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.a.service' 2023-10-08T14:10:03.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:03.215 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-08T14:10:03.215 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-08T14:10:03.216 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.d 2023-10-08T14:10:03.487 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 14:10:03 smithi057 systemd[1]: Stopping Ceph mds.d for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:03.987 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 14:10:03 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-d[156645]: 2023-10-08T14:10:03.601+0000 7f396d82f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:03.987 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 14:10:03 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-d[156645]: 2023-10-08T14:10:03.601+0000 7f396d82f700 -1 mds.d *** got signal Terminated *** 2023-10-08T14:10:06.970 INFO:journalctl@ceph.mds.d.smithi057.stdout:Oct 08 14:10:06 smithi057 bash[191164]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-d 2023-10-08T14:10:07.166 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.d.service' 2023-10-08T14:10:07.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:07.208 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-08T14:10:07.209 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-08T14:10:07.209 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.g 2023-10-08T14:10:07.487 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 08 14:10:07 smithi057 systemd[1]: Stopping Ceph mds.g for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:07.988 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 08 14:10:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-g[157254]: 2023-10-08T14:10:07.592+0000 7fe9ceed2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:07.988 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 08 14:10:07 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-g[157254]: 2023-10-08T14:10:07.592+0000 7fe9ceed2700 -1 mds.g *** got signal Terminated *** 2023-10-08T14:10:10.987 INFO:journalctl@ceph.mds.g.smithi057.stdout:Oct 08 14:10:10 smithi057 bash[191305]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-g 2023-10-08T14:10:11.028 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.g.service' 2023-10-08T14:10:11.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:11.063 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-08T14:10:11.063 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-08T14:10:11.064 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.j 2023-10-08T14:10:11.443 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 14:10:11 smithi057 systemd[1]: Stopping Ceph mds.j for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:11.737 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 14:10:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-j[156304]: 2023-10-08T14:10:11.441+0000 7f3e2e26e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:11.737 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 14:10:11 smithi057 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-j[156304]: 2023-10-08T14:10:11.441+0000 7f3e2e26e700 -1 mds.j *** got signal Terminated *** 2023-10-08T14:10:15.312 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 14:10:15 smithi057 bash[191448]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-j 2023-10-08T14:10:15.536 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.j.service' 2023-10-08T14:10:15.562 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 14:10:15 smithi057 systemd[1]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.j.service: Succeeded. 2023-10-08T14:10:15.563 INFO:journalctl@ceph.mds.j.smithi057.stdout:Oct 08 14:10:15 smithi057 systemd[1]: Stopped Ceph mds.j for 5f25708e-65e0-11ee-8db5-212e2dc638e7. 2023-10-08T14:10:15.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:15.573 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-08T14:10:15.573 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-08T14:10:15.573 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.b 2023-10-08T14:10:15.933 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 14:10:15 smithi099 systemd[1]: Stopping Ceph mds.b for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:16.272 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 14:10:15 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b[140884]: 2023-10-08T14:10:15.931+0000 7f96e3f6b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:16.272 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 14:10:15 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b[140884]: 2023-10-08T14:10:15.931+0000 7f96e3f6b700 -1 mds.b *** got signal Terminated *** 2023-10-08T14:10:19.272 INFO:journalctl@ceph.mds.b.smithi099.stdout:Oct 08 14:10:19 smithi099 bash[148532]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-b 2023-10-08T14:10:19.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.b.service' 2023-10-08T14:10:19.526 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:19.526 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-08T14:10:19.526 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-08T14:10:19.526 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.e 2023-10-08T14:10:19.916 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 14:10:19 smithi099 systemd[1]: Stopping Ceph mds.e for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:20.272 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 14:10:19 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-e[140207]: 2023-10-08T14:10:19.915+0000 7fa9a67d4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:20.272 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 14:10:19 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-e[140207]: 2023-10-08T14:10:19.915+0000 7fa9a67d4700 -1 mds.e *** got signal Terminated *** 2023-10-08T14:10:23.455 INFO:journalctl@ceph.mds.e.smithi099.stdout:Oct 08 14:10:23 smithi099 bash[148677]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-e 2023-10-08T14:10:23.460 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.e.service' 2023-10-08T14:10:23.493 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:23.494 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-08T14:10:23.494 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-08T14:10:23.494 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.h 2023-10-08T14:10:23.772 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 14:10:23 smithi099 systemd[1]: Stopping Ceph mds.h for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:24.272 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 14:10:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-h[140545]: 2023-10-08T14:10:23.863+0000 7fd21f355700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:24.272 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 14:10:23 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-h[140545]: 2023-10-08T14:10:23.863+0000 7fd21f355700 -1 mds.h *** got signal Terminated *** 2023-10-08T14:10:27.272 INFO:journalctl@ceph.mds.h.smithi099.stdout:Oct 08 14:10:26 smithi099 bash[148817]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-h 2023-10-08T14:10:27.325 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.h.service' 2023-10-08T14:10:27.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:27.360 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-08T14:10:27.360 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-08T14:10:27.360 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.k 2023-10-08T14:10:27.754 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 14:10:27 smithi099 systemd[1]: Stopping Ceph mds.k for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:28.022 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 14:10:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-k[139865]: 2023-10-08T14:10:27.753+0000 7f52ea37b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:28.022 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 14:10:27 smithi099 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-k[139865]: 2023-10-08T14:10:27.753+0000 7f52ea37b700 -1 mds.k *** got signal Terminated *** 2023-10-08T14:10:31.272 INFO:journalctl@ceph.mds.k.smithi099.stdout:Oct 08 14:10:31 smithi099 bash[148962]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-k 2023-10-08T14:10:31.377 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.k.service' 2023-10-08T14:10:31.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:31.410 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-08T14:10:31.410 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-08T14:10:31.410 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.c 2023-10-08T14:10:31.767 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 14:10:31 smithi115 systemd[1]: Stopping Ceph mds.c for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:32.186 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 14:10:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-c[141540]: 2023-10-08T14:10:31.765+0000 7fb312a41700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:32.186 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 14:10:31 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-c[141540]: 2023-10-08T14:10:31.765+0000 7fb312a41700 -1 mds.c *** got signal Terminated *** 2023-10-08T14:10:35.125 INFO:journalctl@ceph.mds.c.smithi115.stdout:Oct 08 14:10:34 smithi115 bash[149128]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-c 2023-10-08T14:10:35.131 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.c.service' 2023-10-08T14:10:35.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:35.169 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-08T14:10:35.169 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-08T14:10:35.169 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.f 2023-10-08T14:10:35.436 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 14:10:35 smithi115 systemd[1]: Stopping Ceph mds.f for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:35.937 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 14:10:35 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-f[141199]: 2023-10-08T14:10:35.570+0000 7f73d21da700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:35.937 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 14:10:35 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-f[141199]: 2023-10-08T14:10:35.570+0000 7f73d21da700 -1 mds.f *** got signal Terminated *** 2023-10-08T14:10:39.017 INFO:journalctl@ceph.mds.f.smithi115.stdout:Oct 08 14:10:38 smithi115 bash[149269]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-f 2023-10-08T14:10:39.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.f.service' 2023-10-08T14:10:39.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:39.236 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-08T14:10:39.236 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-08T14:10:39.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.i 2023-10-08T14:10:39.623 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 14:10:39 smithi115 systemd[1]: Stopping Ceph mds.i for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:39.936 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 14:10:39 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-i[141877]: 2023-10-08T14:10:39.621+0000 7fad96e1f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:39.936 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 14:10:39 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-i[141877]: 2023-10-08T14:10:39.621+0000 7fad96e1f700 -1 mds.i *** got signal Terminated *** 2023-10-08T14:10:42.982 INFO:journalctl@ceph.mds.i.smithi115.stdout:Oct 08 14:10:42 smithi115 bash[149409]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-i 2023-10-08T14:10:43.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.i.service' 2023-10-08T14:10:43.204 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:43.204 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-08T14:10:43.204 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-08T14:10:43.204 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.l 2023-10-08T14:10:43.436 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 14:10:43 smithi115 systemd[1]: Stopping Ceph mds.l for 5f25708e-65e0-11ee-8db5-212e2dc638e7... 2023-10-08T14:10:43.936 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 14:10:43 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-l[140851]: 2023-10-08T14:10:43.570+0000 7f9c9e684700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T14:10:43.936 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 14:10:43 smithi115 ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-l[140851]: 2023-10-08T14:10:43.570+0000 7f9c9e684700 -1 mds.l *** got signal Terminated *** 2023-10-08T14:10:47.149 INFO:journalctl@ceph.mds.l.smithi115.stdout:Oct 08 14:10:46 smithi115 bash[149553]: ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7-mds-l 2023-10-08T14:10:47.154 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f25708e-65e0-11ee-8db5-212e2dc638e7@mds.l.service' 2023-10-08T14:10:47.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T14:10:47.188 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-08T14:10:47.188 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-08T14:10:47.517 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T14:10:49.184 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-08T14:10:49.796 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T14:10:51.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-08T14:10:51.728 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T14:10:53.377 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T14:10:53.415 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T14:10:53.451 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T14:10:53.481 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-08T14:10:53.483 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991/remote/smithi057/crash 2023-10-08T14:10:53.483 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/crash -- . 2023-10-08T14:10:53.515 INFO:teuthology.orchestra.run.smithi057.stderr:tar: /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-08T14:10:53.515 INFO:teuthology.orchestra.run.smithi057.stderr:tar: Error is not recoverable: exiting now 2023-10-08T14:10:53.517 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991/remote/smithi099/crash 2023-10-08T14:10:53.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/crash -- . 2023-10-08T14:10:53.563 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-08T14:10:53.563 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2023-10-08T14:10:53.565 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991/remote/smithi115/crash 2023-10-08T14:10:53.565 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/crash -- . 2023-10-08T14:10:53.613 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-08T14:10:53.613 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2023-10-08T14:10:53.614 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-08T14:10:53.615 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-08T14:10:53.652 INFO:teuthology.orchestra.run.smithi057.stderr:grep: /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph.log: No such file or directory 2023-10-08T14:10:53.655 INFO:tasks.cephadm:Compressing logs... 2023-10-08T14:10:53.655 DEBUG:teuthology.orchestra.run.smithi057:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T14:10:53.697 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T14:10:53.700 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T14:10:53.745 INFO:teuthology.orchestra.run.smithi057.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-08T14:10:53.745 INFO:teuthology.orchestra.run.smithi057.stderr:: No such file or directory 2023-10-08T14:10:53.746 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-08T14:10:53.746 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-08T14:10:53.749 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.0.log 2023-10-08T14:10:53.750 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.g.log 2023-10-08T14:10:53.751 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.a.log 2023-10-08T14:10:53.751 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.6.log 2023-10-08T14:10:53.752 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-08T14:10:53.752 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-08T14:10:53.753 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.3.log 2023-10-08T14:10:53.753 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.d.log 2023-10-08T14:10:53.753 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mgr.x.log 2023-10-08T14:10:53.754 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.9.log 2023-10-08T14:10:53.754 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-08T14:10:53.754 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.j.log 2023-10-08T14:10:53.754 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mon.a.log 2023-10-08T14:10:53.754 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-10-08T14:10:53.755 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.10.log 2023-10-08T14:10:53.755 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.h.log 2023-10-08T14:10:53.755 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.e.log 2023-10-08T14:10:53.756 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.10.log: /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.4.log 2023-10-08T14:10:53.756 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.1.log 2023-10-08T14:10:53.756 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.log: 94.3% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.g.log.gz 2023-10-08T14:10:53.757 INFO:teuthology.orchestra.run.smithi057.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-10-08T14:10:53.758 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.k.log 2023-10-08T14:10:53.758 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-08T14:10:53.758 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.7.log 2023-10-08T14:10:53.758 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mon.b.log 2023-10-08T14:10:53.758 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.b.log 2023-10-08T14:10:53.758 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mgr.y.log 2023-10-08T14:10:53.759 INFO:teuthology.orchestra.run.smithi057.stderr: 94.4% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.j.log.gz 2023-10-08T14:10:53.759 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.5.log 2023-10-08T14:10:53.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.2.log 2023-10-08T14:10:53.761 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.11.log 2023-10-08T14:10:53.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mgr.z.log 2023-10-08T14:10:53.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.2.log: /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.8.log 2023-10-08T14:10:53.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.f.log 2023-10-08T14:10:53.761 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.c.log 2023-10-08T14:10:53.762 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mgr.y.log: 92.9% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-08T14:10:53.763 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.i.log 2023-10-08T14:10:53.763 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mon.c.log 2023-10-08T14:10:53.763 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.c.log: /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-08T14:10:53.763 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.l.log 2023-10-08T14:10:53.765 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-volume.log: /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.l.log: 93.1% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-08T14:10:53.779 INFO:teuthology.orchestra.run.smithi057.stderr: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-08T14:10:53.821 INFO:teuthology.orchestra.run.smithi099.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-08T14:10:54.064 INFO:teuthology.orchestra.run.smithi057.stderr: 90.1% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-08T14:10:54.140 INFO:teuthology.orchestra.run.smithi115.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-08T14:10:54.661 INFO:teuthology.orchestra.run.smithi115.stderr: 91.0% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-08T14:10:54.669 INFO:teuthology.orchestra.run.smithi057.stderr: 91.0% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-08T14:10:55.001 INFO:teuthology.orchestra.run.smithi099.stderr: 91.0% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-08T14:10:55.347 INFO:teuthology.orchestra.run.smithi115.stderr: 91.8% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mon.c.log.gz 2023-10-08T14:10:56.219 INFO:teuthology.orchestra.run.smithi099.stderr: 91.9% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mon.b.log.gz 2023-10-08T14:10:56.833 INFO:teuthology.orchestra.run.smithi057.stderr: 90.4% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mon.a.log.gz 2023-10-08T14:11:03.101 INFO:teuthology.orchestra.run.smithi057.stderr: 95.4% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.d.log.gz 2023-10-08T14:11:04.069 INFO:teuthology.orchestra.run.smithi057.stderr: 95.4% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.a.log.gz 2023-10-08T14:11:20.917 INFO:teuthology.orchestra.run.smithi099.stderr: 95.5% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.k.log.gz 2023-10-08T14:11:21.622 INFO:teuthology.orchestra.run.smithi115.stderr: 95.1% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.l.log.gz 2023-10-08T14:11:30.095 INFO:teuthology.orchestra.run.smithi099.stderr: 95.3% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.e.log.gz 2023-10-08T14:12:15.902 INFO:teuthology.orchestra.run.smithi057.stderr: 91.2% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.0.log.gz 2023-10-08T14:12:21.215 INFO:teuthology.orchestra.run.smithi057.stderr: 91.3% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.3.log.gz 2023-10-08T14:12:23.683 INFO:teuthology.orchestra.run.smithi057.stderr: 91.3% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.6.log.gz 2023-10-08T14:12:30.399 INFO:teuthology.orchestra.run.smithi057.stderr: 91.2% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.9.log.gz 2023-10-08T14:12:30.401 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-08T14:12:30.401 INFO:teuthology.orchestra.run.smithi057.stderr:real 1m36.687s 2023-10-08T14:12:30.401 INFO:teuthology.orchestra.run.smithi057.stderr:user 4m19.852s 2023-10-08T14:12:30.401 INFO:teuthology.orchestra.run.smithi057.stderr:sys 0m8.372s 2023-10-08T14:12:42.114 INFO:teuthology.orchestra.run.smithi099.stderr: 91.6% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.10.log.gz 2023-10-08T14:13:03.870 INFO:teuthology.orchestra.run.smithi099.stderr: 91.5% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.1.log.gz 2023-10-08T14:13:29.436 INFO:teuthology.orchestra.run.smithi099.stderr: 91.3% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.7.log.gz 2023-10-08T14:14:03.126 INFO:teuthology.orchestra.run.smithi115.stderr: 91.5% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.8.log.gz 2023-10-08T14:14:03.388 INFO:teuthology.orchestra.run.smithi099.stderr: 91.5% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.4.log.gz 2023-10-08T14:14:12.811 INFO:teuthology.orchestra.run.smithi115.stderr: 91.5% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.11.log.gz 2023-10-08T14:14:17.054 INFO:teuthology.orchestra.run.smithi099.stderr: 94.1% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.h.log.gz 2023-10-08T14:14:35.837 INFO:teuthology.orchestra.run.smithi099.stderr: 94.0% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.b.log.gz 2023-10-08T14:14:35.840 INFO:teuthology.orchestra.run.smithi099.stderr: 2023-10-08T14:14:35.840 INFO:teuthology.orchestra.run.smithi099.stderr:real 3m42.123s 2023-10-08T14:14:35.840 INFO:teuthology.orchestra.run.smithi099.stderr:user 6m11.050s 2023-10-08T14:14:35.840 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m14.824s 2023-10-08T14:14:36.421 INFO:teuthology.orchestra.run.smithi115.stderr: 91.5% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.2.log.gz 2023-10-08T14:14:51.904 INFO:teuthology.orchestra.run.smithi115.stderr: 93.6% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.c.log.gz 2023-10-08T14:15:20.520 INFO:teuthology.orchestra.run.smithi115.stderr: 93.6% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.f.log.gz 2023-10-08T14:15:21.363 INFO:teuthology.orchestra.run.smithi115.stderr: 91.3% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-osd.5.log.gz 2023-10-08T14:16:48.052 INFO:teuthology.orchestra.run.smithi115.stderr: 93.9% -- replaced with /var/log/ceph/5f25708e-65e0-11ee-8db5-212e2dc638e7/ceph-mds.i.log.gz 2023-10-08T14:16:48.065 INFO:teuthology.orchestra.run.smithi115.stderr: 2023-10-08T14:16:48.066 INFO:teuthology.orchestra.run.smithi115.stderr:real 5m54.350s 2023-10-08T14:16:48.066 INFO:teuthology.orchestra.run.smithi115.stderr:user 8m9.322s 2023-10-08T14:16:48.066 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m19.579s 2023-10-08T14:16:48.066 INFO:tasks.cephadm:Archiving logs... 2023-10-08T14:16:48.066 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991/remote/smithi057/log 2023-10-08T14:16:48.067 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T14:17:32.177 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991/remote/smithi099/log 2023-10-08T14:17:32.178 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T14:18:32.799 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991/remote/smithi115/log 2023-10-08T14:18:32.800 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T14:19:50.909 INFO:tasks.cephadm:Removing cluster... 2023-10-08T14:19:50.909 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 --force 2023-10-08T14:19:51.365 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T14:19:52.194 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 --force 2023-10-08T14:19:53.070 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T14:19:54.089 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f25708e-65e0-11ee-8db5-212e2dc638e7 --force 2023-10-08T14:19:54.918 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: 5f25708e-65e0-11ee-8db5-212e2dc638e7 2023-10-08T14:19:56.000 INFO:tasks.cephadm:Removing cephadm ... 2023-10-08T14:19:56.000 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-08T14:19:56.022 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-08T14:19:56.124 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-08T14:19:56.145 INFO:tasks.cephadm:Teardown complete 2023-10-08T14:19:56.145 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-08T14:19:56.159 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-08T14:19:56.160 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T14:19:56.162 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T14:19:56.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T14:19:56.300 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-08T14:19:56.317 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-08T14:19:56.317 DEBUG:teuthology.orchestra.run.smithi057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T14:19:56.321 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T14:19:56.324 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T14:19:56.338 INFO:teuthology.orchestra.run.smithi057.stderr:bash: ntpq: command not found 2023-10-08T14:19:56.340 INFO:teuthology.orchestra.run.smithi115.stderr:bash: ntpq: command not found 2023-10-08T14:19:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2023-10-08T14:19:56.356 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T14:19:56.356 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2023-10-08T14:19:56.356 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 +145us[ +145us] +/- 83ms 2023-10-08T14:19:56.356 INFO:teuthology.orchestra.run.smithi057.stdout:^* hv02.front.sepia.ceph.com 3 6 377 22 -11us[ +21us] +/- 54ms 2023-10-08T14:19:56.356 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -34us[-2583ns] +/- 65ms 2023-10-08T14:19:56.357 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T14:19:56.362 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T14:19:56.362 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-10-08T14:19:56.362 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 +138us[ +166us] +/- 83ms 2023-10-08T14:19:56.362 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 -22us[+6060ns] +/- 54ms 2023-10-08T14:19:56.362 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -32us[-3967ns] +/- 65ms 2023-10-08T14:19:56.362 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T14:19:56.369 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T14:19:56.369 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2023-10-08T14:19:56.369 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 +131us[ +136us] +/- 83ms 2023-10-08T14:19:56.369 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -11us[-5267ns] +/- 54ms 2023-10-08T14:19:56.369 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 84 -15us[ -12us] +/- 64ms 2023-10-08T14:19:56.369 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T14:19:56.370 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-08T14:19:56.381 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-08T14:19:56.381 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-08T14:19:56.391 DEBUG:teuthology.orchestra.run.smithi057:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T14:19:56.546 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T14:19:56.724 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T14:19:56.946 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T14:19:56.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T14:19:56.985 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T14:19:57.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T14:19:57.021 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T14:19:57.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T14:19:57.053 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-08T14:19:57.065 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-08T14:19:57.078 INFO:teuthology.task.internal:Duration was 3040.370613 seconds 2023-10-08T14:19:57.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-08T14:19:57.090 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-08T14:19:57.090 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T14:19:57.093 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T14:19:57.096 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T14:19:57.182 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T14:19:57.185 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T14:19:57.218 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T14:19:57.463 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-08T14:19:57.464 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi057.front.sepia.ceph.com 2023-10-08T14:19:57.464 DEBUG:teuthology.orchestra.run.smithi057:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T14:19:57.489 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2023-10-08T14:19:57.489 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T14:19:57.520 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2023-10-08T14:19:57.520 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T14:19:57.546 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-08T14:19:57.546 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T14:19:57.549 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T14:19:57.562 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T14:19:57.653 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-08T14:19:57.654 DEBUG:teuthology.orchestra.run.smithi057:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T14:19:57.656 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T14:19:57.660 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T14:19:57.782 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-08T14:19:57.794 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-08T14:19:57.794 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T14:19:57.824 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T14:19:57.853 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T14:19:57.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-08T14:19:57.904 DEBUG:teuthology.orchestra.run.smithi057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T14:19:57.906 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T14:19:57.909 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T14:19:57.947 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2023-10-08T14:19:57.948 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = core 2023-10-08T14:19:57.971 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2023-10-08T14:19:57.994 DEBUG:teuthology.orchestra.run.smithi057:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T14:19:58.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T14:19:58.010 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T14:19:58.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T14:19:58.026 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T14:19:58.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T14:19:58.050 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-08T14:19:58.067 INFO:teuthology.task.internal:Transferring archived files... 2023-10-08T14:19:58.068 DEBUG:teuthology.misc:Transferring archived files from smithi057:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991/remote/smithi057 2023-10-08T14:19:58.068 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T14:19:58.115 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991/remote/smithi099 2023-10-08T14:19:58.115 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T14:19:58.154 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991/remote/smithi115 2023-10-08T14:19:58.155 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T14:19:58.194 INFO:teuthology.task.internal:Removing archive directory... 2023-10-08T14:19:58.194 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T14:19:58.196 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T14:19:58.198 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T14:19:58.244 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-08T14:19:58.257 INFO:teuthology.task.internal:Not uploading archives. 2023-10-08T14:19:58.257 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-08T14:19:58.269 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-08T14:19:58.269 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T14:19:58.271 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T14:19:58.274 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T14:19:58.286 INFO:teuthology.orchestra.run.smithi057.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 14:19 /home/ubuntu/cephtest 2023-10-08T14:19:58.288 INFO:teuthology.orchestra.run.smithi099.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 14:19 /home/ubuntu/cephtest 2023-10-08T14:19:58.301 INFO:teuthology.orchestra.run.smithi115.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 14:19 /home/ubuntu/cephtest 2023-10-08T14:19:58.320 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-08T14:19:58.332 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-08T14:19:58.379 INFO:teuthology.nuke:Checking targets against current locks 2023-10-08T14:19:58.405 DEBUG:teuthology.nuke:shortname: smithi057 2023-10-08T14:19:58.406 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T14:19:58.427 DEBUG:teuthology.nuke:shortname: smithi099 2023-10-08T14:19:58.428 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T14:19:58.432 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 13:23:40.455017', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T14:19:58.451 DEBUG:teuthology.nuke:shortname: smithi115 2023-10-08T14:19:58.451 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T14:19:58.454 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 13:23:40.452819', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T14:19:58.486 INFO:teuthology.orchestra.console.smithi057:Power off 2023-10-08T14:19:58.486 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T14:19:58.493 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 13:23:40.454000', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T14:19:58.510 DEBUG:teuthology.orchestra.console.smithi057:power off output: Chassis Power Control: Down/Off 2023-10-08T14:19:58.510 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T14:19:58.525 INFO:teuthology.orchestra.console.smithi099:Power off 2023-10-08T14:19:58.525 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T14:19:58.536 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-10-08T14:19:58.545 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2023-10-08T14:19:58.546 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T14:19:58.563 INFO:teuthology.orchestra.console.smithi115:Power off 2023-10-08T14:19:58.563 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T14:19:58.569 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-10-08T14:19:58.583 DEBUG:teuthology.orchestra.console.smithi115:power off output: Chassis Power Control: Down/Off 2023-10-08T14:19:58.584 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T14:19:58.604 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2023-10-08T14:20:02.538 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T14:20:02.570 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T14:20:03.027 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T14:20:03.036 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-10-08T14:20:03.328 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-10-08T14:20:03.329 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2023-10-08T14:20:07.330 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T14:20:07.338 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T14:20:07.345 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T14:20:07.440 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2023-10-08T14:20:07.446 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2023-10-08T14:20:07.454 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2023-10-08T14:20:07.541 INFO:teuthology.orchestra.console.smithi057:Power off completed 2023-10-08T14:20:07.547 INFO:teuthology.orchestra.console.smithi099:Power off completed 2023-10-08T14:20:11.458 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T14:20:11.568 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off 2023-10-08T14:20:11.668 INFO:teuthology.orchestra.console.smithi115:Power off completed 2023-10-08T14:20:11.773 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/default mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} duration: 3040.370613336563 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2023-10-08T14:20:11.773 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T14:20:11.859 INFO:teuthology.run:FAIL